Got it. Let the race begin. Thank you, again. I will start documenting the social mechanisms outlined for those of us impacted/suspected impacted to plea for validator help to consider in a github project. The list will be manually moderated, but independently verifiable. I don’t think I can help anyone who lost their keys or seed, but for those of us compromised or concerned they may be, it may help give an edge.
Hi. I am the founder of Allnodes. We host a lot of Ethereum 2.0 nodes and I have met a lot of people who exposed their keys to scammers and even who completely lost access to their withdrawal keys. I am sure that we should help these people.
Solution that I propose:
You can execute transaction to change withdrawal credentials from these wallets:
- Initial deposit address (this transaction can be reverted using current withdrawal address and withdrawal key can be changed only to initial deposit address).
- Current withdrawal address (this transaction cannot be reverted, but withdrawal key can be changed only to initial deposit address).
Because of possible revert option these transactions should be possible to execute long before withdrawals are enabled.
Whom this solution will help:
- Those who lost access to their withdrawal keys.
- Those who’s withdrawal keys were breached.
Thank you for your time.
Hi Seph and Allnodes team,
It is wonderful of you to have joined the discussion. Many thanks. However I am thinking you may have scrolled back in our chat at allnodes.com and clicked an old link, I did send you a link via email to a GitHub proposal where this topic has been worked on significantly since this thread. I have sent you the link again via the allnodes.com chat. I hope you can find the time to check it out as it will probably answer many questions you may have, and with luck the Devs and yourself can continue the discussion on the Discord group which I will ask @benjaminchodroff to pass you/me an invite that I can pass to you.
I am super grateful you made the effort to chime into the discussion, when you read the GitHub proposal I think you will see the Devs have come up with something quite special. Consensus Layer Withdrawal Protection.
My understanding is the EIP needs to be reviewed and there is still chance for some changes. It is a very important proposal, and sounds like you agree it could potentially save hundreds of millions of $ of ETH.
Thanks kindly for taking a look and your help.
@3eph1r0th Really appreciate your support from Allnodes. I fully agree that there are many users who are either aware or even unaware their validator mnemonic has been compromised.
I have documented the suggestions from the developers into a draft proposal, which covers your suggestions. None of the proposal changes consensus. Instead, we have suggested mechanisms which favors (not guarantees) legitimate users can likely win through voluntary node operator behavior. This includes using Initial Deposit Address as a tiebreaker for any race conditions, asking clients to voluntarily delay rebroadcasting change addresses which do not match the deposit address, filter rebroadcasting messages which do not match the intended signed change withdrawal address, and providing a mechanism to broadcast signed change address messages to any address on the first available block.
I am now working to rework this proposal into the EIP template, review with Ethereum Cat Herders for feedback, and get more developer feedback. I hope to eventually create a client which demonstrates these ideas in action. If anyone can point me to a client codebase which already supports the change withdrawal address operation, I will try to make a pull request against it.
Anyone can reach out to me via github, or my discord is @benjaminchodroff#5260. I appreciate any feedback and help.
I have submitted an EIP pull request: https://github.com/ethereum/EIPs/pull/4736
Ongoing EIP discussion will happen here: Consensus Layer Withdrawal Protection - EIPs - Fellowship of Ethereum Magicians
I appreciate any help or feedback.
Hey @benjaminchodroff, it’s almost a full year after you posted this reply and I’m stuck in a similar situation as both my eth1 Deposit address keys and the eth2 validator address are compromised. I have messaged you on your website and on LinkedIn as well so please excuse my persistence as I’m very stressed out over the threat of losing my 32eth stake. I’m doing the best I can to educate myself on EIP-4736 and any other related topic but I’m not really sure how to proceed. I want to ensure I beat the hackers to the withdraw when they are finally enabled. I may be at a disadvantage because I have no prior coding experience (I’m using a Plug & Play validator). Is there anything I can do now to prepare myself in the race for my staked eth?
Hi @Tex419 - glad we were able to connect. CLWP has already received hundreds of validators that have been compromised by all number of reasons - hackers, scammers, nation state espionage, bankrupt companies, ex-girlfriends. I hope we can try to help, but I’d encourage getting involved and helping us out too.
I am writing a presentation on Consensus Layer Withdrawal Protection (CLWP) - an optional way for Ethereum Validators to broadcast their set withdrawal address as early as possible to a social community of nodes. I’d greatly appreciate any feedback. CLWP Overview - Google Slides
I will make a video demo of the process later this week.
Ben, you are a legend. Just to confirm to Tex, sorry if you messaged us over christmas period and we did not get back to you, did you message us from www.clwp.xyz?
Anyway, Ben and JGM seem to have the solution, I am currently fumbling away to be the test dummy, feel free to follow our twitter page over the next few days and we will try and demonstrate the process, and while we do that I will learn from my mistakes so to help build a water tight tutorial with Bens help. We are on the case and will be very happy if we can help you and others.
Bare in mind I am running the twitter page https://twitter.com/EthCLWP , so sorry for the random posts, I just cant help myself, Twitter is crazy addictive, but never the less we are on the case, so please keep a close eye, lets see if I can follow Bens instructions over the next few days and with luck we will have full instructions very soon!
All the best
Hi, I have accidentally set my 0x01… withdrawal account to a compromised one after the fork. This was for a partial withdrawal. Just wondering if you guys have a solution for this now so there might be a way to do a full exit and keep my 32 ETH, or was this only pre-Shanghai fork?
I am so sorry to hear this. Sadly this change was a one time change, the only thing you can do now is, exit, you may want to see if you can get help with a bot to withdraw funds from your compromised wallet at lightning speed, with luck you can get it out before the hacker does. Please remember if the wallet is compromised a hacker can watch any movements using Beaconcha.in automated email monitoring service, so you need to be very vigilant, by rights the validator will let you know what Epoch it will exit, so you will know when to move funds at lightning speed. Good luck, and sorry to hear this. Tobes
Hi, I was googling about my problem and saw this. I have a weird issue: I started staking from day 1 with several validators, I used the deposit cli 1.0 and a mnemonic I’ve written down.
I read somewhere at the time that the deposit data json and keystores would be the things needed for withdrawals, they were kept safely with the mnemonic, it has been in cold storage for as long as possible. But for some reason, my mnemonic doesn’t work to update the withdrawal address. I’m in a weird situation:
- I’ve been staking from day one and I’m still staking right now
- I kept all the deposit and keystore files
- I still have exclusive access to the funding address
- I probably made a mistake writing the mnemonic, got the wrong one or there’s another issue I don’t know
- And I can’t even recover my staking earnings
Is there a way with everything I have (running node, files, …) to recover the withdrawal private key.
Or is there some “social” help available as it was specified earlier (to be able to recover the funds when we can prove ownership of everything?) I’m reaching my technical limits here…
I’m afraid I’m in the same spot. Validator since day 1. I wrote down my mnemonic but it does not work with setting my withdrawal address. So my validators are stuck on 0x00 now. I’m assuming the mistake was on me i.e. I used a different mnemonic in the end.
Is there any hope that we will be able to set the withdrawal address in the (near) future? I still have access to the validator keys, deposit data and original deposit address. But no mnemonic.
I really am so sorry to hear this news from you both. Sadly, if you have lost or partly forgotten your withdrawal key, there is no way to retrieve your funds that I know of, which ironically is pretty ridiculous and was something we did not think about at the time of building EIP-4736, however this could be an issue similar to what we started this thread about, ie: there could be a lot of people who have forgotten or lost their withdrawal key.
I have no clue how this can be resolved and can only hope a super brain core dev can come up with a solution, but it does sound very tricky because even if you were able to exit the node, how do you withdraw the ETH without a withdrawal address set
I am so sorry I cannot offer better advice, I hope someone else can.
CLWP was only for pre-Capella fork. Once a withdrawal address is set, it can never be changed. If the withdrawal address is compromised, it is no longer a Consensus Layer race, but instead an Execution Layer race. Your best bet is to contact Flashbots via their whitehat request form:
They can use their EL bots to race against any attacker and attempt to retrieve as much of the funds back. There is a cost, typically measured in a percentage of the funds, but this is technically your only option unless you intend to build the bot yourself. I highly recommend their services and expertise for your situation.
There is no way to set a withdrawal address unless you have the validator seed phrase. The deposit address seed phrase and generated keystore files are not used during the setting of the withdrawal address, which is required to withdraw. However, let’s keep trying some ideas?
I’d encourage retracing your steps. Download the exact same version of staking-deposit-cli you used to generate your validator keys. Redo the process in a fresh directory, but try importing an existing mnemonic. You might find that you can regenerate the same pubkey as an existing validator, and then that means you have found the exact problem?
The main reason I am suggesting this is because staking-deposit-cli has removed a critical “–mnemonic-password” feature which might be the reason you are now hitting issues. Originally, their tooling supported an optional additional (and strongly not recommended, due to confusion) BIP38 passphrase for the mnemonic. This behaves essentially as a 25th word in your mnemonic. If you don’t use the correct mnemonic-password, you won’t be able to generate the correct private keys for the withdrawal.
While incredibly infuriating, I believe this original capability was removed at some point from the staking-deposit-cli – but the same argument is still available as a completely hidden feature if you pass it in to a newer versions of the staking-deposit-cli. It is entirely possible you set this additional passphrase while generating your seed phrase. Please note, the --mnemonic-password password may be completely different than your --keystore_password which is the password used inside the beacon node client to load the keystores (however, it’s entirely possible you set it to be identical). I find the idea that they hide this argument entirely in the documentation very concerning, as there may very well be others hitting this issue.
I am really hoping I guessed your problem correctly and this magically saves your validators. If it doesn’t, feel free to reach out to the EthStaker discord community as there are some incredible people who likely can suggest other clever ideas.
Please see my response for @Sphearis and again, EthStaker discord for more help.
CLWP completed successfully with 100% success on all 2133 validators, but it is no longer useful for any future issues. If you hit issues with withdrawals, please use EthStaker and check out the #withdrawals channel. If you are certain you are compromised, you may also wish to contact Flashbots whitehat request for professional paid help. Keep your seed phrases secure and offline at all times. Best of luck.
Last answer on this message here, I’ll use ethstaker discord from now on.
Thank you for your answer, it doesn’t really help me as I did all those things but some other people in the same situation may need this valuable advice, I was planning to release an article/blog or medium post about everything I’ve done after the facts but I’m not yet out of the woods…
I already checked all this, and learned a lot on the way, let’s take the positive out of it. Used that deposit 1.00 to regenerate the files like I did at that time and so on…
According to everything I have on hand, I’ve got 3 options here:
First one: I’ve got a 24 words mnemonic written on paper indeed, I used it and maybe permutated 2 words just to make it harder to reuse, this is unlikely as the checksum is correct in the current order. It’s not impossible though. I’ve got a python script using ethdo to derive permutations and show the withdrawal credentials for --path=“m/12381/3600/0/0/0”.
Second option, this is the one I “remember”: I had that 24 words mnemonic from a first deposit attempt and I think I finally decided to not go for it, I used my own 12 words one (from a cold stored wallet) thinking it would mean withdrawals would go to that specific address, and a password I’ve written down (several words, more like a passphrase) on that same piece of paper. I tried that with the mnemonic-password parameter with no luck. It’s unlikely I messed up with writing the password as it has to be written in the parameters, it’s shown on the screen in plain view and I know I triple check everything (I’ve got Bitcoin wallets from 2015 still recoverable (with no funds unfortunately). I already tested a few variations with no luck, maybe I can try with a password generator based on what I’ve written down but I didn’t find anything yet to do that.
Third option, the one I want and don’t want at the same time: I had that 24 words mnemonic from a first deposit attempt and I think I finally decided to not go for it, I used my own 12 words one (from ANOTHER cold stored wallet) and that written down password. Both were stored in a (supposedly) fireproof and waterproof safe but we had a flood and some of the material had still been damaged, some material has been relocated somewhere else (it has been 2 years ago and I had brain surgery amongst other things last year, this is clearly not a proper context for self custody, but I didn’t expect to wait for so long and that life would happen in the worst ways in between. I also thought all that time that I was safe with what I had “saved” for withdrawals). So, now I’m also trying to find that lost mnemonic, not knowing if it’s the right one or if it actually still exists somewhere.
That’s the way it is, it was lifechanging money, for me and my whole family who believed in me. We had a lot of dreams, all shattered for now. But a happy ending is still possible, and maybe I’ll be lucky with some proposal at some point. Whatever happens, my node is still running.
Really tough to read your post, I think from experience if enough people encounter similar issues with luck eventually someone will help find a solution so please don’t give up.
Sadly you are not alone, maybe read this post by an Ethereum core dev, could it help?
@ Sphearis I symphatize with you. Just remember you’re not alone on this. Looks like I’ve donated 4 validators to the matrix as well. Use it as motivation to do better. No point in being a defeatist.
Pretty similar in my case. Since the geth account management switch to “clef” recently, i used clef to create the withdrawal address/wallet. A “clef newaccount” gave my a fresh address, i created backups of my password (which i pasted from my password manager) and backups of clef’s “master seed” and if course the generated keyfiles(keystore/UTC… after focusing on getting the json right for the BLS2Execution change I was quite happy when the partial withdrawals finally arrived. But all changed when clef told be that the keyfile can’t be decrypted by the given password. Okay, let’s use clef’s “masterseed” capability to change the password. It turned out clef doesn’t really have HD wallet capabilities! The command “clef setpw” is merely there for setting a environment variable (password within clef context) and not modifying the keystore file like a “geth account update” would. Long story short i lost basically any option to access my funds and even a validator exit would just send everything to my locked withdrawal address. It’s pretty hard to believe that this one mistake (still not sure what happend during copy/paste the pwd from my keepass tool) renders the effort of the last 3 years worthless, especially as i still have full control over the validators themself.
I really hope that there is an option in the future to set e.g. a specific full withdrawl (exit) address or the option to change the withdrawal address one more time.