Preparing withdrawals for compromised validator or withdrawal keys. FAO Zilm

Hey @benjaminchodroff, it’s almost a full year after you posted this reply and I’m stuck in a similar situation as both my eth1 Deposit address keys and the eth2 validator address are compromised. I have messaged you on your website and on LinkedIn as well so please excuse my persistence as I’m very stressed out over the threat of losing my 32eth stake. I’m doing the best I can to educate myself on EIP-4736 and any other related topic but I’m not really sure how to proceed. I want to ensure I beat the hackers to the withdraw when they are finally enabled. I may be at a disadvantage because I have no prior coding experience (I’m using a Plug & Play validator). Is there anything I can do now to prepare myself in the race for my staked eth?

1 Like

Hi @Tex419 - glad we were able to connect. CLWP has already received hundreds of validators that have been compromised by all number of reasons - hackers, scammers, nation state espionage, bankrupt companies, ex-girlfriends. I hope we can try to help, but I’d encourage getting involved and helping us out too.

I am writing a presentation on Consensus Layer Withdrawal Protection (CLWP) - an optional way for Ethereum Validators to broadcast their set withdrawal address as early as possible to a social community of nodes. I’d greatly appreciate any feedback. CLWP Overview - Google Slides

I will make a video demo of the process later this week.

1 Like

Ben, you are a legend. Just to confirm to Tex, sorry if you messaged us over christmas period and we did not get back to you, did you message us from www.clwp.xyz?

Anyway, Ben and JGM seem to have the solution, I am currently fumbling away to be the test dummy, feel free to follow our twitter page over the next few days and we will try and demonstrate the process, and while we do that I will learn from my mistakes so to help build a water tight tutorial with Bens help. We are on the case and will be very happy if we can help you and others.

Bare in mind I am running the twitter page https://twitter.com/EthCLWP , so sorry for the random posts, I just cant help myself, Twitter is crazy addictive, but never the less we are on the case, so please keep a close eye, lets see if I can follow Bens instructions over the next few days and with luck we will have full instructions very soon!

All the best

1 Like

Hi, I have accidentally set my 0x01… withdrawal account to a compromised one after the fork. This was for a partial withdrawal. Just wondering if you guys have a solution for this now so there might be a way to do a full exit and keep my 32 ETH, or was this only pre-Shanghai fork?
Thanks!

I am so sorry to hear this. Sadly this change was a one time change, the only thing you can do now is, exit, you may want to see if you can get help with a bot to withdraw funds from your compromised wallet at lightning speed, with luck you can get it out before the hacker does. Please remember if the wallet is compromised a hacker can watch any movements using Beaconcha.in automated email monitoring service, so you need to be very vigilant, by rights the validator will let you know what Epoch it will exit, so you will know when to move funds at lightning speed. Good luck, and sorry to hear this. Tobes

1 Like

Hi, I was googling about my problem and saw this. I have a weird issue: I started staking from day 1 with several validators, I used the deposit cli 1.0 and a mnemonic I’ve written down.
I read somewhere at the time that the deposit data json and keystores would be the things needed for withdrawals, they were kept safely with the mnemonic, it has been in cold storage for as long as possible. But for some reason, my mnemonic doesn’t work to update the withdrawal address. I’m in a weird situation:

  • I’ve been staking from day one and I’m still staking right now
  • I kept all the deposit and keystore files
  • I still have exclusive access to the funding address
  • I probably made a mistake writing the mnemonic, got the wrong one or there’s another issue I don’t know
  • And I can’t even recover my staking earnings

Is there a way with everything I have (running node, files, …) to recover the withdrawal private key.

Or is there some “social” help available as it was specified earlier (to be able to recover the funds when we can prove ownership of everything?) I’m reaching my technical limits here…

I’m afraid I’m in the same spot. Validator since day 1. I wrote down my mnemonic but it does not work with setting my withdrawal address. So my validators are stuck on 0x00 now. I’m assuming the mistake was on me i.e. I used a different mnemonic in the end.

Is there any hope that we will be able to set the withdrawal address in the (near) future? I still have access to the validator keys, deposit data and original deposit address. But no mnemonic.

@benjaminchodroff @3eph1r0th tagging you out of desperation.

Hi @maverickandy and @Sphearis

I really am so sorry to hear this news from you both. Sadly, if you have lost or partly forgotten your withdrawal key, there is no way to retrieve your funds that I know of, which ironically is pretty ridiculous and was something we did not think about at the time of building EIP-4736, however this could be an issue similar to what we started this thread about, ie: there could be a lot of people who have forgotten or lost their withdrawal key.

I have no clue how this can be resolved and can only hope a super brain core dev can come up with a solution, but it does sound very tricky because even if you were able to exit the node, how do you withdraw the ETH without a withdrawal address set :frowning:

I am so sorry I cannot offer better advice, I hope someone else can.

Good luck

CLWP was only for pre-Capella fork. Once a withdrawal address is set, it can never be changed. If the withdrawal address is compromised, it is no longer a Consensus Layer race, but instead an Execution Layer race. Your best bet is to contact Flashbots via their whitehat request form:

They can use their EL bots to race against any attacker and attempt to retrieve as much of the funds back. There is a cost, typically measured in a percentage of the funds, but this is technically your only option unless you intend to build the bot yourself. I highly recommend their services and expertise for your situation.

1 Like

There is no way to set a withdrawal address unless you have the validator seed phrase. The deposit address seed phrase and generated keystore files are not used during the setting of the withdrawal address, which is required to withdraw. However, let’s keep trying some ideas?

I’d encourage retracing your steps. Download the exact same version of staking-deposit-cli you used to generate your validator keys. Redo the process in a fresh directory, but try importing an existing mnemonic. You might find that you can regenerate the same pubkey as an existing validator, and then that means you have found the exact problem?

The main reason I am suggesting this is because staking-deposit-cli has removed a critical “–mnemonic-password” feature which might be the reason you are now hitting issues. Originally, their tooling supported an optional additional (and strongly not recommended, due to confusion) BIP38 passphrase for the mnemonic. This behaves essentially as a 25th word in your mnemonic. If you don’t use the correct mnemonic-password, you won’t be able to generate the correct private keys for the withdrawal.

While incredibly infuriating, I believe this original capability was removed at some point from the staking-deposit-cli – but the same argument is still available as a completely hidden feature if you pass it in to a newer versions of the staking-deposit-cli. It is entirely possible you set this additional passphrase while generating your seed phrase. Please note, the --mnemonic-password password may be completely different than your --keystore_password which is the password used inside the beacon node client to load the keystores (however, it’s entirely possible you set it to be identical). I find the idea that they hide this argument entirely in the documentation very concerning, as there may very well be others hitting this issue.

I am really hoping I guessed your problem correctly and this magically saves your validators. If it doesn’t, feel free to reach out to the EthStaker discord community as there are some incredible people who likely can suggest other clever ideas.

1 Like

Please see my response for @Sphearis and again, EthStaker discord for more help.

CLWP completed successfully with 100% success on all 2133 validators, but it is no longer useful for any future issues. If you hit issues with withdrawals, please use EthStaker and check out the #withdrawals channel. If you are certain you are compromised, you may also wish to contact Flashbots whitehat request for professional paid help. Keep your seed phrases secure and offline at all times. Best of luck.

1 Like

Last answer on this message here, I’ll use ethstaker discord from now on.

Thank you for your answer, it doesn’t really help me as I did all those things but some other people in the same situation may need this valuable advice, I was planning to release an article/blog or medium post about everything I’ve done after the facts but I’m not yet out of the woods…

I already checked all this, and learned a lot on the way, let’s take the positive out of it. Used that deposit 1.00 to regenerate the files like I did at that time and so on…

According to everything I have on hand, I’ve got 3 options here:

  • First one: I’ve got a 24 words mnemonic written on paper indeed, I used it and maybe permutated 2 words just to make it harder to reuse, this is unlikely as the checksum is correct in the current order. It’s not impossible though. I’ve got a python script using ethdo to derive permutations and show the withdrawal credentials for --path=“m/12381/3600/0/0/0”.

  • Second option, this is the one I “remember”: I had that 24 words mnemonic from a first deposit attempt and I think I finally decided to not go for it, I used my own 12 words one (from a cold stored wallet) thinking it would mean withdrawals would go to that specific address, and a password I’ve written down (several words, more like a passphrase) on that same piece of paper. I tried that with the mnemonic-password parameter with no luck. It’s unlikely I messed up with writing the password as it has to be written in the parameters, it’s shown on the screen in plain view and I know I triple check everything (I’ve got Bitcoin wallets from 2015 still recoverable (with no funds unfortunately). I already tested a few variations with no luck, maybe I can try with a password generator based on what I’ve written down but I didn’t find anything yet to do that.

  • Third option, the one I want and don’t want at the same time: I had that 24 words mnemonic from a first deposit attempt and I think I finally decided to not go for it, I used my own 12 words one (from ANOTHER cold stored wallet) and that written down password. Both were stored in a (supposedly) fireproof and waterproof safe but we had a flood and some of the material had still been damaged, some material has been relocated somewhere else (it has been 2 years ago and I had brain surgery amongst other things last year, this is clearly not a proper context for self custody, but I didn’t expect to wait for so long and that life would happen in the worst ways in between. I also thought all that time that I was safe with what I had “saved” for withdrawals). So, now I’m also trying to find that lost mnemonic, not knowing if it’s the right one or if it actually still exists somewhere.

That’s the way it is, it was lifechanging money, for me and my whole family who believed in me. We had a lot of dreams, all shattered for now. But a happy ending is still possible, and maybe I’ll be lucky with some proposal at some point. Whatever happens, my node is still running.

1 Like

Really tough to read your post, I think from experience if enough people encounter similar issues with luck eventually someone will help find a solution so please don’t give up.

Sadly you are not alone, maybe read this post by an Ethereum core dev, could it help?

Good luck :crossed_fingers:t2:

1 Like

@ Sphearis I symphatize with you. Just remember you’re not alone on this. Looks like I’ve donated 4 validators to the matrix as well. Use it as motivation to do better. No point in being a defeatist.

1 Like

Pretty similar in my case. Since the geth account management switch to “clef” recently, i used clef to create the withdrawal address/wallet. A “clef newaccount” gave my a fresh address, i created backups of my password (which i pasted from my password manager) and backups of clef’s “master seed” and if course the generated keyfiles(keystore/UTC… after focusing on getting the json right for the BLS2Execution change I was quite happy when the partial withdrawals finally arrived. But all changed when clef told be that the keyfile can’t be decrypted by the given password. Okay, let’s use clef’s “masterseed” capability to change the password. It turned out clef doesn’t really have HD wallet capabilities! The command “clef setpw” is merely there for setting a environment variable (password within clef context) and not modifying the keystore file like a “geth account update” would. Long story short i lost basically any option to access my funds and even a validator exit would just send everything to my locked withdrawal address. It’s pretty hard to believe that this one mistake (still not sure what happend during copy/paste the pwd from my keepass tool) renders the effort of the last 3 years worthless, especially as i still have full control over the validators themself.
I really hope that there is an option in the future to set e.g. a specific full withdrawl (exit) address or the option to change the withdrawal address one more time. :pray:

Hi @Sphearis and @maverickandy

I hope the below information helps you tremendously in recovering and changing your withdrawal keys.

I encountered a similar situation and I had all the necessary information recorded down correctly. Read through various guides and tried wagyu tool to no avail.

I used the deposit_CLI_version 1.1.0 to kickstart the process to deposit and stake eth for eth2 and realized that it was using path=“m/12381/3600/0/0/0”

There was a major difference when generating private key using path=“m/12381/3600/0/0/0” vs the widely suggested m/12381/3600/0/0/ (with 1 less 0).

Solution:
Please follow the ethdo steps on changing withdrawal address

ethdo account derive --mnemonic="24 words" --path="m/12381/3600/0/0/0" --show-private-key

Result will be generated as Private key: 0x

ethdo validator credentials set --offline --private-key=0x --withdrawal-address=0x

If it works, a change-operations.json will be created in folder and you can broadcasted to the beaconchain

Good luck and I hope the solution works for those who encountered a similar issue!

One person loosing the mnemonic of an execution layer account will have it’s funds lost indeed. Whereas in the beacon chain validator case, you must admit that there is a key difference: your validator can still be operational, even if you lost the mnemonic.

In a practical sense, there are two options for the affected user:

  • Stop operating the node. The slashing will occur block by block until the validator is automatically exited by the beacon chain when its balance falls down to 16ETH. Two things can happen now. If you had already added the withdrawal credentials, you’ll receive 16ETH when this day come. If you didn’t and still have a 0x00 withdraw type (prior to shapella upgrade) then i don’t know what happens (maybe @benjaminchodroff can help us uncover what happens in this case).

  • Keep operating the node and get the MEV reward only. Consensus rewards will be accumulated in the validator which you can’t access the funds as it requires the lost mnemonic. The MEV reward address is defined by the node operator and therefore you can still receive these. Finally you may hang on to the glimpse of hope that maybe… someday… , there will be a way to unlock the funds.

It is important to understand the two paths. One is deterministic, you stop operating, you lose 50% of the initial deposit and wait for the chain to trigger the validator exit. The second is optimistic, as it relies on random revenue generation (MEV only, which can greatly vary depending on luck to produce a block, and luck to get decent MEV on that particular block).

There is still one last case that we need to figure out: if you lost the seed before shapella and therefore still have a 0x00 withdrawal credential. You won’t be able to set a withdrawal address and if you decide to stop the validator, the 16ETH will be sent to … (please help us here @benjaminchodroff)

Edit, edit, edit sorry, just reread even further up. The situation sucks if the withdrawal address is already been set it’s not possible to set it again.

He would have to do like we did and see if there’s a way to convince the core Devs to be able to change the withdrawal address again.

To be slashed. He would have to fire up a second node with the same credentials that would get him slashed and he would be thrown out within 3 epoches which means he’d lose 1.5 E. Ben may have a better idea… The problem is here he needs to start up another validator with the same keys and if you’re saying he doesn’t have the same keys he can’t be slashed without doing something to be slashed. Though that might be possible!

So the only other thing he can do as you say is slowly bleed the Ethereum out from being switched off. Ben may have a better idea!

Having just read the OP message, sounds like a sticky situation. So I understand correctly doesn’t have access to the withdrawal address?

Sorry I can’t be more helpful, validator keys suck!

Unfortunately this did not work.

@benjaminchodroff is there a way to propose to the core devs that whenever a validator exit is initiated (by using the keystore…json and thus verifying ownership) AND no withdrawal address has been set (i.e. 0x00) then it defaults to the deposit address?

I’ve lost my mnemonic phrase (yes… very very very stupid I know) but I’ve been staking since day 1. Feels a bit harsh that I can exit my validator but I have no way to access the accumulated funds…

I am sorry for your loss. If you don’t have a withdrawal address set (0x00), all original deposit and accumulated consensus layer rewards will remain permanently locked to the 0x00 address until one is set (0x01) by using the validator mnemonic.

What you are proposing is different from a compromised validator withdrawal key. There is no “clean” solution to recovering a lost validator mnemonic – the wider community would never accept it. I did discuss that idea with many ETH R&D and Core Devs while proposing EIP-4736. Their thoughts were that changing consensus to set the execution layer deposit address as the default withdrawal address may inadvertently cause a security issue, and the original execution layer deposit address details are not currently stored in the consensus layer and non-trivial to add.

Depending on your residency and tax situations, you may want to consider filing a police report of missing property, which might help declare a casualty loss for itemized tax deduction purposes in some regions or homeowners insurance reimbursement depending on the nature of a disaster that destroyed the seed phrase (fire, flood, theft, etc.).

By continuing the run the validator, you can be technically still eligible to receive “fee recipient” rewards when your validator makes blocks proposals. It won’t be a lot, but it is more than nothing.

1 Like