Sharding phase 1 spec (RETIRED)

Because the fund is locked up for PROPOSER_LOCKUP_LENGTH. So deregister_proposer simply removes a proposer from the proposer_pool, and locks up its fund. It needs to call release_proposer to actually retrieve the funds after PROPOSER_LOCKUP_LENGTH

It was used for collations to point to a block in the mainchain.
Since we don’t have the field for now, just remove the check?

Yep, I commented it out. Just wondering why we removed it.

I don’t think so. If the proposer calls release_proposer without proposer_withdraw_balance, the funds are lost. proposer_withdraw_balance retrieves the funds, not release_proposer.

Why a rational proposer would call release_proposer?

So as I explained above with the current proposed spec:

  • a proposer can have balances in multiple shards
  • they must withdraw their balances from all shards, one at a time, with proposer_withdraw_balance(shard_id).
  • then once all balances are withdrawn they call release_proposer() to withdraw from the collator pool.

However, a proposer could withdraw all their balances and not call release_proposer(). Then that would fill up empty slots in the proposer_registry. So it seems that there should also be a check to periodically check after the lockup_length to see if proposer_registry[proposer_address].balances == 0, and if so, set proposer_registry[proposer_address].deregistered = 0, otherwise break and do nothing. But this is probably related to storage rent / time to live / pay-to-resurrect schemes, and intuitively I don’t think it poses a security risk.

Actually I just realised that the SMC as I have written it at the moment doesn’t really account that a proposer could have balances in multiple shards, so I’ll need to rewrite it to account for that.

So as I explained above:
The proposer has nothing to gain by calling release_proposer. So rationally this method will never be called, just wastes gas. This whole “periodically check” you’re proposing is a hack to fix a design flaw.

Why assert proposer_registry[proposer_address].balances == 0?
That’s not done in the current implementation, so why it should be done in the other one?
Just transfer whatever is in balances to the proposer.

Also, in the current specification, a proposer can withdraw only one at a time, if I deposited in the 100 shards, I must call deregister_proposer then proposer_withdraw_balance, 100 times. Since the lock up period is not per shard. Assuming proposer_withdraw_balance resets the lockup time (set the proposer as registered again).

P.S.
In your implementation:
self.collator_registry[self.collator_address].deregistered \ = self.collator_lockup_length
This is wrong, you’re saving a constant every time there. See: https://github.com/enriquefynn/go-ethereum/blob/smc_phase1/sharding/contracts/sharding_manager.sol, I’m also doing some stuff there.

OK, fair point. The balances aren’t stored in each shard, so they could be withdrawn with release_collator. But separating functionality also is conventional. In Vyper you could put a @private decorator before proposer_withdraw_balance(), and instead call proposer_withdraw_balance() from within release_proposer(). But a problem with that is that you can’t then pass shard_id directly from proposer_withdraw_balance(), you’d have to pass it to release_proposer(), then internally to proposer_withdraw_balance().

With separate functions, then the assert is there to make sure that balances are empty (from proposer_withdraw_balance() before releasing the collator, to make sure that funds are not lost. But if you call proposer_withdraw_balance() from inside release_proposer(), it still seems like best practice to make that assert to prevent lost funds. Of course I should’ve raised the change with others, but it is there to read.

That’s a fair point. As I mentioned, we can certainly have another function that withdraws all balances from all shards and then releases the proposer. It just adds more complexity because you need to keep track of where proposers are registered in each shard if you want to try to withdraw by shard_id. It certainly is simpler to do it the same way as release_collator and just release all balances, and not bother about withdrawing and adding to individual shards. However, there is an advantage in being able to withdraw and add to one shard, so we could do a similar thing for collators. But withdrawing and adding to individual shards is more of an enhancement that can be done later, as it does add complexity.

It’s good to have others scrutinizing it! I can move it to a separate private function that does this, and call the function. Do you have a better idea? It’s still a WIP, and I will certainly be reviewing it multiple times for security and optimizations, although I am sure that others will too, and I do need to work on the Rust side of things as well, but @ChosunOne has started contributing. Besides, I think all clients should scrutinize the Vyper contract, since the plan is to just interact with it, not have multiple SMCs. I’ll see if I can find time to look at your code.

What’s the advantage of using something different to RLP?

In my understanding, you do release_proposer to get back your deposit?
I mean, it seems balance and deposit are separate concepts.
So you mentioned proposer_withdraw_balance, it only withdraws the balance in a specific shard of that proposer, but the deposit is still locked up in the contract.
So the question might be, why do we need a deposit for each proposer? and why is it needed to be locked?(is there a slashing condition for proposers?)

1 Like

From the sharding spec, the collation pool is defined as “The set of all collators collectively participating in the security of all shards.” Does this mean that a collator must store all the state of all the shards at all times or is it ok for the collator to only keep the state of its currently assigned shard?

Additionally, does a collator need to download the entire state of the assigned shard, or just to the windback depth?

1 Like

Do we need proposer_address as an explicit collation header field? I’m thinking maybe we can save another 32 bytes and just recover the address from the signature, similar to how it’s done with transaction senders in Ethereum right now.

3 Likes

@jamesray1 @efynn @mhchia
If I understand correctly, there are deposit and balance[shard_id] for a proposer and they are two different thing. deposit is used for slashing(if the proposer did misbehave) and balance is used to pay the collator, so each of them will have their own withdraw function(proposer_withdraw_balance for balance withdrawal and release_proposer for deposit withdrawal).

But there are still a few questions left regarding proposer registry/deposit.

  1. Since proposer’s balance on each shard is a mapping, there’s no way to clean up the proposer’s registry entry(because there’s no way to delete a mapping unless you keep track of which shard the proposer added a balance).
  2. IIUC there seems to be no slashing condition applicable to proposer(at least for now?). Both proposal_commitment_slashing and availability slashing are applicable only to collator.
3 Likes

The PROPOSER_LOCKUP_LENGTH would be the same, for both balance and deposit, or just deposit?
Why have both? Why can’t the deposit be used to slash the proposer?
What if a proposer misbehave in different shards, it can be slashed as it was misbehaving in just 1? Since the slashing balance is not bound to the shards.

PROPOSER_LOCKUP_LENGTH is for deposit only and deposit is exactly used to slash the proposer(proposer’s deposit will get slashed if the proposer misbehave).
What are a proposer’s misbehaviors you have in mind? If there is one, it can be added to the slashing conditions.

For 1. proposer registry entry not been cleaned up seems not really harmful to me tho. But regarding mapping deletion without knowing the keys, IIRC it’s currently not supported by either Solidity or Vyper).

The advantage is that you can use a Merkle branch into the collation header structure which then identifies that there is a blob and that blob will definitely be processed. With RLP, you can’t tell if it’s valid without verifying the whole thing.

Fair enough, that should reduce I/O a lot right, which is the main bottleneck on the network at the moment. Good thinking!

It would be good to see how that would work in more detail or develop it.

What I would like to see is formal proofs of security for proposed specifications like this, covering security in fundamental properties, e.g.: “consistency and liveness in an asynchronous network with adversarial delays that are a-priori bounded, within a formal model allowing for adaptive corruption and spawning of new players, assuming that the computational puzzle is modeled as a random oracle.” This paper already proves that Nakamoto consensus blockchains already have these properties, and whatever changes we make should still have these properties. (With Casper, the part about the computational puzzle is not applicable, but the proposal mechanism should still be as random as practical or pseudorandom. Additionally there should be an analysis of security with synchronous cases such as cross-shard synchronous contract calls and internally synchronous zones.) Also do an analysis of the tendency towards centralization vs decentralization would be good, as well as whether there are asymmetries (particularly economic ones) between participants (e.g. like this analysis: Exploring the proposer/collator split), which can lead to a different Nash equilibrium than the intended one. Additionally assess whether there are externalities, which should be internalized for sustainability.

Then again, if you are just going to have a testnet without execution, then these research topics are less important.

The links to this post have broken because the heading has been updated. Please change it back to “Sharding phase 1 spec”. The update at the top of the post is clear enough.

Nevermind, old links work now, although they didn’t before.