That’s a bug in the infographic! Just corrected it, thank you!
@vbuterin, what about including a Bitcoin-NG style approach in the roadmap (e.g. phase 5 or 6)? Blocks are distinguished validating consensus blocks and non-consensus transaction micro-blocks. Additional features include:
- a poison transaction in each microblock pointing to the header of the first block in the pruned branch as a proof of fraud, accompanied with a maturity window for the reward, within which the poison transaction must be placed on the blockchain; and
- rewards split by ~40% for the miner/proposer of a leader block followed by micro-blocks and 60% to the leader of the next block).
I’m not sure if this can be used with Casper, however.
But maybe with this proposal it is not necessary or it would be more prone to adaptive adversaries.
compute_header_hash() also need
height: in128 and
proposer_signature: bytes as arguments to match with the collation header struct fields?
I think floor(block.number / PERIOD_LENGTH) + LOOKAHEAD_LENGTH >= period >= floor(block.number / PERIOD_LENGTH), otherwise we can get only collators for the current period, right?
If this function only gets the collator for the current period, you don’t need to check in
Since this is guaranteed by the
The roadmap is an active area of research. The outline below is only intended to provide flavour.
Phase 1: Basic sharding without EVM
- Blob shard without transactions
- Proposal commitments
- Collation availability challenges
Phase 2: EVM state transition function
- Full nodes only
- Asynchronous cross-contract calls only
- Account abstraction
- Archive accumulators: History, state, and asynchronous accumulators in the stateless model and Batching and cyclic partitioning of logs and Double-batched Merkle log accumulator
Phase 3: Light client state protocol
Phase 4: Cross-shard transactions
- Internally-synchronous zones: mind map including architectures
Phase 5: Tight coupling with main chain security
Phase 6: Super-quadratic sharding
- Recursively, shards within shards within shards…
- Load balancing: Wikipedia, search results. Related: History, state, and asynchronous accumulators in the stateless model, State minimized implementation on current evm
And a lot more: https://ethresear.ch/c/sharding.
Reposting the above roadmap to make it easier to link to it.
Why would a proposer call
The proposer can call
deregister_proposer, later withdraw the funds. and leave the registry untouched, never calling
It would be better if the proposer could withdraw with this method, effectively cleaning the registry. For example:
proposer_withdraw_balance, and modify
for s in
At the end of the call, delete the registry.
So you set the lockup period in
deregister_proposer, then only after the lockup period you can withdraw the funds, which AIUI should be done with
withdraw_balance, since that withdraws a balance for a shard, and a proposer may have balances in more than one shard.
For your proposed implementation you would need to enter each
shard_id as arguments to
release_proposer(), do the asserts for each shard, clear the balances in each shard, and still
assert proposer_registry[proposer_address].balances == 0 before setting
proposer_registry[proposer_address].deregistered = 0. Additionally, you would still need a separate
proposer_withdraw_balance() in case proposers just want to withdraw funds in one shard (but not all shards that they have balances, as may be the case), and then not
release_proposer(). So it seems better to keep the functions separate, but perhaps there could be a separate function to clear funds from all shards and call
release_proposer() at the end of it.
If you want to see how I’ve written the SMC so far, have a look here: https://github.com/Drops-of-Diamond/sharding/blob/develop/smc/Sharding_Manager_Contract.v.py. Speaking of which, I think I need to talk to @mhchia about this, for instance about what to do since there is no
period_start_prevhash. Perhaps you would like to review it and we can discuss on Gitter, e.g. a private chat? I think
expected_period_number is equivalent to
get_eligible_collator(shard_id, period) actually accepts
periods from current period to lookahead_period, while we only want
period to be the current period in
add_header. So we still need that check, if I didn’t understand your meaning wrong.
Because the fund is locked up for
deregister_proposer simply removes a proposer from the
proposer_pool, and locks up its fund. It needs to call
release_proposer to actually retrieve the funds after
It was used for collations to point to a block in the mainchain.
Since we don’t have the field for now, just remove the check?
Yep, I commented it out. Just wondering why we removed it.
I don’t think so. If the proposer calls
proposer_withdraw_balance, the funds are lost.
proposer_withdraw_balance retrieves the funds, not
Why a rational proposer would call
So as I explained above with the current proposed spec:
- a proposer can have balances in multiple shards
- they must withdraw their balances from all shards, one at a time, with
- then once all balances are withdrawn they call
release_proposer()to withdraw from the collator pool.
However, a proposer could withdraw all their balances and not call
release_proposer(). Then that would fill up empty slots in the
proposer_registry. So it seems that there should also be a check to periodically check after the lockup_length to see if
proposer_registry[proposer_address].balances == 0, and if so, set
proposer_registry[proposer_address].deregistered = 0, otherwise
break and do nothing. But this is probably related to storage rent / time to live / pay-to-resurrect schemes, and intuitively I don’t think it poses a security risk.
Actually I just realised that the SMC as I have written it at the moment doesn’t really account that a proposer could have balances in multiple shards, so I’ll need to rewrite it to account for that.
So as I explained above:
The proposer has nothing to gain by calling
release_proposer. So rationally this method will never be called, just wastes gas. This whole “periodically check” you’re proposing is a hack to fix a design flaw.
proposer_registry[proposer_address].balances == 0?
That’s not done in the current implementation, so why it should be done in the other one?
Just transfer whatever is in balances to the proposer.
Also, in the current specification, a proposer can withdraw only one at a time, if I deposited in the 100 shards, I must call
proposer_withdraw_balance, 100 times. Since the lock up period is not per shard. Assuming
proposer_withdraw_balance resets the lockup time (set the proposer as registered again).
In your implementation:
self.collator_registry[self.collator_address].deregistered \ = self.collator_lockup_length
This is wrong, you’re saving a constant every time there. See: https://github.com/enriquefynn/go-ethereum/blob/smc_phase1/sharding/contracts/sharding_manager.sol, I’m also doing some stuff there.
OK, fair point. The balances aren’t stored in each shard, so they could be withdrawn with
release_collator. But separating functionality also is conventional. In Vyper you could put a
@private decorator before
proposer_withdraw_balance(), and instead call
proposer_withdraw_balance() from within
release_proposer(). But a problem with that is that you can’t then pass
shard_id directly from
proposer_withdraw_balance(), you’d have to pass it to
release_proposer(), then internally to
With separate functions, then the assert is there to make sure that balances are empty (from
proposer_withdraw_balance() before releasing the collator, to make sure that funds are not lost. But if you call
proposer_withdraw_balance() from inside
release_proposer(), it still seems like best practice to make that assert to prevent lost funds. Of course I should’ve raised the change with others, but it is there to read.
That’s a fair point. As I mentioned, we can certainly have another function that withdraws all balances from all shards and then releases the proposer. It just adds more complexity because you need to keep track of where proposers are registered in each shard if you want to try to withdraw by
shard_id. It certainly is simpler to do it the same way as
release_collator and just release all balances, and not bother about withdrawing and adding to individual shards. However, there is an advantage in being able to withdraw and add to one shard, so we could do a similar thing for collators. But withdrawing and adding to individual shards is more of an enhancement that can be done later, as it does add complexity.
It’s good to have others scrutinizing it! I can move it to a separate private function that does this, and call the function. Do you have a better idea? It’s still a WIP, and I will certainly be reviewing it multiple times for security and optimizations, although I am sure that others will too, and I do need to work on the Rust side of things as well, but @ChosunOne has started contributing. Besides, I think all clients should scrutinize the Vyper contract, since the plan is to just interact with it, not have multiple SMCs. I’ll see if I can find time to look at your code.
What’s the advantage of using something different to RLP?
In my understanding, you do
release_proposer to get back your deposit?
I mean, it seems
deposit are separate concepts.
So you mentioned
proposer_withdraw_balance, it only withdraws the
balance in a specific shard of that proposer, but the
deposit is still locked up in the contract.
So the question might be, why do we need a
deposit for each proposer? and why is it needed to be locked?(is there a slashing condition for proposers?)
From the sharding spec, the collation pool is defined as “The set of all collators collectively participating in the security of all shards.” Does this mean that a collator must store all the state of all the shards at all times or is it ok for the collator to only keep the state of its currently assigned shard?
Additionally, does a collator need to download the entire state of the assigned shard, or just to the windback depth?