Yes, the SMC will run on the main chain in phase 1. Using Vyper I believe that block.number
will fetch the latest block number, but I am yet to run and test the SMC to confirm this . Later onwith phase 6 when we have an SMC in a shard and recursively have shards within shards and so on, then you would need to do a call to the main/mother shard to get the block number.
AIUI there does not need to be a balances key, because when a collator is added to the collator_registry
in the register_collator
function, they must have a msg.value >= collator_deposit
. So by being added to the collator_registry
then implicitly, their balance must be >= collator_deposit
. Thus, storing the collator_registry.balance
seems to be unnecessary. You can see how I’ve implemented this at https://github.com/Drops-of-Diamond/sharding/blob/develop/smc/Sharding_Manager_Contract.v.py.
That’s a bug in the infographic! Just corrected it, thank you!
@vbuterin, what about including a Bitcoin-NG style approach in the roadmap (e.g. phase 5 or 6)? Blocks are distinguished validating consensus blocks and non-consensus transaction micro-blocks. Additional features include:
- a poison transaction in each microblock pointing to the header of the first block in the pruned branch as a proof of fraud, accompanied with a maturity window for the reward, within which the poison transaction must be placed on the blockchain; and
- rewards split by ~40% for the miner/proposer of a leader block followed by micro-blocks and 60% to the leader of the next block).
I’m not sure if this can be used with Casper, however.
But maybe with this proposal it is not necessary or it would be more prone to adaptive adversaries.
Does compute_header_hash()
also need height: in128
and proposer_signature: bytes[8192]
as arguments to match with the collation header struct fields?
I think floor(block.number / PERIOD_LENGTH) + LOOKAHEAD_LENGTH >= period >= floor(block.number / PERIOD_LENGTH), otherwise we can get only collators for the current period, right?
If this function only gets the collator for the current period, you don’t need to check in add_header
for:
Since this is guaranteed by the get_eligible_collator
The VMC used period_start_prevhash
as a collation_header
field. What’s the alternative? @mhchia @NicLin
Roadmap
The roadmap is an active area of research. The outline below is only intended to provide flavour.
-
Phase 1: Basic sharding without EVM
- Blob shard without transactions
- Proposers
- Proposal commitments
- Collation availability challenges
-
Phase 2: EVM state transition function
- Full nodes only
- Asynchronous cross-contract calls only
- Account abstraction
- eWASM
- Archive accumulators: History, state, and asynchronous accumulators in the stateless model and Batching and cyclic partitioning of logs and Double-batched Merkle log accumulator
Storage rent
-
Phase 3: Light client state protocol
- Executors
- State-minimized clients. Stateless clients are not ideal as we don’t want to offload all storage into secondary markets, rather we can give people a choice to pay storage rent on the blockchain or pay for it in secondary markets.
-
Phase 4: Cross-shard transactions
- Internally-synchronous zones: mind map including architectures
-
Phase 5: Tight coupling with main chain security
- Data availability proofs: A note on data availability and erasure coding, Sharding and data forgetfulness,
- Casper integration: Alpha testnet, papers, wiki post (probably outdated).
- Internally fork-free sharding
- Manager shard
-
Phase 6: Super-quadratic sharding
- Recursively, shards within shards within shards…
- Load balancing: Wikipedia, search results. Related: History, state, and asynchronous accumulators in the stateless model, State minimized implementation on current evm
And a lot more: https://ethresear.ch/c/sharding.
Reposting the above roadmap to make it easier to link to it.
Why would a proposer call release_proposer
?
The proposer can call deregister_proposer
, later withdraw the funds. and leave the registry untouched, never calling release_proposer
.
It would be better if the proposer could withdraw with this method, effectively cleaning the registry. For example:
eliminate proposer_withdraw_balance
, and modify release_proposer()
as release_proposer(uint[] withdraw_shards)
.
for s in withdraw_shards
, withdraw release_proposer.balances[s]
At the end of the call, delete the registry.
So you set the lockup period in deregister_proposer
, then only after the lockup period you can withdraw the funds, which AIUI should be done with withdraw_balance
, since that withdraws a balance for a shard, and a proposer may have balances in more than one shard.
For your proposed implementation you would need to enter each shard_id
as arguments to release_proposer()
, do the asserts for each shard, clear the balances in each shard, and still assert proposer_registry[proposer_address].balances == 0
before setting proposer_registry[proposer_address].deregistered = 0
. Additionally, you would still need a separate proposer_withdraw_balance()
in case proposers just want to withdraw funds in one shard (but not all shards that they have balances, as may be the case), and then not release_proposer()
. So it seems better to keep the functions separate, but perhaps there could be a separate function to clear funds from all shards and call release_proposer()
at the end of it.
If you want to see how I’ve written the SMC so far, have a look here: https://github.com/Drops-of-Diamond/sharding/blob/develop/smc/Sharding_Manager_Contract.v.py. Speaking of which, I think I need to talk to @mhchia about this, for instance about what to do since there is no period_start_prevhash
. Perhaps you would like to review it and we can discuss on Gitter, e.g. a private chat? I think expected_period_number
is equivalent to period
.
get_eligible_collator(shard_id, period)
actually accepts period
s from current period to lookahead_period, while we only want period
to be the current period in add_header
. So we still need that check, if I didn’t understand your meaning wrong.
Because the fund is locked up for PROPOSER_LOCKUP_LENGTH
. So deregister_proposer
simply removes a proposer from the proposer_pool
, and locks up its fund. It needs to call release_proposer
to actually retrieve the funds after PROPOSER_LOCKUP_LENGTH
It was used for collations to point to a block in the mainchain.
Since we don’t have the field for now, just remove the check?
Yep, I commented it out. Just wondering why we removed it.
I don’t think so. If the proposer calls release_proposer
without proposer_withdraw_balance
, the funds are lost. proposer_withdraw_balance
retrieves the funds, not release_proposer
.
Why a rational proposer would call release_proposer
?
So as I explained above with the current proposed spec:
- a proposer can have balances in multiple shards
- they must withdraw their balances from all shards, one at a time, with
proposer_withdraw_balance(shard_id)
. - then once all balances are withdrawn they call
release_proposer()
to withdraw from the collator pool.
However, a proposer could withdraw all their balances and not call release_proposer()
. Then that would fill up empty slots in the proposer_registry
. So it seems that there should also be a check to periodically check after the lockup_length to see if proposer_registry[proposer_address].balances == 0
, and if so, set proposer_registry[proposer_address].deregistered = 0
, otherwise break
and do nothing. But this is probably related to storage rent / time to live / pay-to-resurrect schemes, and intuitively I don’t think it poses a security risk.
Actually I just realised that the SMC as I have written it at the moment doesn’t really account that a proposer could have balances in multiple shards, so I’ll need to rewrite it to account for that.
So as I explained above:
The proposer has nothing to gain by calling release_proposer
. So rationally this method will never be called, just wastes gas. This whole “periodically check” you’re proposing is a hack to fix a design flaw.
Why assert proposer_registry[proposer_address].balances == 0
?
That’s not done in the current implementation, so why it should be done in the other one?
Just transfer whatever is in balances to the proposer.
Also, in the current specification, a proposer can withdraw only one at a time, if I deposited in the 100 shards, I must call deregister_proposer
then proposer_withdraw_balance
, 100 times. Since the lock up period is not per shard. Assuming proposer_withdraw_balance
resets the lockup time (set the proposer as registered again).
P.S.
In your implementation:
self.collator_registry[self.collator_address].deregistered \ = self.collator_lockup_length
This is wrong, you’re saving a constant every time there. See: https://github.com/enriquefynn/go-ethereum/blob/smc_phase1/sharding/contracts/sharding_manager.sol, I’m also doing some stuff there.