If it’s at the p2p layer than canonical serialization doesn’t matter as much; it’s only consensus layer objects where we have to really worry about canonical serialization.
For consensus layer, the existing python beacon chain library has a serialization spec built in. It could probably be improved on I suppose. I think as far as desiderata go, what we’re looking for is:
- For fully statically sized data (int16, hash32, int64, etc), including static-length lists and structs of statically sized data, you can deterministically compute the byte offset to access any specific value.
- Be as simple as possible.
- It’s OK to require a type signature to decode a particular object
For statically sized data, simple concatenation (ie. as done in SimpleSerialize linked) may well be the most practical way to go. For dynamically sized data, we could also consider using hash trees (ie. whenever there is a dynamically sized list, make a Merkle tree of the leaves, and whenever there is a dynamically sized byte array, pad it and make a Merkle tree of the chunks).