Block Arrivals, Home Stakers & Bumping the blob count
Thanks to all the community members that are selflessly sharing their data that allowed this analysis to be possible, to MigaLabs for their validator entity data and to ProbeLab for an early version of their Mainnet bandwidth report.
Special thanks to Parithosh, Toni, Mikel, Andrew, and Matty for their feedback and help throughout this analysis.
TL;DR
Summary
- Timing games have historically made it hard to predict a safe blob count bump
- ethPandaOps recently started receiving block arrival timing data from the community
- Analysis of the most bandwidth sensitive scenario shows a healthy block & blobs arrival time
Outcomes
- When naively extrapolating this data and combining with EIP7623, this analysis supports increasing the blob count in EIP7691 to either 4/8 or 6/9.
- This is only one data point, but it leads to a more optimistic outlook on a potential blob count bump than previous analysis.
Intro
The ethPandaOps team has recently started receiving data from members of the community. Home stakers are one of the Ethereum networkâs most valuable assets, and this scheme is starting to shine a light on how they see the network. A sidecar binary is run alongside a beacon node, and records the events happening on the node via the beacon API event stream. These events are sent to the ethPandaOps team, who then publishes the data (with a small delay & some data scrubbing).
For more info:
The team has been collecting community data for around 6 weeks, and we now have enough data to make some interesting observations that were previously not possible.
Background
With the arrival of EIP4844, a block is only considered valid once the block & the blobs referenced in the block are received by a node. This block/blob bundle has until 4s in the slot to be received by the majority of the network otherwise it runs the risk of being re-orged.
Sophisticated operators (and now MEV-Relays) play block proposal timing games. These operators submit their blocks as late as possible to maximise their profit while minimizing their risk of being re-orged. These timing games have historically obfuscated block arrival timing data.
Unreleased, upcoming research from ProbeLab indicates that the 50th percentile of non-cloud nodes on the network have 20Mbps of upload bandwidth.
Blob usage on Mainnet continues to grow.
Problem statement
Ethereumâs decentralization is paramount to itâs success. EIP7691 aims to increase the blob count, and runs the risk of unintentionally excluding some node operators from itâs set if the parameters are too high.
We need to:
- Ensure that a blob count increase is safe for home stakers, as this group of actors is the âworst caseâ for a blob count increase as they have the lowest available bandwidth.
- Ensure that the network has enough data throughput to support Layer 2 growth.
Given the existing landscape, we can make some assumptions with regards to a potential blob count increase:
Least at risk:
Counterintuitively, if you looked at block arrival data, operators playing timing games would appear to be the most at risk of being impacted by a blob count increase. Being reorged out for proposing late is bad for business, and we can assume theyâll adjust their block proposal timings accordingly. A blob count increase is unlikely to be problematic here.
Most at risk:
A solo staker building a block locally (no mev-relay) and being attested to by other home stakers. In this scenario:
- The proposer:
- needs to publish their block and all blobs to the network. This node needs to publish the block (~100KB), and then all of the blobs (128KB each) to all of its mesh peers as fast as possible.
- when building blocks locally, the proposer does not have the help of the MEV Relay gossiping the block/blobs bundle to its own peers.
- needs to publish their block and all blobs to the network. This node needs to publish the block (~100KB), and then all of the blobs (128KB each) to all of its mesh peers as fast as possible.
- The attesters:
- need to download the block/blobs bundle from the p2p network before 4s in to the slot.
This analysis will ask the following questions:
- Question 1: How is 3/6 performing on Mainnet?
- Question 2: Does arrival time scale with block/blob bundle size?
- Question 3: How much more can we support on Mainnet today?
Weâll answer these questions from the perspective of a home staker as this is our most at-risk group of operators.
Analysis
Start: 2024-10-04T22:00:00Z
End: 2024-11-25T02:00:00Z
Blocks: 366,384
Arrival events: 75,945,392
Countries: 9 (Australia, Bulgaria, Czechia, Germany, Italy, Spain, The Netherlands, United Kingdom, United States)
Check out the juypter notebook here.
The timing data was captured by Xatu Sentry, a sidecar binary that runs alongside a beacon node and records many events via the beacon API event stream. These events can be considered worst case for block arrival times as there is additional processing time and network overhead for the event to be recorded. From analysis, this overhead ranges between 50-300ms but we have kept the timing data as-is in the interest of safety.
Each beacon node implementation emits block, head, and blob_sidecar events in different orders. To address this, we define a block/blob bundle as âarrivedâ only after all associated events for the slot have been recorded from each beacon node. This is once again a worst case scenario.
Question 1
How is 3/6 performing on Mainnet?
TL;DR: Pretty well!
This chart shows block/blob bundle seen arrival times against the combined size of the bundle. When looking at locally built blocks proposed by solo stakers and seen by home users, a lot of block/blob bundles are seen before 4s.
We should also look at blocks provided by MEV Relay as the reality is that a lot of blocks are proposed via this route. We can see an increase in the min, as there are additional round trips involved in this process, but things still look healthy!
Outcome: Block/blob bundles are arriving well within the 4s deadline for our home users.
Question 2
Does arrival time scale with block/blob bundle size?
TL;DR: Yes
The trend lines show the 99th, 95th, and 50th percentiles of arrival times - meaning what percentage of blocks arrive were seen faster than that line.
These percentile trend lines answer our question: as bundle sizes increase, arrival times also increase. This suggests bandwidth is the primary bottleneck for these actors.
Again looking at blocks provided via MEV Relay, we see a similar story.
Outcome: Yes, arrival times scale with block/blob bundles size
Question 3:
How much more can we support on Mainnet today?
TLDR: 4/8 and 6/9 are both achievable
To answer this question we need to check how big the block is. The 99th percentile of compressed beacon block size through our time period is 101KB. Our blobs are fixed at a size of 128KB.
Using these parameters, we can overlay the block/blob count:
We can very naively plot a trend line to see when would cross the 4s attestation deadline.
This trend line assumes a linear relationship between blob count and arrival time. Under this assumption, we can support up to 14 blobs per block while maintaining 95% of block/blob bundles arriving within the deadline. The 95% target provides margin for the 50-300ms processing overhead in our measurements, while also accounting for outliers.
When looking at MEV Relay blocks specifically, the data shows an even more optimistic picture - the 95th percentile trend line indicates support for up to 20 blobs per block. This improved performance can be attributed to MEV Relays being high-bandwidth nodes that help distribute blocks and blobs across the network in parallel with the proposer.
EIP7623
EIP7623 improves the worst case compressed block size to ~720KB - about 7x larger than our average historical block. Letâs analyze if we can still support more blobs with this increased block size.
Even with an absolute worst case block size with EIP7623 we still support a blob increase. Note that the current maximum compressed block size on Mainnet is 1.79MB (and weâre seemingly going ok!), so take this data point with a grain of salt.
The trend for MEV Relay blocks again supports a much higher blob count compared to locally built blocks.
Outcome: The data supports a blob count increase, especially if EIP7623 is included at the same time. 4/8 or 6/9 are both safe to apply. There is potential for a higher blob count, but weâll need to see how the network performs with an initial bump first.
Conclusion
Ethereumâs decentralization is fundamental, and home stakers play a crucial role in this picture. The network is a delicate and intricate system that demands thoughtful and deliberate consideration.
Our analysis indicates that block arrival performance surpasses initial expectations for nodes with limited bandwidth. The community-contributed data offers valuable real-world insights into the networkâs capabilities, and we would like to once again thank those who are contributing their data.
While we naively assumed a linear relationship between blob count and arrival time, this is a simplified view of a highly complex distributed system. Additionally, there are ongoing work streams that could either improve or worsen bandwidth requirements over time. Our analysis is focused on the data available to us now, based on observations from the past six weeks of network performance.
Based on block/blob arrival metrics alone, increasing the blob count from (target 3/max 6) to (target 4/max 8) or (target 6/max 9) appears to be viable. However, this is just one of many factors to evaluate when deciding on blob count adjustments.