Rollup diff compression

What is very efficient when you would like to compress this kind of data, is to sort the values first and then only publish the difference compared to the previous value.

So in your case, you can sort on the “to” value and publish them in order. Since the average value difference is less than 2 you can use say 4 bits to represent the difference between the previous and the current “to” value.

When drop amounts are recurring very often, you can use 4 bits to point backward to the same value, or when the value can not be found in the last 15 amounts we have to display the amount using 16 bits.

Of course you could also go for some kind of gzip, but I guess that will be to complicated.

2 Likes