Re: [PATCH 1/2] vdo: add zstd compression support

[Matthew Sakai <msakai@redhat.com>]

On 7/27/26 9:43 AM, June Park wrote:
> Hello! Thank you for the thoughtful comments.

Thank you for your patience. This took me longer to get to than I had hoped.

>> I have to ask, what is the motivation for this change? Do you have a workload that shows some kind of improvement from this proposal?
> 
> My initial testing in QEMU, storing files from the kernel source tree, showed noticeable improvements in compression ratio (around 2.4 to 3.5), even though vdo uses small block sizes. However, I did notice that certain types of data (such as long synthetic streams of repeating bytes) showed far less improvement with zstd, at the expense of more CPU cycles. I had concluded that it could outperform LZ4 in some workloads in terms of compression ratio, for more CPU, a trade-off that could be made on a case-by-case basis.

If you have data handy from these experiments, it would be interesting 
to see it. This is the sort of information that would go well in a cover 
letter explaining what you want to do, and why. At any rate, if you have 
cases that seem to benefit from this, we can certainly revisit whether 
this is worth adding.

Another thing to consider is that we originally chose the LZ4 algorithm 
because it is fairly cheap to compute. If you can, it's worth trying to 
quantify what the extra computational load does to vdo throughput, 
especially with fast storage. The throughput for a vdo volume will often 
lag the raw storage speed significantly (due to the deduplication 
machinery) and it's worth knowing if changing the algorithm will make 
that worse.

> The discussion you linked to proposes the ability to swap the algorithm without reformatting. My current implementation tries to reduce breaking changes as much as possible, so I had decided on storing the compression algorithm directly in the volume geometry, instead of for every block. This has the downside of requiring reformatting, but since the focus of vdo is deduplication, I don't think the large amounts of additional machinery needed to support live changes is justified.

I appreciate that you're attempting to minimize disruption. I admit that 
it is simpler, in terms of pure implementation, to make this a 
format-time choice. However, imagine what happen next:  Long-time vdo 
users will inquire whether they can use this new feature, and we will 
have to tell them no. For new users, I think they may not know all the 
data they will store on a volume up front, but they will be locked into 
their first choice. Given the case-by-case variability of the tradeoff, 
I expect users will appreciate being able to change this setting to fit 
their current needs.

In short, doing this as a format-only option looks like implementing 
half a feature to me, and I think we would be better off starting with 
full flexibility. Also remember that every version of this feature that 
we expose to users is a feature we will have to maintain for the 
lifetime of the dm-vdo driver, and I would rather not have to support 
both versions.

(I believe the difference in complexity is also not that large, but 
that's a bit more subjective. Setting the algorithm as a run-time option 
means extending the compressed block format, but adding a format-time 
option involves more work updating the user space tools, including the 
formatter. Both options also require updating the table line and the 
super block format, so there's also considerable overlap.)

So anyway. If you can show there is utility in doing this, we can look 
at adding it. I will probably want to do it by building on what we did 
last year, though. You can look at what I've already done on the branch 
feature/allow-compression-configuration in the vdo-devel project. I 
haven't rebased the branch in a while, but you can get an idea of how I 
was planning the table line and super block changes, at least.

> June
> 

Matt