Re: [f2fs-dev] [PATCHv2 0/5] direct-io file extended attributes

Re: [f2fs-dev] [PATCHv2 0/5] direct-io file extended attributes

[Eric Biggers]

On Fri, Jul 10, 2026 at 02:06:41PM -0700, Keith Busch via Linux-f2fs-devel wrote:
> From: Keith Busch <kbusch@kernel.org>
> 
> The attributes reported through statx are incomplete for applications to
> fully know exactly how IO construction is valid or not. The statx call
> can report minimum memory alignment and total granularity, but it
> doesn't show the underlying gap boundary requirements or max segments
> per granule.
> 
> This series adds the minimum to the extended file attributes through
> file_getattr. I hear this is the preferred interface for reporting such
> things over adding more fields to statx. In order to get everything
> under a single syscall, some of the attributes are duplicated from
> statx.

Okay, in v2 we at least now know that the existing statx UAPI was
considered.  Could you give a specific real-world example (with the
actual values of each parameter) where it's not sufficient?  Without
that there isn't really any way to evaluate this proposal.

- Eric

Re: [f2fs-dev] [PATCHv2 0/5] direct-io file extended attributes

[Keith Busch]

On Fri, Jul 10, 2026 at 05:53:28PM -0400, Eric Biggers wrote:
> On Fri, Jul 10, 2026 at 02:06:41PM -0700, Keith Busch via Linux-f2fs-devel wrote:
> > From: Keith Busch <kbusch@kernel.org>
> > 
> > The attributes reported through statx are incomplete for applications to
> > fully know exactly how IO construction is valid or not. The statx call
> > can report minimum memory alignment and total granularity, but it
> > doesn't show the underlying gap boundary requirements or max segments
> > per granule.
> > 
> > This series adds the minimum to the extended file attributes through
> > file_getattr. I hear this is the preferred interface for reporting such
> > things over adding more fields to statx. In order to get everything
> > under a single syscall, some of the attributes are duplicated from
> > statx.
> 
> Okay, in v2 we at least now know that the existing statx UAPI was
> considered.  Could you give a specific real-world example (with the
> actual values of each parameter) where it's not sufficient?  Without
> that there isn't really any way to evaluate this proposal.

Yes, we can consider nvme. This protocol supports two different transfer
modes called PRP and SGL. PRP requires 4k aligned segments, though you
can have an arbitrary 4-byte aligned offset at the start. SGL on the
other hand allows completely arbitrary size and alignments for each
segment.

statx reports information sufficient to know that you can have dword
aligned page offsets for a virtually contiguous buffer, but it doesn't
report PRP's boundary gap requirement, so applications can't tell if the
file follows PRP or SGL rules for direct-io.

And if you have a device using SGL, statx doesn't report the max number
of sub-sector segments you can submit in a single command.

This series provides both limits so user space has the complete picture.

A typical nvme that supports only PRP has a DMA alignment of 4 bytes, a
dio offset alignment of 4k, and a virtual boundary of 4k.

If SGL were supported, there would be no virtual boundary gap, and max
segments is 256.

Re: [PATCHv2 0/5] direct-io file extended attributes

[Eric Biggers]

On Fri, Jul 10, 2026 at 04:58:12PM -0600, Keith Busch wrote:
> On Fri, Jul 10, 2026 at 05:53:28PM -0400, Eric Biggers wrote:
> > On Fri, Jul 10, 2026 at 02:06:41PM -0700, Keith Busch via Linux-f2fs-devel wrote:
> > > From: Keith Busch <kbusch@kernel.org>
> > > 
> > > The attributes reported through statx are incomplete for applications to
> > > fully know exactly how IO construction is valid or not. The statx call
> > > can report minimum memory alignment and total granularity, but it
> > > doesn't show the underlying gap boundary requirements or max segments
> > > per granule.
> > > 
> > > This series adds the minimum to the extended file attributes through
> > > file_getattr. I hear this is the preferred interface for reporting such
> > > things over adding more fields to statx. In order to get everything
> > > under a single syscall, some of the attributes are duplicated from
> > > statx.
> > 
> > Okay, in v2 we at least now know that the existing statx UAPI was
> > considered.  Could you give a specific real-world example (with the
> > actual values of each parameter) where it's not sufficient?  Without
> > that there isn't really any way to evaluate this proposal.
> 
> Yes, we can consider nvme. This protocol supports two different transfer
> modes called PRP and SGL. PRP requires 4k aligned segments, though you
> can have an arbitrary 4-byte aligned offset at the start. SGL on the
> other hand allows completely arbitrary size and alignments for each
> segment.
> 
> statx reports information sufficient to know that you can have dword
> aligned page offsets for a virtually contiguous buffer, but it doesn't
> report PRP's boundary gap requirement, so applications can't tell if the
> file follows PRP or SGL rules for direct-io.
> 
> And if you have a device using SGL, statx doesn't report the max number
> of sub-sector segments you can submit in a single command.
> 
> This series provides both limits so user space has the complete picture.
> 
> A typical nvme that supports only PRP has a DMA alignment of 4 bytes, a
> dio offset alignment of 4k, and a virtual boundary of 4k.

So each segment's length has to be a multiple of 4k, *and* it has to end
on a 4k aligned memory address?  That implies the segment begins at a 4k
aligned memory address as well, which is just stx_dio_mem_align=4k.

What am I missing?

What is a specific example of an I/O request that you'd like to be able
to submit that the existing UAPI can't declare support for?

> If SGL were supported, there would be no virtual boundary gap, and max
> segments is 256.

Can you elaborate on why DIO users need to know max_segments?

I'm worried about the UAPI duplication, as well as it going to be very
difficult for userspace to correctly use this information.  With just
the two alignments there's at least a chance of them getting it right.
If we throw virt_boundary_mask and max_segments into the mix, I don't
think there's much chance.

- Eric

Re: [PATCHv2 0/5] direct-io file extended attributes

[Keith Busch]

On Fri, Jul 10, 2026 at 08:24:12PM -0400, Eric Biggers wrote:
> On Fri, Jul 10, 2026 at 04:58:12PM -0600, Keith Busch wrote:
> > dio offset alignment of 4k, and a virtual boundary of 4k.
> 
> So each segment's length has to be a multiple of 4k, *and* it has to end
> on a 4k aligned memory address?  That implies the segment begins at a 4k
> aligned memory address as well, which is just stx_dio_mem_align=4k.
> 
> What am I missing?

I'm apparently poorly explaining PRPs, but I hear this is a common
experience.

Simply put, any virtually contiguous buffer that starts at a dword
aligned address is a valid io vector, no matter how many pages it spans.
It doesn't matter where it starts or where it ends, but every page in
the middle obviously starts and ends on their page boundary.

Simple case: pread/pwrite. You can consult statx to know you can provide
any dword aligned buffer with an aligned length, and that's a valid
direct IO.

What I'm trying to enable here is the vectored preadv/pwritev type paths
for hardware that don't need to subscribe to PRP constraints.
 
> What is a specific example of an I/O request that you'd like to be able
> to submit that the existing UAPI can't declare support for?

I want to support NVMe SGL. This allows virtually *discontiguous*
segments that we currently can't distinguish with what statx reports.
I'm trying to report limits that let applications know what constraints
they're dealing with.

> > If SGL were supported, there would be no virtual boundary gap, and max
> > segments is 256.
> 
> Can you elaborate on why DIO users need to know max_segments?

 * Logical block is 4k.

 * DMA granule is 4 bytes.

 * Max segments is 256.

That's very typical NVMe device contraints under SGL capabilities with
the linux driver.

You can provide 4 byte vectors as needed, but you'll hit the max
segments limit before you have a valid IO if they're all that small.
The average size needs to be larger, so we need to communicate that
somehow.

To be clear, I'm not interested in trying to enable applications
dispatching thousands of 4-byte vectors to do an IO. That's a stupid
application. The applications I'm trying to enable have unpredictable
offsets such that a tiny fraction of vectors are indeed that small, but
that's not a typical vector for the payload. But I can't enable just
some without generically enabling all.

> I'm worried about the UAPI duplication, as well as it going to be very
> difficult for userspace to correctly use this information.  With just
> the two alignments there's at least a chance of them getting it right.
> If we throw virt_boundary_mask and max_segments into the mix, I don't
> think there's much chance.

The blktests framework test case "block/043" does this with great
success with these exact parameters, but it takes these paramters from
the sysfs attributes. The same test works with filesystems too, but I
haven't gotten around to porting it to fstests because it's gating on
having this series.