[PATCH v2 0/2] file-posix: alignment probing improvements

[PATCH v2 0/2] file-posix: alignment probing improvements

[Stefan Hajnoczi]

v2:
- Make sure that XFS_IOC_DIOINFO takes priority over logical blocksize [Eric Biggers]
- Included a link to Eric's linux-block regression email

These patches fix alignment probing with dm-crypt and add support for the new
Linux statx(STATX_DIOALIGN) interface.

Given that Linux v6.0 kernels with dm-crypt returning EIO are out there, I
think we might as well rejig the alignment probing loop to handle that.

Let's also add support for the new STATX_DIOALIGN interface so probing is not
required on new kernels.

Stefan Hajnoczi (2):
  file-posix: fix Linux alignment probing when EIO is returned
  file-posix: add statx(STATX_DIOALIGN) support

 block/file-posix.c | 102 ++++++++++++++++++++++++---------------------
 1 file changed, 55 insertions(+), 47 deletions(-)

-- 
2.38.1

[PATCH v2 1/2] file-posix: fix Linux alignment probing when EIO is returned

[Stefan Hajnoczi]

Linux v6.0 dm-crypt returns errno EIO from unaligned O_DIRECT pread(2)
calls. Alignment probing fails on dm-crypt devices because the code
expects EINVAL. This is a kernel regression that is expected to be fixed
upstream:
https://lore.kernel.org/linux-block/20221103193837.3b5b4bac@xps.demsh.org/T/#t

Treating any errno as an "unaligned" indicator would be easy, but breaks
commit 22d182e82b4b ("block/raw-posix: fix launching with failed
disks"). Offline disks return EIO for correctly aligned requests and
EINVAL for unaligned requests.

It's possible to make both v6.0 dm-crypt and offline disks work: look
for the transition from EINVAL to EIO instead of for a single EINVAL
value.

Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1290
Fixes: 22d182e82b4b ("block/raw-posix: fix launching with failed disks")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 block/file-posix.c | 42 +++++++++++++++---------------------------
 1 file changed, 15 insertions(+), 27 deletions(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index b9647c5ffc..b9d62f52fe 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -355,31 +355,6 @@ static bool raw_needs_alignment(BlockDriverState *bs)
     return s->force_alignment;
 }
 
-/* Check if read is allowed with given memory buffer and length.
- *
- * This function is used to check O_DIRECT memory buffer and request alignment.
- */
-static bool raw_is_io_aligned(int fd, void *buf, size_t len)
-{
-    ssize_t ret = pread(fd, buf, len, 0);
-
-    if (ret >= 0) {
-        return true;
-    }
-
-#ifdef __linux__
-    /* The Linux kernel returns EINVAL for misaligned O_DIRECT reads.  Ignore
-     * other errors (e.g. real I/O error), which could happen on a failed
-     * drive, since we only care about probing alignment.
-     */
-    if (errno != EINVAL) {
-        return true;
-    }
-#endif
-
-    return false;
-}
-
 static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
 {
     BDRVRawState *s = bs->opaque;
@@ -426,34 +401,47 @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
      * try to detect buf_align, which cannot be detected in some cases (e.g.
      * Gluster). If buf_align cannot be detected, we fallback to the value of
      * request_alignment.
+     *
+     * The probing loop keeps track of the last errno so that the alignment of
+     * offline disks can be probed. On Linux pread(2) returns with errno EINVAL
+     * for most file descriptors when O_DIRECT alignment constraints are unmet.
+     * Offline disks fail correctly aligned pread(2) with EIO. Therefore it's
+     * possible to detect alignment on offline disks by observing when the
+     * errno changes from EINVAL to something else.
      */
 
     if (!bs->bl.request_alignment) {
+        int last_errno = 0;
         int i;
         size_t align;
         buf = qemu_memalign(max_align, max_align);
         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
             align = alignments[i];
-            if (raw_is_io_aligned(fd, buf, align)) {
+            if (pread(fd, buf, align, 0) >= 0 ||
+                (errno != EINVAL && last_errno == EINVAL)) {
                 /* Fallback to safe value. */
                 bs->bl.request_alignment = (align != 1) ? align : max_align;
                 break;
             }
+            last_errno = errno;
         }
         qemu_vfree(buf);
     }
 
     if (!s->buf_align) {
+        int last_errno = 0;
         int i;
         size_t align;
         buf = qemu_memalign(max_align, 2 * max_align);
         for (i = 0; i < ARRAY_SIZE(alignments); i++) {
             align = alignments[i];
-            if (raw_is_io_aligned(fd, buf + align, max_align)) {
+            if (pread(fd, buf + align, max_align, 0) >= 0 ||
+                (errno != EINVAL && last_errno == EINVAL)) {
                 /* Fallback to request_alignment. */
                 s->buf_align = (align != 1) ? align : bs->bl.request_alignment;
                 break;
             }
+            last_errno = errno;
         }
         qemu_vfree(buf);
     }
-- 
2.38.1

[PATCH v2 2/2] file-posix: add statx(STATX_DIOALIGN) support

[Stefan Hajnoczi]

Linux v6.1 commit 825cf206ed51 ("statx: add direct I/O alignment
information") added an interface to fetch O_DIRECT alignment values for
block devices and file systems.

Prefer STATX_DIOALIGN to older interfaces and probing, but keep them as
fallbacks in case STATX_DIOALIGN cannot provide the information.

Testing shows the status of STATX_DIOALIGN support in Linux 6.1-rc3
appears to be:
- btrfs: no
- ext4: yes
- XFS: yes
- NVMe block devices: yes
- dm-crypt: yes

Cc: Eric Biggers <ebiggers@google.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
v2:
- Make sure that XFS_IOC_DIOINFO takes priority over logical blocksize [Eric Biggers]
---
 block/file-posix.c | 60 ++++++++++++++++++++++++++++++----------------
 1 file changed, 40 insertions(+), 20 deletions(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index b9d62f52fe..b7e5a08e41 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -372,29 +372,49 @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)
 
     bs->bl.request_alignment = 0;
     s->buf_align = 0;
+
+#if defined(__linux__) && defined(STATX_DIOALIGN)
+    struct statx stx;
+
+    /*
+     * Linux 6.1 introduced an interface for both block devices and file
+     * systems. The system call returns with the STATX_DIOALIGN bit cleared
+     * when the information is unavailable.
+     */
+    if (statx(fd, "", AT_EMPTY_PATH, STATX_DIOALIGN, &stx) == 0 &&
+        (stx.stx_mask & STATX_DIOALIGN)) {
+        bs->bl.request_alignment = stx.stx_dio_offset_align;
+        s->buf_align = stx.stx_dio_mem_align;
+    }
+#endif /* defined(__linux__) && defined(STATX_DIOALIGN) */
+
+#ifdef __linux__
+    if (!bs->bl.request_alignment) {
+        /*
+         * The XFS ioctl definitions are shipped in extra packages that might
+         * not always be available. Since we just need the XFS_IOC_DIOINFO ioctl
+         * here, we simply use our own definition instead:
+         */
+        struct xfs_dioattr {
+            uint32_t d_mem;
+            uint32_t d_miniosz;
+            uint32_t d_maxiosz;
+        } da;
+        if (ioctl(fd, _IOR('X', 30, struct xfs_dioattr), &da) >= 0) {
+            bs->bl.request_alignment = da.d_miniosz;
+            /* The kernel returns wrong information for d_mem */
+            /* s->buf_align = da.d_mem; */
+        }
+    }
+#endif /* __linux__ */
+
     /* Let's try to use the logical blocksize for the alignment. */
-    if (probe_logical_blocksize(fd, &bs->bl.request_alignment) < 0) {
-        bs->bl.request_alignment = 0;
+    if (!bs->bl.request_alignment) {
+        if (probe_logical_blocksize(fd, &bs->bl.request_alignment) < 0) {
+            bs->bl.request_alignment = 0;
+        }
     }
 
-#ifdef __linux__
-    /*
-     * The XFS ioctl definitions are shipped in extra packages that might
-     * not always be available. Since we just need the XFS_IOC_DIOINFO ioctl
-     * here, we simply use our own definition instead:
-     */
-    struct xfs_dioattr {
-        uint32_t d_mem;
-        uint32_t d_miniosz;
-        uint32_t d_maxiosz;
-    } da;
-    if (ioctl(fd, _IOR('X', 30, struct xfs_dioattr), &da) >= 0) {
-        bs->bl.request_alignment = da.d_miniosz;
-        /* The kernel returns wrong information for d_mem */
-        /* s->buf_align = da.d_mem; */
-    }
-#endif
-
     /*
      * If we could not get the sizes so far, we can only guess them. First try
      * to detect request alignment, since it is more likely to succeed. Then
-- 
2.38.1

Re: [PATCH v2 1/2] file-posix: fix Linux alignment probing when EIO is returned

[Kevin Wolf]

Am 03.11.2022 um 19:36 hat Stefan Hajnoczi geschrieben:
> Linux v6.0 dm-crypt returns errno EIO from unaligned O_DIRECT pread(2)
> calls. Alignment probing fails on dm-crypt devices because the code
> expects EINVAL. This is a kernel regression that is expected to be fixed
> upstream:
> https://lore.kernel.org/linux-block/20221103193837.3b5b4bac@xps.demsh.org/T/#t
> 
> Treating any errno as an "unaligned" indicator would be easy, but breaks
> commit 22d182e82b4b ("block/raw-posix: fix launching with failed
> disks"). Offline disks return EIO for correctly aligned requests and
> EINVAL for unaligned requests.
> 
> It's possible to make both v6.0 dm-crypt and offline disks work: look
> for the transition from EINVAL to EIO instead of for a single EINVAL
> value.
> 
> Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1290
> Fixes: 22d182e82b4b ("block/raw-posix: fix launching with failed disks")
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

I haven't tested it myself, but from the comments in the bug tracker and
on the mailing this, I don't think this can actually distinguish the two
cases: AIUI, even with the buggy dm-crypt, you get a transition from
EINVAL to EIO because everything < 512 (such as our first probed value
1) will still return EINVAL.

So even with this patch, I think we would still probe the incorrect
512 bytes when the dm-crypt bug is present.

Kevin

Re: [PATCH v2 2/2] file-posix: add statx(STATX_DIOALIGN) support

[Kevin Wolf]

Am 03.11.2022 um 19:36 hat Stefan Hajnoczi geschrieben:
> Linux v6.1 commit 825cf206ed51 ("statx: add direct I/O alignment
> information") added an interface to fetch O_DIRECT alignment values for
> block devices and file systems.
> 
> Prefer STATX_DIOALIGN to older interfaces and probing, but keep them as
> fallbacks in case STATX_DIOALIGN cannot provide the information.
> 
> Testing shows the status of STATX_DIOALIGN support in Linux 6.1-rc3
> appears to be:
> - btrfs: no
> - ext4: yes
> - XFS: yes
> - NVMe block devices: yes
> - dm-crypt: yes
> 
> Cc: Eric Biggers <ebiggers@google.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

I think it's worth noting in the commit message that this essentially
disables patch 1 again on Linux 6.1 because it doesn't even use the code
any more that patch 1 modified to work around the dm-crypt bug.

This is only okay because we think that the final 6.1 release is going
to have the bug fixed, and it's also the first version to support
STATX_DIOALIGN, so you won't have both STATX_DIOALIGN and the bug in a
stable kernel.

Kevin