[PATCH] libceph: Reset sparse-read on fault

[PATCH] libceph: Reset sparse-read on fault

[Sam Edwards]

When a fault occurs, the connection is abandoned, reestablished, and any
pending operations are retried. The OSD client tracks the progress of a
sparse-read reply using a separate state machine, largely independent of
the messenger's state.

If a connection is lost mid-payload or the sparse-read state machine
returns an error, the sparse-read state is not reset. The OSD client
will then interpret the beginning of a new reply as the continuation of
the old one. If this makes the sparse-read machinery enter a failure
state, it may never recover, producing loops like:

  libceph:  [0] got 0 extents
  libceph: data len 142248331 != extent len 0
  libceph: osd0 (1)...:6801 socket error on read
  libceph: data len 142248331 != extent len 0
  libceph: osd0 (1)...:6801 socket error on read

Therefore, reset the sparse-read state in osd_fault(), ensuring retries
start from a clean state.

Cc: stable@vger.kernel.org
Signed-off-by: Sam Edwards <CFSworks@gmail.com>
---
 net/ceph/osd_client.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c
index 3667319b949d..1a7be2f615dc 100644
--- a/net/ceph/osd_client.c
+++ b/net/ceph/osd_client.c
@@ -4281,6 +4281,9 @@ static void osd_fault(struct ceph_connection *con)
 		goto out_unlock;
 	}
 
+	osd->o_sparse_op_idx = -1;
+	ceph_init_sparse_read(&osd->o_sparse_read);
+
 	if (!reopen_osd(osd))
 		kick_osd_requests(osd);
 	maybe_request_map(osdc);
-- 
2.51.2

Re: [PATCH] libceph: Reset sparse-read on fault

[Ilya Dryomov]

On Wed, Dec 31, 2025 at 5:05 AM Sam Edwards <cfsworks@gmail.com> wrote:
>
> When a fault occurs, the connection is abandoned, reestablished, and any
> pending operations are retried. The OSD client tracks the progress of a
> sparse-read reply using a separate state machine, largely independent of
> the messenger's state.
>
> If a connection is lost mid-payload or the sparse-read state machine
> returns an error, the sparse-read state is not reset. The OSD client
> will then interpret the beginning of a new reply as the continuation of
> the old one. If this makes the sparse-read machinery enter a failure
> state, it may never recover, producing loops like:
>
>   libceph:  [0] got 0 extents
>   libceph: data len 142248331 != extent len 0
>   libceph: osd0 (1)...:6801 socket error on read
>   libceph: data len 142248331 != extent len 0
>   libceph: osd0 (1)...:6801 socket error on read
>
> Therefore, reset the sparse-read state in osd_fault(), ensuring retries
> start from a clean state.
>
> Cc: stable@vger.kernel.org
> Signed-off-by: Sam Edwards <CFSworks@gmail.com>
> ---
>  net/ceph/osd_client.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c
> index 3667319b949d..1a7be2f615dc 100644
> --- a/net/ceph/osd_client.c
> +++ b/net/ceph/osd_client.c
> @@ -4281,6 +4281,9 @@ static void osd_fault(struct ceph_connection *con)
>                 goto out_unlock;
>         }
>
> +       osd->o_sparse_op_idx = -1;
> +       ceph_init_sparse_read(&osd->o_sparse_read);
> +
>         if (!reopen_osd(osd))
>                 kick_osd_requests(osd);
>         maybe_request_map(osdc);
> --
> 2.51.2
>

Hi Sam,

Good catch!  Applied (with the sad note that support for sparse
reads is officially the most problematic patchset that ever made it
into libceph).

Thanks,

                Ilya