Linux kernel -stable discussions
 help / color / mirror / Atom feed
* [PATCH v4] ceph: fix OOB read in ceph_osdc_list_watchers via uncapped outdata_len
       [not found] <20260702114034.917507-13-amarkuze@redhat.com>
@ 2026-07-07 10:57 ` Pavitra Jha
  2026-08-12 19:17   ` Ilya Dryomov
  0 siblings, 1 reply; 2+ messages in thread
From: Pavitra Jha @ 2026-07-07 10:57 UTC (permalink / raw)
  To: idryomov, amarkuze, slava; +Cc: ceph-devel, linux-kernel, stable, Pavitra Jha

The OSD reply header field op->payload_len is wire-controlled and is
copied directly into m->outdata_len[i] without any bounds check:

  m->outdata_len[i] = le32_to_cpu(op->payload_len);

This value propagates unchecked to req->r_ops[0].outdata_len and is
then used to set the decode boundary in ceph_osdc_list_watchers():

  void *const end = p + req->r_ops[0].outdata_len;

The actual data allocation is always exactly one page:
  ceph_alloc_page_vector(1, GFP_NOIO)
  ceph_osd_data_pages_init(..., PAGE_SIZE, ...)

The messenger caps the copy to PAGE_SIZE bytes, but the decode window
end is set from the uncapped wire value. A malicious OSD can send
outdata_len=0x10000, causing _safe decoder boundary checks to pass
while the physical reads cross the slab allocation boundary.
KASAN report (kernel 7.0.0-rc7, QEMU/x86_64, KASLR disabled):

  ==================================================================
  BUG: KASAN: slab-out-of-bounds in ceph_oob2_init+0x23d/0xff0 [ceph_oob2_poc]
  Read of size 4 at addr ffff88800a229f9e by task insmod/57

  CPU: 0 UID: 0 PID: 57 Comm: insmod Tainted: G           O        7.0.0-rc7-g9c2abf69da83-dirty #15 PREEMPT(lazy)
  Tainted: [O]=OOT_MODULE
  Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
  Call Trace:
   <TASK>
   dump_stack_lvl+0x4d/0x70
   print_report+0x170/0x4f3
   kasan_report+0xda/0x110
   ceph_oob2_init+0x23d/0xff0 [ceph_oob2_poc]
   do_one_initcall+0x9a/0x3a0
   do_init_module+0x27c/0x790
   load_module+0x4a9a/0x6350
   init_module_from_file+0x15c/0x180
   idempotent_init_module+0x21f/0x750
   __x64_sys_finit_module+0xba/0x120
   do_syscall_64+0xe2/0x570
   entry_SYSCALL_64_after_hwframe+0x77/0x7f

  Allocated by task 57:
   kasan_save_stack+0x30/0x50
   kasan_save_track+0x14/0x30
   __kasan_kmalloc+0x7f/0x90
   ceph_oob2_init+0x44/0xff0 [ceph_oob2_poc]
   do_one_initcall+0x9a/0x3a0
   do_init_module+0x27c/0x790
   load_module+0x4a9a/0x6350
   init_module_from_file+0x15c/0x180
   idempotent_init_module+0x21f/0x750
   __x64_sys_finit_module+0xba/0x120
   do_syscall_64+0xe2/0x570
   entry_SYSCALL_64_after_hwframe+0x77/0x7f

  The buggy address belongs to the object at ffff88800a229000
   which belongs to the cache kmalloc-4k of size 4096
  The buggy address is located 3998 bytes inside of
   allocated 4000-byte region [ffff88800a229000, ffff88800a229fa0)

  Memory state around the buggy address:
   ffff88800a229e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   ffff88800a229f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  >ffff88800a229f80: 00 00 00 00 fc fc fc fc fc fc fc fc fc fc fc fc
                                 ^
   ffff88800a22a000: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
   ffff88800a22a080: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
  ==================================================================

  val=0xccccaaaa (OOB garbage from KASAN redzone)

Fix by introducing buf_len to hold the allocation size, using it in
both ceph_osd_data_pages_init() and the min_t() decode boundary cap,
so the two are guaranteed to stay in sync if the buffer size changes.
buf_len is declared as u32 to match the type of outdata_len used in
the min_t() expression.

Attacker model: a malicious or compromised OSD in a multi-tenant
Ceph deployment can trigger this against any client issuing
CEPH_OSD_OP_LIST_WATCHERS without further privileges beyond OSD
session establishment.

Fixes: a4ed38d7a180 ("libceph: support for CEPH_OSD_OP_LIST_WATCHERS")
Cc: stable@vger.kernel.org
Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>
Signed-off-by: Pavitra Jha <jhapavitra98@gmail.com>
---
v4: Rebase against current linux/master and ceph-testing/testing,
    confirmed clean git am -3 apply against both. No functional
    changes from Slava's reviewed v3.
v3: Change buf_len type from size_t to u32 to match outdata_len type
    in min_t(), per Viacheslav Dubeyko's review.
v2: Introduce buf_len variable instead of hardcoding PAGE_SIZE
    independently in ceph_osd_data_pages_init() and the min_t() cap,
    per Viacheslav Dubeyko's review.
---
 net/ceph/osd_client.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c
index 2ff00070c..62bec6ad8 100644
--- a/net/ceph/osd_client.c
+++ b/net/ceph/osd_client.c
@@ -5060,6 +5060,7 @@ int ceph_osdc_list_watchers(struct ceph_osd_client *osdc,
 	struct ceph_osd_request *req;
 	struct page **pages;
 	int ret;
+	const u32 buf_len = PAGE_SIZE;
 
 	req = ceph_osdc_alloc_request(osdc, NULL, 1, false, GFP_NOIO);
 	if (!req)
@@ -5078,7 +5079,7 @@ int ceph_osdc_list_watchers(struct ceph_osd_client *osdc,
 	osd_req_op_init(req, 0, CEPH_OSD_OP_LIST_WATCHERS, 0);
 	ceph_osd_data_pages_init(osd_req_op_data(req, 0, list_watchers,
 						 response_data),
-				 pages, PAGE_SIZE, 0, false, true);
+				 pages, buf_len, 0, false, true);
 
 	ret = ceph_osdc_alloc_messages(req, GFP_NOIO);
 	if (ret)
@@ -5088,7 +5089,8 @@ int ceph_osdc_list_watchers(struct ceph_osd_client *osdc,
 	ret = ceph_osdc_wait_request(osdc, req);
 	if (ret >= 0) {
 		void *p = page_address(pages[0]);
-		void *const end = p + req->r_ops[0].outdata_len;
+		void *const end = p +
+			min_t(u32, req->r_ops[0].outdata_len, buf_len);
 
 		ret = decode_watchers(&p, end, watchers, num_watchers);
 	}
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 2+ messages in thread

* Re: [PATCH v4] ceph: fix OOB read in ceph_osdc_list_watchers via uncapped outdata_len
  2026-07-07 10:57 ` [PATCH v4] ceph: fix OOB read in ceph_osdc_list_watchers via uncapped outdata_len Pavitra Jha
@ 2026-08-12 19:17   ` Ilya Dryomov
  0 siblings, 0 replies; 2+ messages in thread
From: Ilya Dryomov @ 2026-08-12 19:17 UTC (permalink / raw)
  To: Pavitra Jha; +Cc: amarkuze, slava, ceph-devel, linux-kernel, stable

On Tue, Jul 7, 2026 at 12:58 PM Pavitra Jha <jhapavitra98@gmail.com> wrote:
>
> The OSD reply header field op->payload_len is wire-controlled and is
> copied directly into m->outdata_len[i] without any bounds check:
>
>   m->outdata_len[i] = le32_to_cpu(op->payload_len);
>
> This value propagates unchecked to req->r_ops[0].outdata_len and is

Hi Pavitra,

IIRC it's not completely unchecked: if the sum of m.outdata_len values
for all OSD ops in the request doesn't match msg->hdr.data_len,
handle_reply() should fail and hence ceph_osdc_wait_request() in
ceph_osdc_list_watchers() too.

> then used to set the decode boundary in ceph_osdc_list_watchers():
>
>   void *const end = p + req->r_ops[0].outdata_len;
>
> The actual data allocation is always exactly one page:
>   ceph_alloc_page_vector(1, GFP_NOIO)
>   ceph_osd_data_pages_init(..., PAGE_SIZE, ...)
>
> The messenger caps the copy to PAGE_SIZE bytes, but the decode window
> end is set from the uncapped wire value. A malicious OSD can send

I don't think the messenger ever silently caps anything (or at least it
shouldn't).  The messenger wants to read exactly msg->hdr.data_len bytes
off the wire and if the underlying buffers aren't big enough the message
gets discarded.

> outdata_len=0x10000, causing _safe decoder boundary checks to pass
> while the physical reads cross the slab allocation boundary.
> KASAN report (kernel 7.0.0-rc7, QEMU/x86_64, KASLR disabled):

Could you please elaborate on what fields were set to what values and
what payload was injected and fed to decode_watchers() because it for
sure wasn't just outdata_len set to 0x10000.

Thanks,

                Ilya

>
>   ==================================================================
>   BUG: KASAN: slab-out-of-bounds in ceph_oob2_init+0x23d/0xff0 [ceph_oob2_poc]
>   Read of size 4 at addr ffff88800a229f9e by task insmod/57
>
>   CPU: 0 UID: 0 PID: 57 Comm: insmod Tainted: G           O        7.0.0-rc7-g9c2abf69da83-dirty #15 PREEMPT(lazy)
>   Tainted: [O]=OOT_MODULE
>   Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
>   Call Trace:
>    <TASK>
>    dump_stack_lvl+0x4d/0x70
>    print_report+0x170/0x4f3
>    kasan_report+0xda/0x110
>    ceph_oob2_init+0x23d/0xff0 [ceph_oob2_poc]
>    do_one_initcall+0x9a/0x3a0
>    do_init_module+0x27c/0x790
>    load_module+0x4a9a/0x6350
>    init_module_from_file+0x15c/0x180
>    idempotent_init_module+0x21f/0x750
>    __x64_sys_finit_module+0xba/0x120
>    do_syscall_64+0xe2/0x570
>    entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
>   Allocated by task 57:
>    kasan_save_stack+0x30/0x50
>    kasan_save_track+0x14/0x30
>    __kasan_kmalloc+0x7f/0x90
>    ceph_oob2_init+0x44/0xff0 [ceph_oob2_poc]
>    do_one_initcall+0x9a/0x3a0
>    do_init_module+0x27c/0x790
>    load_module+0x4a9a/0x6350
>    init_module_from_file+0x15c/0x180
>    idempotent_init_module+0x21f/0x750
>    __x64_sys_finit_module+0xba/0x120
>    do_syscall_64+0xe2/0x570
>    entry_SYSCALL_64_after_hwframe+0x77/0x7f
>
>   The buggy address belongs to the object at ffff88800a229000
>    which belongs to the cache kmalloc-4k of size 4096
>   The buggy address is located 3998 bytes inside of
>    allocated 4000-byte region [ffff88800a229000, ffff88800a229fa0)
>
>   Memory state around the buggy address:
>    ffff88800a229e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    ffff88800a229f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   >ffff88800a229f80: 00 00 00 00 fc fc fc fc fc fc fc fc fc fc fc fc
>                                  ^
>    ffff88800a22a000: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>    ffff88800a22a080: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>   ==================================================================
>
>   val=0xccccaaaa (OOB garbage from KASAN redzone)
>
> Fix by introducing buf_len to hold the allocation size, using it in
> both ceph_osd_data_pages_init() and the min_t() decode boundary cap,
> so the two are guaranteed to stay in sync if the buffer size changes.
> buf_len is declared as u32 to match the type of outdata_len used in
> the min_t() expression.
>
> Attacker model: a malicious or compromised OSD in a multi-tenant
> Ceph deployment can trigger this against any client issuing
> CEPH_OSD_OP_LIST_WATCHERS without further privileges beyond OSD
> session establishment.
>
> Fixes: a4ed38d7a180 ("libceph: support for CEPH_OSD_OP_LIST_WATCHERS")
> Cc: stable@vger.kernel.org
> Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>
> Signed-off-by: Pavitra Jha <jhapavitra98@gmail.com>
> ---
> v4: Rebase against current linux/master and ceph-testing/testing,
>     confirmed clean git am -3 apply against both. No functional
>     changes from Slava's reviewed v3.
> v3: Change buf_len type from size_t to u32 to match outdata_len type
>     in min_t(), per Viacheslav Dubeyko's review.
> v2: Introduce buf_len variable instead of hardcoding PAGE_SIZE
>     independently in ceph_osd_data_pages_init() and the min_t() cap,
>     per Viacheslav Dubeyko's review.
> ---
>  net/ceph/osd_client.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c
> index 2ff00070c..62bec6ad8 100644
> --- a/net/ceph/osd_client.c
> +++ b/net/ceph/osd_client.c
> @@ -5060,6 +5060,7 @@ int ceph_osdc_list_watchers(struct ceph_osd_client *osdc,
>         struct ceph_osd_request *req;
>         struct page **pages;
>         int ret;
> +       const u32 buf_len = PAGE_SIZE;
>
>         req = ceph_osdc_alloc_request(osdc, NULL, 1, false, GFP_NOIO);
>         if (!req)
> @@ -5078,7 +5079,7 @@ int ceph_osdc_list_watchers(struct ceph_osd_client *osdc,
>         osd_req_op_init(req, 0, CEPH_OSD_OP_LIST_WATCHERS, 0);
>         ceph_osd_data_pages_init(osd_req_op_data(req, 0, list_watchers,
>                                                  response_data),
> -                                pages, PAGE_SIZE, 0, false, true);
> +                                pages, buf_len, 0, false, true);
>
>         ret = ceph_osdc_alloc_messages(req, GFP_NOIO);
>         if (ret)
> @@ -5088,7 +5089,8 @@ int ceph_osdc_list_watchers(struct ceph_osd_client *osdc,
>         ret = ceph_osdc_wait_request(osdc, req);
>         if (ret >= 0) {
>                 void *p = page_address(pages[0]);
> -               void *const end = p + req->r_ops[0].outdata_len;
> +               void *const end = p +
> +                       min_t(u32, req->r_ops[0].outdata_len, buf_len);
>
>                 ret = decode_watchers(&p, end, watchers, num_watchers);
>         }
> --
> 2.53.0
>

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-08-12 19:17 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <20260702114034.917507-13-amarkuze@redhat.com>
2026-07-07 10:57 ` [PATCH v4] ceph: fix OOB read in ceph_osdc_list_watchers via uncapped outdata_len Pavitra Jha
2026-08-12 19:17   ` Ilya Dryomov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox