[BUG] librdmacm: Accessing out-of-bounds memory

[BUG] librdmacm: Accessing out-of-bounds memory

[Korb, Andreas]

[-- Attachment #1: Type: text/plain, Size: 2308 bytes --]

The function `ds_init_ep` in librdmacm/rsocket.c may access memory via an object that is not allocated for this object.

Relevant lines from this function:

// (1): Prepare `struct rsocket` 
ds_set_qp_size(rs);
// (2): Allocation
rs->sbuf = calloc(rs->sq_size, RS_SNDLOWAT);
// (3): Copy pointer to rs->smsg_free
rs->smsg_free = (struct ds_smsg *) rs->sbuf;
// (4): Copy pointer to msg
msg = rs->smsg_free;
// (5): Write to msg->next
msg->next = NULL;

Within my podman container:
Before (1): rs->sq_size = rs->rq_size = 384
After (1): rs->sq_size = rs->rq_size = 0
Therefore, (2) does not reserve a buffer, but still returns a pointer which can be freed later, as described by man-page calloc(3p).
(5) writes data to the buffer allocated in (2). If no actual buffer is allocated, it overwrites arbitrary data, yielding undefined
behavior.

I found it by executing /usr/bin/udpong (without any arguments) with libscudo on an arm64 server with memory tagging enabled. It
immediately crashes with a segmentation fault, then. Without memory tagging, the bug stays undetected, and execution continues.
The code behavior described above also happens on x86-64, there it doesn't result in a crash and is silently ignored because of the
lack of MemoryTagging. Valgrind also detects this violation, however.

In summary:
The libc man-page states that if the allocated buffer size is 0, then either:
>        *  A null pointer shall be returned and errno may be set to an
>        implementation-defined value, or
>        *  A pointer to the allocated space shall be returned. The
>        application shall ensure that the pointer is not used to
>        access an object.

and the function `ds_init_ep` misses to ensure to work for the second case.

* Linux distribution and version
Debian Trixie (in podman container) 

* Linux kernel and version
Linux arm01 6.12.22+bpo-arm64 #1 SMP Debian 6.12.22-1~bpo12+1 (2025-04-25) aarch64 GNU/Linux

* InfiniBand hardware and firmware version
None available in podman container

* How to reproduce
As I'm assuming most people use x86_64 machines, this shows the bug with valrind:
podman run --rm -e DEBUGINFOD_URLS="https://debuginfod.debian.net" debian:trixie bash -lc 'apt-get update && apt-get install -y
valgrind rdmacm-utils debuginfod && valgrind /usr/bin/udpong' 


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 8022 bytes --]

Re: [BUG] librdmacm: Accessing out-of-bounds memory

[Leon Romanovsky]

On Tue, Apr 07, 2026 at 01:14:45PM +0000, Korb, Andreas wrote:
> The function `ds_init_ep` in librdmacm/rsocket.c may access memory via an object that is not allocated for this object.
> 
> Relevant lines from this function:
> 
> // (1): Prepare `struct rsocket` 
> ds_set_qp_size(rs);
> // (2): Allocation
> rs->sbuf = calloc(rs->sq_size, RS_SNDLOWAT);
> // (3): Copy pointer to rs->smsg_free
> rs->smsg_free = (struct ds_smsg *) rs->sbuf;
> // (4): Copy pointer to msg
> msg = rs->smsg_free;
> // (5): Write to msg->next
> msg->next = NULL;
> 
> Within my podman container:
> Before (1): rs->sq_size = rs->rq_size = 384
> After (1): rs->sq_size = rs->rq_size = 0
> Therefore, (2) does not reserve a buffer, but still returns a pointer which can be freed later, as described by man-page calloc(3p).
> (5) writes data to the buffer allocated in (2). If no actual buffer is allocated, it overwrites arbitrary data, yielding undefined
> behavior.
> 
> I found it by executing /usr/bin/udpong (without any arguments) with libscudo on an arm64 server with memory tagging enabled. It
> immediately crashes with a segmentation fault, then. Without memory tagging, the bug stays undetected, and execution continues.
> The code behavior described above also happens on x86-64, there it doesn't result in a crash and is silently ignored because of the
> lack of MemoryTagging. Valgrind also detects this violation, however.
> 
> In summary:
> The libc man-page states that if the allocated buffer size is 0, then either:
> >        *  A null pointer shall be returned and errno may be set to an
> >        implementation-defined value, or
> >        *  A pointer to the allocated space shall be returned. The
> >        application shall ensure that the pointer is not used to
> >        access an object.

Can you please provide a link to these sentences in the manual?

You provided invalid value as sq_size and rq_size. It is expected that
library won't work after that.

Thanks

RE: [BUG] librdmacm: Accessing out-of-bounds memory

[Korb, Andreas]

> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: Sonntag, 12. April 2026 16:04
> To: Korb, Andreas <andreas.korb@aisec.fraunhofer.de>
> Cc: linux-rdma@vger.kernel.org
> Subject: Re: [BUG] librdmacm: Accessing out-of-bounds memory
> 
> On Tue, Apr 07, 2026 at 01:14:45PM +0000, Korb, Andreas wrote:
> > The function `ds_init_ep` in librdmacm/rsocket.c may access memory via an object that is not allocated for this object.
> >
> > Relevant lines from this function:
> >
> > // (1): Prepare `struct rsocket`
> > ds_set_qp_size(rs);
> > // (2): Allocation
> > rs->sbuf = calloc(rs->sq_size, RS_SNDLOWAT);
> > // (3): Copy pointer to rs->smsg_free
> > rs->smsg_free = (struct ds_smsg *) rs->sbuf;
> > // (4): Copy pointer to msg
> > msg = rs->smsg_free;
> > // (5): Write to msg->next
> > msg->next = NULL;
> >
> > Within my podman container:
> > Before (1): rs->sq_size = rs->rq_size = 384
> > After (1): rs->sq_size = rs->rq_size = 0
> > Therefore, (2) does not reserve a buffer, but still returns a pointer which can be freed later, as described by man-page calloc(3p).
> > (5) writes data to the buffer allocated in (2). If no actual buffer is allocated, it overwrites arbitrary data, yielding undefined
> > behavior.
> >
> > I found it by executing /usr/bin/udpong (without any arguments) with libscudo on an arm64 server with memory tagging enabled. It
> > immediately crashes with a segmentation fault, then. Without memory tagging, the bug stays undetected, and execution continues.
> > The code behavior described above also happens on x86-64, there it doesn't result in a crash and is silently ignored because of the
> > lack of MemoryTagging. Valgrind also detects this violation, however.
> >
> > In summary:
> > The libc man-page states that if the allocated buffer size is 0, then either:
> > >        *  A null pointer shall be returned and errno may be set to an
> > >        implementation-defined value, or
> > >        *  A pointer to the allocated space shall be returned. The
> > >        application shall ensure that the pointer is not used to
> > >        access an object.
> 
> Can you please provide a link to these sentences in the manual?
> 
> You provided invalid value as sq_size and rq_size. It is expected that
> library won't work after that.
> 
> Thanks

These sentences are from: https://man7.org/linux/man-pages/man3/calloc.3p.html

When the incoming values of sq_size and rq_size are wrong, that is a
bug in udpong then. However, since librdmacm is doing the calloc allocation,
it should still refrain from undefined behavior with the own allocated buffer.

Re: [BUG] librdmacm: Accessing out-of-bounds memory

[Leon Romanovsky]

On Mon, Apr 13, 2026 at 06:54:32AM +0000, Korb, Andreas wrote:
> > -----Original Message-----
> > From: Leon Romanovsky <leon@kernel.org>
> > Sent: Sonntag, 12. April 2026 16:04
> > To: Korb, Andreas <andreas.korb@aisec.fraunhofer.de>
> > Cc: linux-rdma@vger.kernel.org
> > Subject: Re: [BUG] librdmacm: Accessing out-of-bounds memory
> > 
> > On Tue, Apr 07, 2026 at 01:14:45PM +0000, Korb, Andreas wrote:
> > > The function `ds_init_ep` in librdmacm/rsocket.c may access memory via an object that is not allocated for this object.
> > >
> > > Relevant lines from this function:
> > >
> > > // (1): Prepare `struct rsocket`
> > > ds_set_qp_size(rs);
> > > // (2): Allocation
> > > rs->sbuf = calloc(rs->sq_size, RS_SNDLOWAT);
> > > // (3): Copy pointer to rs->smsg_free
> > > rs->smsg_free = (struct ds_smsg *) rs->sbuf;
> > > // (4): Copy pointer to msg
> > > msg = rs->smsg_free;
> > > // (5): Write to msg->next
> > > msg->next = NULL;
> > >
> > > Within my podman container:
> > > Before (1): rs->sq_size = rs->rq_size = 384
> > > After (1): rs->sq_size = rs->rq_size = 0
> > > Therefore, (2) does not reserve a buffer, but still returns a pointer which can be freed later, as described by man-page calloc(3p).
> > > (5) writes data to the buffer allocated in (2). If no actual buffer is allocated, it overwrites arbitrary data, yielding undefined
> > > behavior.
> > >
> > > I found it by executing /usr/bin/udpong (without any arguments) with libscudo on an arm64 server with memory tagging enabled. It
> > > immediately crashes with a segmentation fault, then. Without memory tagging, the bug stays undetected, and execution continues.
> > > The code behavior described above also happens on x86-64, there it doesn't result in a crash and is silently ignored because of the
> > > lack of MemoryTagging. Valgrind also detects this violation, however.
> > >
> > > In summary:
> > > The libc man-page states that if the allocated buffer size is 0, then either:
> > > >        *  A null pointer shall be returned and errno may be set to an
> > > >        implementation-defined value, or
> > > >        *  A pointer to the allocated space shall be returned. The
> > > >        application shall ensure that the pointer is not used to
> > > >        access an object.
> > 
> > Can you please provide a link to these sentences in the manual?
> > 
> > You provided invalid value as sq_size and rq_size. It is expected that
> > library won't work after that.
> > 
> > Thanks
> 
> These sentences are from: https://man7.org/linux/man-pages/man3/calloc.3p.html
> 
> When the incoming values of sq_size and rq_size are wrong, that is a
> bug in udpong then. However, since librdmacm is doing the calloc allocation,
> it should still refrain from undefined behavior with the own allocated buffer.

I don't think so. This buffer is allocated in application memory, and if  
the application is buggy, that is its problem, not the library's.

Thanks

RE: [BUG] librdmacm: Accessing out-of-bounds memory

[Korb, Andreas]

> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: Montag, 13. April 2026 18:02
> To: Korb, Andreas <andreas.korb@aisec.fraunhofer.de>
> Cc: linux-rdma@vger.kernel.org
> Subject: Re: [BUG] librdmacm: Accessing out-of-bounds memory
> 
> On Mon, Apr 13, 2026 at 06:54:32AM +0000, Korb, Andreas wrote:
> > > -----Original Message-----
> > > From: Leon Romanovsky <leon@kernel.org>
> > > Sent: Sonntag, 12. April 2026 16:04
> > > To: Korb, Andreas <andreas.korb@aisec.fraunhofer.de>
> > > Cc: linux-rdma@vger.kernel.org
> > > Subject: Re: [BUG] librdmacm: Accessing out-of-bounds memory
> > >
> > > On Tue, Apr 07, 2026 at 01:14:45PM +0000, Korb, Andreas wrote:
> > > > The function `ds_init_ep` in librdmacm/rsocket.c may access memory via an object that is not allocated for this object.
> > > >
> > > > Relevant lines from this function:
> > > >
> > > > // (1): Prepare `struct rsocket`
> > > > ds_set_qp_size(rs);
> > > > // (2): Allocation
> > > > rs->sbuf = calloc(rs->sq_size, RS_SNDLOWAT);
> > > > // (3): Copy pointer to rs->smsg_free
> > > > rs->smsg_free = (struct ds_smsg *) rs->sbuf;
> > > > // (4): Copy pointer to msg
> > > > msg = rs->smsg_free;
> > > > // (5): Write to msg->next
> > > > msg->next = NULL;
> > > >
> > > > Within my podman container:
> > > > Before (1): rs->sq_size = rs->rq_size = 384
> > > > After (1): rs->sq_size = rs->rq_size = 0
> > > > Therefore, (2) does not reserve a buffer, but still returns a pointer which can be freed later, as described by man-page calloc(3p).
> > > > (5) writes data to the buffer allocated in (2). If no actual buffer is allocated, it overwrites arbitrary data, yielding undefined
> > > > behavior.
> > > >
> > > > I found it by executing /usr/bin/udpong (without any arguments) with libscudo on an arm64 server with memory tagging enabled.
> It
> > > > immediately crashes with a segmentation fault, then. Without memory tagging, the bug stays undetected, and execution
> continues.
> > > > The code behavior described above also happens on x86-64, there it doesn't result in a crash and is silently ignored because of
> the
> > > > lack of MemoryTagging. Valgrind also detects this violation, however.
> > > >
> > > > In summary:
> > > > The libc man-page states that if the allocated buffer size is 0, then either:
> > > > >        *  A null pointer shall be returned and errno may be set to an
> > > > >        implementation-defined value, or
> > > > >        *  A pointer to the allocated space shall be returned. The
> > > > >        application shall ensure that the pointer is not used to
> > > > >        access an object.
> > >
> > > Can you please provide a link to these sentences in the manual?
> > >
> > > You provided invalid value as sq_size and rq_size. It is expected that
> > > library won't work after that.
> > >
> > > Thanks
> >
> > These sentences are from: https://man7.org/linux/man-pages/man3/calloc.3p.html
> >
> > When the incoming values of sq_size and rq_size are wrong, that is a
> > bug in udpong then. However, since librdmacm is doing the calloc allocation,
> > it should still refrain from undefined behavior with the own allocated buffer.
> 
> I don't think so. This buffer is allocated in application memory, and if
> the application is buggy, that is its problem, not the library's.
> 
> Thanks

Maybe we're not talking about the same thing. I'm specifically talking about this line of code:
https://github.com/linux-rdma/rdma-core/blob/de593a02932a5ee7c729bd92df90e1fe8892b584/librdmacm/rsocket.c#L1169

Here, the buffer is allocated within the library, and the violating access happens in the same function in line 1186.
Both appear to me to be part of librdmacm. However, I'm not that familiar about this entire system.

Thanks

Re: [BUG] librdmacm: Accessing out-of-bounds memory

[Leon Romanovsky]

On Tue, Apr 14, 2026 at 08:37:38AM +0000, Korb, Andreas wrote:
> > -----Original Message-----
> > From: Leon Romanovsky <leon@kernel.org>
> > Sent: Montag, 13. April 2026 18:02
> > To: Korb, Andreas <andreas.korb@aisec.fraunhofer.de>
> > Cc: linux-rdma@vger.kernel.org
> > Subject: Re: [BUG] librdmacm: Accessing out-of-bounds memory
> > 
> > On Mon, Apr 13, 2026 at 06:54:32AM +0000, Korb, Andreas wrote:
> > > > -----Original Message-----
> > > > From: Leon Romanovsky <leon@kernel.org>
> > > > Sent: Sonntag, 12. April 2026 16:04
> > > > To: Korb, Andreas <andreas.korb@aisec.fraunhofer.de>
> > > > Cc: linux-rdma@vger.kernel.org
> > > > Subject: Re: [BUG] librdmacm: Accessing out-of-bounds memory
> > > >
> > > > On Tue, Apr 07, 2026 at 01:14:45PM +0000, Korb, Andreas wrote:
> > > > > The function `ds_init_ep` in librdmacm/rsocket.c may access memory via an object that is not allocated for this object.
> > > > >
> > > > > Relevant lines from this function:
> > > > >
> > > > > // (1): Prepare `struct rsocket`
> > > > > ds_set_qp_size(rs);
> > > > > // (2): Allocation
> > > > > rs->sbuf = calloc(rs->sq_size, RS_SNDLOWAT);
> > > > > // (3): Copy pointer to rs->smsg_free
> > > > > rs->smsg_free = (struct ds_smsg *) rs->sbuf;
> > > > > // (4): Copy pointer to msg
> > > > > msg = rs->smsg_free;
> > > > > // (5): Write to msg->next
> > > > > msg->next = NULL;
> > > > >
> > > > > Within my podman container:
> > > > > Before (1): rs->sq_size = rs->rq_size = 384
> > > > > After (1): rs->sq_size = rs->rq_size = 0
> > > > > Therefore, (2) does not reserve a buffer, but still returns a pointer which can be freed later, as described by man-page calloc(3p).
> > > > > (5) writes data to the buffer allocated in (2). If no actual buffer is allocated, it overwrites arbitrary data, yielding undefined
> > > > > behavior.
> > > > >
> > > > > I found it by executing /usr/bin/udpong (without any arguments) with libscudo on an arm64 server with memory tagging enabled.
> > It
> > > > > immediately crashes with a segmentation fault, then. Without memory tagging, the bug stays undetected, and execution
> > continues.
> > > > > The code behavior described above also happens on x86-64, there it doesn't result in a crash and is silently ignored because of
> > the
> > > > > lack of MemoryTagging. Valgrind also detects this violation, however.
> > > > >
> > > > > In summary:
> > > > > The libc man-page states that if the allocated buffer size is 0, then either:
> > > > > >        *  A null pointer shall be returned and errno may be set to an
> > > > > >        implementation-defined value, or
> > > > > >        *  A pointer to the allocated space shall be returned. The
> > > > > >        application shall ensure that the pointer is not used to
> > > > > >        access an object.
> > > >
> > > > Can you please provide a link to these sentences in the manual?
> > > >
> > > > You provided invalid value as sq_size and rq_size. It is expected that
> > > > library won't work after that.
> > > >
> > > > Thanks
> > >
> > > These sentences are from: https://man7.org/linux/man-pages/man3/calloc.3p.html
> > >
> > > When the incoming values of sq_size and rq_size are wrong, that is a
> > > bug in udpong then. However, since librdmacm is doing the calloc allocation,
> > > it should still refrain from undefined behavior with the own allocated buffer.
> > 
> > I don't think so. This buffer is allocated in application memory, and if
> > the application is buggy, that is its problem, not the library's.
> > 
> > Thanks
> 
> Maybe we're not talking about the same thing. I'm specifically talking about this line of code:
> https://github.com/linux-rdma/rdma-core/blob/de593a02932a5ee7c729bd92df90e1fe8892b584/librdmacm/rsocket.c#L1169
> 
> Here, the buffer is allocated within the library, and the violating access happens in the same function in line 1186.
> Both appear to me to be part of librdmacm. However, I'm not that familiar about this entire system.

Yes, we are talking about the same. "rs" is allocated in the user context. It will crash user-application
and not whole librdmacm library.

Thanks

> 
> Thanks