Typecasting a void pointer to unsigned long in zsmalloc.c

Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

While looking through the zsmalloc.c file I stumbled upon something.

On the following line, in the zs_malloc() function:
handle = cache_alloc_handle(pool, gfp);
https://elixir.bootlin.com/linux/v6.13-rc3/source/mm/zsmalloc.c#L1356

the handle is a result of a typecast from a void pointer to an unsigned long.

return (unsigned long)kmem_cache_alloc(...);
https://elixir.bootlin.com/linux/v6.13-rc3/source/mm/zsmalloc.c#L333

What's the guarantee that an unsigned long can hold the data
from the void pointer? Is this safe to do? Shouldn't there be a special type
for doing this type of conversion?

I understand the reason for not using the void pointer directly and why
they need a "handle" instead, but I'm just a bit confused
by the method that has been chosen here.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Greg KH]

On Thu, Jan 23, 2025 at 04:52:24PM +0100, Sotir Danailov wrote:
> What's the guarantee that an unsigned long can hold the data
> from the void pointer?

This is a requirement that Linux makes for the compiler to work with
it for any platform that Linux runs on.

> Is this safe to do?

Yes.

> Shouldn't there be a special type for doing this type of conversion?

That type is "unsigned long" :)

thanks,

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

> This is a requirement that Linux makes for the compiler to work with
> it for any platform that Linux runs on.

Ah I see, thank you! Is this documented somewhere? I was searching
for information on this, but nothing came up in the kernel docs.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Greg KH]

On Thu, Jan 23, 2025 at 05:27:03PM +0100, Sotir Danailov wrote:
> > This is a requirement that Linux makes for the compiler to work with
> > it for any platform that Linux runs on.
> 
> Ah I see, thank you! Is this documented somewhere? I was searching
> for information on this, but nothing came up in the kernel docs.

It's just a "known thing" for Linux platforms.  It's documented
somewhere I think but I can't remember where, sorry.

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

I see, well if I find some info, I'll link it in the thread if that's
okay, in case someone stumbles over this in the future.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Leam Hall]

[-- Attachment #1.1: Type: text/plain, Size: 508 bytes --]

Would you mind posting it on the Kernel Newbies website? That would let
people fing it for years to come.

Thanks!

Leam

On Thu, Jan 23, 2025, 11:34 Sotir Danailov <sndanailov@gmail.com> wrote:

> I see, well if I find some info, I'll link it in the thread if that's
> okay, in case someone stumbles over this in the future.
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>

[-- Attachment #1.2: Type: text/html, Size: 1116 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Alex Hogen]

>> Shouldn't there be a special type for doing this type of conversion?

> That type is "unsigned long" :)

Wouldn't "uintptr_t" be a more explicit type for storing pointer addresses as integers? It seems to me it has the advantage of expressing intent better for us human readers. See <linux/types.h>. Yes, it is currently defined as an "unsigned long."
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Greg KH]

On Thu, Jan 23, 2025 at 05:58:10PM +0000, Alex Hogen wrote:
> >> Shouldn't there be a special type for doing this type of conversion?
> 
> > That type is "unsigned long" :)
> 
> Wouldn't "uintptr_t" be a more explicit type for storing pointer addresses as integers? It seems to me it has the advantage of expressing intent better for us human readers. See <linux/types.h>. Yes, it is currently defined as an "unsigned long."

"unitptr_t" came years, if not decades, after the policy of "a pointer
fits into an unsigned long" was the rule (i.e. from the very first
release of Linux.

And those crazy _t types are for userspace code, don't use them in the
kernel, there's no need.

thanks,

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

On Thu, Jan 23, 2025 at 6:41 PM Leam Hall <leamhall@gmail.com> wrote:
> Would you mind posting it on the Kernel Newbies website? That would let people fing it for years to come.

I don't mind, but I'm not allowed to edit the wiki and I'm not sure
where I'm supposed to add it on the website anyway.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[jim.cromie]

On Thu, Jan 23, 2025 at 9:27 AM Sotir Danailov <sndanailov@gmail.com> wrote:
>
> > This is a requirement that Linux makes for the compiler to work with
> > it for any platform that Linux runs on.
>
> Ah I see, thank you! Is this documented somewhere? I was searching
> for information on this, but nothing came up in the kernel docs.
>

theres a chance that it is implemented as a BUILD_BUG,
which would then be kinda self-documenting.
These are compiled in assertions, and break the build early.
Look at container_of() and marvel at its mechanics.

> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Leam Hall]

On 1/23/25 12:30, Sotir Danailov wrote:
> On Thu, Jan 23, 2025 at 6:41 PM Leam Hall <leamhall@gmail.com> wrote:
>> Would you mind posting it on the Kernel Newbies website? That would let people fing it for years to come.
> 
> I don't mind, but I'm not allowed to edit the wiki and I'm not sure
> where I'm supposed to add it on the website anyway.


Sotir,

I can edit the wiki and I don't mind taking care of it, if you can let me know what it should say. Your kernel skill is much higher than mine. If you want, we can discuss it here so others can help us get the wording and location right.

Thanks!

Leam


Linux Software Engineer   (reuel.net/career)
Scribe: The Domici War    (domiciwar.net)
Coding Ne'er-do-well      (github.com/LeamHall)

Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh

Practice allows options and foresight. -- lh

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

On Thu, Jan 23, 2025 at 8:55 PM Leam Hall <leamhall@gmail.com> wrote:
> I can edit the wiki and I don't mind taking care of it, if you can let me know what it should say.

Okay sounds good, I just need to dig around and find out more about
the topic and see if there's proper documentation for it on
kernel.org. I hope someone, that knows something about it in more
details, can just mention in the thread.

> Your kernel skill is much higher than mine.

No, definitely not, I'm a complete newbie when it comes to kernel development.

> If you want, we can discuss it here so others can help us get the wording and location right.

Yeah of course! I'd prefer this be a public discussion for sure.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Costa Shulyupin]

[-- Attachment #1.1: Type: text/plain, Size: 516 bytes --]

Hi. I've found this:

Pointers are __u64, cast from/to a uintptr_t on the userspace side and
from/to a void __user * in the kernel. Try really hard not to delay this
conversion or worse, fiddle the raw __u64 through your code since that
diminishes the checking tools like sparse can provide. The macro
u64_to_user_ptr can be used in the kernel to avoid warnings about integers
and pointers of different sizes.

https://origin.kernel.org/doc/html/latest/process/botching-up-ioctls.html#:~:text=Pointers%20are%20__u64

[-- Attachment #1.2: Type: text/html, Size: 827 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Greg KH]

On Thu, Jan 23, 2025 at 11:09:47PM +0200, Costa Shulyupin wrote:
> Hi. I've found this:
> 
> Pointers are __u64, cast from/to a uintptr_t on the userspace side and
> from/to a void __user * in the kernel. Try really hard not to delay this
> conversion or worse, fiddle the raw __u64 through your code since that
> diminishes the checking tools like sparse can provide. The macro
> u64_to_user_ptr can be used in the kernel to avoid warnings about integers
> and pointers of different sizes.
> 
> https://origin.kernel.org/doc/html/latest/process/botching-up-ioctls.html#:~:text=Pointers%20are%20__u64

That is talking about the ioctl user/kernel boundry and passing pointers
across it.  Not the use of pointers directly in the kernel tree.

The goal of that paragraph is to tell people to always use a 64bit value
for a pointer they are putting into an ioctl, which ensures that the
structure remains the same size no matter if you are running on a system
that has 32bit or 64bit pointers.

So while this is great advice, and should be followed, it doesn't really
discuss the implicit rule that "unsigned long can always hold a pointer"
that Linux relies on.

hope this helps,

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

I found this page which looks a bit legacy and doesn't seem to be
linked on kernel.org anywhere: https://www.kernel.org/doc/
In here, under "Standards documents applicable to the Linux kernel",
there's an interesting line:
"LP64 standard defining the size of char, short, int, and long on
32-bit and 64-bit platforms." and it links to this:
https://unix.org/whitepapers/64bit.html
In the link, under "64-bit Data Models", there's a table, which shows
that LP64 means:
char 8 bits, short 16 bits, int 32 bits, long 64 bits, pointer 64 bits

This lead me to:
https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-ABI/abi.pdf?job=build
which explains that for "System V AMD64 ABI":
on page 12: "all pointer types are 64-bit objects (LP64)"
on page 17: there's a table showing that "unsigned long (LP64)" is 8 bytes.

Finally, in GCC's manual for x86:
https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html#index-mabi-7
"The default is to use the Microsoft ABI when targeting Microsoft
Windows and the SysV ABI on all other systems."

I was wondering how it worked for other architectures, so I decided to
grep the Makefiles in the Linux tree and apparently for some
architectures it's explicitly stated:
./arch/arm64/Makefile:53:KBUILD_CFLAGS    += $(call cc-option,-mabi=lp64)
./arch/riscv/Makefile:33:    KBUILD_CFLAGS += -mabi=lp64

What I found in kernelnewbies: https://kernelnewbies.org/InternalKernelDataTypes
"One thing that has caused a lot of problems, as 64-bit machines are
getting more popular, is the fact that the size of a pointer is not
the same as the size of an unsigned integer. The size of a pointer is
equal to the size of an unsigned long."

I think there should be an explanation both on kernel.org and
kernelnewbies what determines the sizes, so that people can have a
better idea of what's going on. It should be something that is easy to
find and part of "introductions" to kernel development, because it's
not obvious at all how all of these things are setup. To me it seems
fundamental, to be able to quickly find easy to digest information
regarding the data types you're working with.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Leam Hall]

On 1/24/25 15:45, Sotir Danailov wrote:
> I found this page which looks a bit legacy and doesn't seem to be
> linked on kernel.org anywhere: https://www.kernel.org/doc/
> In here, under "Standards documents applicable to the Linux kernel",
> there's an interesting line:
> "LP64 standard defining the size of char, short, int, and long on
> 32-bit and 64-bit platforms." and it links to this:
> https://unix.org/whitepapers/64bit.html
> In the link, under "64-bit Data Models", there's a table, which shows
> that LP64 means:
> char 8 bits, short 16 bits, int 32 bits, long 64 bits, pointer 64 bits
> 
> This lead me to:
> https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-ABI/abi.pdf?job=build
> which explains that for "System V AMD64 ABI":
> on page 12: "all pointer types are 64-bit objects (LP64)"
> on page 17: there's a table showing that "unsigned long (LP64)" is 8 bytes.
> 
> Finally, in GCC's manual for x86:
> https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html#index-mabi-7
> "The default is to use the Microsoft ABI when targeting Microsoft
> Windows and the SysV ABI on all other systems."
> 
> I was wondering how it worked for other architectures, so I decided to
> grep the Makefiles in the Linux tree and apparently for some
> architectures it's explicitly stated:
> ./arch/arm64/Makefile:53:KBUILD_CFLAGS    += $(call cc-option,-mabi=lp64)
> ./arch/riscv/Makefile:33:    KBUILD_CFLAGS += -mabi=lp64
> 
> What I found in kernelnewbies: https://kernelnewbies.org/InternalKernelDataTypes
> "One thing that has caused a lot of problems, as 64-bit machines are
> getting more popular, is the fact that the size of a pointer is not
> the same as the size of an unsigned integer. The size of a pointer is
> equal to the size of an unsigned long."
> 
> I think there should be an explanation both on kernel.org and
> kernelnewbies what determines the sizes, so that people can have a
> better idea of what's going on. It should be something that is easy to
> find and part of "introductions" to kernel development, because it's
> not obvious at all how all of these things are setup. To me it seems
> fundamental, to be able to quickly find easy to digest information
> regarding the data types you're working with.


Sotir, great research!

Sadly, https://kernelnewbies.org is giving a "ConfigurationError".  :(  So I'm not sure if the reference to https://www.kernel.org/doc/ is already in place, or not. Something to check when KN is back up.

I'd think pointing to good documents is better than replicating them and risk staleness, isn't it?

Leam


Linux Software Engineer   (reuel.net/career)
Scribe: The Domici War    (domiciwar.net)
Coding Ne'er-do-well      (github.com/LeamHall)

Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh

Practice allows options and foresight. -- lh

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

On Mon, Jan 27, 2025 at 12:43 AM Leam Hall <leamhall@gmail.com> wrote:
> I'd think pointing to good documents is better than replicating them and risk staleness, isn't it?

Yes, but then the issue becomes, what if the documentation you're
referencing moves or is removed. People would lose the information
completely. I think it's good to point to it as a "source", but still
have a basic description.
The LP32/LP64 data models seem like one of those things which
shouldn't change anytime soon for the kernel anyway.

I would like to gather info regarding uintptr_t and other similar
types as well. Why they aren't used and have it all compiled into one
place.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Leam Hall]

On 1/27/25 04:39, Sotir Danailov wrote:
> On Mon, Jan 27, 2025 at 12:43 AM Leam Hall <leamhall@gmail.com> wrote:
>> I'd think pointing to good documents is better than replicating them and risk staleness, isn't it?
> 
> Yes, but then the issue becomes, what if the documentation you're
> referencing moves or is removed. People would lose the information
> completely. I think it's good to point to it as a "source", but still
> have a basic description.
> The LP32/LP64 data models seem like one of those things which
> shouldn't change anytime soon for the kernel anyway.
> 
> I would like to gather info regarding uintptr_t and other similar
> types as well. Why they aren't used and have it all compiled into one
> place.


My thinking is that a single authoritative source reduces time and change based risk. You really dug into this and found the information, the facts. Those facts can't actually be removed because they are part of the kernel code itself. Well, they can be removed but then they are no longer relevant facts. KernelNewbies is the gentle introduction to working on Linux, but much of its information is dated. Kernel 2.x is referenced often, and we're currently up to 6.x! That's one of the things I want to fix.

If you want my opinion, I'll offer it. The research you did was great, and useful for kernel programmers. I feel it is a more advanced topic, something a newbie wouldn't have a significant mental frame of reference for. Here are the sites that seem most relevant:


kernel.org
  - authoritative source of facts
  - encourage those who have access (which is not me) and responsibility to improve the documentation of those facts

tldp.org
  - In-depth information on Linux facts
  - - Sadly, it seems to be aging as the latest Guides update is almost ten years old

kernelnewbies.org
  - Help newbies find and use facts


Would you consider writing up a guide for either tldp.org or kernel.org?

Leam


Linux Software Engineer   (reuel.net/career)
Scribe: The Domici War    (domiciwar.net)
Coding Ne'er-do-well      (github.com/LeamHall)

Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh

Practice allows options and foresight. -- lh
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

> Kernel 2.x is referenced often, and we're currently up to 6.x! That's one of the things I want to fix.

Yeah this definitely seems more high priority currently. Is there
anything outside people can do, to help the effort?

> something a newbie wouldn't have a significant mental frame of reference for.

You're probably right, come to think of it now maybe describing all of
these details in kernelnewbies isn't appropriate.

What my concern with documentation usually is, when it's hard to
navigate or when fundamental information is missing or is spread
outside of the documentation, it becomes really discouraging to engage
with a project, especially for new people.

I'll look into what's missing, if anything, at kernel.org regarding
data types and attempt to send a patch for the docs there, when I have
the time.

> tldp.org
>   - In-depth information on Linux facts
>   - - Sadly, it seems to be aging as the latest Guides update is almost ten years old

I didn't even know about this one.

> kernelnewbies.org
>   - Help newbies find and use facts

This makes sense, I guess I interpreted it more as a small summary of
kernel.org, but helping people find the facts is the more practical
solution. There's definitely no way to keep track of everything
happening and updating it constantly, now that I think of it.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Costa Shulyupin]

[-- Attachment #1.1: Type: text/plain, Size: 129 bytes --]

Hi

You are welcome to contribute to
https://en.wikibooks.org/wiki/The_Linux_Kernel/Memory#Pointers_and_addresses


Thanks
Costa

[-- Attachment #1.2: Type: text/html, Size: 292 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

On Tue, Jan 28, 2025 at 9:59 AM Costa Shulyupin <costa.shul@redhat.com> wrote:
> You are welcome to contribute to
> https://en.wikibooks.org/wiki/The_Linux_Kernel/Memory#Pointers_and_addresses

Nice, I wasn't aware of this as well. Thank you!

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Sotir Danailov]

I'm sorry, I didn't intend for this thread to go beyond the initial
email and title! The rest of the thread should've been separate I feel
like.

Sorry if I filled the mailing list with noise, also wasn't my intention!

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Typecasting a void pointer to unsigned long in zsmalloc.c

[Leam Hall]

On 1/28/25 03:24, Sotir Danailov wrote:
> I'm sorry, I didn't intend for this thread to go beyond the initial
> email and title! The rest of the thread should've been separate I feel
> like.
> 
> Sorry if I filled the mailing list with noise, also wasn't my intention!


I have enjoyed it, and I think it has been educational. Which is a KernelNewbies thing.

Leam

Linux Software Engineer   (reuel.net/career)
Scribe: The Domici War    (domiciwar.net)
Coding Ne'er-do-well      (github.com/LeamHall)

Between "can" and "can't" is a gap of "I don't know", a place of discovery. For the passionate, much of "can't" falls into "yet". -- lh

Practice allows options and foresight. -- lh

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies