readlink() example sometimes fails

readlink() example sometimes fails

[Ursache Vladimir]

 Hi,
the readlink() example from:

http://man7.org/linux/man-pages/man2/readlink.2.html

relies on lstat()'s st_size, but that doesn't work for files contents
created dynamically by the Linux kernel which often report a zero
size, for example the link at:
/sys/block/sda

the example code will fail because stat.st_size reports zero and you
try to read (stat.st_size + 1) which will succeed, which will generate
the error : "symlink increased in size between lstat() and
readlink()".

Somewhat related, the same issue is true for reading regular text
files, e.g: "/proc/filesystems" which will report stat.st_size = 0.

My quick workaround:

if (stat.st_size != 0)
    // work as usual
else if (file_is_a_link)
    // malloc 4K of ram and try to readlink() into it
else if (is_regular_file)
    // read() into a byte array that grows accordingly

In case it matters, I'm using Ubuntu 16.04 amd64.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: readlink() example sometimes fails

[Michael Kerrisk (man-pages)]

Hello Vladimir,

On 08/18/2016 05:30 PM, Ursache Vladimir wrote:
>  Hi,
> the readlink() example from:
> 
> http://man7.org/linux/man-pages/man2/readlink.2.html
> 
> relies on lstat()'s st_size, but that doesn't work for files contents
> created dynamically by the Linux kernel which often report a zero
> size, for example the link at:
> /sys/block/sda
> 
> the example code will fail because stat.st_size reports zero and you
> try to read (stat.st_size + 1) which will succeed, which will generate
> the error : "symlink increased in size between lstat() and
> readlink()".
> 
> Somewhat related, the same issue is true for reading regular text
> files, e.g: "/proc/filesystems" which will report stat.st_size = 0.
> 
> My quick workaround:
> 
> if (stat.st_size != 0)
>     // work as usual
> else if (file_is_a_link)
>     // malloc 4K of ram and try to readlink() into it
> else if (is_regular_file)
>     // read() into a byte array that grows accordingly
> 
> In case it matters, I'm using Ubuntu 16.04 amd64.

Thanks for this report. I agree that the page needs to be fixed.
I modified the example program to be as follows:

#include <sys/types.h>
#include <sys/stat.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int
main(int argc, char *argv[])
{
    struct stat sb;
    char *linkname;
    ssize_t r, bufsiz;

    if (argc != 2) {
        fprintf(stderr, "Usage: %s <pathname>\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    if (lstat(argv[1], &sb) == -1) {
        perror("lstat");
        exit(EXIT_FAILURE);
    }

    bufsiz = sb.st_size + 1;

    /* Some magic symlinks under (for example) /proc and /sys
       report 'st_size' as zero. In that case, take PATH_MAX as
       a "good enough" estimate */

    if (sb.st_size == 0)
	bufsiz = PATH_MAX;

    printf("%zd\n", bufsiz);

    linkname = malloc(bufsiz);
    if (linkname == NULL) {
        perror("malloc");
        exit(EXIT_FAILURE);
    }

    r = readlink(argv[1], linkname, bufsiz);
    if (r == -1) {
        perror("readlink");
        exit(EXIT_FAILURE);
    }

    linkname[r] = '\0';

    printf("'%s' points to '%s'\n", argv[1], linkname);

    if (r == bufsiz)
        printf("(Returned buffer may have been truncated)\n");

    free(linkname);
    exit(EXIT_SUCCESS);
}

Seem okay?

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: readlink() example sometimes fails

[Ursache Vladimir]

Seems fine, works for me, thank you.

On Sat, Aug 20, 2016 at 6:32 AM, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hello Vladimir,
>
> On 08/18/2016 05:30 PM, Ursache Vladimir wrote:
>>  Hi,
>> the readlink() example from:
>>
>> http://man7.org/linux/man-pages/man2/readlink.2.html
>>
>> relies on lstat()'s st_size, but that doesn't work for files contents
>> created dynamically by the Linux kernel which often report a zero
>> size, for example the link at:
>> /sys/block/sda
>>
>> the example code will fail because stat.st_size reports zero and you
>> try to read (stat.st_size + 1) which will succeed, which will generate
>> the error : "symlink increased in size between lstat() and
>> readlink()".
>>
>> Somewhat related, the same issue is true for reading regular text
>> files, e.g: "/proc/filesystems" which will report stat.st_size = 0.
>>
>> My quick workaround:
>>
>> if (stat.st_size != 0)
>>     // work as usual
>> else if (file_is_a_link)
>>     // malloc 4K of ram and try to readlink() into it
>> else if (is_regular_file)
>>     // read() into a byte array that grows accordingly
>>
>> In case it matters, I'm using Ubuntu 16.04 amd64.
>
> Thanks for this report. I agree that the page needs to be fixed.
> I modified the example program to be as follows:
>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <limits.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>
>
> int
> main(int argc, char *argv[])
> {
>     struct stat sb;
>     char *linkname;
>     ssize_t r, bufsiz;
>
>     if (argc != 2) {
>         fprintf(stderr, "Usage: %s <pathname>\n", argv[0]);
>         exit(EXIT_FAILURE);
>     }
>
>     if (lstat(argv[1], &sb) == -1) {
>         perror("lstat");
>         exit(EXIT_FAILURE);
>     }
>
>     bufsiz = sb.st_size + 1;
>
>     /* Some magic symlinks under (for example) /proc and /sys
>        report 'st_size' as zero. In that case, take PATH_MAX as
>        a "good enough" estimate */
>
>     if (sb.st_size == 0)
>         bufsiz = PATH_MAX;
>
>     printf("%zd\n", bufsiz);
>
>     linkname = malloc(bufsiz);
>     if (linkname == NULL) {
>         perror("malloc");
>         exit(EXIT_FAILURE);
>     }
>
>     r = readlink(argv[1], linkname, bufsiz);
>     if (r == -1) {
>         perror("readlink");
>         exit(EXIT_FAILURE);
>     }
>
>     linkname[r] = '\0';
>
>     printf("'%s' points to '%s'\n", argv[1], linkname);
>
>     if (r == bufsiz)
>         printf("(Returned buffer may have been truncated)\n");
>
>     free(linkname);
>     exit(EXIT_SUCCESS);
> }
>
> Seem okay?
>
> Cheers,
>
> Michael
>
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: readlink() example sometimes fails

[Ursache Vladimir]

Here's another issue, this time related to stat(), st_size and regular files.

I think stat(2) should be updated in relation to exceptions with
st_size and regular files.
Currently the man pages describe this exception (corner case) as:

"For most files under the /proc directory, stat() does not return the
file size in the st_size
field; instead the field is returned with the value 0."

However, I found at /sys/devices/cpu/ and in other (Linux kernel)
directories (all) regular files
to report 4096 bytes yet their real size is only a few bytes (for
example the regular
 file "/sys/devices/cpu/type" st_size=4096 bytes yet one can only read 2 bytes).

Hence in some cases (st_size=0) it reports a smaller size, and in the
latter case - a bigger size
(st_size=4096).

I don't know if these cases are posix compliant and since you have a
lot more experience I hope you'll
write the proper explanation in place of the one mentioned at the top.
I would write something
like this:

"Many regular files generated by the (Linux) kernel at /proc or /sys
return an st_size that has nothing
to do with the real file size so one should try to read as much as one
can and append
a '\0' at the end if it's a text file";

But then the regular file at /proc/kcore reports st_size=many terabytes.






On Sat, Aug 20, 2016 at 10:57 AM, Ursache Vladimir <f35f22fan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Seems fine, works for me, thank you.
>
> On Sat, Aug 20, 2016 at 6:32 AM, Michael Kerrisk (man-pages)
> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> Hello Vladimir,
>>
>> On 08/18/2016 05:30 PM, Ursache Vladimir wrote:
>>>  Hi,
>>> the readlink() example from:
>>>
>>> http://man7.org/linux/man-pages/man2/readlink.2.html
>>>
>>> relies on lstat()'s st_size, but that doesn't work for files contents
>>> created dynamically by the Linux kernel which often report a zero
>>> size, for example the link at:
>>> /sys/block/sda
>>>
>>> the example code will fail because stat.st_size reports zero and you
>>> try to read (stat.st_size + 1) which will succeed, which will generate
>>> the error : "symlink increased in size between lstat() and
>>> readlink()".
>>>
>>> Somewhat related, the same issue is true for reading regular text
>>> files, e.g: "/proc/filesystems" which will report stat.st_size = 0.
>>>
>>> My quick workaround:
>>>
>>> if (stat.st_size != 0)
>>>     // work as usual
>>> else if (file_is_a_link)
>>>     // malloc 4K of ram and try to readlink() into it
>>> else if (is_regular_file)
>>>     // read() into a byte array that grows accordingly
>>>
>>> In case it matters, I'm using Ubuntu 16.04 amd64.
>>
>> Thanks for this report. I agree that the page needs to be fixed.
>> I modified the example program to be as follows:
>>
>> #include <sys/types.h>
>> #include <sys/stat.h>
>> #include <limits.h>
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <unistd.h>
>>
>> int
>> main(int argc, char *argv[])
>> {
>>     struct stat sb;
>>     char *linkname;
>>     ssize_t r, bufsiz;
>>
>>     if (argc != 2) {
>>         fprintf(stderr, "Usage: %s <pathname>\n", argv[0]);
>>         exit(EXIT_FAILURE);
>>     }
>>
>>     if (lstat(argv[1], &sb) == -1) {
>>         perror("lstat");
>>         exit(EXIT_FAILURE);
>>     }
>>
>>     bufsiz = sb.st_size + 1;
>>
>>     /* Some magic symlinks under (for example) /proc and /sys
>>        report 'st_size' as zero. In that case, take PATH_MAX as
>>        a "good enough" estimate */
>>
>>     if (sb.st_size == 0)
>>         bufsiz = PATH_MAX;
>>
>>     printf("%zd\n", bufsiz);
>>
>>     linkname = malloc(bufsiz);
>>     if (linkname == NULL) {
>>         perror("malloc");
>>         exit(EXIT_FAILURE);
>>     }
>>
>>     r = readlink(argv[1], linkname, bufsiz);
>>     if (r == -1) {
>>         perror("readlink");
>>         exit(EXIT_FAILURE);
>>     }
>>
>>     linkname[r] = '\0';
>>
>>     printf("'%s' points to '%s'\n", argv[1], linkname);
>>
>>     if (r == bufsiz)
>>         printf("(Returned buffer may have been truncated)\n");
>>
>>     free(linkname);
>>     exit(EXIT_SUCCESS);
>> }
>>
>> Seem okay?
>>
>> Cheers,
>>
>> Michael
>>
>>
>> --
>> Michael Kerrisk
>> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
>> Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: readlink() example sometimes fails

[Michael Kerrisk (man-pages)]

On 08/27/2016 05:47 AM, Ursache Vladimir wrote:
> Here's another issue, this time related to stat(), st_size and regular files.
> 
> I think stat(2) should be updated in relation to exceptions with
> st_size and regular files.
> Currently the man pages describe this exception (corner case) as:
> 
> "For most files under the /proc directory, stat() does not return the
> file size in the st_size
> field; instead the field is returned with the value 0."
> 
> However, I found at /sys/devices/cpu/ and in other (Linux kernel)
> directories (all) regular files
> to report 4096 bytes yet their real size is only a few bytes (for
> example the regular
>  file "/sys/devices/cpu/type" st_size=4096 bytes yet one can only read 2 bytes).
> 
> Hence in some cases (st_size=0) it reports a smaller size, and in the
> latter case - a bigger size
> (st_size=4096).
> 
> I don't know if these cases are posix compliant and since you have a
> lot more experience I hope you'll
> write the proper explanation in place of the one mentioned at the top.
> I would write something
> like this:
> 
> "Many regular files generated by the (Linux) kernel at /proc or /sys
> return an st_size that has nothing
> to do with the real file size so one should try to read as much as one
> can and append
> a '\0' at the end if it's a text file";
> 
> But then the regular file at /proc/kcore reports st_size=many terabytes.

So, it's not going to be possible to explain every corner case here, but
text such as you suggest is good to alert the programmer. Starting from 
your suggestion, I reworked this to:

       For many pseudofiles that are autogenerated by the kernel,  stat()
       does not return an accurate value in the st_size field.  For exam‐
       ple, the value 0 is returned for many files under the /proc direc‐
       tory,  while various files under /sys report a size of 4096 bytes,
       even though the file content is  smaller.   For  such  files,  one
       should  simply  try  to read as many bytes as possible (and append
       '\0' to the returned buffer if  it  is  to  be  interpreted  as  a
       string).

Okay?

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: readlink() example sometimes fails

[Ursache Vladimir]

 yeah, thanks.

On Sun, Aug 28, 2016 at 11:48 PM, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On 08/27/2016 05:47 AM, Ursache Vladimir wrote:
>> Here's another issue, this time related to stat(), st_size and regular files.
>>
>> I think stat(2) should be updated in relation to exceptions with
>> st_size and regular files.
>> Currently the man pages describe this exception (corner case) as:
>>
>> "For most files under the /proc directory, stat() does not return the
>> file size in the st_size
>> field; instead the field is returned with the value 0."
>>
>> However, I found at /sys/devices/cpu/ and in other (Linux kernel)
>> directories (all) regular files
>> to report 4096 bytes yet their real size is only a few bytes (for
>> example the regular
>>  file "/sys/devices/cpu/type" st_size=4096 bytes yet one can only read 2 bytes).
>>
>> Hence in some cases (st_size=0) it reports a smaller size, and in the
>> latter case - a bigger size
>> (st_size=4096).
>>
>> I don't know if these cases are posix compliant and since you have a
>> lot more experience I hope you'll
>> write the proper explanation in place of the one mentioned at the top.
>> I would write something
>> like this:
>>
>> "Many regular files generated by the (Linux) kernel at /proc or /sys
>> return an st_size that has nothing
>> to do with the real file size so one should try to read as much as one
>> can and append
>> a '\0' at the end if it's a text file";
>>
>> But then the regular file at /proc/kcore reports st_size=many terabytes.
>
> So, it's not going to be possible to explain every corner case here, but
> text such as you suggest is good to alert the programmer. Starting from
> your suggestion, I reworked this to:
>
>        For many pseudofiles that are autogenerated by the kernel,  stat()
>        does not return an accurate value in the st_size field.  For exam‐
>        ple, the value 0 is returned for many files under the /proc direc‐
>        tory,  while various files under /sys report a size of 4096 bytes,
>        even though the file content is  smaller.   For  such  files,  one
>        should  simply  try  to read as many bytes as possible (and append
>        '\0' to the returned buffer if  it  is  to  be  interpreted  as  a
>        string).
>
> Okay?
>
> Cheers,
>
> Michael
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: readlink() example sometimes fails

[Mats Wichmann]

On 08/28/2016 02:48 PM, Michael Kerrisk (man-pages) wrote:

> So, it's not going to be possible to explain every corner case here, but
> text such as you suggest is good to alert the programmer. Starting from 
> your suggestion, I reworked this to:
> 
>        For many pseudofiles that are autogenerated by the kernel,  stat()
>        does not return an accurate value in the st_size field.  For exam‐
>        ple, the value 0 is returned for many files under the /proc direc‐
>        tory,  while various files under /sys report a size of 4096 bytes,
>        even though the file content is  smaller.   For  such  files,  one
>        should  simply  try  to read as many bytes as possible (and append
>        '\0' to the returned buffer if  it  is  to  be  interpreted  as  a
>        string).
> 
> Okay?

Is it possibly better to not suggest that for some /proc files it might
still be useful? (I don't actually know if there are cases where it is):

"For pseudofiles that are autogenerated by the kernel, stat() does not
return a meaningful value in the st_size field".
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: readlink() example sometimes fails

[Michael Kerrisk (man-pages)]

Hello Mats,

On 30 August 2016 at 03:17, Mats Wichmann <mats-mBRmNHn34rVzbRFIqnYvSA@public.gmane.org> wrote:
> On 08/28/2016 02:48 PM, Michael Kerrisk (man-pages) wrote:
>
>> So, it's not going to be possible to explain every corner case here, but
>> text such as you suggest is good to alert the programmer. Starting from
>> your suggestion, I reworked this to:
>>
>>        For many pseudofiles that are autogenerated by the kernel,  stat()
>>        does not return an accurate value in the st_size field.  For exam‐
>>        ple, the value 0 is returned for many files under the /proc direc‐
>>        tory,  while various files under /sys report a size of 4096 bytes,
>>        even though the file content is  smaller.   For  such  files,  one
>>        should  simply  try  to read as many bytes as possible (and append
>>        '\0' to the returned buffer if  it  is  to  be  interpreted  as  a
>>        string).
>>
>> Okay?
>
> Is it possibly better to not suggest that for some /proc files it might
> still be useful? (I don't actually know if there are cases where it is):
>
> "For pseudofiles that are autogenerated by the kernel, stat() does not
> return a meaningful value in the st_size field".

Yes, some scripting confirms you are right. I changed the wording as
you suggest. Thanks for catching that!

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html