[uml-devel] UML hostfs bugs

[uml-devel] UML hostfs bugs

[wang lianwei]

[-- Attachment #1.1: Type: text/plain, Size: 1292 bytes --]

Hi all,

There is a bug with UML kernel, some host directory can not be read, for
example "ls /mnt/host/usr/lib", this command is dead until you press CTRL+c.
I test it with Redhat Enterprise Linux 4.


This is because the UML read host directory through opendir, seekdir,
readdir and telldir. When get the dir stream offset through telldir, it
always return 0 on some directory, for example".", I don't know why? But
we can fix it as follows although the next dir entry offset is not exact:


Modify the fs/hostfs/hostfs_user.c file, update the read_dir function,

char *read_dir(void *stream, unsigned long long *pos,
               unsigned long long *ino_out, int *len_out)
{
        DIR *dir = stream;
        struct dirent *ent;
        unsigned long long oldpos;

        oldpos = *pos;

        seekdir(dir, *pos);
        ent = readdir(dir);
        if(ent == NULL)
        {
                return(NULL);
        }
        *len_out = strlen(ent->d_name);
        *ino_out = ent->d_ino;
        *pos = telldir(dir);

        /* Fix the telldir return 0 error */
        if (*pos == 0)
                *pos = oldpos + *len_out;

        return(ent->d_name);
}

although the next offset is not exact, but the readdir can get the next
entry correct.

Can anyone give a better solution?

[-- Attachment #1.2: Type: text/html, Size: 4202 bytes --]

[-- Attachment #2: Type: text/plain, Size: 373 bytes --]

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 194 bytes --]

_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML hostfs bugs

[Blaisorblade]

On Thursday 10 August 2006 17:42, wang lianwei wrote:
> Hi all,
>
> There is a bug with UML kernel, some host directory can not be read, for
> example "ls /mnt/host/usr/lib", this command is dead until you press
> CTRL+c. I test it with Redhat Enterprise Linux 4.

Yes, this has been reported various times but difficult to reproduce, so we're 
very happy somebody diagnosed it, and I'd be interested in understanding it 
better.

On which filesystem (ext3, reiserfs, what else) lays /usr/lib on the host?

> This is because the UML read host directory through opendir, seekdir,
> readdir and telldir. When get the dir stream offset through telldir, it
> always return 0 on some directory, for example".",

Can you elaborate on this? (And if possible post some details, like a strace)

What I've understood is:

position 0 -> you read "." -> you call telldir -> it _again_ says you're at 0?

I can maybe understand why it could happen. I'd be more interested in checking 
if a simple sequence of readdir() calls (without seekdir()) would work 
(probably write a standalone test program, it will be easier to use and 
debug) and why read_dir needs to do seekdir (I think it's done to handle 
calls to seekdir by the underlying process, but there should be different 
ways to do it).

> I don't know why? But 
> we can fix it as follows although the next dir entry offset is not exact:

> Modify the fs/hostfs/hostfs_user.c file, update the read_dir function,
>
> char *read_dir(void *stream, unsigned long long *pos,
>                unsigned long long *ino_out, int *len_out)
> {
>         DIR *dir = stream;
>         struct dirent *ent;
>         unsigned long long oldpos;
>
>         oldpos = *pos;
>
>         seekdir(dir, *pos);
>         ent = readdir(dir);
>         if(ent == NULL)
>         {
>                 return(NULL);
>         }
>         *len_out = strlen(ent->d_name);
>         *ino_out = ent->d_ino;
>         *pos = telldir(dir);
>
>         /* Fix the telldir return 0 error */
>         if (*pos == 0)
>                 *pos = oldpos + *len_out;
>
>         return(ent->d_name);
> }
>
> although the next offset is not exact, but the readdir can get the next
> entry correct.

This probably depends heavily on the underlying filesystem...

> Can anyone give a better solution?

I can't yet, but maybe with your help we'll be able.
-- 
Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
Paolo Giarrusso, aka Blaisorblade
http://www.user-mode-linux.org/~blaisorblade
Chiacchiera con i tuoi amici in tempo reale! 
 http://it.yahoo.com/mail_it/foot/*http://it.messenger.yahoo.com 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML hostfs bugs

[Blaisorblade]

On Saturday 12 August 2006 20:22, Blaisorblade wrote:
> On Thursday 10 August 2006 17:42, wang lianwei wrote:
> > Hi all,
> >
> > There is a bug with UML kernel, some host directory can not be read, for
> > example "ls /mnt/host/usr/lib", this command is dead until you press
> > CTRL+c. I test it with Redhat Enterprise Linux 4.

> I can maybe understand why it could happen. I'd be more interested in
> checking if a simple sequence of readdir() calls (without seekdir()) would
> work (probably write a standalone test program, it will be easier to use
> and debug) and 

> why read_dir needs to do seekdir (I think it's done to 
> handle calls to seekdir by the underlying process, but there should be
> different ways to do it).

I want to propose as an alternative modifying hostfs_dir_fops.llseek to point 
to a wrapper of generic_file_llseek doing what we need (i.e. seeking the 
underlying pointer), and avoiding seekdir and telldir in read_dir.
This should fix the issue (I think).

I'm sorry but I will not have the time to code the patch, so please do it 
yourselves; Wang, you're welcome to code it yourself if you feel like doing 
it.

Bye
-- 
Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
Paolo Giarrusso, aka Blaisorblade
http://www.user-mode-linux.org/~blaisorblade
Chiacchiera con i tuoi amici in tempo reale! 
 http://it.yahoo.com/mail_it/foot/*http://it.messenger.yahoo.com 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML hostfs bugs

[wang lianwei]

[-- Attachment #1.1: Type: text/plain, Size: 4022 bytes --]

Answers:

1. I test it on ext3 filesystem.

2. You are right. Sometimes the pos telldir returned is not the next entry
offset. just as your point out:
      position 0 -> readdir "." -> you call telldir -> it _again_ says
you're at 0.

3. I also try to resolve it by remove the seekdir(only use the
sequence readdir call), It is also worked.

--- linux-2.6.x/fs/hostfs/hostfs_kern.c 2006-04-12 15:12:51.000000000 +0800
+++ linux-2.6.10-src/fs/hostfs/hostfs_kern.c      2006-08-14 18:25:
13.772237744 +0800
@@ -338,6 +338,7 @@
        kfree(name);
        if(dir == NULL) return(-error);
        next = file->f_pos;
+       seekdir(dir,next);
        while((name = read_dir(dir, &next, &ino, &len)) != NULL){
                error = (*filldir)(ent, name, len, file->f_pos,
                                   ino, DT_UNKNOWN);


--- linux-2.6.x/fs/hostfs/hostfs_user.c 2006-04-12 15:12:52.000000000 +0800
+++ linux-2.6.10-src/fs/hostfs/hostfs_user.c      2006-08-14 18:36:
24.276305632 +0800
@@ -122,7 +122,6 @@
        DIR *dir = stream;
        struct dirent *ent;

-       seekdir(dir, *pos);
        ent = readdir(dir);
        if(ent == NULL) return(NULL);
        *len_out = strlen(ent->d_name);




2006/8/13, Blaisorblade <blaisorblade@yahoo.it>:
>
> On Thursday 10 August 2006 17:42, wang lianwei wrote:
> > Hi all,
> >
> > There is a bug with UML kernel, some host directory can not be read, for
>
> > example "ls /mnt/host/usr/lib", this command is dead until you press
> > CTRL+c. I test it with Redhat Enterprise Linux 4.
>
> Yes, this has been reported various times but difficult to reproduce, so
> we're
> very happy somebody diagnosed it, and I'd be interested in understanding
> it
> better.
>
> On which filesystem (ext3, reiserfs, what else) lays /usr/lib on the host?
>
> This is because the UML read host directory through opendir, seekdir,
> > readdir and telldir. When get the dir stream offset through telldir, it
> > always return 0 on some directory, for example".",
>
> Can you elaborate on this? (And if possible post some details, like a
> strace)
>
> What I've understood is:
>
> position 0 -> you read "." -> you call telldir -> it _again_ says you're
> at 0?
>
> I can maybe understand why it could happen. I'd be more interested in
> checking
> if a simple sequence of readdir() calls (without seekdir()) would work
> (probably write a standalone test program, it will be easier to use and
> debug) and why read_dir needs to do seekdir (I think it's done to handle
> calls to seekdir by the underlying process, but there should be different
> ways to do it).
>
> > I don't know why? But
> > we can fix it as follows although the next dir entry offset is not
> exact:
>
> > Modify the fs/hostfs/hostfs_user.c file, update the read_dir function,
> >
> > char *read_dir(void *stream, unsigned long long *pos,
> >                unsigned long long *ino_out, int *len_out)
> > {
> >         DIR *dir = stream;
> >         struct dirent *ent;
> >         unsigned long long oldpos;
> >
> >         oldpos = *pos;
> >
> >         seekdir(dir, *pos);
> >         ent = readdir(dir);
> >         if(ent == NULL)
> >         {
> >                 return(NULL);
> >         }
> >         *len_out = strlen(ent->d_name);
> >         *ino_out = ent->d_ino;
> >         *pos = telldir(dir);
> >
> >         /* Fix the telldir return 0 error */
> >         if (*pos == 0)
> >                 *pos = oldpos + *len_out;
> >
> >         return(ent->d_name);
> > }
> >
> > although the next offset is not exact, but the readdir can get the next
> > entry correct.
>
> This probably depends heavily on the underlying filesystem...
>
> > Can anyone give a better solution?
>
> I can't yet, but maybe with your help we'll be able.
> --
> Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
> Paolo Giarrusso, aka Blaisorblade
> http://www.user-mode-linux.org/~blaisorblade
> Chiacchiera con i tuoi amici in tempo reale!
> http://it.yahoo.com/mail_it/foot/*http://it.messenger.yahoo.com
>
>

[-- Attachment #1.2: Type: text/html, Size: 6873 bytes --]

[-- Attachment #2: Type: text/plain, Size: 373 bytes --]

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 194 bytes --]

_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML hostfs bugs

[Blaisorblade]

On Monday 14 August 2006 13:07, wang lianwei wrote:
> Answers:
>
> 1. I test it on ext3 filesystem.
>
> 2. You are right. Sometimes the pos telldir returned is not the next entry
> offset. just as your point out:
>       position 0 -> readdir "." -> you call telldir -> it _again_ says
> you're at 0.
>
> 3. I also try to resolve it by remove the seekdir(only use the
> sequence readdir call), It is also worked.

I can guess what the patch does but please generate with the additional -p 
option so it's easier to read (otherwise the recipient _has_ to read the 
source). Compliments for the solution, it's really simpler and better than my 
proposal.

Also, you have to regenerate it anyway for another (tiny) reason: it is not 
always safe to use "seekdir" in kernelspace code, you need (sadly) a new 
wrapper in hostfs_user.c and to call it instead of seekdir in hostfs_kern.c; 
almost surely you'll get an "undeclared function" warning but miscompilations 
in some rare cases are also possible, say for stat(): it is defined almost 
only as a static inline function, or for some structures having different 
definitions in userspace and kernelspace.

> --- linux-2.6.x/fs/hostfs/hostfs_kern.c 2006-04-12 15:12:51.000000000 +0800
> +++ linux-2.6.10-src/fs/hostfs/hostfs_kern.c      2006-08-14 18:25:
> 13.772237744 +0800
> @@ -338,6 +338,7 @@
>         kfree(name);
>         if(dir == NULL) return(-error);
>         next = file->f_pos;
> +       seekdir(dir,next);
>         while((name = read_dir(dir, &next, &ino, &len)) != NULL){
>                 error = (*filldir)(ent, name, len, file->f_pos,
>                                    ino, DT_UNKNOWN);
>
>
> --- linux-2.6.x/fs/hostfs/hostfs_user.c 2006-04-12 15:12:52.000000000 +0800
> +++ linux-2.6.10-src/fs/hostfs/hostfs_user.c      2006-08-14 18:36:
> 24.276305632 +0800
> @@ -122,7 +122,6 @@
>         DIR *dir = stream;
>         struct dirent *ent;
>
> -       seekdir(dir, *pos);
>         ent = readdir(dir);
>         if(ent == NULL) return(NULL);
>         *len_out = strlen(ent->d_name);

-- 
Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
Paolo Giarrusso, aka Blaisorblade
http://www.user-mode-linux.org/~blaisorblade
Chiacchiera con i tuoi amici in tempo reale! 
 http://it.yahoo.com/mail_it/foot/*http://it.messenger.yahoo.com 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML hostfs bugs

[wang lianwei]

[-- Attachment #1.1: Type: text/plain, Size: 4613 bytes --]

yes, you are right. see this patches.


diff -urNp linux-2.6.10/fs/hostfs/hostfs.h linux-2.6.10-src
/fs/hostfs/hostfs.h
--- linux-2.6.10/fs/hostfs/hostfs.h      2006-04-12 15:12:51.000000000 +0800
+++ linux-2.6.10-src/fs/hostfs/hostfs.h   2006-08-16 15:58:59.149590168+0800
@@ -40,6 +40,7 @@ extern int access_file(char *path, int r
 extern int open_file(char *path, int r, int w, int append);
 extern int file_type(const char *path, int *maj, int *min);
 extern void *open_dir(char *path, int *err_out);
+extern void seek_dir(void *stream, unsigned long long pos);
 extern char *read_dir(void *stream, unsigned long long *pos,
                      unsigned long long *ino_out, int *len_out);
 extern void close_file(void *stream);



diff -urNp linux-2.6.10/fs/hostfs/hostfs_kern.c linux-2.6.10-src
/fs/hostfs/hostfs_kern.c
--- linux-2.6.10/fs/hostfs/hostfs_kern.c 2006-04-12 15:12:51.000000000 +0800
+++ linux-2.6.10-src/fs/hostfs/hostfs_kern.c      2006-08-16 15:51:
08.105199800 +0800
@@ -338,6 +338,7 @@ int hostfs_readdir(struct file *file, vo
        kfree(name);
        if(dir == NULL) return(-error);
        next = file->f_pos;
+      seek_dir(dir, next);
        while((name = read_dir(dir, &next, &ino, &len)) != NULL){
                error = (*filldir)(ent, name, len, file->f_pos,
                                   ino, DT_UNKNOWN);



diff -urNp linux-2.6.10/fs/hostfs/hostfs_user.c linux-2.6.10-src
/fs/hostfs/hostfs_user.c
--- linux-2.6.x/fs/hostfs/hostfs_user.c 2006-04-12 15:12:52.000000000 +0800
+++ linux-2.6.10-src/fs/hostfs/hostfs_user.c      2006-08-16 15:53:
31.618382480 +0800
@@ -116,13 +116,19 @@ void *open_dir(char *path, int *err_out)
        return(dir);
 }

+void seek_dir(void *stream, unsigned long long pos)
+{
+        DIR *dir = stream;
+
+        seekdir(dir, pos);
+}
+
 char *read_dir(void *stream, unsigned long long *pos,
               unsigned long long *ino_out, int *len_out)
 {
        DIR *dir = stream;
        struct dirent *ent;

-       seekdir(dir, *pos);
        ent = readdir(dir);
        if(ent == NULL) return(NULL);
        *len_out = strlen(ent->d_name);





2006/8/16, Blaisorblade <blaisorblade@yahoo.it>:
>
> On Monday 14 August 2006 13:07, wang lianwei wrote:
> > Answers:
> >
> > 1. I test it on ext3 filesystem.
> >
> > 2. You are right. Sometimes the pos telldir returned is not the next
> entry
> > offset. just as your point out:
> >       position 0 -> readdir "." -> you call telldir -> it _again_ says
> > you're at 0.
> >
> > 3. I also try to resolve it by remove the seekdir(only use the
> > sequence readdir call), It is also worked.
>
> I can guess what the patch does but please generate with the additional -p
> option so it's easier to read (otherwise the recipient _has_ to read the
> source). Compliments for the solution, it's really simpler and better than
> my
> proposal.
>
> Also, you have to regenerate it anyway for another (tiny) reason: it is
> not
> always safe to use "seekdir" in kernelspace code, you need (sadly) a new
> wrapper in hostfs_user.c and to call it instead of seekdir in
> hostfs_kern.c;
> almost surely you'll get an "undeclared function" warning but
> miscompilations
> in some rare cases are also possible, say for stat(): it is defined almost
> only as a static inline function, or for some structures having different
> definitions in userspace and kernelspace.
>
> > --- linux-2.6.x/fs/hostfs/hostfs_kern.c 2006-04-12 15:12:51.000000000+0800
> > +++ linux-2.6.10-src/fs/hostfs/hostfs_kern.c      2006-08-14 18:25:
> > 13.772237744 +0800
> > @@ -338,6 +338,7 @@
> >         kfree(name);
> >         if(dir == NULL) return(-error);
> >         next = file->f_pos;
> > +       seekdir(dir,next);
> >         while((name = read_dir(dir, &next, &ino, &len)) != NULL){
> >                 error = (*filldir)(ent, name, len, file->f_pos,
> >                                    ino, DT_UNKNOWN);
> >
> >
> > --- linux-2.6.x/fs/hostfs/hostfs_user.c 2006-04-12 15:12:52.000000000+0800
> > +++ linux-2.6.10-src/fs/hostfs/hostfs_user.c      2006-08-14 18:36:
> > 24.276305632 +0800
> > @@ -122,7 +122,6 @@
> >         DIR *dir = stream;
> >         struct dirent *ent;
> >
> > -       seekdir(dir, *pos);
> >         ent = readdir(dir);
> >         if(ent == NULL) return(NULL);
> >         *len_out = strlen(ent->d_name);
>
> --
> Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
> Paolo Giarrusso, aka Blaisorblade
> http://www.user-mode-linux.org/~blaisorblade
> Chiacchiera con i tuoi amici in tempo reale!
> http://it.yahoo.com/mail_it/foot/*http://it.messenger.yahoo.com
>
>

[-- Attachment #1.2: Type: text/html, Size: 7299 bytes --]

[-- Attachment #2: Type: text/plain, Size: 373 bytes --]

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

[-- Attachment #3: Type: text/plain, Size: 194 bytes --]

_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel