linux-um archives
 help / color / mirror / Atom feed
* [uml-devel] Host panic triggered by UML
@ 2003-11-20 18:15 BlaisorBlade
  2003-11-20 18:56 ` Frank Dekervel
  2003-11-20 20:06 ` [uml-devel] Host panic triggered by UML Matt Zimmerman
  0 siblings, 2 replies; 12+ messages in thread
From: BlaisorBlade @ 2003-11-20 18:15 UTC (permalink / raw)
  To: user-mode-linux-devel

I reported that if I leave an UML sleeping, it segfaults(once I saw the killed 
by SIGSEGV from the shell, the other time nothing, maybe I had detached the 
job from the shell).

Now I've seen that what happens is that the host kernel oopses. I'm posting it 
here to collect a bit more infos before posting it to the LKML, and to know 
why only UML is able to trigger it. Since the bug seems related only to 
/dev/shm, maybe you could have some enlightenment about it and how UML uses 
shared memory to make this happen.

Nov 19 04:40:24 blaisorblade kernel: Unable to handle kernel NULL pointer 
dereference at virtual address 00000000
Nov 19 04:40:24 blaisorblade kernel:  printing eip:
Nov 19 04:40:24 blaisorblade kernel: c015421b
Nov 19 04:40:24 blaisorblade kernel: *pde = 00000000
Nov 19 04:40:24 blaisorblade kernel: Oops: 0002
Nov 19 04:40:24 blaisorblade kernel: CPU:    0
Nov 19 04:40:24 blaisorblade kernel: EIP:    0010:[dcache_dir_lseek+299/384]    
Tainted: PF
Nov 19 04:40:24 blaisorblade kernel: EIP:    0010:[<c015421b>]    Tainted: PF
Nov 19 04:40:24 blaisorblade kernel: EFLAGS: 00010286
Nov 19 04:40:24 blaisorblade kernel: eax: 00000000   ebx: c0d24160   ecx: 
d6b15f08   edx: c0d24160
Nov 19 04:40:24 blaisorblade kernel: esi: 00000002   edi: 00000000   ebp: 
c0d24140   esp: d67cbf84
Nov 19 04:40:24 blaisorblade kernel: ds: 0018   es: 0018   ss: 0018
Nov 19 04:40:24 blaisorblade kernel: Process linux (pid: 18847, 
stackpage=d67cb000)
Nov 19 04:40:24 blaisorblade kernel: Stack: fffffff2 c0d24140 00000000 
00000000 c01540f0 d2473f20 c01433f8 d2473f20 
Nov 19 04:40:24 blaisorblade kernel:        00000002 00000000 00000000 
d67ca000 4014b008 00000002 a2533abc c01075b7 
Nov 19 04:40:24 blaisorblade kernel:        0000002f 00000002 00000000 
4014b008 00000002 a2533abc 00000013 0000002b 
Nov 19 04:40:24 blaisorblade kernel: Call Trace:    [dcache_dir_lseek+0/384] 
[sys_lseek+120/192] [system_call+51/56]
Nov 19 04:40:24 blaisorblade kernel: Call Trace:    [<c01540f0>] [<c01433f8>] 
[<c01075b7>]
Nov 19 04:40:24 blaisorblade kernel: 
Nov 19 04:40:24 blaisorblade kernel: Code: 89 10 89 42 04 b8 00 e0 ff ff 21 e0 
ff 48 04 8b 50 14 39 50 

(I've had another oops in the same call path and point, so I guess this was 
the reason even the other time).

This means that it crashes in fs/readdir.c:dcache_dir_lseek; after some checks 
on the disassembled code, I could see that it happens here, about line 80:

                         list_del(&cursor->d_child);
                         list_add_tail(&cursor->d_child, p);

and exactly, on list_add_tail: the offending EIP means p->prev->next = 
cursor->d_child, but it fails when trying to dereference p->prev, which is 
NULL. This means that very likely, the bug happens when the list p points 
into(d_subdirs) was built.

The dumped code is correct, as long as I see, so the kernel text section 
hasn't been corrupted.

When this oops happens, a semaphore is held, and so I guess it never gets 
released(I don't think the oops handler would be able to do anything or does 
anything).
In fact I've had, once, an astonishing number of processes in D state: I could 
guess that they were all trying to access /dev/shm. I don't remember if any 
UML had crashed, but I think it's very likely. Anyhow, that time I found a 
UML and a process(msec_find from Mandrake) which was walking on the whole 
filesystem. I could start any find / and it always stopped and went into D 
state a bit before going to /dev/shm.

So, probably the oops happened while working on that folder: what needs to be 
checked is if the tmpfs code creates an inconsistent list(exactly, an 
inconsistent struct dentry.d_subdirs one, since p walks into it). Do you 
think this is reasonable or not?

-- 
cat <<EOSIGN
Paolo Giarrusso, aka Blaisorblade
Linux Kernel 2.4.21/2.6.0-test on an i686; Linux registered user n. 292729
EOSIGN



-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?  SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [uml-devel] Host panic when UML reads host /dev/shm
@ 2003-11-23 23:08 James W McMechan
  0 siblings, 0 replies; 12+ messages in thread
From: James W McMechan @ 2003-11-23 23:08 UTC (permalink / raw)
  To: user-mode-linux-devel

I have been tracing down the Oops on a vanilla 2.4.22 kernel
This appears to be a Oops with a semaphore held that will
lock out all other accesses to the tmpfs/shmfs filesystem.

I traced it to read_dir in hostfs_user.c by setting
breakpoints on all the hostfs_user.c functions.

Ok I now have a simple test program which Oops
the host kernel instantly (3 times through loop)

This is very strange my attached test program when run
as a unprivileged user Oops the kernel and locks /dev/shm
in general a unprivileged user should not be able to do this

It has taken a while in order to check that the kernel I was
running was without patches (i.e. no skas)
It still Oops on the third loop :(

I have also noted that the tmpfs/shmfs does not have a set
of files like other filesystems under fs/ it seems to be located
in mm/shmem.c instead... very strange

Oh well it is a problem with the vanilla 2.4.22 kernel so
more testing and off to the LKML...

Anybody have a better guess where to send this?

For your Enjoyment the test program

/* by James_McMechan at hotmail com */
/* test program to Oops shmfs usually mounted at /dev/shm */
/* yes it is dumb but unprivileged users should not be able */
/* to Oops the kernel regardless of how dumb the program */
#include <sys/types.h>                                                   
      
#include <dirent.h>
#include <stdio.h>

main()
{
        DIR *dir;
        struct dirent *ent;
        off_t pos = 0;

        do {
                dir = opendir("/dev/shm");
                seekdir(dir, pos);
                ent = readdir(dir);
                if (ent == 0) {
                        printf("end of directory\n");
                        perror("readdir ended with");
                } else {
                        printf("d_name is %s\n",ent->d_name);
                        pos = telldir(dir);
                        if (pos < 0)
                                perror("telldir failed with");
                }
                closedir(dir);
        } while (ent != 0);
}

________________________________________________________________
The best thing to hit the internet in years - Juno SpeedBand!
Surf the web up to FIVE TIMES FASTER!
Only $14.95/ month - visit www.juno.com to sign up today!


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?  SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread
* Re: [uml-devel] Host panic when UML reads host /dev/shm
@ 2003-11-25  3:02 James W McMechan
  0 siblings, 0 replies; 12+ messages in thread
From: James W McMechan @ 2003-11-25  3:02 UTC (permalink / raw)
  To: blaisorblade_spam; +Cc: user-mode-linux-devel

> > I have been tracing down the Oops on a vanilla 2.4.22 kernel
> > This appears to be a Oops with a semaphore held that will
> > lock out all other accesses to the tmpfs/shmfs filesystem.
> In fact, I wrote the exact point of the Oops in the host kernel in 
> the first 
> message... it's not clear if you missed this.

No I did not miss it I was working on what tasks were occurring
before the Oops, I used it to generate the test program.

> > I traced it to read_dir in hostfs_user.c by setting
> > breakpoints on all the hostfs_user.c functions.
> Yes, I've get to that point, too.
> > This is very strange my attached test program when run
> > as a unprivileged user Oops the kernel and locks /dev/shm
> > in general a unprivileged user should not be able to do this
> Yes, but it's a kernel bug: provided the kernel doesn't oops, 
> checking for priviledges with semaphore isn't needed. And 
> if someone wants to implement this, he will receive as answer
> "fix the oops and post that. No overhead, please". And 
> actually implementing the fix wouldn't be easy(how do you
> steal a semaphore that a priviledged process holds?)...
> 
> Maybe, releasing semaphores on Oops could be useful. IIRC, the code 
> already releases spinlocks(but not sure at all). But when oopsing, it's
not 
> easy to do anything, since we have already problems.

Ah, no I was not trying to complain about the semaphore
I think the Oops needs fixing also.
It is not that it needs a permission check, but rather that any commands
issued by a unprivileged user (no matter how dumb) should not Oops
the kernel.

> > It has taken a while in order to check that the kernel I was
> > running was without patches (i.e. no skas)
> So, it happens even without SKAS mode.
> The other possible issue(i.e. something unusual which happens only 
> with UML) is with threading. If I have time, I'll check whether a
> program creating threads on the host the way UML does can
> obtain the same Oops. At that point, LKML won't be able to
> ignore it. I mean just a userspace program to run on the host.

You don't need to bother that is what my test program does it will
Oops the host kernel without requiring any UML to be present.
That is why I checked with a vanilla kernel rather then my
normal skas patched version.

> > I have also noted that the tmpfs/shmfs does not have a set
> > of files like other filesystems under fs/ it seems to be located
> > in mm/shmem.c instead... very strange
> Actually, it's based on fs/ramfs... which has only one file because 
> nothing is saved by the FS. Normally dentries(an object which
> represents anything contained inside a dir, a "directory entry")
> are just used to cache access to the FS; for ramfs and tmpfs
> (or better, for all ram-based fs, even the 2.6 sysfs), they are 
> locked in memory and never unloaded.

Yes, I understand mostly how it works, but the layout of the
shmem_* inode ops in mm/shmem.c was confusing I was expecting
TMPFS to be under fs/tmpfs or fs/ramfs, I had to grep the whole
tree to find the CONFIG_TMPFS ifdefs

> > Oh well it is a problem with the vanilla 2.4.22 kernel so
> > more testing and off to the LKML...
> 
> > Anybody have a better guess where to send this?
> Maybe there is a linux-mm mailing list... check Maintainers file. 
> But before LKML we probably need to write a simple test
> program... like the threaded one 
> I said above(if threading is the issue).

Threading is not a issue the test program I attached to yesterdays
e-mail can be run on the host system to Oops the host kernel
without requiring UML and can be run inside of UML to Oops
the UML kernel with all the nice debugging features, while letting
the host system keep running without any Oopses.

It is much simpler to use than using UML to Oops the host kernel

The biggest problem I have been having is all of the inlines hiding
where exactly it is going wrong, it appears that the
dcache_readdir blows up when a invalid ->prev pointer is in
one of the lists it may relate to the list_del(q); list_add(q,
&dentry->d_subdirs); on a empty list that I think is occurring

MAINTAINERS, mm/shmem.c don't seem to list contact
addresses for tmpfs/shmfs :(

here is a even shorter version of the test program
it blows up on the first real entry
The first version is easier to understand what
UML is doing though.

/* by James_McMechan at hotmail com */                                   
      
/* test2 program to Oops shmfs mounted at /dev/shm */
/* yes it is dumb but unprivileged users should not be able */
/* to Oops the kernel regardless of how dumb the program */
#include <sys/types.h>
#include <dirent.h>
main()
{/* off 0 is "." off 1 is ".." off 2 is empty */
seekdir(opendir("/dev/shm"), (off_t) 2);
}

________________________________________________________________
The best thing to hit the internet in years - Juno SpeedBand!
Surf the web up to FIVE TIMES FASTER!
Only $14.95/ month - visit www.juno.com to sign up today!


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
Does SourceForge.net help you be more productive?  Does it
help you create better code?  SHARE THE LOVE, and help us help
YOU!  Click Here: http://sourceforge.net/donate/
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread
end of thread, other threads:[~2003-11-25  3:09 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-11-20 18:15 [uml-devel] Host panic triggered by UML BlaisorBlade
2003-11-20 18:56 ` Frank Dekervel
2003-11-20 19:37   ` [uml-devel] Host panic when UML reads host /dev/shm BlaisorBlade
2003-11-20 22:32     ` Frank Dekervel
2003-11-20 22:47       ` Frank Dekervel
2003-11-22 18:11         ` BlaisorBlade
2003-11-24  3:36           ` Frank Dekervel
2003-11-21  2:22     ` Jeff Dike
2003-11-22 18:12       ` BlaisorBlade
2003-11-20 20:06 ` [uml-devel] Host panic triggered by UML Matt Zimmerman
  -- strict thread matches above, loose matches on Subject: below --
2003-11-23 23:08 [uml-devel] Host panic when UML reads host /dev/shm James W McMechan
2003-11-25  3:02 James W McMechan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox