[PATCH] fully preemptible kernel

[PATCH] fully preemptible kernel

[Robert Love]

Updated preempt-kernel patches for 2.4.16 and 2.4.17-pre6 are now
available at:

 	ftp://ftp.kernel.org/pub/linux/kernel/people/rml/preempt-kernel

patches for various previous kernels are there as well, but not in sync
with this release.

This patch enables a fully preemptible linux kernel -- userspace
processes are preemptible by higher priority tasks, even if running in
kernel space.  Nice gains in response _and_ throughput are observed.

The main change in this release is support for the SH architecture. 
i386 and ARM are also supported.

Testing and subsequent feedback is welcome.  Enjoy,

	Robert Love

Re: [PATCH] fully preemptible kernel

[Robert Love]

On Mon, 2001-12-10 at 05:54, vda wrote:

> I reported a problem with preemptible 2.4.13 and Samba server (oops, problems 
> with creation of files from win clients).
> Is this issue addressed?

No, because I could not reproduce it.  Could you see if it occurs on the
current kernel with the current patch?  If so, send me the relevant
information.

	Robert Love

Re: [PATCH] fully preemptible kernel

[vda]

On Sunday 09 December 2001 18:41, Robert Love wrote:
> Updated preempt-kernel patches for 2.4.16 and 2.4.17-pre6 are now
> available at:
>
>  	ftp://ftp.kernel.org/pub/linux/kernel/people/rml/preempt-kernel
>
> patches for various previous kernels are there as well, but not in sync
> with this release.
>
> This patch enables a fully preemptible linux kernel -- userspace
> processes are preemptible by higher priority tasks, even if running in
> kernel space.  Nice gains in response _and_ throughput are observed.
>
> The main change in this release is support for the SH architecture.
> i386 and ARM are also supported.

I reported a problem with preemptible 2.4.13 and Samba server (oops, problems 
with creation of files from win clients).
Is this issue addressed?
--
vda

Re: [PATCH] fully preemptible kernel

[vda]

On Monday 10 December 2001 05:03, Robert Love wrote:
> On Mon, 2001-12-10 at 05:54, vda wrote:
> > I reported a problem with preemptible 2.4.13 and Samba server (oops,
> > problems with creation of files from win clients).
> > Is this issue addressed?
>
> No, because I could not reproduce it.  Could you see if it occurs on the
> current kernel with the current patch?  If so, send me the relevant
> information.
>
> 	Robert Love

This is my report dating back to 2.4.10.
Since getting oops is not easy (actually, oops is rare, but buggy behavios 
described below is consistent) I don't have newer oops.
I have ksymoops compiled from sources and it does not work right
("Error (Oops_bfd_perror): set_section_contents Section has no contents").
Can you enlighten me why I'm getting ths message?
Or just send me working ksymoops binary, we can sort out this later.

Ok, I'll compile and install 2.4.17-pre6-pr now.

--------------------------------------------------------------------------
On first attempt to create/copy files from win box to linux samba server it 
says that "file/dir already exists" or that "connection terminated". File/dir 
is indeed exists, in case of a file it is of zero length. Subsequent copy with
overwrite succeeds.

It seems that smbd dies after it has created file/dir but before
reporting this fact to the client. New smbd gets spawned by inetd
then and it see that file/dir is already there.

It might be related to newer kernel.
I am willing to track it further.

Samba: 2.2.1a
Kernel: 2.4.10

Also I've got two oopses so far. One of them decoded below
(my first ksymoops, I don't know is it done right on not)

--------------------------------------------
ksymoops 2.4.3 on i686 2.4.10.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.10/ (default)
     -m /usr/src/linux/System.map (default)

Warning: You did not tell me where to find symbol information....
<snip>

Unable to handle kernel paging request at virtual address 4008e6ed
*pde = 01100067
Oops: 0003
CPU:    0
EIP:    0010:[<c010655b>]
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010246
eax: 00000000   ebx: 4008e76d   ecx: 4008e6cd   edx: 0000e865
esi: 00000000   edi: 00000128   ebp: 4008e6c9   esp: c1c5bf74
ds: 0018   es: 0018   ss: 0018
Process smbd (pid: 213, stackpage=c1c5b000)
Stack: 5650faf0 c1c5a000 c029afd0 00000128 c01068d0 4008e6cd 4008e76d c1c5bf94
       c02a23b2 55c3c85e ec83e589 4000bcd9 40015f67 4000bcd9 40015f67 c1c5a000
       00000000 00000000 bfffea74 c0107603 00000000 bfffe6a3 00000001 00000000
Call Trace: [<c01068d0>] [<c0107603>]
Code: 89 51 20 31 c0 66 8b 53 0c 81 e2 ff ff 00 00 09 c6 89 51 1c
Error (Oops_bfd_perror): set_section_contents Section has no contents

>>EIP; c010655a <restore_sigcontext+3a/140>   <=====
Trace; c01068d0 <sys_rt_sigreturn+110/180>
Trace; c0107602 <system_call+32/40>

1 warning and 1 error issued.  Results may not be reliable.
--
vda

Re: [PATCH] fully preemptible kernel

[Wilson]

----- Original Message -----
From: "Robert Love" <rml@tech9.net>
To: "vda" <vda@port.imtp.ilyichevsk.odessa.ua>
Cc: <linux-kernel@vger.kernel.org>; <kpreempt-tech@lists.sourceforge.net>
Sent: Monday, December 10, 2001 2:03 AM
Subject: Re: [PATCH] fully preemptible kernel


> On Mon, 2001-12-10 at 05:54, vda wrote:
>
> > I reported a problem with preemptible 2.4.13 and Samba server (oops,
problems
> > with creation of files from win clients).
> > Is this issue addressed?
>
> No, because I could not reproduce it.  Could you see if it occurs on the
> current kernel with the current patch?  If so, send me the relevant
> information.
>
> Robert Love
>

I saw this as well, with 2.4.12-ac3+preempt and Samba 2.2.2.
Errors like this:
  ===============================================================
[2001/10/21 18:53:24, 0] lib/util.c:smb_panic(1055)
  PANIC: internal error
[2001/10/21 18:55:56, 0] smbd/nttrans.c:call_nt_transact_ioctl(1762)
  call_nt_transact_ioctl: Currently not implemented.
[2001/10/21 21:16:05, 0] lib/fault.c:fault_report(40)
  ===============================================================
[2001/10/21 21:16:05, 0] lib/fault.c:fault_report(41)
  INTERNAL ERROR: Signal 11 in pid 7540 (2.2.2)
  Please read the file BUGS.txt in the distribution
[2001/10/21 21:16:05, 0] lib/fault.c:fault_report(43)
  ===============================================================

I'll recompile 2.4.16 with the new preempt patch, and see if this problem
has gone away.

Re: [PATCH] fully preemptible kernel

[vda]

Hi Robert,

I just installed 2.4.17pre7+preempt and it still shows the bug.
No oops yet (oops doesn't show up frequently).
2.4.17pre7 without preempt is being compiled right now.

> > > I reported a problem with preemptible 2.4.13 and Samba server (oops,
> > > problems with creation of files from win clients).
> > > Is this issue addressed?
> >
> > No, because I could not reproduce it.  Could you see if it occurs on the
> > current kernel with the current patch?  If so, send me the relevant
> > information.
...
...
> I have ksymoops compiled from sources and it does not work right
> ("Error (Oops_bfd_perror): set_section_contents Section has no contents").
> Can you enlighten me why I'm getting ths message?
> Or just send me working ksymoops binary, we can sort out this later.

In preparation for oops I need ksymoops binary. Mine is not ok:

> Unable to handle kernel paging request at virtual address 4008e6ed
> *pde = 01100067
> Oops: 0003
> CPU:    0
> EIP:    0010:[<c010655b>]
> Using defaults from ksymoops -t elf32-i386 -a i386
> EFLAGS: 00010246
> eax: 00000000   ebx: 4008e76d   ecx: 4008e6cd   edx: 0000e865
> esi: 00000000   edi: 00000128   ebp: 4008e6c9   esp: c1c5bf74
> ds: 0018   es: 0018   ss: 0018
> Process smbd (pid: 213, stackpage=c1c5b000)
> Stack: 5650faf0 c1c5a000 c029afd0 00000128 c01068d0 4008e6cd 4008e76d
> c1c5bf94 c02a23b2 55c3c85e ec83e589 4000bcd9 40015f67 4000bcd9 40015f67
> c1c5a000 00000000 00000000 bfffea74 c0107603 00000000 bfffe6a3 00000001
> 00000000 Call Trace: [<c01068d0>] [<c0107603>]
> Code: 89 51 20 31 c0 66 8b 53 0c 81 e2 ff ff 00 00 09 c6 89 51 1c
> Error (Oops_bfd_perror): set_section_contents Section has no contents
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Dunno is it a result of miscompiled ksymoops or what...
--
vda

Re: [PATCH] fully preemptible kernel

[vda]

> I just installed 2.4.17pre7+preempt and it still shows the bug.
> No oops yet (oops doesn't show up frequently).
> 2.4.17pre7 without preempt is being compiled right now.

2.4.17pre7 without preempt runs fine (no bug).

Was my description of bug detailed enough? What additional info do you need?
--
vda

Re: [PATCH] fully preemptible kernel

[Steve Bergman]

On Sun, 2001-12-09 at 14:41, Robert Love wrote:
> Updated preempt-kernel patches for 2.4.16 and 2.4.17-pre6 are now
> available at:
> 
>  	ftp://ftp.kernel.org/pub/linux/kernel/people/rml/preempt-kernel
> 

Hi,

I just compiled 2.4.17 with the patch from your site that looks to be
for 2.4.17-final.  Unfortunately, several modules (e.g. unix.o) fail on
load with an undefined symbol error (preempt_schedule).  FWIW, I also
have the patch that started the recent "Make highly niced processes run
only when idle" thread.  Which reminds me, I'm anxious to try out your
"fixed" version of SCHED_IDLE when it's ready. ;-) 

Re: [PATCH] fully preemptible kernel

[Robert Love]

On Wed, 2001-12-26 at 15:02, Steve Bergman wrote:

> I just compiled 2.4.17 with the patch from your site that looks to be
> for 2.4.17-final.  Unfortunately, several modules (e.g. unix.o) fail on
> load with an undefined symbol error (preempt_schedule).  FWIW, I also
> have the patch that started the recent "Make highly niced processes run
> only when idle" thread.  Which reminds me, I'm anxious to try out your
> "fixed" version of SCHED_IDLE when it's ready. ;-) 

If there is no compile error, it is probably mismatched modules, because
preempt_schedule is properly exported.

Make sure your kernel tree is fully patched and CONFIG_PREEMPT is
enabled.  Rerun `make dep' and then recompile (including your modules). 
Make sure your modules are properly installed.

Either your running kernel doesn't have preemption enabled or your
kernel does but your modules don't (i.e. they are old modules).

	Robert Love

Re: [PATCH] fully preemptible kernel

[Heinz Diehl]

On Wed Dec 26 2001, Steve Bergman wrote:

[Preempt-kernel patch for 2.4.17 final]
> I just compiled 2.4.17 with the patch from your site that looks to be
> for 2.4.17-final.  Unfortunately, several modules (e.g. unix.o) fail on
> load with an undefined symbol error (preempt_schedule).

No problems here. Are you shure you built the kernel modules and they were
installed properly?

-- 
# Heinz Diehl, 68259 Mannheim, Germany

Re: [PATCH] fully preemptible kernel

[ncw]

In linux-kernel, Heinz Diehl <hd@cavy.de> wrote:
>  On Wed Dec 26 2001, Steve Bergman wrote:
>  
>  [Preempt-kernel patch for 2.4.17 final]
> > I just compiled 2.4.17 with the patch from your site that looks to be
> > for 2.4.17-final.  Unfortunately, several modules (e.g. unix.o) fail on
> > load with an undefined symbol error (preempt_schedule).
>  
>  No problems here. Are you shure you built the kernel modules and they were
>  installed properly?

It might be because you didn't do a 'make mrproper' on a kernel tree
you've used before for a non-preempt build.  As I understand it the
preempt patch effectively makes your kernel SMP - this changes the
architecture of the kernel so you need a 'make mrproper'.

make mrproper deletes your .config file so save that first if you
wanted it ;-)

Also you need a seperate modules directory - the preempt modules
aren't compatible with non-preempt.  Edit the top level makefile and
put a suffix in.

BTW The preempt patch is working fine here too!
-- 
Nick Craig-Wood
ncw@axis.demon.co.uk