Re: 3rd version of R/W mmap_sem patch available

Re: 3rd version of R/W mmap_sem patch available

[Manfred Spraul]

>
> Besides, the fair semaphores would potentially slow things down, while
> this potentially speeds things up. So.. It looks obvious enough.
>

Rik, did you check that {pte,pmd}_alloc are thread safe? At least in
2.4.2 they aren't (include/asm-i386/pgalloc.h), and your patch doesn't
touch pgalloc.

{pte,pmd}_alloc are called from handle_mm_fault, and that function is
now running with down_read().
--

    Manfred

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

On Tue, 20 Mar 2001, Manfred Spraul wrote:
>
> Rik, did you check that {pte,pmd}_alloc are thread safe? At least in
> 2.4.2 they aren't (include/asm-i386/pgalloc.h), and your patch doesn't
> touch pgalloc.

Excellent point. We used to do all the looping and re-trying, but it got
ripped out a long time ago (and in any case, it historically didn't do
SMP, so the old code doesn't really work).

		Linus

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

On Mon, 19 Mar 2001, Linus Torvalds wrote:
>
> Excellent point. We used to do all the looping and re-trying, but it got
> ripped out a long time ago (and in any case, it historically didn't do
> SMP, so the old code doesn't really work).

Actually, funnily enough, I see that the old thread-safe stuff is still
there in get_pte_kernel_slow(). The only thing that breaks it is that we
don't hold any locks, so it's only UP-safe, not SMP-safe.

However, it definitely looks like we should just un-inline that thing
completely, and make a lot of it architecture-independent anyway.

		Linus

Re: 3rd version of R/W mmap_sem patch available

[Rik van Riel]

On Mon, 19 Mar 2001, Linus Torvalds wrote:
> On Mon, 19 Mar 2001, Linus Torvalds wrote:
> >
> > Excellent point. We used to do all the looping and re-trying, but it got
> > ripped out a long time ago (and in any case, it historically didn't do
> > SMP, so the old code doesn't really work).
>
> Actually, funnily enough, I see that the old thread-safe stuff is still
> there in get_pte_kernel_slow(). The only thing that breaks it is that we
> don't hold any locks, so it's only UP-safe, not SMP-safe.
>
> However, it definitely looks like we should just un-inline that thing
> completely, and make a lot of it architecture-independent anyway.

Also, because lots of architectures seem to have exactly
the same code, we might as well remove the duplicates and
put them in the same place...

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: 3rd version of R/W mmap_sem patch available

[Rik van Riel]

On Tue, 20 Mar 2001, Manfred Spraul wrote:
> >
> > Besides, the fair semaphores would potentially slow things down, while
> > this potentially speeds things up. So.. It looks obvious enough.
>
> Rik, did you check that {pte,pmd}_alloc are thread safe? At
> least in 2.4.2 they aren't (include/asm-i386/pgalloc.h), and
> your patch doesn't touch pgalloc.

I checked and they're not. This still needs to be fixed...

(ie the patch really isn't ready yet to be included in the
main kernel ... OTOH, the changes needed to make it ready
are all trivial and tedious ;))

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

On Tue, 20 Mar 2001, Rik van Riel wrote:
>
> (ie the patch really isn't ready yet to be included in the
> main kernel ... OTOH, the changes needed to make it ready
> are all trivial and tedious ;))

They are trivial and tedious only if done wrong - which will also add tons
of new places where we lock and unlock only to lock again.

My -pre5 has the non-trivial "fix the calling convention and require that
pmd/pgd_alloc() be called with the lock held" version of the patch.

		Linus

3rd version of R/W mmap_sem patch available

[Rik van Riel]

On Mon, 19 Mar 2001, Mike Galbraith wrote:

> @@ -1135,6 +1170,7 @@
	[large patch]

I've been finding small bugs in both my late-night code and in
Mike's code and have redone the changes in do_anonymous_page(),
do_no_page() and do_swap_page() much more carefully...

Now the code is beautiful and it might even be bugfree ;)

If you feel particularly adventurous, please help me test the
patch; it is available from:

	http://www.surriel.com/patches/2.4/2.4.2-ac20-rwmmap_sem3

regards,

Rik
--
Linux MM bugzilla: http://linux-mm.org/bugzilla.shtml

Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

> Now the code is beautiful and it might even be bugfree ;)

I'm applying this to my tree - I'm not exactly comfortable with this
during the 2.4.x timeframe, but at the same time I'm even less comfortable
with the current alternative, which is to make the regular semaphores
fairer (we tried it once, and the implementation had problems, I'm not
going to try that again during 2.4.x).

Besides, the fair semaphores would potentially slow things down, while
this potentially speeds things up. So.. It looks obvious enough.

		Linus

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

There is a 2.4.3-pre5 in the test-directory on ftp.kernel.org.

The complete changelog is appended, but the biggest recent change is the
mmap_sem change, which I updated with new locking rules for pte/pmd_alloc
to avoid the race on the actual page table build.

This has only been tested on i386 without PAE, and is known to break other
architectures. Ingo, mind checking what PAE needs? Generally, the changes
are simple, and really only implies changing the pte/pmd allocation
functions to _only_ allocate (ie removing the stuff that actually modifies
the page tables, as that is now handled by generic code), and to make sure
that the "pgd/pmd_populate()" functions do the right thing.

I have also removed the xxx_kernel() functions - for architectures that
need them, I suspect that the right approach is to just make the
"populate" funtions notice when "mm" is "init_mm", the kernel context.
That removed a lot of duplicate code that had little good reason.

This pre-release is meant mainly as a synchronization point for mm
developers, not for generic use.

	Thanks,

		Linus


-----
-pre5:
  - Rik van Riel and others: mm rw-semaphore (ps/top ok when swapping)
  - IDE: 256 sectors at a time is legal, but apparently confuses some
    drives. Max out at 255 sectors instead.
  - Petko Manolov: USB pegasus driver update
  - make the boottime memory map printout at least almost readable.
  - USB driver updates
  - pte_alloc()/pmd_alloc() need page_table_lock.

-pre4:
  - Petr Vandrovec, Al Viro: dentry revalidation fixes
  - Stephen Tweedie / Manfred Spraul: kswapd and ptrace race
  - Neil Brown: nfsd/rpc/raid cleanups and fixes

-pre3:
  - Alan Cox: continued merging
  - Urban Widmark: smbfs fix (d_add on already hashed dentry - no-no).
  - Andrew Morton: 3c59x update
  - Jeff Garzik: network driver cleanups and fixes
  - Gérard Roudier: sym-ncr drivers update
  - Jens Axboe: more loop cleanups and fixes
  - David Miller: sparc update, some networking fixes

-pre2:
  - Jens Axboe: fix loop device deadlocks
  - Greg KH: USB updates
  - Alan Cox: continued merging
  - Tim Waugh: parport and documentation updates
  - Cort Dougan: PowerPC merge
  - Jeff Garzik: network driver updates
  - Justin Gibbs: new and much improved aic7xxx driver 6.1.5

-pre1:
  - Chris Mason: reiserfs, another null bytes bug
  - Andrea Arkangeli: make SMP Athlon build
  - Alexander Zarochentcev: reiserfs directory fsync SMP locking fix
  - Jeff Garzik: PCI network driver updates
  - Alan Cox: continue merging
  - Ingo Molnar: fix RAID AUTORUN ioctl, scheduling improvements

Re: 3rd version of R/W mmap_sem patch available

[Marcelo Tosatti]

On Mon, 19 Mar 2001, Linus Torvalds wrote:

> 
> There is a 2.4.3-pre5 in the test-directory on ftp.kernel.org.
> 
> The complete changelog is appended, but the biggest recent change is the
> mmap_sem change, which I updated with new locking rules for pte/pmd_alloc
> to avoid the race on the actual page table build.
> 
> This has only been tested on i386 without PAE, and is known to break other
> architectures. Ingo, mind checking what PAE needs? Generally, the changes
> are simple, and really only implies changing the pte/pmd allocation
> functions to _only_ allocate (ie removing the stuff that actually modifies
> the page tables, as that is now handled by generic code), and to make sure
> that the "pgd/pmd_populate()" functions do the right thing.
> 
> I have also removed the xxx_kernel() functions - for architectures that
> need them, I suspect that the right approach is to just make the
> "populate" funtions notice when "mm" is "init_mm", the kernel context.
> That removed a lot of duplicate code that had little good reason.
> 
> This pre-release is meant mainly as a synchronization point for mm
> developers, not for generic use.
> 
> 	Thanks,
> 
> 		Linus
> 
> 
> -----
> -pre5:
>   - Rik van Riel and others: mm rw-semaphore (ps/top ok when swapping)
>   - IDE: 256 sectors at a time is legal, but apparently confuses some
>     drives. Max out at 255 sectors instead.

Could the IDE one cause corruption ?

EXT2-fs error (device ide0(3,1)): ext2_free_blocks: bit already cleared
for block 6211

Just hitted this now with pre3. 

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

On Tue, 20 Mar 2001, Marcelo Tosatti wrote:
>
> Could the IDE one cause corruption ?

Only with broken disks, as far as we know right now. There's been so far
just one report of this problem, and nobody has heard back about which
disk this was.. And it should be noisy about it when it happens -
complaining about lost interrupts and resetting the IDE controller.

So unlikely.

		Linus

Re: 3rd version of R/W mmap_sem patch available

[Marcelo Tosatti]

On Mon, 19 Mar 2001, Linus Torvalds wrote:

> 
> 
> On Tue, 20 Mar 2001, Marcelo Tosatti wrote:
> >
> > Could the IDE one cause corruption ?
> 
> Only with broken disks, as far as we know right now. There's been so far
> just one report of this problem, and nobody has heard back about which
> disk this was.. And it should be noisy about it when it happens -
> complaining about lost interrupts and resetting the IDE controller.
> 
> So unlikely.

Ok, so I think we have a problem. The disk is OK -- no lost interrupts or
resets. Just this message on syslog and pgbench complaining about
corruption of the database.

I'll put pre5 in and try to reproduce the problem (I hitted it while
running pgbench + shmtest). 

Damn. 

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

On Tue, 20 Mar 2001, Marcelo Tosatti wrote:
>
> I'll put pre5 in and try to reproduce the problem (I hitted it while
> running pgbench + shmtest).

I found a case where pre5 will forget to unlock the page_table_lock (in
copy_page_range()), and one place where I had missed the lock altogether
(in ioremap()), so I'll make a pre6 (neither is a problem on UP, though,
so pre5 is not unusable - even on SMP it works really well until you hit
the case where it forgets to unlock ;).

Although I'd prefer to see somebody check out the other architectures, to
do the (pretty trivial) changes to make them support properly threaded
page faults. I'd hate to have two pre-patches without any input from other
architectures..

		Linus

Re: 3rd version of R/W mmap_sem patch available

[Linus Torvalds]

On Tue, 20 Mar 2001, Andrew Morton wrote:
> Linus Torvalds wrote:
> >
> > There is a 2.4.3-pre5 in the test-directory on ftp.kernel.org.
>
> I can't see it.  Where did you hide it?

Ahh. The mirroring is apparently broken. I put my stuff on a faster local
connection to "master.kernel.org", and depend on it being mirrored
automatically. And apparently the mirroring has been down for a few days
now. Ugh. I'll ping Peter.

		Linus