2.4.21pre2aa1

2.4.21pre2aa1

[Andrea Arcangeli]

URL:

	http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.21pre2aa1.gz
	http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.21pre2aa1/

diff between 2.4.20aa1 and 2.4.21pre2aa1:

Only in 2.4.20aa1: 00_extraversion-14
Only in 2.4.21pre2aa1: 00_extraversion-15
Only in 2.4.20aa1: 00_poll-nfds-2
Only in 2.4.21pre2aa1: 00_poll-nfds-3
Only in 2.4.20aa1: 00_readahead-got-broken-somewhere-2
Only in 2.4.21pre2aa1: 00_readahead-got-broken-somewhere-3
Only in 2.4.20aa1: 00_rwsem-fair-33
Only in 2.4.20aa1: 00_rwsem-fair-33-recursive-8
Only in 2.4.21pre2aa1: 00_rwsem-fair-34
Only in 2.4.21pre2aa1: 00_rwsem-fair-34-recursive-8
Only in 2.4.20aa1: 00_sched-O1-aa-2.4.19rc3-5.gz
Only in 2.4.21pre2aa1: 00_sched-O1-aa-2.4.19rc3-7.gz
Only in 2.4.20aa1: 00_sd_softerr-1
Only in 2.4.20aa1: 70_alloc_inode-1
Only in 2.4.21pre2aa1: 70_alloc_inode-2
Only in 2.4.20aa1: 70_delalloc-1
Only in 2.4.21pre2aa1: 70_delalloc-2
Only in 2.4.20aa1: 70_dmapi-stuff-1
Only in 2.4.21pre2aa1: 70_dmapi-stuff-2
Only in 2.4.20aa1: 70_quota-backport-2
Only in 2.4.21pre2aa1: 70_quota-backport-3
Only in 2.4.20aa1: 90_s390-aa-3
Only in 2.4.21pre2aa1: 90_s390-aa-4
Only in 2.4.20aa1: 90_s390x-aa-2
Only in 2.4.21pre2aa1: 90_s390x-aa-3
Only in 2.4.20aa1: 93_NUMAQ-6
Only in 2.4.21pre2aa1: 93_NUMAQ-8
Only in 2.4.20aa1: 9930_io_request_scale-4
Only in 2.4.21pre2aa1: 9930_io_request_scale-5
Only in 2.4.20aa1: 9931_io_request_scale-drivers-1
Only in 2.4.21pre2aa1: 9931_io_request_scale-drivers-2
Only in 2.4.20aa1: 9970_corename-1.gz
Only in 2.4.21pre2aa1: 9970_corename-2.gz

	Rediffed.

Only in 2.4.20aa1: 00_flock-posix-2001-1
Only in 2.4.20aa1: 00_loop-fixes-2.4.19pre7ac2-1
Only in 2.4.20aa1: 00_loop-handling-pages-in-cache-1
Only in 2.4.20aa1: 00_scsi-largelun-2
Only in 2.4.20aa1: 30_01_nfs-call-start-1
Only in 2.4.20aa1: 9960_cyclone-3

	Merged in mainline.

Only in 2.4.21pre2aa1: 00_free_pages-lru-no_irq-1

	Allow aio to release pages queued in the plage replacement lru.

Only in 2.4.21pre2aa1: 00_irda-compile-1

	Fix compilation trouble.

Only in 2.4.21pre2aa1: 00_rbtree-cleanups-1

	Merged rbtree cleanups/microoptimizations from Érsek László after
	verifying their math correctness also with the help of Paolo Carlini
	and of some gentle reminder from Rusty, they are obviously right,
	thanks.

Only in 2.4.20aa1: 00_sd-cleanup-2
Only in 2.4.21pre2aa1: 00_sd-cleanup-3

	Pert of it merged in mainline.

Only in 2.4.21pre2aa1: 00_semop-timeout-1

	Added semop timeout to allow some app to skip a syscall in some fast
	path, same as in 2.5.53.

Only in 2.4.21pre2aa1: 00_small-vma-1

	Reduce the size of a vma to 68bytes, the last private field isn't
	interesting and in turn there shouldn't be that much false sharing on
	this big struct.

Only in 2.4.21pre2aa1: 00_vma-file-merge-1

	Implement (finish) the vma merging for file backed vmas too (all of them).
	This implements total back and forward merging too, not only forward
	merging with a quick hack. this is a showstopper need for some app that
	wouldn't run at all otherwise in some configuration. Only mmap() does
	the total merging at the moment, it could be extended to all other
	vma-related syscalls but this app doesn't care about those other syscalls,
	so I did the minimalistic thing so far for 2.4 (but done right and in the
	most powerful manner, so that it can be extended to other syscalls too
	over time if needed).

Only in 2.4.21pre2aa1: 00_writeoute_one_page-b_flushtime-1

	There's no point to set bflushtime there.

Only in 2.4.20aa1: 10_rawio-vary-io-13
Only in 2.4.21pre2aa1: 10_rawio-vary-io-15

	Fix several (though very minor) varyio bugs.

Only in 2.4.21pre2aa1: 21_sched-o1-yield-rt-1

	Fix RT handling in the o1 scheduler, so keventd will run rt as it
	pretends to according to the code, and fix sched_yield for RT tasks
	so that it stops deadlocking hard the box with some important app,
	when some lower prio RT task can't run despite the higher prio runs
	sched_yield in a loop etc... This also has the chance to make staroffice
	under kernel compile more interactive: we didn't recalculated the effective
	prio always after a sched_yield, so it could be queued again with
	lesser priority at the next timeslice expire. They're all o1 bugs.

Only in 2.4.20aa1: 9900_aio-13.gz
Only in 2.4.21pre2aa1: 9900_aio-14.gz
Only in 2.4.20aa1: 9910_shm-largepage-6.gz
Only in 2.4.21pre2aa1: 9910_shm-largepage-9.gz

	More fixes (i.e. try to boot with bigpages > 4G, now you can, and try
	to swap with plenty of bigpages, now it's solid).

Only in 2.4.20aa1: 9920_kgdb-4.gz
Only in 2.4.21pre2aa1: 9920_kgdb-5.gz

	Dropped the superflous PAGE_OFFSET check in mainline and rely only
	on the code segment value.

Only in 2.4.21pre2aa1: 9985_blk-atomic-aa4

	New design to avoid fractured blocks for the rawio, this works on all
	device drivers and through whatever blkdev mapper (lvm/md/..) as far as
	it's possible to merge the I/O in the biggest possible atomic dma
	operations provided by the hardware (this of course also requires that
	the device remapper has extents aligned with the database blocksize and that
	the hardware supports SG dma operations as large as the blocksize of the
	db, so some care must be necessary in userspace while choosing the hardware,
	i.e. if you stripe in raid0 we can't merge the I/O in a single operation
	if the stripe size is lower than the blocksize etc..).  You can imagine
	this feature for rawio, similar to journaling for a filesystem. this is
	a requirement for some database.

	This obsolete the superbh patch, that was meant to achieve the same thing
	but it was working only for direct scsi devices (no lvm/md), and only
	for a few of them (no mylex,cpqarray etc..). This is also much simpler and
	much safer, since all the merging code is unchanged and the lowlevel device
	drivers doesn't need any change to make this work. This also doesn't invalidate
	the elevator (my first version did that, and I planned to add it later but Jens
	beated me sending after a few minutes a 10 liner incremental that
	refiles stuff in order rather than putting at the end of the queue like
	my first lazy version did ;). btw, Jens implemented the sequence number
	management part too.

Only in 2.4.20aa1: 9981_elevator-lowlatency-2
Only in 2.4.21pre2aa1: 9981_elevator-lowlatency-3

	Rediffed. If you get any sign of low I/O performance (on both read or
	writes) please try to backout this patch (called
	9981_elevator-lowlatency-3) as first thing. I think we should limit
	the number of requests too, but I hadn't time to implement it yet, and
	the current patch is stable and it should work better on lowmem machines
	(and my 50mbyte scsi works as fast as always even with this patch but
	slower cpus than 2.5Ghz xeon might notice the I/O [not cpu :) ]
	pipeline is shorter than in mainline).

Only in 2.4.21pre2aa1: 9990_hack-drop-x86-fast-pte-2

	I can reproduce the mm corruption problem without the above workaround
	applied and dri in the X server enabled, playing gltron for a few seconds and
	restarting the X server a few times in between is enough to get an
	oops, so I applied this band-aid that just reverse the x86-fast-pte,
	that is only a theorical microoptimization (since we allocate the quick
	pgd we could as well use it), the frequency of pgd allocations is small
	enough that it shouldn't matter, so it's not urgent to fix it right,
	but still I didn't delete the x86-fast-pte because I would like to
	debug and understand it completely first. It might be related to the
	vmalloc part of the pgd but I'm not sure yet. It may even be that the
	band-aid is making harder to reproduce a bug that is still there.

	I never noticed this problem before because I rarely use 3d (and usually
	I had mesasoft setup anyways). It's not specific to a certain graphics card,
	so it looks more like an agp generic problem or something, I can
	reproduce myself on my laptop i830 graphics card and i830 agp, on my
	desktop g450 with amd agp, and on my test box on a ati radeon 7500 and
	intel agp, so it doesn't look like a lowlevel driver problem, and it
	only hurts while using the agp and/or drm somehow. Many thanks to
	Srihari Vijayaraghavan who found the offending patch in the whole kit
	originally some time ago.

Andrea

Re: 2.4.21pre2aa1

[Eyal Lebedinsky]

Andrea Arcangeli wrote:
> 
> URL:
> 
>         http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.21pre2aa1.gz

At the package in www.kernel.org says it is a gzip but is really a
bzip2.

FYI

--
Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>

Re: 2.4.21pre2aa1: compile error in fs/buffer.c

[Eyal Lebedinsky]

[-- Attachment #1: Type: text/plain, Size: 207 bytes --]

A declaration at the wrong place was introduced by pre2aa1 in
fs/buffer.c. I simply moved the declaration tot the top of the
relevant block.

--
Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>

[-- Attachment #2: 2.4.21-pre2-aa1-buffer.patch --]
[-- Type: text/plain, Size: 406 bytes --]

--- linux/fs/buffer.c.orig	Thu Dec 26 15:10:26 2002
+++ linux/fs/buffer.c	Thu Dec 26 15:10:51 2002
@@ -2334,8 +2334,8 @@
 				}
 				if (iobuf->varyio &&
 				    (!(offset & RAWIO_BLOCKMASK))) {
-					iosize = RAWIO_BLOCKSIZE; 
 					int block_iter;
+					iosize = RAWIO_BLOCKSIZE; 
 					if (iosize > length)
 						iosize = length;
 					for (block_iter = 1; block_iter < iosize / size; block_iter++) {

Re: 2.4.21pre2aa1: compile error in DAC960.c

[Eyal Lebedinsky]

As is the case in the last few -aa patches, we still have the
problem in drivers/block/DAC960.c.

I suggest that someone who knows fixes it, or else remove it
from the -aa collection.

--
Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>

Re: 2.4.21pre2aa1: compile error in DAC960.c

[Dave Olien]

I have a patch that works at the URL:

	http://www.osdl.org/archive/dmo/DAC960_2.4.20aa1/

I originally developed it for the aa1 patch.  But it works
on the newer aa patches as well.

On Thu, Dec 26, 2002 at 03:33:09PM +1100, Eyal Lebedinsky wrote:
> As is the case in the last few -aa patches, we still have the
> problem in drivers/block/DAC960.c.
> 
> I suggest that someone who knows fixes it, or else remove it
> from the -aa collection.
> 
> --
> Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: 2.4.21pre2aa1: compile error in DAC960.c

[Andrea Arcangeli]

On Mon, Dec 30, 2002 at 04:49:11PM -0800, Dave Olien wrote:
> 
> I have a patch that works at the URL:
> 
> 	http://www.osdl.org/archive/dmo/DAC960_2.4.20aa1/
> 
> I originally developed it for the aa1 patch.  But it works
> on the newer aa patches as well.

Thanks for the help, obviously merged ;)

Andrea

Re: 2.4.21pre2aa1

[J.A. Magallón]

On 2002.12.26 Andrea Arcangeli wrote:
[...]
> 
> Only in 2.4.21pre2aa1: 9990_hack-drop-x86-fast-pte-2
> 
[...]
> 	I never noticed this problem before because I rarely use 3d (and usually
> 	I had mesasoft setup anyways). It's not specific to a certain graphics card,
> 	so it looks more like an agp generic problem or something, I can
> 	reproduce myself on my laptop i830 graphics card and i830 agp, on my
> 	desktop g450 with amd agp, and on my test box on a ati radeon 7500 and
> 	intel agp, so it doesn't look like a lowlevel driver problem, and it
> 	only hurts while using the agp and/or drm somehow. Many thanks to
> 	Srihari Vijayaraghavan who found the offending patch in the whole kit
> 	originally some time ago.
> 

I saw it also using nVidia drivers, that do not touch drm. So I would vote for
agpgart.

-- 
J.A. Magallon <jamagallon@able.es>      \                 Software is like sex:
werewolf.able.es                         \           It's better when it's free
Mandrake Linux release 9.1 (Cooker) for i586
Linux 2.4.21-pre2-jam1 (gcc 3.2.1 (Mandrake Linux 9.1 3.2.1-2mdk))

Re: 2.4.21pre2aa1

[Marc-Christian Petersen]

[-- Attachment #1: Type: text/plain, Size: 828 bytes --]

On Thursday 26 December 2002 16:13, J.A. Magallón wrote:

Hi J.A.

> > 	I never noticed this problem before because I rarely use 3d (and usually
> > 	I had mesasoft setup anyways). It's not specific to a certain graphics
> > card, so it looks more like an agp generic problem or something, I can
> > reproduce myself on my laptop i830 graphics card and i830 agp, on my
> > desktop g450 with amd agp, and on my test box on a ati radeon 7500 and
> > intel agp, so it doesn't look like a lowlevel driver problem, and it only
> > hurts while using the agp and/or drm somehow. Many thanks to Srihari
> > Vijayaraghavan who found the offending patch in the whole kit originally
> > some time ago.
> I saw it also using nVidia drivers, that do not touch drm. So I would vote
> for agpgart.
try this please.

ciao, Marc

[-- Attachment #2: 281_use-after-free-mremap-fix.patch --]
[-- Type: text/x-diff, Size: 3155 bytes --]

 mm/mremap.c |   31 +++++++++++++++++++++++--------
 1 files changed, 23 insertions(+), 8 deletions(-)

--- 24/mm/mremap.c~move_vma-use-after-free	Thu Dec 19 01:29:52 2002
+++ 24-akpm/mm/mremap.c	Thu Dec 19 01:31:43 2002
@@ -134,14 +134,16 @@ static inline unsigned long move_vma(str
 	next = find_vma_prev(mm, new_addr, &prev);
 	if (next) {
 		if (prev && prev->vm_end == new_addr &&
-		    can_vma_merge(prev, vma->vm_flags) && !vma->vm_file && !(vma->vm_flags & VM_SHARED)) {
+				can_vma_merge(prev, vma->vm_flags) &&
+				!(vma->vm_flags & VM_SHARED)) {
 			spin_lock(&mm->page_table_lock);
 			prev->vm_end = new_addr + new_len;
 			spin_unlock(&mm->page_table_lock);
 			new_vma = prev;
 			if (next != prev->vm_next)
 				BUG();
-			if (prev->vm_end == next->vm_start && can_vma_merge(next, prev->vm_flags)) {
+			if (prev->vm_end == next->vm_start &&
+					can_vma_merge(next, prev->vm_flags)) {
 				spin_lock(&mm->page_table_lock);
 				prev->vm_end = next->vm_end;
 				__vma_unlink(mm, next, prev);
@@ -151,7 +153,8 @@ static inline unsigned long move_vma(str
 				kmem_cache_free(vm_area_cachep, next);
 			}
 		} else if (next->vm_start == new_addr + new_len &&
-			   can_vma_merge(next, vma->vm_flags) && !vma->vm_file && !(vma->vm_flags & VM_SHARED)) {
+					can_vma_merge(next, vma->vm_flags) &&
+					!(vma->vm_flags & VM_SHARED)) {
 			spin_lock(&mm->page_table_lock);
 			next->vm_start = new_addr;
 			spin_unlock(&mm->page_table_lock);
@@ -160,7 +163,8 @@ static inline unsigned long move_vma(str
 	} else {
 		prev = find_vma(mm, new_addr-1);
 		if (prev && prev->vm_end == new_addr &&
-		    can_vma_merge(prev, vma->vm_flags) && !vma->vm_file && !(vma->vm_flags & VM_SHARED)) {
+				can_vma_merge(prev, vma->vm_flags) &&
+				!(vma->vm_flags & VM_SHARED)) {
 			spin_lock(&mm->page_table_lock);
 			prev->vm_end = new_addr + new_len;
 			spin_unlock(&mm->page_table_lock);
@@ -177,11 +181,15 @@ static inline unsigned long move_vma(str
 	}
 
 	if (!move_page_tables(vma, new_addr, addr, old_len)) {
+		unsigned long must_fault_in;
+		unsigned long fault_in_start;
+		unsigned long fault_in_end;
+
 		if (allocated_vma) {
 			*new_vma = *vma;
 			new_vma->vm_start = new_addr;
 			new_vma->vm_end = new_addr+new_len;
-			new_vma->vm_pgoff += (addr - vma->vm_start) >> PAGE_SHIFT;
+			new_vma->vm_pgoff += (addr-vma->vm_start) >> PAGE_SHIFT;
 			new_vma->vm_raend = 0;
 			if (new_vma->vm_file)
 				get_file(new_vma->vm_file);
@@ -189,12 +197,19 @@ static inline unsigned long move_vma(str
 				new_vma->vm_ops->open(new_vma);
 			insert_vm_struct(current->mm, new_vma);
 		}
+
+		must_fault_in = new_vma->vm_flags & VM_LOCKED;
+		fault_in_start = new_vma->vm_start;
+		fault_in_end = new_vma->vm_end;
+
 		do_munmap(current->mm, addr, old_len);
+
+		/* new_vma could have been invalidated by do_munmap */
+
 		current->mm->total_vm += new_len >> PAGE_SHIFT;
-		if (new_vma->vm_flags & VM_LOCKED) {
+		if (must_fault_in) {
 			current->mm->locked_vm += new_len >> PAGE_SHIFT;
-			make_pages_present(new_vma->vm_start,
-					   new_vma->vm_end);
+			make_pages_present(fault_in_start, fault_in_end);
 		}
 		return new_addr;
 	}

Re: 2.4.21pre2aa1

[J.A. Magallon]

On 2002.12.26 Marc-Christian Petersen wrote:
> On Thursday 26 December 2002 16:13, J.A. Magallón wrote:
> 
> Hi J.A.
> 
> > > 	I never noticed this problem before because I rarely use 3d (and usually
> > > 	I had mesasoft setup anyways). It's not specific to a certain graphics
> > > card, so it looks more like an agp generic problem or something, I can
> > > reproduce myself on my laptop i830 graphics card and i830 agp, on my
> > > desktop g450 with amd agp, and on my test box on a ati radeon 7500 and
> > > intel agp, so it doesn't look like a lowlevel driver problem, and it only
> > > hurts while using the agp and/or drm somehow. Many thanks to Srihari
> > > Vijayaraghavan who found the offending patch in the whole kit originally
> > > some time ago.
> > I saw it also using nVidia drivers, that do not touch drm. So I would vote
> > for agpgart.
> try this please.
> 

Sorry to say this, but it just gives me more hangs ;).
Without pte_fast and your patch, I can run gl apps, and even strace gears
(I was trying to know why it takes 4 seconds to show the first image onto
the gl window, and other 4 betwwen a Ctrl-C and the app really dying...)
With both, gl apps lock the box, and strace gears segfaults after a big
mprotect...

Thanks anyways.

-- 
J.A. Magallon <jamagallon@able.es>      \                 Software is like sex:
werewolf.able.es                         \           It's better when it's free
Mandrake Linux release 9.1 (Cooker) for i586
Linux 2.4.21-pre2-jam2 (gcc 3.2.1 (Mandrake Linux 9.1 3.2.1-2mdk))