* 4g/4g for 2.6.6 @ 2004-05-23 19:43 Phy Prabab 2004-05-23 20:32 ` Linus Torvalds 2004-05-24 1:33 ` Martin J. Bligh 0 siblings, 2 replies; 106+ messages in thread From: Phy Prabab @ 2004-05-23 19:43 UTC (permalink / raw) To: linux-kernel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 546 bytes --] Hello, Please cc me as I am not on this mailling list. I have been researching the 4g patches for kernels. Seems there was a rift between people over this. Is there any plan to resume publishing 4g patches for developing kernels? I am currently trying to get 4g to work with 2.6.6-mm5 but of course running into issues,so any help on this would be great! Thank you for your time. Phy __________________________________ Do you Yahoo!? Yahoo! Domains Claim yours for only $14.70/year http://smallbusiness.promotions.yahoo.com/offer ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 19:43 4g/4g for 2.6.6 Phy Prabab @ 2004-05-23 20:32 ` Linus Torvalds 2004-05-23 20:51 ` Jeff Garzik ` (3 more replies) 2004-05-24 1:33 ` Martin J. Bligh 1 sibling, 4 replies; 106+ messages in thread From: Linus Torvalds @ 2004-05-23 20:32 UTC (permalink / raw) To: Phy Prabab; +Cc: linux-kernel On Sun, 23 May 2004, Phy Prabab wrote: > > I have been researching the 4g patches for kernels. > Seems there was a rift between people over this. Is > there any plan to resume publishing 4g patches for > developing kernels? Quite frankly, a number of us are hoping that we can make them unnecessary. The cost of the 4g/4g split is absolutely _huge_ on some things, including basic stuff like kernel compiles. The only valid reason for the 4g split is that the VM doesn't always behave well with huge amounts of highmem. The anonvma stuff in 2.6.7-pre1 is hoped to make that much less of an issue. Personally, if we never need to merge 4g for real, I'll be really really happy. I see it as a huge ugly hack. Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 20:32 ` Linus Torvalds @ 2004-05-23 20:51 ` Jeff Garzik 2004-05-24 1:55 ` Linus Torvalds 2004-05-23 21:55 ` Phy Prabab ` (2 subsequent siblings) 3 siblings, 1 reply; 106+ messages in thread From: Jeff Garzik @ 2004-05-23 20:51 UTC (permalink / raw) To: Linus Torvalds; +Cc: Phy Prabab, linux-kernel Linus Torvalds wrote: > > On Sun, 23 May 2004, Phy Prabab wrote: > >>I have been researching the 4g patches for kernels. >>Seems there was a rift between people over this. Is >>there any plan to resume publishing 4g patches for >>developing kernels? > > > Quite frankly, a number of us are hoping that we can make them > unnecessary. The cost of the 4g/4g split is absolutely _huge_ on some > things, including basic stuff like kernel compiles. Sorta like I'm hoping that cheap and prevalent 64-bit CPUs make PAE36 and PAE40 on ia32 largely unnecessary. Addressing more memory than 32 bits of memory on a 32-bit CPU always seemed like a hack to me, and a source of bugs and lost performance... Jeff ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 20:51 ` Jeff Garzik @ 2004-05-24 1:55 ` Linus Torvalds 2004-05-24 2:19 ` Jeff Garzik 2004-06-01 5:52 ` Eric W. Biederman 0 siblings, 2 replies; 106+ messages in thread From: Linus Torvalds @ 2004-05-24 1:55 UTC (permalink / raw) To: Jeff Garzik; +Cc: Phy Prabab, linux-kernel On Sun, 23 May 2004, Jeff Garzik wrote: > > Sorta like I'm hoping that cheap and prevalent 64-bit CPUs make PAE36 > and PAE40 on ia32 largely unnecessary. Addressing more memory than 32 > bits of memory on a 32-bit CPU always seemed like a hack to me, and a > source of bugs and lost performance... I agree. I held out on PAE for a longish while, in the unrealistic hope that people would switch to alpha's. Oh, well. I don't expect _everybody_ to switch to x86-64 immediately, but I hope we can hold out long enough without 4g that it does work out this time. Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 1:55 ` Linus Torvalds @ 2004-05-24 2:19 ` Jeff Garzik 2004-05-24 2:33 ` Wim Coekaerts 2004-05-24 3:30 ` Martin J. Bligh 2004-06-01 5:52 ` Eric W. Biederman 1 sibling, 2 replies; 106+ messages in thread From: Jeff Garzik @ 2004-05-24 2:19 UTC (permalink / raw) To: Linus Torvalds; +Cc: Phy Prabab, linux-kernel Linus Torvalds wrote: > Oh, well. I don't expect _everybody_ to switch to x86-64 immediately, but > I hope we can hold out long enough without 4g that it does work out this > time. I think the switchover will happen fairly rapidly, since it is positioned "the upgrade you get next time you buy a new computer", similar to the current PATA->SATA switchover. Since these CPUs run in 32-bit mode just fine, I bet you wind up with people running Intel or AMD 64-bit CPUs long they abandon their 32-bit OS. I recall people often purchasing gigabit ethernet cards long before they had a gigabit switch, simply because it was the volume technology that was being at the time. I think the same thing is going to happen here, with AMD64 and EM64T. AMD got a lot of things right with this one... Jeff ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 2:19 ` Jeff Garzik @ 2004-05-24 2:33 ` Wim Coekaerts 2004-05-31 9:51 ` Pavel Machek 2004-05-24 3:30 ` Martin J. Bligh 1 sibling, 1 reply; 106+ messages in thread From: Wim Coekaerts @ 2004-05-24 2:33 UTC (permalink / raw) To: Jeff Garzik; +Cc: Linus Torvalds, Phy Prabab, linux-kernel > I think the switchover will happen fairly rapidly, since it is > positioned "the upgrade you get next time you buy a new computer", > similar to the current PATA->SATA switchover. you're thinking what people do at home, the large business side doesn't throw stuff out.. let me know when rhat replaces every desktop with amd64, and ll buy you a drink if that's within the next few years ;) > Since these CPUs run in 32-bit mode just fine, I bet you wind up with > people running Intel or AMD 64-bit CPUs long they abandon their 32-bit > OS. I recall people often purchasing gigabit ethernet cards long before > they had a gigabit switch, simply because it was the volume technology > that was being at the time. I think the same thing is going to happen > here, with AMD64 and EM64T. yeah well what we can do (and do) is advice that hey 64bit, in some form is really a much better environment to run servers with lots of gb on. just takes time for products to mature on the market, even today you can't get a lot of those boxes, not in bulk, and I am not talking about a 1 or 2 way or a desktop with 2gb. don't confuse the systems where 4/4gb matters with your home box. on the plus side, all the changes that went in recently seem to work mighty well on 32gb, and I still have to see the first 64gb ia32 boxes, m sure they are out there but I doubt that really matters any more. anyhow, the current VM seems to be capable of 32gb systems on ia32 happily without needing ugly 4/4. which is awesome. it's also interesting timing. anywys, 2.6 kicks butt quite decently :) Wim ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 2:33 ` Wim Coekaerts @ 2004-05-31 9:51 ` Pavel Machek 0 siblings, 0 replies; 106+ messages in thread From: Pavel Machek @ 2004-05-31 9:51 UTC (permalink / raw) To: Wim Coekaerts; +Cc: Jeff Garzik, Linus Torvalds, Phy Prabab, linux-kernel Hi! > > I think the switchover will happen fairly rapidly, since it is > > positioned "the upgrade you get next time you buy a new computer", > > similar to the current PATA->SATA switchover. > > you're thinking what people do at home, the large business side doesn't > throw stuff out.. let me know when rhat replaces every desktop with > amd64, and ll buy you a drink if that's within the next few years ;) > Desktops do not tent to have 4GB+ memory. -- 64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 2:19 ` Jeff Garzik 2004-05-24 2:33 ` Wim Coekaerts @ 2004-05-24 3:30 ` Martin J. Bligh 1 sibling, 0 replies; 106+ messages in thread From: Martin J. Bligh @ 2004-05-24 3:30 UTC (permalink / raw) To: Jeff Garzik, Linus Torvalds; +Cc: Phy Prabab, linux-kernel > Since these CPUs run in 32-bit mode just fine, I bet you wind up with > people running Intel or AMD 64-bit CPUs long they abandon their 32-bit OS. > I recall people often purchasing gigabit ethernet cards long before they > had a gigabit switch, simply because it was the volume technology that > was being at the time. I think the same thing is going to happen here, > with AMD64 and EM64T. Except some bright spark at Intel decided to fix the low end stuff first, and not the high end stuff for ages. Marvellous - all those 64GB 2 CPU boxes are fixed. Triples all round. > AMD got a lot of things right with this one... If the rest of the world would catch up, I'd be a happy man ... M. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 1:55 ` Linus Torvalds 2004-05-24 2:19 ` Jeff Garzik @ 2004-06-01 5:52 ` Eric W. Biederman 1 sibling, 0 replies; 106+ messages in thread From: Eric W. Biederman @ 2004-06-01 5:52 UTC (permalink / raw) To: Linus Torvalds; +Cc: Jeff Garzik, Phy Prabab, linux-kernel Linus Torvalds <torvalds@osdl.org> writes: > On Sun, 23 May 2004, Jeff Garzik wrote: > > > > Sorta like I'm hoping that cheap and prevalent 64-bit CPUs make PAE36 > > and PAE40 on ia32 largely unnecessary. Addressing more memory than 32 > > bits of memory on a 32-bit CPU always seemed like a hack to me, and a > > source of bugs and lost performance... > > I agree. I held out on PAE for a longish while, in the unrealistic hope > that people would switch to alpha's. > > Oh, well. I don't expect _everybody_ to switch to x86-64 immediately, but > I hope we can hold out long enough without 4g that it does work out this > time. Sounds sane. One of the real problems on machines with more than 4GB in 32bit mode is where do you put all of the pci resources. Especially when you start getting into machines with large memory mapped resources of 128MB or more. On Intel chipsets that is usually not a problem because you can relocate the memory from under those resources and still get at it with PAE36. Other chipsets don't really have that kind of capability. >From what I can tell all of the large PCI memory mapped resources are 64bit. Which gives another solution of simply moving all of those large memory mapped I/O resources above the memory entirely. Besides solving my immediate problem of loosing 1/2GB of memory in some cases, that looks like the only sane long term solution. So to encourage x86-64 usage, I am going to implement that in LinuxBIOS and encourage any other BIOS vendor I run into to follow suit. That way the only painful customer questions I will get will be why doesn't this high performance card work with a 32bit kernel :) Which is much easier to explain :) Can anyone think of a reason that would not be a good solution? Eric ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 20:32 ` Linus Torvalds 2004-05-23 20:51 ` Jeff Garzik @ 2004-05-23 21:55 ` Phy Prabab 2004-05-24 7:05 ` Arjan van de Ven 2004-05-24 2:39 ` William Lee Irwin III 2004-05-24 8:25 ` Ingo Molnar 3 siblings, 1 reply; 106+ messages in thread From: Phy Prabab @ 2004-05-23 21:55 UTC (permalink / raw) To: Linus Torvalds; +Cc: linux-kernel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 1186 bytes --] So do I understand this correctly, in 2.6.7(+) it will no longer be necessary to have the 4g patches? I will be able to get 4g/process with the going forward kernels? Thank you for your time. Phy --- Linus Torvalds <torvalds@osdl.org> wrote: > > > On Sun, 23 May 2004, Phy Prabab wrote: > > > > I have been researching the 4g patches for > kernels. > > Seems there was a rift between people over this. > Is > > there any plan to resume publishing 4g patches for > > developing kernels? > > Quite frankly, a number of us are hoping that we can > make them > unnecessary. The cost of the 4g/4g split is > absolutely _huge_ on some > things, including basic stuff like kernel compiles. > > The only valid reason for the 4g split is that the > VM doesn't always > behave well with huge amounts of highmem. The > anonvma stuff in 2.6.7-pre1 > is hoped to make that much less of an issue. > > Personally, if we never need to merge 4g for real, > I'll be really really > happy. I see it as a huge ugly hack. > > Linus __________________________________ Do you Yahoo!? Yahoo! Domains Claim yours for only $14.70/year http://smallbusiness.promotions.yahoo.com/offer ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 21:55 ` Phy Prabab @ 2004-05-24 7:05 ` Arjan van de Ven 2004-05-24 7:11 ` Phy Prabab 0 siblings, 1 reply; 106+ messages in thread From: Arjan van de Ven @ 2004-05-24 7:05 UTC (permalink / raw) To: Phy Prabab; +Cc: Linus Torvalds, linux-kernel [-- Attachment #1: Type: text/plain, Size: 580 bytes --] On Sun, 2004-05-23 at 23:55, Phy Prabab wrote: > So do I understand this correctly, in 2.6.7(+) it will > no longer be necessary to have the 4g patches? I will > be able to get 4g/process with the going forward > kernels? The kernel RPMs I do (http://people.redhat.com/arjanv/2.6/) pretty much will have it always for 2.6. Not just for large memory configs, but because several userspace applications (databases, java etc) really like that extra Gb of virtual space too. As for the cost; 4:4 split seems to be hardly expensive at all, only in some microbenchmarks. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 7:05 ` Arjan van de Ven @ 2004-05-24 7:11 ` Phy Prabab 2004-05-24 7:24 ` Arjan van de Ven 0 siblings, 1 reply; 106+ messages in thread From: Phy Prabab @ 2004-05-24 7:11 UTC (permalink / raw) To: arjanv; +Cc: Linus Torvalds, linux-kernel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 966 bytes --] Is this source, or precompiled targeting just RH setups? Thank you for your help. Phy --- Arjan van de Ven <arjanv@redhat.com> wrote: > On Sun, 2004-05-23 at 23:55, Phy Prabab wrote: > > So do I understand this correctly, in 2.6.7(+) it > will > > no longer be necessary to have the 4g patches? I > will > > be able to get 4g/process with the going forward > > kernels? > > The kernel RPMs I do > (http://people.redhat.com/arjanv/2.6/) pretty much > will have it always for 2.6. Not just for large > memory configs, but > because several userspace applications (databases, > java etc) really like > that extra Gb of virtual space too. > As for the cost; 4:4 split seems to be hardly > expensive at all, only in > some microbenchmarks. > > ATTACHMENT part 2 application/pgp-signature name=signature.asc __________________________________ Do you Yahoo!? Yahoo! Domains Claim yours for only $14.70/year http://smallbusiness.promotions.yahoo.com/offer ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 7:11 ` Phy Prabab @ 2004-05-24 7:24 ` Arjan van de Ven 2004-05-24 7:27 ` Phy Prabab 0 siblings, 1 reply; 106+ messages in thread From: Arjan van de Ven @ 2004-05-24 7:24 UTC (permalink / raw) To: Phy Prabab; +Cc: Linus Torvalds, linux-kernel [-- Attachment #1: Type: text/plain, Size: 142 bytes --] On Mon, May 24, 2004 at 12:11:26AM -0700, Phy Prabab wrote: > Is this source, or precompiled targeting just RH > setups? it's both actually. [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 7:24 ` Arjan van de Ven @ 2004-05-24 7:27 ` Phy Prabab 2004-05-24 12:01 ` Dave Jones 0 siblings, 1 reply; 106+ messages in thread From: Phy Prabab @ 2004-05-24 7:27 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Linus Torvalds, linux-kernel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 539 bytes --] Thats great! Do you include a change file to see what other patches have been applied? Thank you very much for your help. Phy --- Arjan van de Ven <arjanv@redhat.com> wrote: > On Mon, May 24, 2004 at 12:11:26AM -0700, Phy Prabab > wrote: > > Is this source, or precompiled targeting just RH > > setups? > > it's both actually. > > ATTACHMENT part 2 application/pgp-signature __________________________________ Do you Yahoo!? Yahoo! Domains Claim yours for only $14.70/year http://smallbusiness.promotions.yahoo.com/offer ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 7:27 ` Phy Prabab @ 2004-05-24 12:01 ` Dave Jones 0 siblings, 0 replies; 106+ messages in thread From: Dave Jones @ 2004-05-24 12:01 UTC (permalink / raw) To: Phy Prabab; +Cc: Arjan van de Ven, Linus Torvalds, linux-kernel On Mon, May 24, 2004 at 12:27:05AM -0700, Phy Prabab wrote: > Thats great! Do you include a change file to see what > other patches have been applied? It's in the kernel-2.6.spec in those RPMs. Dave ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 20:32 ` Linus Torvalds 2004-05-23 20:51 ` Jeff Garzik 2004-05-23 21:55 ` Phy Prabab @ 2004-05-24 2:39 ` William Lee Irwin III 2004-05-24 8:25 ` Ingo Molnar 3 siblings, 0 replies; 106+ messages in thread From: William Lee Irwin III @ 2004-05-24 2:39 UTC (permalink / raw) To: Linus Torvalds; +Cc: Phy Prabab, linux-kernel On Sun, May 23, 2004 at 01:32:19PM -0700, Linus Torvalds wrote: > Quite frankly, a number of us are hoping that we can make them > unnecessary. The cost of the 4g/4g split is absolutely _huge_ on some > things, including basic stuff like kernel compiles. > The only valid reason for the 4g split is that the VM doesn't always > behave well with huge amounts of highmem. The anonvma stuff in 2.6.7-pre1 > is hoped to make that much less of an issue. > Personally, if we never need to merge 4g for real, I'll be really really > happy. I see it as a huge ugly hack. The performance can be improved by using a u area to store and map things like vmas, kernel stacks, pagetables, file handles and descriptor tables, and the like with supervisor privileges in the same set of pagetables as the user context so that system calls may be serviced without referencing the larger global kernel data area, which would require the %cr3 reload. This does, however, seem at odds with Linux' design in a number of respects, e.g. vmas etc. are on lists containing elements belonging to different contexts. I suspect kernels doing this would have to architect their page replacement algorithms and truncate() semantics so as to avoid these out-of-context accesses or otherwise suffer these operations being inefficient. -- wli ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 20:32 ` Linus Torvalds ` (2 preceding siblings ...) 2004-05-24 2:39 ` William Lee Irwin III @ 2004-05-24 8:25 ` Ingo Molnar 2004-05-24 12:48 ` Andrea Arcangeli 3 siblings, 1 reply; 106+ messages in thread From: Ingo Molnar @ 2004-05-24 8:25 UTC (permalink / raw) To: Linus Torvalds; +Cc: Phy Prabab, linux-kernel * Linus Torvalds <torvalds@osdl.org> wrote: > Quite frankly, a number of us are hoping that we can make them > unnecessary. The cost of the 4g/4g split is absolutely _huge_ on some > things, including basic stuff like kernel compiles. > > The only valid reason for the 4g split is that the VM doesn't always > behave well with huge amounts of highmem. The anonvma stuff in > 2.6.7-pre1 is hoped to make that much less of an issue. > > Personally, if we never need to merge 4g for real, I'll be really > really happy. I see it as a huge ugly hack. i agree with the hack part - but the performance aspect has been blown out of proportion. 4:4 has the same cost on kernel compiles as highpte. There are also real workloads where it actually helps performance. also, the 4:4 overhead is really a hardware problem - and there are x86-compatible CPUs (amd64) where the TLB flush problem has already been solved: on amd64 the 4:4 feature has no noticeable overhead. So as long as people opt for a 32-bit OS (even on a 64-bit CPU, for whatever weird compatibility reason), 4:4 can be useful. For the other workloads i as much hope as everyone else that people switch to a 64-bit OS on x86-64 ASAP! also, while a quick transition to x86-64 will most likely happen, the large installed base of big x86 boxes is a matter of fact too - and they wont vanish into thin air. Also, there will always be specific user workloads where lowmem grows to large values. Not to mention the fact that 4:4 is a nice debugging/security tool as you cannot dereference user pointers ;) - it has caught countless bugs already. Plus there are specific workloads that want 4GB of userspace (and no, 3.5 GB wont do). So 4:4 will have its niches where it will live on. We could argue on and on how quickly 'x86 with more than 4GB of RAM' and 'x86 with 4GB of a userspace' will become a niche, hopefully it happens fast but we've got to keep our options open. Ingo ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 8:25 ` Ingo Molnar @ 2004-05-24 12:48 ` Andrea Arcangeli 2004-05-25 19:15 ` Rik van Riel 0 siblings, 1 reply; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-24 12:48 UTC (permalink / raw) To: Ingo Molnar; +Cc: Linus Torvalds, Phy Prabab, linux-kernel On Mon, May 24, 2004 at 10:25:22AM +0200, Ingo Molnar wrote: > on how quickly 'x86 with more than 4GB of RAM' and s/4GB/32GB/ my usual x86 test box has 48G of ram (though to keep an huge margin of safety we assume 32G is the very safe limit). ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 12:48 ` Andrea Arcangeli @ 2004-05-25 19:15 ` Rik van Riel 2004-05-25 19:41 ` Andrea Arcangeli 0 siblings, 1 reply; 106+ messages in thread From: Rik van Riel @ 2004-05-25 19:15 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: Ingo Molnar, Linus Torvalds, Phy Prabab, linux-kernel On Mon, 24 May 2004, Andrea Arcangeli wrote: > On Mon, May 24, 2004 at 10:25:22AM +0200, Ingo Molnar wrote: > > on how quickly 'x86 with more than 4GB of RAM' and > > s/4GB/32GB/ > > my usual x86 test box has 48G of ram (though to keep an huge margin of > safety we assume 32G is the very safe limit). Just how many 3GB sized processes can you run on that system, if each of them have a hundred or so VMAs ? -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 19:15 ` Rik van Riel @ 2004-05-25 19:41 ` Andrea Arcangeli 2004-05-25 19:50 ` Rik van Riel 0 siblings, 1 reply; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-25 19:41 UTC (permalink / raw) To: Rik van Riel; +Cc: Ingo Molnar, Linus Torvalds, Phy Prabab, linux-kernel On Tue, May 25, 2004 at 03:15:14PM -0400, Rik van Riel wrote: > On Mon, 24 May 2004, Andrea Arcangeli wrote: > > On Mon, May 24, 2004 at 10:25:22AM +0200, Ingo Molnar wrote: > > > on how quickly 'x86 with more than 4GB of RAM' and > > > > s/4GB/32GB/ > > > > my usual x86 test box has 48G of ram (though to keep an huge margin of > > safety we assume 32G is the very safe limit). > > Just how many 3GB sized processes can you run on that > system, if each of them have a hundred or so VMAs ? with 400m of normal zone we can allocate 5000000 vmas, if each task uses 100 of them that's around 50000 processes. The 3G size doesn't matter. I think you meant each task using some _dozen_thousand_ of vmas (not hundreds), that's what actually happens with 32k large vmas spread over 2G of shared memory on some database (assuming no merging, with merging it goes down to the few thousands), but remap_file_pages has been designed exactly to avoid allocating several thousands of VMA per task, no? So it's just 1 vma per task (plus a few more vmas for the binary shared libs and anonymous memory). Clearly by opening enough files or enough network sockets or enough vmas or similar, you can still run out of normal zone, even on a 2G system, but this is not the point or you would be shipping 4:4 on the 2G systems too, no? We're not trying to make it impossible to run out of zone normal, even 4:4 can't make that impossible to happen on >4G boxes. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 19:41 ` Andrea Arcangeli @ 2004-05-25 19:50 ` Rik van Riel 2004-05-25 20:10 ` Rik van Riel 2004-05-25 21:04 ` Bill Davidsen 0 siblings, 2 replies; 106+ messages in thread From: Rik van Riel @ 2004-05-25 19:50 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: Ingo Molnar, Linus Torvalds, Phy Prabab, linux-kernel On Tue, 25 May 2004, Andrea Arcangeli wrote: > Clearly by opening enough files or enough network sockets or enough vmas > or similar, you can still run out of normal zone, even on a 2G system, > but this is not the point or you would be shipping 4:4 on the 2G systems > too, no? The point is, people like to run bigger workloads on bigger systems. Otherwise they wouldn't bother buying those bigger systems. -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 19:50 ` Rik van Riel @ 2004-05-25 20:10 ` Rik van Riel 2004-05-25 21:15 ` Andrea Arcangeli 2004-05-25 21:16 ` 4g/4g for 2.6.6 Andrew Morton 2004-05-25 21:04 ` Bill Davidsen 1 sibling, 2 replies; 106+ messages in thread From: Rik van Riel @ 2004-05-25 20:10 UTC (permalink / raw) To: Andrea Arcangeli; +Cc: Ingo Molnar, Linus Torvalds, Phy Prabab, linux-kernel On Tue, 25 May 2004, Rik van Riel wrote: > The point is, people like to run bigger workloads on > bigger systems. Otherwise they wouldn't bother buying > those bigger systems. Btw, you're right about the VMAs. Looking through customer stuff a bit more the more common issues are low memory being eaten by dentry / inode cache - which you can't always reclaim due to files being open, and don't always _want_ to reclaim because that could well be a bigger performance hit than the 4:4 split. The primary impact of the dentry / inode cache using memory isn't lowmem exhaustion, btw. It's lowmem fragmentation. Fragmentation causes fork trouble (gone with the 4k stacks) and trouble for the network layer and kiobuf allocation, which still do need higher order allocations. Sure, the 4/4 kernel could also have problems with lowmem fragmentation, but it just seems to be nowhere near as bad. -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 20:10 ` Rik van Riel @ 2004-05-25 21:15 ` Andrea Arcangeli 2004-05-26 10:33 ` 4k stacks in 2.6 Ingo Molnar 2004-05-25 21:16 ` 4g/4g for 2.6.6 Andrew Morton 1 sibling, 1 reply; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-25 21:15 UTC (permalink / raw) To: Rik van Riel; +Cc: Ingo Molnar, Linus Torvalds, Phy Prabab, linux-kernel On Tue, May 25, 2004 at 04:10:29PM -0400, Rik van Riel wrote: > Fragmentation causes fork trouble (gone with the 4k stacks) btw, the 4k stacks sounds not safe to me, most people only tested with 8k stacks so far, I wouldn't make that change in a production tree without an unstable cycle of testing in between. I'd rather risk a an allocation failure than a stack memory corruption. x86-64 has per-irq stacks that allowed to reduce the stack size to 8k (which is very similar to 4k for an x86, but without per-irq stack it's too risky). as for the dcache size, I can certainly imagine you can measure slowdown bigger than 4:4 in some very vfs intensive workload, but the stuff that needs 32G of ram normally uses only rawio or a few huge files as storage anyways. as for the kiobufs they're long gone in 2.6. Possibly some webserving could use 32G as pure pagecache for the webserver with tons of tiny files, for such an app the 2:2 or 1:3 model should be better than 4:4. ^ permalink raw reply [flat|nested] 106+ messages in thread
* 4k stacks in 2.6 2004-05-25 21:15 ` Andrea Arcangeli @ 2004-05-26 10:33 ` Ingo Molnar 2004-05-26 12:50 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Ingo Molnar @ 2004-05-26 10:33 UTC (permalink / raw) To: Andrea Arcangeli Cc: Rik van Riel, Linus Torvalds, Arjan van de Ven, linux-kernel * Andrea Arcangeli <andrea@suse.de> wrote: > On Tue, May 25, 2004 at 04:10:29PM -0400, Rik van Riel wrote: > > Fragmentation causes fork trouble (gone with the 4k stacks) > > btw, the 4k stacks sounds not safe to me, most people only tested with > 8k stacks so far, I wouldn't make that change in a production tree > without an unstable cycle of testing in between. I'd rather risk a an > allocation failure than a stack memory corruption. 4k stacks is a cool and useful feature and tons of effort that went into making them as safe as possible. Sure, we couldnt fix up bin-only modules, but all the kernel drivers are audited for stack footprint, and many months of beta testing has gone into this as well. Anyway, if you prefer you can turn on 8k stacks - especially if you tree has lots of not-yet-upstream driver patches. > x86-64 has per-irq stacks that allowed to reduce the stack size to 8k > (which is very similar to 4k for an x86, but without per-irq stack > it's too risky). do you realize that the 4K stacks feature also adds a separate softirq and a separate hardirq stack? So the maximum footprint is 4K+4K+4K, with a clear and sane limit for each type of context, while the 2.4 kernel has 6.5K for all 3 contexts combined. (Also, in 2.4 irq contexts pretty much assumed that there's 2K of stack for them - leaving a de-facto 4K stack for the process and softirq contexts.) So in fact there is more space in 2.6 for all, and i dont really understand your fears. Ingo ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 10:33 ` 4k stacks in 2.6 Ingo Molnar @ 2004-05-26 12:50 ` Jörn Engel 2004-05-26 12:53 ` Arjan van de Ven 2004-05-26 18:12 ` David S. Miller 0 siblings, 2 replies; 106+ messages in thread From: Jörn Engel @ 2004-05-26 12:50 UTC (permalink / raw) To: Ingo Molnar Cc: Andrea Arcangeli, Rik van Riel, Linus Torvalds, Arjan van de Ven, linux-kernel On Wed, 26 May 2004 12:33:03 +0200, Ingo Molnar wrote: > * Andrea Arcangeli <andrea@suse.de> wrote: > > On Tue, May 25, 2004 at 04:10:29PM -0400, Rik van Riel wrote: > > > Fragmentation causes fork trouble (gone with the 4k stacks) > > > > btw, the 4k stacks sounds not safe to me, most people only tested with > > 8k stacks so far, I wouldn't make that change in a production tree > > without an unstable cycle of testing in between. I'd rather risk a an > > allocation failure than a stack memory corruption. > > 4k stacks is a cool and useful feature and tons of effort that went into > making them as safe as possible. Sure, we couldnt fix up bin-only > modules, but all the kernel drivers are audited for stack footprint, and > many months of beta testing has gone into this as well. Anyway, if you > prefer you can turn on 8k stacks - especially if you tree has lots of > not-yet-upstream driver patches. > > > x86-64 has per-irq stacks that allowed to reduce the stack size to 8k > > (which is very similar to 4k for an x86, but without per-irq stack > > it's too risky). > > do you realize that the 4K stacks feature also adds a separate softirq > and a separate hardirq stack? So the maximum footprint is 4K+4K+4K, with > a clear and sane limit for each type of context, while the 2.4 kernel > has 6.5K for all 3 contexts combined. (Also, in 2.4 irq contexts pretty > much assumed that there's 2K of stack for them - leaving a de-facto 4K > stack for the process and softirq contexts.) So in fact there is more > space in 2.6 for all, and i dont really understand your fears. Experience indicates that for whatever reason, big stack consumers for all three contexts never hit at the same time. Big stack consumers for one context happen too often, though. "Too often" may be quite rare, but considering the result of a stack overflow, even "quite rare" is too much. "Never" is the only acceptable target. Change gcc to catch stack overflows before the fact and disallow module load unless modules have those checks as well. If that is done, a stack overflow will merely cause a kernel panic. Until then, I am just as conservative as Andreas. Jörn -- And spam is a useful source of entropy for /dev/random too! -- Jasmine Strong ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 12:50 ` Jörn Engel @ 2004-05-26 12:53 ` Arjan van de Ven 2004-05-26 13:00 ` Jörn Engel 2004-05-26 18:12 ` David S. Miller 1 sibling, 1 reply; 106+ messages in thread From: Arjan van de Ven @ 2004-05-26 12:53 UTC (permalink / raw) To: Jörn Engel Cc: Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel [-- Attachment #1: Type: text/plain, Size: 846 bytes --] On Wed, May 26, 2004 at 02:50:14PM +0200, Jörn Engel wrote: > > Experience indicates that for whatever reason, big stack consumers for > all three contexts never hit at the same time. Big stack consumers > for one context happen too often, though. "Too often" may be quite > rare, but considering the result of a stack overflow, even "quite > rare" is too much. "Never" is the only acceptable target. Actually it's not mever in 2.4. It does get here there by our customers once in a while. Esp with several NICs hitting an irq on the same CPU (eg the irq context goes over it's 2Kb limit) > done, a stack overflow will merely cause a kernel panic. Until then, > I am just as conservative as Andreas. actually the 4k stacks approach gives MORE breathing room for the problem cases that are getting hit by our customers... [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 12:53 ` Arjan van de Ven @ 2004-05-26 13:00 ` Jörn Engel 2004-05-26 13:05 ` Arjan van de Ven 2004-06-07 18:14 ` 4k stacks in 2.6 Timothy Miller 0 siblings, 2 replies; 106+ messages in thread From: Jörn Engel @ 2004-05-26 13:00 UTC (permalink / raw) To: Arjan van de Ven Cc: Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel On Wed, 26 May 2004 14:53:00 +0200, Arjan van de Ven wrote: > On Wed, May 26, 2004 at 02:50:14PM +0200, Jörn Engel wrote: > > > > Experience indicates that for whatever reason, big stack consumers for > > all three contexts never hit at the same time. Big stack consumers > > for one context happen too often, though. "Too often" may be quite > > rare, but considering the result of a stack overflow, even "quite > > rare" is too much. "Never" is the only acceptable target. > > Actually it's not mever in 2.4. It does get here there by our customers once > in a while. Esp with several NICs hitting an irq on the same CPU (eg the irq > context goes over it's 2Kb limit) > > > done, a stack overflow will merely cause a kernel panic. Until then, > > I am just as conservative as Andreas. > > actually the 4k stacks approach gives MORE breathing room for the problem > cases that are getting hit by our customers... For the cases you described, yes. For some others like nvidia, no. Not sure if we want to make things worse for some users in order to improve things for others (better paying ones?). I want the seperate interrupt stacks, sure. I'm just not comfy with 4k per process yet. But I'll shut up now and see if I can generate better data over the weekend. -test11 still had fun stuff like 3k stack consumption over some code paths in a pretty minimal kernel. Wonder what 2.6.6 will do with allyesconfig. ;) Jörn -- He who knows that enough is enough will always have enough. -- Lao Tsu ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 13:00 ` Jörn Engel @ 2004-05-26 13:05 ` Arjan van de Ven 2004-05-26 16:41 ` Jörn Engel 2004-06-07 18:14 ` 4k stacks in 2.6 Timothy Miller 1 sibling, 1 reply; 106+ messages in thread From: Arjan van de Ven @ 2004-05-26 13:05 UTC (permalink / raw) To: Jörn Engel Cc: Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel [-- Attachment #1: Type: text/plain, Size: 1331 bytes --] On Wed, May 26, 2004 at 03:00:47PM +0200, Jörn Engel wrote: > > > Experience indicates that for whatever reason, big stack consumers for > > > all three contexts never hit at the same time. Big stack consumers > > > for one context happen too often, though. "Too often" may be quite > > > rare, but considering the result of a stack overflow, even "quite > > > rare" is too much. "Never" is the only acceptable target. > > > > actually the 4k stacks approach gives MORE breathing room for the problem > > cases that are getting hit by our customers... > > For the cases you described, yes. For some others like nvidia, no. > Not sure if we want to make things worse for some users in order to > improve things for others (better paying ones?). I want the seperate You used the word "Never" and now you go away from it.... It wasn't Never, and it will never be never if you want to include random binary only modules. However in 2.4 for all intents and pruposes there was 4Kb already, and now there still is, for user context. Because those interrupts DO happen. NVidia was a walking timebomb, and with one function using 4Kb that's an obvious Needs-Fix case. The kernel had a few of those in rare drivers, most of which have been fixed by now. It'll never be never, but it never was never either. [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 13:05 ` Arjan van de Ven @ 2004-05-26 16:41 ` Jörn Engel 2004-05-27 12:45 ` Ingo Molnar 2004-06-01 5:56 ` 4k stacks in 2.6 [worst offenders] Jörn Engel 0 siblings, 2 replies; 106+ messages in thread From: Jörn Engel @ 2004-05-26 16:41 UTC (permalink / raw) To: Arjan van de Ven Cc: Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel On Wed, 26 May 2004 15:05:00 +0200, Arjan van de Ven wrote: > > You used the word "Never" and now you go away from it.... It wasn't Never, > and it will never be never if you want to include random binary only > modules. However in 2.4 for all intents and pruposes there was 4Kb already, > and now there still is, for user context. Because those interrupts DO > happen. NVidia was a walking timebomb, and with one function using 4Kb > that's an obvious Needs-Fix case. The kernel had a few of those in rare > drivers, most of which have been fixed by now. It'll never be never, but it > never was never either. In a way, you are right. nVidia was and is a walking timebomb and making bugs more likely to happen is a good thing in general. Except that this bug can eat filesystems, so making it more likely will cause more filesystems to be eaten. Anyway, whether we go for 4k in 2.6 or not, we should do our best to fix bad code and I will go looking for some more so others can go and fix some more. There's still enough horror in mainline for more than one amusement park, we just haven't found it yet. Jörn -- All art is but imitation of nature. -- Lucius Annaeus Seneca ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 16:41 ` Jörn Engel @ 2004-05-27 12:45 ` Ingo Molnar 2004-05-27 13:59 ` Andrea Arcangeli 2004-06-01 5:56 ` 4k stacks in 2.6 [worst offenders] Jörn Engel 1 sibling, 1 reply; 106+ messages in thread From: Ingo Molnar @ 2004-05-27 12:45 UTC (permalink / raw) To: Jörn Engel Cc: Arjan van de Ven, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel * Jörn Engel <joern@wohnheim.fh-wedel.de> wrote: > Anyway, whether we go for 4k in 2.6 or not, [...] 4K stacks have been added to the 2.6 kernel more than a month ago, are in the official 2.6.6 kernel and are used by FC2 happily, so objections are a bit belated. I only reacted to Andrea's mail to clear up apparent misunderstandings about the impact and implementation of this feature. Ingo ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 12:45 ` Ingo Molnar @ 2004-05-27 13:59 ` Andrea Arcangeli 2004-05-27 14:03 ` Arjan van de Ven 2004-05-27 14:18 ` Brian Gerst 0 siblings, 2 replies; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-27 13:59 UTC (permalink / raw) To: Ingo Molnar Cc: Jörn Engel, Arjan van de Ven, Rik van Riel, Linus Torvalds, linux-kernel On Thu, May 27, 2004 at 02:45:51PM +0200, Ingo Molnar wrote: > are a bit belated. I only reacted to Andrea's mail to clear up apparent > misunderstandings about the impact and implementation of this feature. note that there is something relevant to improve in the implementation, that is the per-cpu irq stack size should be bigger than 4k, we use 16k on x86-64, on x86 it should be 8k. Currently you're decreasing _both_ the normal kernel context and even the irq stack in some condition. There's no good reason to decrease the irq stack too, that's cheap, it's per-cpu. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 13:59 ` Andrea Arcangeli @ 2004-05-27 14:03 ` Arjan van de Ven 2004-05-27 14:42 ` Andrea Arcangeli 2004-05-27 14:18 ` Brian Gerst 1 sibling, 1 reply; 106+ messages in thread From: Arjan van de Ven @ 2004-05-27 14:03 UTC (permalink / raw) To: Andrea Arcangeli Cc: Ingo Molnar, Jörn Engel, Rik van Riel, Linus Torvalds, linux-kernel [-- Attachment #1: Type: text/plain, Size: 1231 bytes --] On Thu, May 27, 2004 at 03:59:30PM +0200, Andrea Arcangeli wrote: > On Thu, May 27, 2004 at 02:45:51PM +0200, Ingo Molnar wrote: > > are a bit belated. I only reacted to Andrea's mail to clear up apparent > > misunderstandings about the impact and implementation of this feature. > > note that there is something relevant to improve in the implementation, > that is the per-cpu irq stack size should be bigger than 4k, we use 16k > on x86-64, on x86 it should be 8k. Currently you're decreasing _both_ > the normal kernel context and even the irq stack in some condition. > There's no good reason to decrease the irq stack too, that's cheap, it's > per-cpu. In theory you are absolutely right, problem is the current macro..... it's SO much easier to have one stacksize everywhere (and cheaper too) for this... (and it hasn't been a problem so far, esp since the softirq's have their own stack, irq handlers seem to be all really light on the stack already since they punt all the heavy lifting to tasklets etc. Tasklets don't recurse stack wise, and have their own stack, so that ought to be fine. On x86_64 you have the PDA for current so that's not a problem, and you can do the bigger stacks easily but for x86 you don't... [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:03 ` Arjan van de Ven @ 2004-05-27 14:42 ` Andrea Arcangeli 2004-06-02 19:40 ` Bill Davidsen 0 siblings, 1 reply; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-27 14:42 UTC (permalink / raw) To: Arjan van de Ven Cc: Ingo Molnar, Jörn Engel, Rik van Riel, Linus Torvalds, linux-kernel On Thu, May 27, 2004 at 04:03:22PM +0200, Arjan van de Ven wrote: > In theory you are absolutely right, problem is the current macro..... it's > SO much easier to have one stacksize everywhere (and cheaper too) for > this... (and it hasn't been a problem so far, esp since the softirq's have I see the problem, but then why don't we wait to implement it right, to allow 8k irq-stacks before merging into mainline? grep for "~s 4k" (i.e. the word "4[kK]" in the subject) on l-k and you'll see there's more than just nvidia. one user reported not being able to boot at all with 4k stacks since 2.6.6 doesn't have a stack overflow in the oops, so I hope he tested w/ and w/o 4KSTACKS option enabled to be able to claim what broke his machine is the 4KSTACKS option. (his oops doesn't reveal a stack overflow, the thread_info is at 0xf000 and the esp is at 0xffxx) Making it a config option, is a sort of proof that you agree it can break something, or you wouldn't make it a config option in the first place. What's the point of making it a configuration option if it cannot break anything and if it's not risky? Making it a config option is not good, because then some developer may develop leaving 4KSTACKS disabled, and then his kernel code might break on the users with 4KSTACKS enabled (it's not much different from PREEMPT). Amittedly code that overflows 4k is likely to be not legitimate but not all code is good (the most common error is to allocate big strutures locally on the stack with local vars), and if developers are able to notice the overflow on their own testing it's better. Clearly it's more relaxed to merge something knowing with a config option you can choose if to use 4k or 8k stacks, but I'm not sure if it's the right thing to do for the long term. If we go 4k stacks, then I'd prefer that you drop the 4KSTACKS option and force people to reduce the stack usage in their code, and secondly that we fixup the irqstack to be 8k. Plus the allocation errors you had, could be just 2.6 vm issues with order > 0 allocations, we never had issues with 8k stacks in 2.4, so using the 4k stacks may just hide the real problem. archs like x86-64 have to use order > 0 allocations for kernel stack, no way around it, so order > 0 must work reliably regardless of whatever code we change in x86. > On x86_64 you have the PDA for current so that's not a problem, and > you can do the bigger stacks easily but for x86 you don't... yep. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:42 ` Andrea Arcangeli @ 2004-06-02 19:40 ` Bill Davidsen 0 siblings, 0 replies; 106+ messages in thread From: Bill Davidsen @ 2004-06-02 19:40 UTC (permalink / raw) To: linux-kernel Andrea Arcangeli wrote: > On Thu, May 27, 2004 at 04:03:22PM +0200, Arjan van de Ven wrote: > >>In theory you are absolutely right, problem is the current macro..... it's >>SO much easier to have one stacksize everywhere (and cheaper too) for >>this... (and it hasn't been a problem so far, esp since the softirq's have > > > I see the problem, but then why don't we wait to implement it right, to > allow 8k irq-stacks before merging into mainline? > > grep for "~s 4k" (i.e. the word "4[kK]" in the subject) on l-k and > you'll see there's more than just nvidia. one user reported not being > able to boot at all with 4k stacks since 2.6.6 doesn't have a stack > overflow in the oops, so I hope he tested w/ and w/o 4KSTACKS option > enabled to be able to claim what broke his machine is the 4KSTACKS > option. (his oops doesn't reveal a stack overflow, the thread_info is at > 0xf000 and the esp is at 0xffxx) > > Making it a config option, is a sort of proof that you agree it can > break something, or you wouldn't make it a config option in the first > place. What's the point of making it a configuration option if it cannot > break anything and if it's not risky? Making it a config option is not > good, because then some developer may develop leaving 4KSTACKS disabled, > and then his kernel code might break on the users with 4KSTACKS enabled > (it's not much different from PREEMPT). Amittedly code that overflows > 4k is likely to be not legitimate but not all code is good (the most > common error is to allocate big strutures locally on the stack with > local vars), and if developers are able to notice the overflow on their > own testing it's better. We have lots of options which may cause problems but are useful for special situations, why is this one any different? The only actual benefit I've seen quoted for 4k stack is that it improves fork performance if memory is so fragmented that there is no 8k block left. And my first thought on hearing that was if that's common the VM should be investigated. This is a stable kernel, and breaking even such an abomination as a binary-only driver for the sake of whoever has this vastly fragmented memory seems to be the anthesis of stable. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 13:59 ` Andrea Arcangeli 2004-05-27 14:03 ` Arjan van de Ven @ 2004-05-27 14:18 ` Brian Gerst 2004-05-27 14:50 ` Andrea Arcangeli 1 sibling, 1 reply; 106+ messages in thread From: Brian Gerst @ 2004-05-27 14:18 UTC (permalink / raw) To: Andrea Arcangeli Cc: Ingo Molnar, Jörn Engel, Arjan van de Ven, Rik van Riel, Linus Torvalds, linux-kernel Andrea Arcangeli wrote: > On Thu, May 27, 2004 at 02:45:51PM +0200, Ingo Molnar wrote: > >>are a bit belated. I only reacted to Andrea's mail to clear up apparent >>misunderstandings about the impact and implementation of this feature. > > > note that there is something relevant to improve in the implementation, > that is the per-cpu irq stack size should be bigger than 4k, we use 16k > on x86-64, on x86 it should be 8k. Currently you're decreasing _both_ > the normal kernel context and even the irq stack in some condition. > There's no good reason to decrease the irq stack too, that's cheap, it's > per-cpu. The problem on i386 (unlike x86-64) is that the thread_info struct sits at the bottom of the stack and is referenced by masking bits off %esp. So the stack size must be constant whether in process context or IRQ context. -- Brian Gerst ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:18 ` Brian Gerst @ 2004-05-27 14:50 ` Andrea Arcangeli 2004-05-27 14:55 ` Linus Torvalds 0 siblings, 1 reply; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-27 14:50 UTC (permalink / raw) To: Brian Gerst Cc: Ingo Molnar, Jörn Engel, Arjan van de Ven, Rik van Riel, Linus Torvalds, linux-kernel On Thu, May 27, 2004 at 10:18:40AM -0400, Brian Gerst wrote: > The problem on i386 (unlike x86-64) is that the thread_info struct sits > at the bottom of the stack and is referenced by masking bits off %esp. > So the stack size must be constant whether in process context or IRQ > context. so what, that's a minor implementation detail, pda is a software thing. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:50 ` Andrea Arcangeli @ 2004-05-27 14:55 ` Linus Torvalds 2004-05-27 15:39 ` Andrea Arcangeli 2004-05-27 18:31 ` Guy Sotomayor 0 siblings, 2 replies; 106+ messages in thread From: Linus Torvalds @ 2004-05-27 14:55 UTC (permalink / raw) To: Andrea Arcangeli Cc: Brian Gerst, Ingo Molnar, Jörn Engel, Arjan van de Ven, Rik van Riel, linux-kernel On Thu, 27 May 2004, Andrea Arcangeli wrote: > > On Thu, May 27, 2004 at 10:18:40AM -0400, Brian Gerst wrote: > > The problem on i386 (unlike x86-64) is that the thread_info struct sits > > at the bottom of the stack and is referenced by masking bits off %esp. > > So the stack size must be constant whether in process context or IRQ > > context. > > so what, that's a minor implementation detail, pda is a software thing. "minor implementation detail"? You need to get to the thread info _some_ way, and you need to get to it _fast_. There are really no sane alternatives. I certainly do not want to play games with segments. Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:55 ` Linus Torvalds @ 2004-05-27 15:39 ` Andrea Arcangeli 2004-05-27 18:31 ` Guy Sotomayor 1 sibling, 0 replies; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-27 15:39 UTC (permalink / raw) To: Linus Torvalds Cc: Brian Gerst, Ingo Molnar, Jörn Engel, Arjan van de Ven, Rik van Riel, linux-kernel On Thu, May 27, 2004 at 07:55:36AM -0700, Linus Torvalds wrote: > > > On Thu, 27 May 2004, Andrea Arcangeli wrote: > > > > On Thu, May 27, 2004 at 10:18:40AM -0400, Brian Gerst wrote: > > > The problem on i386 (unlike x86-64) is that the thread_info struct sits > > > at the bottom of the stack and is referenced by masking bits off %esp. > > > So the stack size must be constant whether in process context or IRQ > > > context. > > > > so what, that's a minor implementation detail, pda is a software thing. > > "minor implementation detail"? > > You need to get to the thread info _some_ way, and you need to get to it > _fast_. There are really no sane alternatives. I certainly do not want to > play games with segments. If the page is "even" the thread_info is at the top of the stack. If the page is "odd" the thread_info is at the bottom of the stack (or the other way around depending what you mean with "odd" and "even"). the per-cpu irq stack will have the thread_info at both the top and the bottom of the 8k naturally aligned order1 compound page. The regular kernel stack will have it at the top or the bottom depending if it's odd or even. this should allow 8k irqstack and bh stack fine at in-cpu-core speed w/o segments or similar. The only downside is that itadds a branch to current_thread_info that will have to check the 12th bitflag in the esp before doing andl, the second downside is having to update two thread_info during irq, instead of just one. It would be probably better if the thread_info was just a pointer to a "pda" instead of being the PDA itself so there are just two writes into the kernel stack for every irq. In x86-64 this is much more natural since the pda-pointer is in the cpu 64bit %gs register and that saves a branch and defereference on the stack for every "current" invocation, and two writes for every first-irq or first-bh. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:55 ` Linus Torvalds 2004-05-27 15:39 ` Andrea Arcangeli @ 2004-05-27 18:31 ` Guy Sotomayor 2004-05-27 19:26 ` Brian Gerst 1 sibling, 1 reply; 106+ messages in thread From: Guy Sotomayor @ 2004-05-27 18:31 UTC (permalink / raw) To: Linus Torvalds Cc: Andrea Arcangeli, Brian Gerst, Ingo Molnar, Jörn Engel, Arjan van de Ven, Rik van Riel, linux-kernel On Thu, 2004-05-27 at 07:55, Linus Torvalds wrote: > "minor implementation detail"? > > You need to get to the thread info _some_ way, and you need to get to it > _fast_. There are really no sane alternatives. I certainly do not want to > play games with segments. While segments on x86 are in general to be avoided (aka the 286 segmented memory models) they can be useful for some things in the kernel. Here's a couple of examples: * dereference gs:0 to get the thread info. The first element in the structure is its linear address (ie usable for being deref'd off of DS). * use SS to enforce the stack limit. This way you'd absolutely get an exception when there was a stack overflow (underflow). SS gets reloaded on entry into the kernel and on interrupts anyway so there really shouldn't be a performance impact. I haven't looked at all the (potential) gcc implications here so this one may not be completely doable. -- TTFN - Guy ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 18:31 ` Guy Sotomayor @ 2004-05-27 19:26 ` Brian Gerst 0 siblings, 0 replies; 106+ messages in thread From: Brian Gerst @ 2004-05-27 19:26 UTC (permalink / raw) To: Guy Sotomayor Cc: Linus Torvalds, Andrea Arcangeli, Ingo Molnar, Jörn Engel, Arjan van de Ven, Rik van Riel, linux-kernel Guy Sotomayor wrote: > On Thu, 2004-05-27 at 07:55, Linus Torvalds wrote: > > >>"minor implementation detail"? >> >>You need to get to the thread info _some_ way, and you need to get to it >>_fast_. There are really no sane alternatives. I certainly do not want to >>play games with segments. > > > While segments on x86 are in general to be avoided (aka the 286 > segmented memory models) they can be useful for some things in the > kernel. > > Here's a couple of examples: > * dereference gs:0 to get the thread info. The first element in > the structure is its linear address (ie usable for being deref'd > off of DS). The only problem with using %gs as a base register is that reloading it on every entry and exit is rather expensive (GDT access and priviledge checks) compared to masking bits off %esp. x86-64 can get away with it because it has the swapgs instruction which makes it efficient to use. > * use SS to enforce the stack limit. This way you'd absolutely > get an exception when there was a stack overflow (underflow). > SS gets reloaded on entry into the kernel and on interrupts > anyway so there really shouldn't be a performance impact. I > haven't looked at all the (potential) gcc implications here so > this one may not be completely doable. Not possible. GCC completely assumes that we are working with a single flat address space. It has no concept of segmentation at all. -- Brian Gerst ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 [worst offenders] 2004-05-26 16:41 ` Jörn Engel 2004-05-27 12:45 ` Ingo Molnar @ 2004-06-01 5:56 ` Jörn Engel 2004-06-01 6:02 ` [RFC PATCH] explicitly mark recursion count Jörn Engel 1 sibling, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-01 5:56 UTC (permalink / raw) To: Arjan van de Ven Cc: Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel [-- Attachment #1: Type: text/plain, Size: 14303 bytes --] On Wed, 26 May 2004 18:41:29 +0200, Jörn Engel wrote: > > Anyway, whether we go for 4k in 2.6 or not, we should do our best to > fix bad code and I will go looking for some more so others can go and > fix some more. There's still enough horror in mainline for more than > one amusement park, we just haven't found it yet. Here's some. My tool is still buggy, so if any results don't make sense, please tell me. Full results are a bit verbose (2M), but compress quite well, so I have them attached. For the lazy, here are a few interesting things. First the recursions that shouldn't iterate too often: WARNING: recursion detected: 0 default_wake_function 36 try_to_wake_up 0 task_rq_unlock 0 preempt_schedule 68 schedule 52 load_balance 0 find_busiest_queue 0 double_lock_balance 0 __preempt_spin_lock 0 _raw_spin_lock 0 printk 0 printk 16 release_console_sem 16 __wake_up 0 __wait_queue->func WARNING: recursion detected: 12 kfree 12 cache_flusharray 20 free_block 12 slab_destroy 0 kernel_map_pages 20 change_page_attr 12 __change_page_attr 16 split_large_page 0 alloc_pages_node 24 __alloc_pages 284 try_to_free_pages 0 backing_dev_info->congested_fn 0 dm_any_congested 0 dm_table_put 0 table_destroy 0 vfree 0 __vunmap WARNING: recursion detected: 0 kernel_map_pages 20 change_page_attr 12 __change_page_attr 16 split_large_page 0 alloc_pages_node 24 __alloc_pages 284 try_to_free_pages 0 backing_dev_info->congested_fn 0 dm_any_congested 0 dm_table_put 0 table_destroy 0 vfree 0 __vunmap 0 __free_pages 0 free_hot_page 0 free_hot_cold_page WARNING: recursion detected: 0 dm_table_any_congested 0 backing_dev_info->congested_fn 0 dm_any_congested WARNING: recursion detected: 12 kmem_cache_free 12 cache_flusharray 20 free_block 12 slab_destroy WARNING: recursion detected: 68 schedule 0 finish_task_switch 0 __put_task_struct 0 free_task 12 kfree 12 cache_flusharray 20 free_block 12 slab_destroy 0 kernel_map_pages 20 change_page_attr 12 __change_page_attr 16 split_large_page 0 alloc_pages_node 24 __alloc_pages 284 try_to_free_pages 12 shrink_caches 12 shrink_zone 124 shrink_cache 176 shrink_list 0 handle_write_error 0 lock_page 72 __lock_page 0 io_schedule WARNING: recursion detected: 0 kmem_cache_alloc 16 cache_alloc_refill 36 cache_grow 0 alloc_slabmgmt WARNING: recursion detected: 24 __alloc_pages 284 try_to_free_pages 12 shrink_caches 12 shrink_zone 124 shrink_cache 176 shrink_list 0 add_to_swap 0 __add_to_swap_cache 16 radix_tree_insert 0 radix_tree_node_alloc 0 kmem_cache_alloc 0 kmem_cache_alloc 16 cache_alloc_refill 36 cache_grow 0 kmem_getpages 0 __get_free_pages 0 alloc_pages_node WARNING: recursion detected: 28 qla2x00_handle_port_rscn 28 qla2x00_send_login_iocb 0 qla2x00_issue_marker 28 qla2x00_marker 0 __qla2x00_marker 24 qla2x00_req_pkt 0 qla2x00_poll 28 qla2x00_intr_handler 100 qla2x00_async_event WARNING: recursion detected: 0 qla2x00_process_iodesc 28 qla2x00_handle_port_rscn 28 qla2x00_send_login_iocb 0 qla2x00_issue_marker 28 qla2x00_marker 0 __qla2x00_marker 24 qla2x00_req_pkt 0 qla2x00_poll 28 qla2x00_intr_handler 100 qla2x00_async_event 0 qla2x00_process_response_queue WARNING: recursion detected: 28 qla2x00_marker 0 __qla2x00_marker 24 qla2x00_req_pkt 0 qla2x00_poll 28 qla2x00_intr_handler 88 qla2x00_next 32 qla2x00_start_scsi WARNING: recursion detected: 92 qla2x00_mailbox_command 40 qla2x00_abort_isp 24 qla2x00_restart_isp 24 qla2x00_setup_chip 96 qla2x00_verify_checksum WARNING: recursion detected: 96 qla2x00_issue_iocb 92 qla2x00_mailbox_command 40 qla2x00_abort_isp 24 qla2x00_restart_isp 0 qla2x00_configure_loop 80 qla2x00_configure_fabric 0 qla2x00_rff_id WARNING: recursion detected: 72 qla2x00_rsnn_nn 96 qla2x00_issue_iocb 92 qla2x00_mailbox_command 40 qla2x00_abort_isp 24 qla2x00_restart_isp 0 qla2x00_configure_loop 80 qla2x00_configure_fabric WARNING: recursion detected: 16 acpi_ut_remove_reference 24 acpi_ut_update_object_reference 16 acpi_ut_update_ref_count 16 acpi_ut_delete_internal_obj WARNING: recursion detected: 32 acpi_ex_field_datum_io 76 acpi_ex_insert_into_field 52 acpi_ex_write_with_update_rule WARNING: recursion detected: 72 acpi_ex_extract_from_field 32 acpi_ex_field_datum_io WARNING: recursion detected: 32 acpi_ex_read_data_from_field 72 acpi_ex_extract_from_field 32 acpi_ex_field_datum_io 32 acpi_ex_access_region 20 acpi_ex_setup_region 16 acpi_ds_get_region_arguments 28 acpi_ds_execute_arguments 24 acpi_ps_parse_aml 36 acpi_ps_parse_loop 0 acpi_walk_state->ascending_callback 24 acpi_ds_exec_end_op 40 acpi_ex_resolve_operands 20 acpi_ex_resolve_to_value 28 acpi_ex_resolve_object_to_value WARNING: recursion detected: 28 acpi_ds_execute_arguments 24 acpi_ps_parse_aml 36 acpi_ps_parse_loop 0 acpi_walk_state->ascending_callback 24 acpi_ds_exec_end_op 40 acpi_ex_resolve_operands 20 acpi_ex_resolve_to_value 28 acpi_ex_resolve_object_to_value 16 acpi_ds_get_package_arguments WARNING: recursion detected: 32 acpi_ex_resolve_node_to_value 32 acpi_ex_read_data_from_field 72 acpi_ex_extract_from_field 32 acpi_ex_field_datum_io 32 acpi_ex_access_region 20 acpi_ex_setup_region 16 acpi_ds_get_region_arguments 28 acpi_ds_execute_arguments 24 acpi_ps_parse_aml 36 acpi_ps_parse_loop 0 acpi_walk_state->ascending_callback 24 acpi_ds_exec_end_op 40 acpi_ex_resolve_operands 20 acpi_ex_resolve_to_value WARNING: recursion detected: 28 acpi_ns_evaluate_by_handle 20 acpi_ns_get_object_value 32 acpi_ex_resolve_node_to_value 32 acpi_ex_read_data_from_field 72 acpi_ex_extract_from_field 32 acpi_ex_field_datum_io 32 acpi_ex_access_region 20 acpi_ex_setup_region 16 acpi_ds_get_region_arguments 28 acpi_ds_execute_arguments 24 acpi_ps_parse_aml 36 acpi_ps_parse_loop 0 acpi_walk_state->ascending_callback 24 acpi_ds_exec_end_op 32 acpi_ds_load2_end_op 20 acpi_ex_create_table_region 28 acpi_ev_initialize_region 36 acpi_ev_execute_reg_method WARNING: recursion detected: 24 acpi_ps_parse_aml 36 acpi_ds_call_control_method There are more, but this shows a few ugly spots. It also shows bugs in my tool, I'll have to look into those. Next month. Now some of the top stack killers: stackframes for call path too long (3136): size function 0 radeonfb_pci_resume 2576 radeonfb_set_par 0 preempt_schedule 68 schedule 0 __put_task_struct 0 audit_free 0 audit_log_end 12 audit_log_end_fast 12 netlink_unicast 76 netlink_attachskb 0 __kfree_skb 0 ip_conntrack_put 0 ip_conntrack_put 12 kfree 0 kernel_map_pages 20 change_page_attr 24 __alloc_pages 284 try_to_free_pages 0 out_of_memory 0 mmput 0 exit_aio 52 aio_cancel_all 0 list_kiocb stackframes for call path too long (3056): size function 720 ncp_ioctl 616 ncp_conn_logged_in 24 ncp_lookup_volume 0 ncp_request2 164 sock_sendmsg 0 wait_on_sync_kiocb 68 schedule 0 __put_task_struct 0 audit_free 0 audit_log_end 12 audit_log_end_fast 12 netlink_unicast 76 netlink_attachskb 0 __kfree_skb 0 ip_conntrack_put 0 ip_conntrack_put 12 kfree 0 kernel_map_pages 20 change_page_attr 24 __alloc_pages 284 try_to_free_pages 0 out_of_memory 0 mmput 0 exit_aio 0 __put_ioctx 16 do_munmap 0 fput 0 __fput 0 locks_remove_flock 0 panic 0 sys_sync 0 sync_inodes 308 sync_inodes_sb 0 do_writepages 128 mpage_writepages 4 write_boundary_block 0 ll_rw_block 28 submit_bh 0 bio_alloc 88 mempool_alloc 256 wakeup_bdflush 20 pdflush_operation 0 printk 16 release_console_sem 16 __wake_up 0 printk 0 vscnprintf 32 vsnprintf 112 number stackframes for call path too long (3024): size function 0 acpi_device_ops->bind 292 acpi_pci_bind 292 acpi_pci_irq_add_prt 20 acpi_get_irq_routing_table 20 acpi_rs_get_prt_method_data 24 acpi_ut_evaluate_object 32 acpi_ns_evaluate_relative 28 acpi_ns_evaluate_by_handle 20 acpi_ns_get_object_value 32 acpi_ex_resolve_node_to_value 32 acpi_ex_read_data_from_field 72 acpi_ex_extract_from_field 32 acpi_ex_field_datum_io 32 acpi_ex_access_region 20 acpi_ex_setup_region 16 acpi_ds_get_region_arguments 28 acpi_ds_execute_arguments 24 acpi_ps_parse_aml 36 acpi_ds_call_control_method 24 acpi_ps_parse_aml 36 acpi_ps_parse_loop 24 acpi_ds_exec_end_op 32 acpi_ds_load2_end_op 20 acpi_ex_create_table_region 24 acpi_tb_find_table 224 acpi_get_firmware_table 68 acpi_tb_get_table 24 acpi_tb_get_table_body 36 acpi_tb_table_override 36 acpi_tb_get_this_table 0 acpi_os_map_memory 0 __ioremap 0 __pmd_alloc 0 preempt_schedule 68 schedule 0 __put_task_struct 0 audit_free 0 audit_log_end 12 audit_log_end_fast 12 netlink_unicast 76 netlink_attachskb 0 __kfree_skb 0 ip_conntrack_put 0 ip_conntrack_put 12 kfree 0 kernel_map_pages 20 change_page_attr 24 __alloc_pages 284 try_to_free_pages 0 out_of_memory 0 mmput 0 exit_aio 0 __put_ioctx 16 do_munmap 0 fput 0 __fput 0 locks_remove_flock 0 panic 0 sys_sync 0 sync_inodes 308 sync_inodes_sb 0 do_writepages 128 mpage_writepages 4 write_boundary_block 0 ll_rw_block 28 submit_bh 0 bio_alloc 88 mempool_alloc 256 wakeup_bdflush 20 pdflush_operation 0 printk 0 preempt_schedule 68 schedule stackframes for call path too long (3104): size function 0 client_reg_t->event_handler 1168 ide_config 12 ide_register_hw 1596 ide_unregister 0 device_unregister 0 device_del 0 kobject_del 0 kobject_hotplug 132 call_usermodehelper 80 wait_for_completion 68 schedule 0 __put_task_struct 0 audit_free 16 audit_filter_syscall 32 audit_filter_rules stackframes for call path too long (3144): size function 148 generic_ide_ioctl 12 ide_register_hw 1596 ide_unregister 0 device_unregister 0 device_del 0 kobject_del 0 kobject_hotplug 132 call_usermodehelper 80 wait_for_completion 68 schedule 0 __put_task_struct 0 audit_free 0 audit_log_end 12 audit_log_end_fast 12 netlink_unicast 76 netlink_attachskb 0 __kfree_skb 0 ip_conntrack_put 0 ip_conntrack_put 12 kfree 0 kernel_map_pages 20 change_page_attr 24 __alloc_pages 284 try_to_free_pages 0 out_of_memory 0 mmput 0 exit_aio 0 __put_ioctx 16 do_munmap 0 fput 0 __fput 0 locks_remove_flock 0 panic 0 sys_sync 0 sync_inodes 308 sync_inodes_sb 0 do_writepages 128 mpage_writepages 204 mpage_writepage 12 mpage_alloc 0 bio_alloc tackframes for call path too long (3004): size function 0 ____FAKE.Name.Chip.stat.Regi.LILP.Opti.high.lowe->ProcessIMQEntry 2076 CpqTsProcessIMQEntry 12 cpqfcTSCompleteExchange 12 kfree 0 kernel_map_pages 20 change_page_attr 24 __alloc_pages 284 try_to_free_pages 0 out_of_memory 0 mmput 0 exit_aio 0 __put_ioctx 16 do_munmap 0 fput 0 __fput 0 locks_remove_flock 0 panic 0 sys_sync 0 sync_inodes 308 sync_inodes_sb 0 do_writepages 128 mpage_writepages 4 write_boundary_block 0 ll_rw_block 28 submit_bh 36 submit_bio 56 generic_make_request 0 bdev_get_queue Not too many above 3k and none above 4k. Actually, intermezzo had quite a few paths that went above 4k, but that one's gone. So effectively, it comes down to the recursive paths. Unless someone comes up with a semantical parser that can figure out the maximum number of iterations, we have to look at them manually. Jörn -- My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed. -- Edsger W. Dijkstra [-- Attachment #2: data.nointermezzo.cs2.bz2 --] [-- Type: text/plain, Size: 27812 bytes --] BZh91AY&SY8yèS\x02î7ßx\x10@c\x7fñ?ïþÀ¿ÿÿð`f\x1cà@\0ø@\x17Ò\x1a\0 \0\0>ø¨DQ@Ý\0^Ãl\x16`fÛ\0 \0H\x05\0\x01-\x1aP*\0h\x1e\0\0Ð\0\0\x03 :\0\0\0\04$( =\04õ¶4\x1c\0)\f\x10\0Ð\x05 Ð\x01ªª*Ð\0Ò\x05\x14\x1aÐ3`:\0\0P\0\0\x05h\x17e)\x06T\x01GAÒ@\x0e¨d\x1a4\0VØ{Ù@éAè\x02%p\0u \x1a [\x1a\x15@(¡Ñ\x1a\x04d&D\x1a\0\0\0\0\0\x1a Ò\x10jmPõ\r\0\0\04\0\bª\x7fïUTh\r\r4\x06@42\0\0Ð\0\x01'ª¥14zh\x1a\0\0Ð\0\0\0\x13Ti¤Ñ¤ÐÒ\x1a\x0fP\0õ\r=CAê\0)H\b\x04\b Â0$ôÒ3\x13SƨóZµWËj³Uk[j¿ßó\x02\x05\0\x15@\x01\x1fö¿Ç\ç.\0\0\0!\0\x04(DU\b P\x04\0\x12\x11\0\0\0\0\0\0\0$Q\b\0!\x01"\b\0\0*\x10$P\x04$P\0\0\0*Õ*I\x01 \x10\x12\x10\x12\0I \x12\x04 \x10\x04 \x04HH\0\0\x04\x04H\0I$$!! B\x01 \x01\b\0\0\x12\0\x12\x12\x12\x04 \x12HH\x10\x04\x02\0\x04!$B\x12\x12\x10\x02\x01\x02\0\0\0\0\x04 H@\0\0\0\x12\x01\b\0\0\0\0\x1fSVS©[ø:ýöôðÿ·\x0eIÿh\x7fõ\x11¯úe\x0fúÖ\a<Â\x7f÷D4jD\x1dÆRTÓvÓÒÿðòsÃþrU5¬b¦\x7fýwI\x1c¸qÝv|TéwãG)#«\x18c,e°É&L2dÌÉ*dÊdɶH©©&L2dÉ&m2LÌÌÌÌ̦S333)Êe2!LÂS)ÊfdÉLÍ2L2dÉ&L2dÉ\x11U½/^ÖêÕâÛJ±ÁåjjJ¦røgr¶×<꯷n*.ÓÍâòÒ\x05ËNn¹¼ô\x1a\x7f\x05[çå[÷c®uxÿ\x1dúc¯iöÝÝ\x1eêK·ZÒÍ}Z\x05¶\få5»Òß\x16Å*»ÀMúfê_ú[ïéUg\x7f 9&ôïßÉÆ:`ÈU^Qè¸ÑƱֽmù¥Æ½®\x0eËÙ\x1c6ØÃfxXâί\x1añm®;DV}gÅø·5óN=xãѧ;Ó;(ü çÓ\x0f¾._í2ñäÓê+*ï{yò×k\x1d0vÆ1¿f·Vtֵƶ£8©qªç<çÑ S\f«%ª"$$$I""""""""I$I",I$ÚÜZ׺õn·nÒU=Ö7§ÓÕÖ#Zó·yéíþg¿·Oóõ¨õoD].N¥µu>ïÌ|/Åpè_\r'ÊÇ.+é§³£iÕôËLcèêå²½;Îîèñйé\x7foÛÄO§ªê;\x1dM=_ømÄá\x1ez0+2½gµÇÀª\x1c¤\x17Ù=ÏMÙÒzV\Y<W»£M¼-ÊôôNf.ü:8jii¡ÃwU4»»;bÜcùë»~î/gìôW_,´íw\x7fÝ<¹q%S*3ßSÏLMB/¼äNLj²pÂc&6ôXøÊíÑk^\x1eÏ&;|½§Yôøz¿bâh~«of^[apÆUçÕ§\x0f§Xæ½×ѧVγÔSyCkn\v×cÊ#\x0eO3àävô6dYïw@}é\x14Üò±à:®½ÏÃë=g[Þéöw¹M/K¢({tÆftá×:çFFÝ\x19ëÍ~[¦-\v¦ßß³« uKt9ªÖL|°èû4ºÿ\x0f\x1aáèÓ´îx~¾Kðøz8ï·âpòþ\x1ag£µ.TçÝÕ5ÑåÑ¿\vQå¿ÝϽEߤNÒU9kF5Ýó\x1aLpÛ8Þ=s¶8zk9"\fo·gwL¯¥3\x0e^¯³\fêâ¾è]¶ü<úsÝîá¡\x16.'fîb+¿YÝÕïÉ©T]dèiìn\x1cpâÛ\x7fS»ÞB¯Wìêc®¾tré¯L{~Ú¾¹áðñÝðòÓO/wCØð¯\x0ee*½C±cÒeSÒǤ(º¼yz´:²Î«ZÓ\x1eμh¤]Ù\K¬·íTXâ4³öIìK¤õmÓ×4p<°Ó\x15ö\x1cóßËѾ¯w\x16óß˪uw<¿wu÷öÿSǧVÞí·<q¿[Ø;W¤êåÁo~¸q½fûcq¶qìÛ¹ÿ Ю=\x1fjü<\x1e=\vOC²iÆëÚµÙå\x1f\x1eNÌ<|¿/ºõz½½~Já}^[¼g©ñû»§ ùpðåöyháë:6|<;=aTêeJ¦Rty®\fªz®ý\x7f]^&=ËÑiÆV<=^§òz¾<û³ÝÝÈ==Ó'Øvp¦G³·Ùv9Þ1Ãô{Uv4÷\x11§u§w ;¼>õÁ¥ÛÎöÖpÚµ(¯k\x18\x02¶úUÍâW®9·çB«¥Ý\x1c¿wHà}d²ï©FãÌêy ð\òþáF´Úw=¢·dfn|´¶=x5AL¨©ÄÔÏ S'³§[®äijFxç\x15ÍmMIs\x1cPWNøÆ¸pr1KJ\x10\a¡Í#ÕyFyK¢.ëë@¨&Wo> ¬ò\x10\x1e\x7fCârd\x18\x05P \x11a<45k®ðæ¼êðªn³ü´V¥)K¸zÏ]F§é\x1e÷o\x16ñð\x1a?LP¨³\x10¸>±¯\x11½ðÇ6Ø×W=¸º<Õ2 ©¤bk7®;x\x14;v|Õ]êú»]?c®\x05\x0eb*ø]d«µ\>½gQt\fw·àí$f{Ï\a¤úÏ \x1fÙ~Yù°û®ª9[i.Òw\x11Ñz'ÊzVD.Æ{Ñ\x1càt65¡¦}åj¾\x1arÏG¶qÃÜÖ"cvmrͪrdÊqu\x0e·wA\x16ï¾Z`4^ CV3gi5èNEFÃEÁæ\x11¶ýH;\XìNm9Ûm¬µz;½ùí>§:º3ãÙÀ¼íxîÅI¦»Ã\x17eW×½9Ag¦ß^ÏÆµ|º|(z>ýCùûV§0Ä`üc=\x139N\rL,wåh\x18ªBì\x17]`¹A\x17|Ûck vnnâí[^KÓ}Cô6\x1dz]ǤÓÔ1\x18>ÛÇb¬dÉÍïÖ æ\x18_]FsW¹\x03d:\ÄÕwÔ\x12G\vT±ç*å\x04y\a]Ý|&$\x120' A\x15æX8]Õ5r¾Ì³7Ë\x0f\=ªÀçJ!aX4.\x1f²úyÊÝBG'0cÒÜü¯¤ù29ÕèÄ\x01\x0fI¯\x1cÈUKã»z¯¢ø yZß¾éç¯*¯}[«¡\x06A\x1d·Â<¼ùypøm÷«\x1aÒ\x15«0ÃvïÂD1Æ5TÈ¿.\x05Äj÷\x19,ÎFFmÝU]QY!:phܸ+ìû\x1có¿\x7f\x11-·î!+\x126üýüFLÜÌ?.õîCùFÌ£.ª\x18\x1aó㩼7Ìz>ÿtD¥þ¥ D\x7f³ü1þ|ÐQ¾Æ[ÌÊ&¬£I\x15Y\x06#&Ú$Vo$\vXªÚ±Sÿt3x¡KQ4Ù®\0±Mäpphã5«µ\x19mƵ§\x13 ª\x18ZÓRµ0·%S2À\x06ôÑ^[\x18ÆÕ4´°j#*Ì\x02\x1abbÀÓ"ÖÛ¶\x12FD¸ß\x19¾\x18«X5P,ââ¨Ù*ãK#LLÁ\x15f^[Ö¸×\r·770oZN2+\x11QÅPhavRg9\ ÞlrD[yÜâäw9Òsu[\v\hµº·\x11Æðo\x15¥k\x01®5½ëÛm\x14²\rªAdá¶µ\x19)W\f# ¥%¡\vKK[Ò\x10ÚáRáJ-ï*¸H,´µ¶Ó·mné\x1a¹¹¹Ú%£µ Vµ½«y¦7Ìe\x10Û\x12)¥\x16e\x13¦¶jp³{o\kIª¤Ì,ʵ\x11¢4Òʦ*ã\x17\f\x1aÇ\x12ÊÒ\x14±q\Fa4W\x01 nYñvkv´ª\x17 U®ùÇ:êÜËjü©q«u¬+\x14£\x1a¤©¢BÄ Ç%jÕp¹&ð¬jD\x1a6HÄ3"Û\x15¡iF#\êm\x06µHÚÅ,¢\x19\x1cJ+\r±o&¶Óy\x0e\x15ûÄC½(üWç% he \x1aÊ f@¿R\x06* p¥*À |¥X.2̨J³,Ëñ¦-df\x12I}\x14RÕZ´·X\x18Ê\x11$ÈY4Ò0ÒidBIRÍ"&-&³d%\x14c^[\x18ǹmJ8ÄT¬Å æ"\x19pJÉ)MåE\x0f\x1ctý~?]ý=wÞJUýt÷ö÷ÓYòÖò76³ß\x13Y½³\x1a²Åµ®5¼\x1a·Î ¶¬pàÕi¾0Æî3W\x068Û{¦V¬6ÃMå4Ë6Åâ[Ô¸Î8¸k\x1aÐàÙ¼«f²q»MâÖZÕ»\x198ãVôÖe\kx5o7^[K-Î#qi8ânuÆ·nºs2În\x16Ýó¥\Ý:ÕÖ»L|ÅsÆ'\x19ÆìËLdÍ5\x1cëyVÍÆç\x04ÛFÖ8pj´ß\x18cw\x19«\x1cm½Ó+JÖ^[a¦òebÌq-ê\g\x1c\5hq[àÝk\x17^[jÞMcZm.8Ózµ®5¼\x1a·Î%ç\x11¸ò ÷¾\x1fyèþªjr¶2Ç,icïtãK \x7f\x06W.\x1aûÝ\x0e¶Ú\x7f\fZ7ei\vh_ü¡\x7fÕ\vª\x16пt/Ý\výÂúbKîZÊ_V EV©+l\x12ÝD\x18²ä\x1aXK"X$³5) ̨ÌQ+\x04Á\x0f¾ Þ$jÅ"\3ï\f\x13\x12%Dª±k\x12\x11¤¨Å\x125mÃb©Ã Ùe\x17\x19)5¶£9ÂhE,Àd+L\x15s\x118Äq³\x06ãSG1¥m\pÌÜ\.^[8MÍ\x1aM\x1cFAµqÃ2cq\x7f¯øbM\x06Q`dr\x0føIþuLÓ\x18bÉ;B`æ»NÕjolâÀd¤b¥Mp³Î´oZ[Ö%^[q7n59Êý±pãFÆ\fÉί¶¿ßÞüZG{ZëçøpTæÕ¤\x7f| 9ÈÌ \fX±TwÕ<]mÿÅÐä¹¹¹å×¢&\x19\x11LÿPÔFÌ\x19®<afS£|\rðòRG\x19K1*;´^Z51ai£Ûª\x1aL\fÏVøòö?¡æ\x7fïúæ>Z~/Y}shѽ±U5bØZÞ±¡3\x19Uf¼÷$´¬^[ÄÖ\x1aÊÛík%p2½j\r,e%`kMã×\rZLÔÝnÔáÕÆdÇ\ræ6pÞ8MpÕ¤ÍMÖíN\x19\fLp÷\x02e\x15=5\x1f8Òdk\x1eð!G¹þòÔxº$:äL&Vwï¯\x12ØÖo[*ñD0ðROÓCf+´\x14Â¥uµ}øÕkC¤:+íïý?»÷Ö÷5gæ_ÇÜgß\væ½X ;2y1\x03÷ìù\x0fh\x15|£Pê´Ô4º";b§j1\f\x06KXF(éZÀç\x12¦\x1f\vö©ùlÚÝNaÐ\x0fáQ¤\x06ð0öoCi=\x17\x14zö=s\x14 Ìß\x1dðµUzBc|jµèÖ\x12 þe\x15ÆøÖL¸VùfMÍêÇ\a\x16nZâ8áZÛ2l5·Ã2l6oTÎ88³rÔ{\x0flDýÓ ¸Ú½R\x7f«dç&¶®·6Ã)¨Î3aÂî¨Í*9µÉâ|OSSGGE×çÅÜ\rí§IÛÁpâo;\x15t\x15âcµ½³v¶ÍØ\x14\x11ý»Di©·\x0eò.pÌkF`ÁYH«2®ôÖp¥cXÎ\x12\vKN5N\x1cQ%§\x19\x10ÕV´±F#\bajp¤8k2cc3\x18ÖÚ[eN4e]k¼×Z£K»ïtiwWj%´¶ÊhÈÜâî¹:\x1d«RѪÑt`¿'.YÙvÎCX\x15¤6pÂ)xvK\x1c»?õËÑQþÎ\x17ZÆ;®§¥Ókfîbq Éy"WHä¤\x03KÂëåq:]y\x1cò¶\x1f»ÐáG0Æ<búÖc2ÍìBÚD· ¬·*&[oRQØ·jÓF£Kq¥î&\x1ajÖ8á£ZâZܵ¥î&\x1ajÖ8á£Z<"x<qåäÞöì8vTtëÄ;#¥²ÈéÂBé£^[hÆ\x1fÙÁÐèuÑ£wBm#¥ÚkÌu[Ü\x7f¹©Øê÷ÄCÚ@ÿî\x10ÿ#þÿ÷ÿþh>ØÿÕJ³îþÛþ²ý\x7f¤½/õqûn#ÒÊý½»ÿÑ\v¯DGüüièÿÖký?ô\x7fÙ\vð¿ð\x7f'ç¯êWå*z½ÿñR®oùûoÎ? Ãü¿H\Õ\vÒï\bwwv#õܯò}ê?/üú?ÒÅü¡|¡}|´¸¬?ù9Bÿ¾ËH_d/ Ù^o¯Úÿº\x16$è¿Ë\x7fî¯\x10\x7fD/\x0f?ø\x7fTû_o;v8÷~Õ\x1düÿý¨Þ\x7fQò¿äª<b+ÃËþìýн¿\x03û0ÿ)ÿÚÃ÷rû^[+ú¾u=×O\x04/YHbª0XÃ\x19EJÆ \x1a¼¿éx{ºñÙ£ò»Ñ²è?êü]\fcþLa\x7fûwÐr8]«©·þÍG\x1e\x19!WWû_w9\buZ\x1aOÿ-[\x18ÝEÙ¶ÙcĤ\x1fþ`Ðÿïjv^\x15î¿ÅýtÙ9tnz±»Ñ\½\r%Ý8/{;=ßw+RU8I\a\x17íåo3§ãý>?ÿ;~ö<îG\x7fJ!ÄüoÓq\x15·ÃQ_s\x15ÊÏ|ff´Ö8á råiÉÎH½u··âóÜÑ \x7fâ"º\x1a[4uiÝÁǵuƺLDO\x0e¾Ý?Üètºû:º^[®Bw=Ö¡<\x11Ä㯼9ÌÍýwq:SÜ<]ôiu[ÑîWIXÄÁªt+±ü.W^§ø'SòïÒöfÏKÃ\x19w[ÓõÓë·×¯¿gÃ\x1dµü²¾/YÍêÆXÇ\b)9)9\f\x10êZ\x03t\x0fY\x04ÈÀàGËçüùë 'ÛzÏ[éþî/ô:\x17Ùíó¿ñéITáíû£òð¤½\x12÷<óû\x1e:÷Æ1íõô½xxE.="®8\x11H\vÌÁàMf\x1c´Õ\b|\x04\x01@¶\x06Dþ7\x11Ø©úòßf{¸þuöp¿ÃÞLLNúªÓÏWNçï>¿«ûÝ<u;ûkÎf2?3\f°Ã\x17·>ÿ=oc³®\x7f\x7fáÜ\x7fLãÙÔëôî\x1f¾Ñ§¯jßáû¿3|1æN¿*¿Ë^[©Äùõ?¨\x7f\x1fG\fç¯ôÎϦ¯\x0fÛ¦ûüyÖÿn°g~Ç\f>º×ðºp¶çO¶¬g\x1fsMnJ§Ã_þºÎ®¸³\x1d¾½nýkÂù4®Õ_/2¨¹5iS÷vw¼1¼ôwyKÛÙâÇç<öâVU\x1fÃ\x15ÉTÒàÛͳÌéáÃ÷^[öüô{©:/§O§\x05~Ø[F\fÇÎBªç\x19©Ñö³3ã\x1d]Íö7q{¼=ïÚ£«åñcâÛÕáùïè¹rÛ(éóñíó:;:y»øù:þ±®6õuõ_\x7f^ïRFR;ô_·ÞïypÓ*:Ä}ÞîÎõ;9vúÕ~?¯/Nµðý\x1d³Ùû½}ÿ\x0eÄÚøéGHüL[t\¿\vÏã¿ÆygÓ¡Ù>q^þ\x1ck=9Þk÷<RTß iRTþüjã1\x0fo©×û=4gë3Us\0\0\a-sòyòêÖýïÃéûS4îòå½û«÷ëÏ\x7f»Ãèãáý\x1etááËM\x0eÇñ÷¶~½¾¾}¾'\x1dª~ǯʸaåóúý»³ã¿Ëw¤ÉñᾯÝý1ÕÖ»}j6í¦\x1e\x15Øç§¥ç¹Ôòü/Æ-\x18ÅÚöhÍækyÔyéitð\x15SWÚyCáÊÅlê>oÛówK¥K^9xðø=bA¯K¥aäþí²\x7f°®®¼OF4þÛ~\x19û²¼}f\x1ai§âR«gÍ1Ã\x1e¾\x7fÏ]«ÓòüOÓ×\x7f\x1aÖ»»N\x11¿ã\x1d½]¨*üj¼O/Äê½_§.Ñóß»;ók8wñ?õܶóZsÉëæ}W÷|íí'\wñp¬ÉJ¯gðß\x0fWÃÙÕ\\x17'IëßnÒ\aÒám¦aû¼On;ýuûéøõkÏfêíRv}Ùn¶\x1cbjpÜÓ³ùno]Úuc«Ìà¤\x1fuö}ñ;&\x1eÑZy4^$AÚ'5\x10b5,§8ªªÑìÑÃ\x05³:ªèt0¤úü<FzÎse$ôèçíwcwC£ãåÿ¡ùþæ0ùW¬½ç\x1f\x1fê÷_¾Áñ¦×pº¿øû=÷0®¬{8}dþ\x7fxõpöuGñ?SÙîôITú¯ ³|ßÇ/\ry «.ÃTªú?[¼tuïÇO¥idıË_ùUiu?'G»¢y5¼Ïð¼?Wé»±þF¾ÏükOÖ÷õ?©åì1¥×KCkU8°¯\x1c¾Ï§çè~\x1fÇg©öûÏgW×ë:±êÎÔ×Ã\x7f+ê¹\x132SÚõýÜËðõÌ|'_O_ÖÑÉæúåI=çWïÏU{6ÅcÞûÕ;:ºþ=\x12+þ}}±¯\x1c]\f}WZJ~_\x7fÕòöüÖaY)&@ÅGÁX\x7f%¦¶\fc\vO7\x1eû\x1e\x17VÔ[åÇ׺\x1e¦{äû#=úfß9ÌÌÌÏ_kFÊIÂøÅÕçñ|zûüvÒ¾,]Ì\x19ééï·îîõìO>\x13KÂD\x1d\x02q4\x06C\x01 ÄÅ@b{O4DD¨PÄ(F=)>ùX°Æ0±æwXáïê¾|Ü#ôÅÓ\x1dfÍw]LÞz¹t©^¦Î×n£OhÒË\x18êìèëLL4åòm\r;ÉTþ¬á?^\x1ez1ä\x17Ã'Ã\x1eËõÐl\x1a¬,bÊÙjk\x19 ºôÇde*§Àw×Xô\x1d®333333;Î+H}×ÊÒ8êÚé~ÓÀÓ»tª\x1aiÓL?]\x03m¶ÏÕdÝ*ûüÜæ´ã:t'·\a%òë:¬ÛõÊôÇ\x0f-;?£GÆß{:\x0eîòU9óøôêé×ûPõNÅ÷th¾vDwÖ¥)ìxëN¾g:Ï´GËîz¯W¯áé;ãêô¯<WÑwLO\x0eâzðÓOÛFDÔ¯~ÓìÔ÷|×+¯k¾æ vx¥^ ØL\x7fOë¾Æµ§¥û½}/^6÷zW:Öm®³¯>ÓæÇðþ\x18iür_ 1\x14ÊTÆ*\a¯<Lvf8~åe6Æ9ÏÉËMc\x1acT´Ò4iÎ4ü)kìü-\_\r)Ôa¾ü;»\x11·\x0f³M:1ÜÊã%4b§XÅÒõ±åýaËS«\x1f¦C«Ñø 2¦¤ª~ÛÆõ4Þ7ITÙn·HSÄ«\f\x18¹¦Ñ²Õ\x18O\x18Ìdtn×*úÆe;2k1&\fe+LkA©8\x1d\x15:°bÅCTþ\x1dÔÔê¹åENÕÒ{àFa3\x19c\x15öÅ\bv²$)ÉÝÊõ´¬\x17Ù5;ÈUá·×àCã ®¿lUi\x02lõû¾Ç.\x1frVÖ°Èú1ð£jvcàí=Ýîã8hêÓ%$fGKË\x19ÿ¾¦1Á ì`ã\x06¦_Tºz¼Ç_Ûþ(ZHYç»ÍÕ;ã>ê4ë=+3°\x1d¾ôW\x16$Ê(>*Qu ëÄæGdê]¥îÄL02d±2¬°ÊÊeAþ°ºØ6É,:rôÆUà<\x14´ç\x1eV üÙßõù*9´ê/\x0554ý½á9\x1fhôG·´»xTʺÔ̦1½u<Q+ªmXù|\x1eI}0Q¬x²¼¤í\x1920ªÊ0Q;qâï}É\x05ðH.·\x15ßWd/XuÆûû3õÒ,ÌÆ\x14ïÿ¦Ýø4k&òo\x15{Ùc&c1úêèáÊ!¡#îõ=TÄ¡_Ã\aÖÒp CàªÑ¯àõaôðUÕËë.l¬²nO\x17Ôi\x14_\vRU4c³Uºþý\k]7M¾Ñï¦ÜWÛ«´ÞãMíÐlߦäÙÃ\x1eí½{frrÛgN\x1f IK¡Ç\x05$Üàø2Éeä±eauhWµÁ²ô¼kü31zÀÕ"º\x18ÏÚdÔVF\x18¡_Z*i»\x04²ø\x03æé®c\x15yÃ2=ÖWÇ7D\x03}ÙzDø9»ð]¾vZdzaù¿4:ÖL±¤`0\x17a|el\x19]!\x0fvRïÙ¤ èȰ\x18ûrj\x15´eÃ5aº*RÆ0\x0e1\x19u\a$¨\x14\x04T\x16Ëeù"¤´:k|·Ç\x1co©Ýþ¹×ñDYw{-GËËLüµuyu/Û£LF Tã\a\x15.Ó\\x1c®©Po߬êp8££¡88îÚj\Î\rÛp.+GdO¡XA\x18°Er§Ãn"!\x16S)\x19.vØÆÙýï^[8ë©ÒJ¦£¬ízS]\x177\x19#ëMïZc1Q4ÂÜDmË\x1eN.3\x0eÁ}2²Ë\x19í\x0eÙVe_=\x19î2½\f°1trJ5ø.\x14Ó\x15r"sÍ\x1e«6,º>\rÉTé5_ÃëLÁI¬¿\x06_'^Ã\x1eäwLW\x18±±\x19 ZeªÚÜç$÷5Å%\x1dMs*LL5¦2¦f\x1amR\x05¬±Ëli0IV%\x13V¯6¯)³4dU\x1a_cgªhþ\x19;ÌwHþx©ñúµ/++ã\x11<:±XÅÃonÆk\vw: ©ÂÅA 0?¤Á-ÝôD§\x11ëOLË&YÄðsÍåDìíeÄâãm3ðS\x19aaßJNÓ\x12?&B^[0²Bu\r,\x1e¯vÎ\x0fWyÒ¯ÉÁn½*ÒÒõË5=½ôéY%SÌGÜó×yLøü±Å±£ïªÖî[õçzǶ+\x18±Â\x12Ûd22\x04B zÃá¶K\x05\x14à\x19\b«\x12\x10`À X@b1Q¡X@ÿÏ|ÔsÏB¸ê^ÈÛÆ\x16¯¶<K£öçÑ«'ÙÏ\x16\aÙ«NZÆT\v©¨ZYñËvm^ó\x1a]n:îáë£ÕªóWS\x1cÌo\rìáY]Ý~yxLÆPÁ*º½í^[ÃkÔ}6\x1aãÙ¨Û}0õö\Ñ+kle}ݧì8+\x1dôuRºIt`ı^[e¼;*·\x11ºøc=³4/]\at\x01·«¦\x0eU*\a\©r±ãR\x06¿«F}þÓsð¸Ùn#2#'\x0eõêÕ\x10ø¨\x1dãË\¦tåØjâªPõ'ûºv¬Etèß´§Sn\x19\x15c(ªe~\x1e\x1a³7c\x18Á\f¿\x16\aØI¢¾,\My·¨âÇ\x0f9óv®ÊH½êÇ'K\x18Áæg«\\x1e\x14ïüãØÁÍ/pá§»¦=Þé#JZ¨!@óÏÀhN\x18ÈÀIb¨EX\x11?iàñ1¬WU\x1cý´([^KE¹ë¢wé(.nnÙÞºSm¤\x17×K\x10Øë]·:NÌ«£'F5Ã\x1eξÎ6î;9zv\r×lféÕÐéÚª\a\x13®8w]^[+¼\x1ez³TæÇvïO\x0e'#¡å^áçW5¾ ÚdìØ°,\x118^[8\x04P c\x17¦Îù\x12<ö\mq\x03´o¨\x1c@ËìTæU\x0eÑvá\x12ñ7GUÀE¡\x04H$\x12;)]u]fRy[\b»\f,rÐ\f\b\f\x10v8ç)7ýC±Á¦¬=sÎa§Ç78q\x1cm \x1d\vY<\x14V\x11$é{ÄCEÏ÷BäîíÖÝFÛé«¢î Ôøë)\x1e\x1f èiÔzên½½ÊÜÊ|as6æ¾\astpêÔ¹äM³¿IEiqñ¤'NzuѼzqin8yxãº÷ìân±ÐãYá\x1cöµÇ~5¿\x19háÜiÒU\x17¢¦¬Hrc4ý®Í¸]wfHó>]Z.ìc\x1680êμ2a¦¼^[6rÆ«ly9;GWYÕÕÇ.^[s§~Ü»Gk\x1c4\fRE¼öéèáÆü\x1c¯\x0e!vìîÅ`g²w:óçô5×®Õ=ÊrôUuð\x16t4»ÕÒ$Ï:2 ·»KÛÖï'\x0e:÷\x1dV層{Ë\v\x16QÄzÜÙÑÃ;LÜíâ#YJq9Gi®ÌÍ(MNÄxuH«,feÇÿ\x1a¨»CÙfV1ONx^[)\x1f:´òÃ<´²ÈþZ°Á³\x03[дÝOÔï§Éàötór¼\x1fçÌÆSÌ¢ãú-:;\x16)\x0f3¿n\x18'ÓéÍqêÓ9ÌÍtÌÍï÷ßÎ:1γ5Ò@Âdô¾¨à½µF$ÉBªâ±¢\x14õ±vÏ«*þ\x06#\x17Ý\vî4¡{¿ÍÑÖ#åîô½oáàû>nôGLM²µE\x062dv«èÔñ'1àüyõmÅÔ üL,ukY2\x14ÖMêÜá|\ÔÂç¤Ú q\bT^²x§få°k\x14³\x1fδ_mwnÍåfM_mÇ\x1d]«ë¤ã%è\x18M>Ú²`^u¯£\x0eDïJíçËÅßâð̲öÖýx|ç;²Æk\áµé8\x1doðÅò1Ñ3H\x03]|V]£³C\x18Ö\x12xcÔÅ\x05rÊ2)\%Á\vLi#zR_½ü±õqa\x18\a¶ÐëÍ\x01#ÖçY_\0\x02\x10\b^aã\x04ôËEß¶©\x1cÒsUtuNns6êâu¿_+£¹ôÁ#ïÁwðö¥xôîÝ"4ò*\x0f\x14Ò«¼ê°Å\x16BtxuW\x11Ï5TWr0¯r±RF\x18\x161SåÆ\01*+Ó;ÞoLc:2µfaí\x11\r¥òmM0þµèª}ÕYÑ`uGv*Æ4ÆVµ\bµY2þþ&¯OT¯2zbj\x0e±x£ÂU$ôuã>°Ì±Ó\x17ÿV¦¤ÅzÍ\x1aÇöÖ¨Æ7+&L3«K\x19L K\x16÷½Ì²°Æ\x13*ìõ\x19ö/\x1d|\x15G\x12\a-î*Î"\v&?㢹®Æ®\x10¯2C¿®tb~2Τ°~ãj' Ãb·PÉ»¤Ë+e©mºµhÆKxup»â\faW'«F2öýüqy|Ñ\aVºêݯ*u í\0<\x1fÅ\x11zÝ\x1eJ¯ß*ì`-ê\x0e]$WûCùgYÑæ:\x1d%Üô\rF\x7fäÇB˨Ì\x18Ó1 ÊȲ0Ãs\x02&½RUj@Á¤\x1d\x05i"\x1a¤b2Ä¢e\x11;böRbV\x18°Ã\x19Hþ%e\x01jª\x06\x06:´Æc)\x18TÆ2FX°ª\fY^[*©ªÓlfM=ÚN|µµG\aðÌÌMi{pÍÙ¯=;èÞï\x1ag®Tfz¤£)VD1Izð_1\x1d´¯ñwêÕ»Vô:æôW ®O%pjÕªã\x1c®f£I*.dÛL\x1cçô1·ªÑ·/\x17á»\x1fÑrô»$f+ï¨\fdÆC&51Åec\x060¢\x19^^[m+{±$\x1a äîtîoT#©\x1d(éÙLY0¬«ÞÒÓ\x13\v\x16213ºkXîá¶aÔWE\x17£%ê í«S\x18Ó\x12ÑÄ´ý®$Ö5mªë\x1e˧ Ç\x05ïüå@ó\x15Óa \x16Ë&1cöý7m1¨ké\v1¾YÇ\vfæôÑA5Ã\x05ÁÀðùÆ0¥¢¤âáSò{ß\x1e¼uU/;\x1a?\x1c³\x12v¬ +wX!Ò7¥ø¡lßçBê ºé6rÑÙÃa]£×ÙýµÕÍÇ\x1d®Ê8¡j©´åZí=p\Lwtû/Ó²\x17t.Ù¿gyÏ\x13T7½\x16,`¤ÖptXnÃ3gÚ:¿D®§ç»éK®Rªf,wV<55RÑÐÙ óÌNó\x1fÈÒ£\x18¨þnþã/Ök¤ýÜzÇ1ìö\x1fÑ®KíÉóÉÔÚéq½Ë\x0eùÔ;\x18G\x1eP½,2Ó\x1ff§$ñtiÅu/díýö'Ã\x18¬_Á\x1eð꯿ëM1OÉìÃ\x1af®ëN\x1cÔÝm½\r.)áé÷[N\x1dñ'õ¸c7èÅ2t}ã\x0eHý}ç£+ÍÌÛò7Ãl}õK°~2fgÒô*©ø\v,+\0,£µ+i\x157»DÄâÄ>ͨ\x03%,|éÁY}µK\ÐU¶ç¤ádX±5hÆ6¹9hN\f®q,¶{±J\x1dEU|æ\x04õÕ¤RɲP4°,j@Ë5XYC¯\x1fo\fB껪pè9ÛF1õã_¸teý*4Õ}+÷®?wc\x18ô®\x16^[1£\x1aiÀâcKá÷ôKµò}Tð»\x7f·ó³ð/ÝYë)3ËÅ`ÜÕcK5Ës]\rͤ¨µÔÝ-\ãYoUÒB \x1fcnb<ITÓGW'\x12AnºÙÄW3ë\x15ôJl¤óM.>ó±]èï§^\x15ÊW|a¬_\x1aÐÖö ãï^íj¹x$w\x179I>¼aêÉfa¹cXûp\x1c©-.Ve¹µ¢cSÃ\b¸V\x1cL<än·8-\x16¶´miie\x014²²FbçUºÙ,k]U©®ÕãÇ\x11ä3Îs¸s²\x17Xi-\x1a-\x0e¹sZk3\x19¿³Ã\x1aW"\v8£UMXÁÖØjöìiodc\x7f!§Eª£\x15*Ý\x16ªh\x0eÔäâS0¯\x1cÂ\x17\x1eiqR\v\x19§.ñ!ªwMõN ÉÂæHi¨mR\r)TÌ31ùaÓÓªÖ¹TMk ã(%³^[§ï1ù#·|Õ-O©§®ÝeáâÈÞ`dg\x18B\x1d\x12àµò$ÌBI \x02$à\x1c\x0eb!É+¯HÿAçÝþßyº®î¢\x16®òc4°õãC)Å\x7fw.\x1f£fMôbpËf$W3\x14ábFÝ\x14¾¢}u\v \x1f\x0fKo=\x1dÇ^g¼Gªú¼©ÑÕ7¶§¨tÇ\x1e;Í=\x1f®ûû" à ÆI¼b³5¬Æ4ff3\x16ÌÓ'\x05Õpn²µÊM#ã¥yåÇ\x04êíéÕÐÅêèqI¸R[å¸íË\x01=\x11\x0eÌ\x160ô±^Ï/\x16çªûß«M:x)z®´y¬wwöɼc\x12F,ö=\r1Nµpîðõµhø\x15SIU;è¾2XÅúi¦\x18´Ê´ÆâeÛ½ò¦=\x163¹eq¦oK1fôÚ©\fÃ\x06Û\x06®ñ\x15®\x19±¦V8¢lÄR¬Ê\^ÿ;ÙÝãkêRU³WUn]V?¹³}Æ©\x15\x1aj\x14\3\x0e$âÞk3+DÕ5\x046H¶!\x18%6Ûc\x10\x18\v\x19:«q¡\x06/ïª*§97\x11\fIã·sûGò!vb?/\x7fGr\x18mÊëKæUkä+ú*{Rë=ìçÔwP¿¤`wéÃ[¸\x19^~Ãü\x1eÏÞi/õöë"rüµ;=9ö®\x1cº\x1e\fxu½Úpkõw²Óøw\x1eø÷½\x1e\x0f\x0e¤»|örÝ\x19#1kE¦\x1e\x1eqL\x15ý'v¶y\x1f®\x16\x06\x19w¶ÖóY¾ÖÓ>õ¶f%ÌH94\x1aÂX®ìÄçÂB¶«Ó«¦Æ?n§Àú·ÁôälûÆ?ÒÍ\aÊk²~\x17ã"^Ö\x7f/Ä©ÿÓÐÉ\0ø¢,õ÷§§ó\x1f»Ò"\x1a].¸^owíòÇÅNøïo_¹¶w@=Ýü-iÛËÒj\x1e)¹©Y!V¹Ó<Äeª»Bª×mM0D»ä\x1dºò`Å\x18´éU\v³£ -Ý(K S.ò\x15nî®\x1f³õKRý\x0f«\x1daõ «GzèWºµf\x12´~Z*·½D|åFØ µc\x18Æþf\x03f6µ^[m¹%¼RLÌM6Ù¶\x14Ç\x1a6ápÏÇ\x1aÖ´Þc\x19¦i\rm¶Ûk2ܿޣݵñtMYZÓL®=±W\x15ѧ}ôÝ\x1dMKa\f\b\x16Æ\x17\x16$5rA\x04E\v\b\0(8\x02±ôæ½|ãmMGðÝr\aÌÌ¢©f¬ÌË\v\x18\0ºÝâ¸Õ£« 8R©Åã\r&Ë Üܯ»vò¥S\x15j2mæã+¿Chhä,:Þir\âj·Np¦´¥pXLeqT½\x17ù·àT\x1ex¥v{\x1aïÔìÖU4¬Åc1É^[Ë/\x18Ë\x1ac"%d;ªQh%Ó7Å`bMµé{9ݬ[1\x13Qdª7i\bK\0ÈBÚèàê¬àʰ^[i)\x0fôu·ë\x1dµ)T㢲ðç^[`_\x1c½}kkm®kÆ·Ìáû1¨ï\x1e¯+é\x06\x15bîÜ·Ð:LèHÕwÆ\x1cçJ=-B¬bÐÖÆ¶ëÅWLåÄÒ%ræ»Úë7%¸ÚÔµ^[_¸8b^[F÷j`\x1cÕ \x0e½wµ*p?\bÐO ¸ãòIL'k.]§kÊ[uÐéÁ~kt\x18²¨«\fXȱMS$HÉw«V©®m}^[%(¾±&k4Õ*LcY!VZ±¶50ÛI-{¶Ò1\rèZ1PÊE=úýß ê/cøDÖ\x04ÄtpÅ|åán¡ZöÉ\f\x18 ¯3 ¿Ý¹Ô^¹+x/ÇRV¡\x05ÓJ¾\apõØz^Ó \x05Þ~>Ê{Ò¾R¬Å"÷ï© ðð¿¯TtæûüM\x1d\x13´¬qÜ\ÆIY)|@©ÞÁ\x06\x195£NÕ4´nac'úÿÑò屫S»\x1aųç]\x0fÝ\0`mQ\f£¡ªé4ÃßT\x13Ö"\x1d\bܨ¿Aáµôj|:w4i\x1a°ÆejéÏòÉHׯ|\x1c^[½ó®ëM3\x19õ\x11\fÈÆkR¿³O\x0eóm8{:çuýU*±'Ç\x7fJÛOö\x7fgÓ£^[\x0fZÉ=çô0þÜU®"8t}{=ú\x17¬Ò%\x7fsî¯\x0eo©ü~¾¢\a/~\x15 óím¼²Æ^Õf`§Âÿ×Úè¤þíWÙ\x18OÎ ªù7µ\x19=»¾îÈçn\x1aaC\x0e*)i§»mŦ ÐêÀÑa¶\x01a x¤^[\x02A'\0Z\x01²'d(\fã3ýºõ£K¥\x10ê·.ÆûÞîNx|jº?sLyÓ¾;Ù:\x19uÛ£¦G=\\x16¬¨C¡LFV\x0e\x1dý°å\x0epô×n·\vn£³£.¸Kk¦z¦¦\x13&\v&\fÊ\x18\x184ÛK4ÒC\x18°Uæb\x1fS(ÈÛJÆb`2am(ZJ\x16àC\x14TC,½õËeH¤Ö²J¶ÛdÙRkjÙ¯\ã\x15·vËfU4¤(°Æ\x14PåMã2c'\x1a¨vzW§w\vb`Ì=\x0fA¶ä\x0fºtÂç¹þ\vòz\x19\x1f³ó\x11à\x144Ç áªG4y¨ð°Oêìø,ï\x01t2ÈQ\x1f´i;xîØ®ï\r6Ê1üéw§m'"Óí¶,\fO\x01ÝÝíýznU'PÊa\fYeYC²C»rTÚ_v%k&2fu²6Þðªò¨¦ÍV\x18Áå*p½¶ºÝ\vÎ1·XöµY\vêþµ*XÅèûJÚu«\x1cÄ\x17\x16\r ¦YfYZ°,ÌÎliÑ© ºÒÏl ,ÉÖï^1\x18|}/³AF´eK&PÂóº"ëÝånÅ9mòÕ½údGLY{M51¡§¦\x15¢ô\½]Y\x10m|\x14\x176HG4ã|Uµ&½çk§\x1dûþÎyy¼½¬]éê¹çD9ÖõɬÍo3q\x10Ö\£U½V\x19¸«5ñ½õ5)i\vÒ¹Ñ:1uÛS19Æaj\x1d?\x1a1µ\x16´g:Õµ®2ma·\x0eX©è¤\x0fv\fpÙ$?yÔöë4æ¶ÐTb<BÉÚey<_/\bÊßnkN,Cß\fb¹f1mDá_v\x0eÇçSjÈÕ Þ4úc0r>OÊ\x1e\x195\biÝ#ep|ÔJùÈQ\x12jÖÆÚÉemRe ²²ûÊ)Ý%¨ân9\x18ñ0\x1fdFfTÞµk\x14zfcJK\v&,,ËNOG\x01Åe\x1e\x0e¿32®+)ß*¦\x06\x16!¼-vh>©`Týóü1åk3Ç UåªVF{pøENÒáñçþL\x18\0>åðñïGRWÙåí\ræ'1÷ôvt{\x17Á§JFËo-ᦠ¾d¯ïT,\x1a÷ðõ£>ôæy3±~عRoÓùZø\v¡c(dömS í*¸amHk)+ "æËn|3sã©nÍĹ-ÌÌ2'\x12>Hñ@]}9õUv® FXzYÕÕ)Ò¬D®q¤¡/g\x03nI«ebµj×I!8Ê¡·Þ·Þa|Y\x15\aZÁÆG\v,05XY\r\x18ia±ª¦YHË«\x188pÓlYTªÄÄrÛ¿\r\x1c»\x15´êÁ \x02+:Ñ\fØ\ìdÔ@ÐkF F\x18m$ùQStC1\x04Çp/Ý3\x18d" \x14A<ë3ùVDCêitcLc\x1eì£tùu£Óý÷\x15BÉHöâ]z|½|:8sPApH8 À\x18\fxÎ"D\x03=Ʃֲ\0¶s\fí]ÝÏ&¶ÎúVõ^¶ã®[îlÊúiï\x18åÏ9·kÑé*¾Ý)*\x7fg^8§©ÄOï¾qݧݡµ×ì~\x18âd¥M§õiOw>fîl6L,±BĪ`\x15VkMZÖffÖfw ;¸\x12Ó? _ì¨æÞ¯}\x05²\x11\x13à¹|3Û²l¤ñ®¸óûù\x1f\r4a\x15efTc\x17¯FÙvÈôsZq©ó«óéúÌ×\x18i7w³J9zu[¤R¥c\x15$ü5¶+döwU\x0fI¡\x03SPÐÉ©#U?<n^[e\x18aìeªÈµu8Âë|?þDC¥;Â\x7f\f2b¢X±hûÉTÀ\vÅaæÂÖ xhò¯w;¤ïô^:ïyKÉO·j\x7f\x1cN\x17áv&MU\x19\x13ªsݵ8cìâ½ò©%/¦\r\x18í»\r¤ªdܦ2bbÃ\x19mSF4ÅyÚ\x19XÉJÉÓF©öÅÝì\x0e\rÒÙ§\x0f\vîÇÿ\x11\x10Ç'Ý5ÕLYg|¿Ò MÃú/\a>M\x123*v&=\x1f=§\c\x18^[Rj¹É´\x03ÔãÕðçjIÊcö\x1a{)ðàc©¨51ÕãoW\x16ÉÞL¯\(,Ë2ÕîYB\x16YKe\v+Òç¿ï^©\x05ëÇK<}÷K7ønâÍÚê ¾»âÕåqrjE.m\x0e^[G5ÝÄàéË*%¥694\x18Ê¢®k.úíÑÇ\x1dRék&]R\v×.m}-|®\x13@SµËSF3Ó\x168rëZuYI´è\x03Ã\x10¤²2âÁ\x10pê"äh=ä\x1c©8'\x02Î \0í"3\rëBA32ë\x14,{ëP#\x176ݼ}\x1dÐ1fSZ¼I" À q\x10I!8Æ \x05`D \x05%GK¸+\x04\x12A Ü\x12V±[Î\x1cõäf´ÇÎ;!v]ýÝÝÞ/nþVú±ÒÈUiÁÄá\0âÞ¼à\x10s\ gV89¦Ò¹ÐëëÖ7Ò#¢o\a\x1fÇ~V1¬æëÚ¸º¹"7U;Ú£©"7^[åÇ=øâ¬4¸8dsdÎ\x15S·26bèåÄ7-ÊUrÍÆldÉ×^["§]rµ[uË©Òvuov8¾;;âáʤÕwVÕÙ²:\x1dw\x15m\r.Ã# ªj¶æ.+^[W=øpºe:.9o¦Îª9ÅôUk©ÝË®[jçà=/d÷%¿UeKè]ÒPèØæYYÅ&óc«J\x7fí\x187Z·@\x01-M¶Z«-Í\x03u\x19t0Öt{¾qzú´ïãÞ«\x18ëS-/._áy\õ̺Ö0±ìÓU½,k®ñ¯ß\fÇ;kÞ^[Û\x1aaN\x12F(¸1©¬náºÌ¸cLjg Íqѱ¶\x1cbÞÚKr\aBáõ jiâLiÎ\x1d°ô§\x1aì\ªtn¤ygÙ 2aëÆÀÒµXÂ`b{Û³38\x1f]Ò!÷²é¬9<X£¼©o~0n¼Ö+çQãÞ\x15N\x19Hx¥,©&$˶2z·C¢½©ÒI×´4ÿ6\x1e+ÊaË\x19)Tî¼Ã®PIéô{N²ºDCÚ7áîͲáæfvÏ\x03\x184|¨¼ÏL¼1þ¦:\x153Îím\x10^ë©~¦%âñHü;6²ÊuíK\x05\x16f0ÄæC\x19R:\x06«ÙÝ1¶Ïg*øú'\x0f\x02ó®ÓA*ºX³f2Ó¹T4émå,½T¤¦¾\x1eSª÷µ~_M»?aÉÙ\x15c\x0f×¼÷Ä\x03^[ .ÇÆ{D¨tödÄ÷ÊÓ\x04ìe\x1fÝÍ#ì¯þ0\x15Ñ\x13\aò¼hjú¯¼¥'\x15Ä_\v Ó½\aÆZ?\x13=Æ)'ò;Ñò\x0fPcÄO¬(£(U¤õѨªg×\x19k®´ã!e^_Ö-\x12k))`c4l°±\x13ÓOícl¢3½V¾V#õY'K\b|-ËV«-\x18Éb\x1d÷ÔÛ\x1ehNÎ^[sh§,ss\8Ou\<AµÑySéNmÖW¾ò³\x1fajËçÅvÇ£¶á>î°\x18f(N½r\rLl§9\x11V¥*°bcCUÜÕs^îøo#ï2^[²¿\x1d;SÑÆ£ô©×Oââ^\x18²Ï4ÆyÉ\x01-\x18ÓÚ·[«f¸ÅÅ\x1a\x19óá{±S´çÛ\x7fëÌÅ:Î9Äs^[ðôÒéШ&l+\f1ZxÕ´·+~õ;G§}»Yeâ.\x14£NeÆrGQU+F1bÊ`È\x15\x13£Q0p Õ¨Î+º¶á:éý±\x1fÖQZ,4Õ¬2ÓK\x18Ì¥50J,³Ç̪\x1dº¾9úÂñiúkÃzÂé4f¾áeõÊYßÏÊHÉí¹´?|;Î *Ç\x15ÌÚïX\x13³R,<L·ú7GQ=¹ýV[]Ltjt8Vi¤Ô YøÒ¶ÆÜ²c\bRC±`\x19 \x14CE/\x10Á\x042\x02;üa¼Þ·ÌÍæi¥\x15ÜÙúd×v²}Ë:t\x7f.Zuí¶ÞÚÒþºõ\x7f®lZòú{CÑ\x12GÉè§îÝ*§}ºÑî$«¡/zÇ+ÿÌPVIÖvÃ5ÔÁ1Ü×à\x1e\x04\x10\x18üK·/ÿÿü\x18\x13\a\0\0÷Ì|N8V\0\0\0ð\x03äKÁÀ$P.ÚÊ\x0fMP\x15BE$D \0\x14\0\0\0\0\0P\x14\0 \0£ hPVØ\0U\f \0\x15 \0\0\x01@\0 \0\x14\x14 \x02\x1a\0ÉF\0\0PPPP\x01C \0\x01Ó \0\x14\fZ\04h(\0\0\r\x1cE<\x01\x015 z\x1a@\0\0\0\0Jd\x11FD§£Iê\x1a\0\0\0\0\x03UOýêª\0\0\r\x01 \0\x1a\0\x04§ê¤ h\0\x01 \0\r\0\0 ªBÂÒQþ©Õ\x19\x1adÄzd\a©êiú)H\x04\x04\b\x14hÀdÓOIoYfVWÛç\x02\x04\0\x02I\x02\x10\0 !\0\0$BH@\x02HBH\x10$\0 \0\0\0\0\0\x02@\b\x10\x02@ \b@\0\0 \0 \x02I\0$\0 @\0$\x02\x10$\0\0$\0\x02\x10\0\0\0\0$ \0\b@$\0\0 \x10$\b\x12@ @\x02\x10\x0fÝYVgf¬ÊÌöÖ\x7f\x05õjWä\x7f*\x1eÔñdr*ÿr¤Ô¢\\aûèÝH<¿â\x7f¼ìéJ¥ÿ6ÿåD'ü±Ìb""""""*"""""""ª"(3 Ê"""""[1¦y\x04\x10K`\b Ë`[-Ëe²Ø \b \b \b [-ËeEjÖµ\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x16U³fùfRy»\x0eÊX&F¯Sm\x14'Z5z¤ó*£ÍÈünÐ˵_P Qêó[t¶<B¯\fÞ×ãq\ãUe^`à$»IÂÄ\x1fò"uQ¢|Ò%Ñ\x1dÜUª¾*íWz¾\x03æ¼®Õl0u6T:Uô\x1dà =¹T}Ô¶\x1c\x1e\x06JÔü+ܫȫÍ{Þ®Õ1\r\x1aUÂDf\x19e+0Ì3\fÊ(¢(¢¨ÌªÅf]+ \x1eÔ){Æ1± ù \8;ê|¾¯Såâwi<æâðÓãùvpÆ:8h:Ý]QÙÈÿ«NÕrøK³Áý\x1a{Ü#rá\UÝÅR§5"¾¯ONOFÞ^[o\r6ð·SÊqu|6q:'CNYW±bòè41ü~\x1dV9¼=×[ââKðÆÃW\x164nù\x1a:EWµÀ¹´Æ¦\0±6ìÉÂø|»ós|½\x1eÅÅ TÒüÚ|°Û\fc2y\x7fÕÒóm|4t¾Ô4´Úôx\bW\a'¹ÈòhÉ\x1d.Aø©U³ÄÇpè»\x0fÍûÓÛ´¿7w\x14ª_\r·~>Z/Ë\x1f·»Ý\Â&iÄ}°åôÒìÓ\x1cÝ\x0e¯ÓØ}½ÝÝ{Ýï\x0eï\r;:ºÌ®Î§â+\x12sTK\x18Ó\r::º±¥¹pÓa%ÙÙÕÕ˰÷KNî·Ã&:8_! ŧӣöôÒÇ£\x0eVÎJK³£fD£1é\x1ai«O£Ô*\x1d®®\x1eì\x1e^[möôèÓNÏN§åyWâ ååäó\x18à´²Ë)æÇR:¶8}<ÕÈÜy/Q\rªÉù¡ê¥¹Íá·Ë\r\f4ź±ËgG²ÿ½¥uqz»{ÞᱦO§/N\x1a|\x1fó//åÜò±äæ´îÝíuwc¥>\aÛÓf0ú]Þå£äìtc«Næ:®ÓåË»³\x0e\x0f\r\x1cßϫԨ:,¡\fËÅÁó:Ïâ]\x1e^\x0fáÝåáå°y^oØûp¦GÑÐÕáê.ÅZt4èá\x1d]w\x06Q\0\0\x06À\x03ø6\x14:Õ\x0fù^¾\x7fDs[\a"\x02\vôæ| ²üò\ÏàBï5L¿à¬M¹^[\x15fÊwêÄ0°VèÈ #¾í|3\x18VÁ\x01\x1c3\x15á\x04^[9¢Ê\x16\x0eÐùuêíVÅ«´BÎ\béªÀV8Ò\x10%i*.â\x15ä§\x1d#¨ìvÆ\x10Û±ª° °å)ÇJ ± pR^/NÂ\îØ \x06¡éQ^6#\bA+Æ`Tl('-ÆÙUù$Æ9½°ß×ËVämJ\x12\x1eN±Þ0=\x16j@×¶`õ\x0eàÜ\x0fS¾GKqKN¢À¤x¼"5Ì,|&ÙÀS\x02\b)ÅUÁ Âê)°\x1eÆis3.Ó*@áó\x01±<ÕÊÔWÔÕÁ&Åd0«\x01ªàÁn¦.÷ãEW}×k讽ïy/+ÉX/GG}wÛØ½+¥LÒÏ\x17\x19æûP³6ÒËóº§ÞfÎÒÐe/·\x1a!Ⱦjý\x05n\S\x12Î4!^âH O)4iÃ\x15\x06\03V\x03eTBwè.t\x0e\x12[uÔ°\r\aNÞY1)` \x06\x1eG|'6ÍS\x11 \x13Ó1Ñ»Í^[j\x01\x1c¶%ñtÌRw¢KÍÒñêt9æzf%&\x19U×{Í"´ÞÖ3Y`ZÔö®èõøõ}ûæW*¸&v¸Mû2Ú.\x06ÜÓ¢±Ê\x18&ÃØÖ¡\x19Â\x04\x0e\x12\x05J¶K§\ÎÛÏê\x05D'íH +üÅú!\x7f]Y_ÂÔDµXTFBþ2±\x15',\x15rÝp\r¢ÁÁ±¡ÀàʪÁÁ\ªÝV*ÃuZ\x1c\x187\f\rÃ\x13\x18«^[hR©bc\x16Shl--,Z\x10¬\x15KF\x16PQª¥L#f@°\x11nȱXDZO\x02Ç\x04® Ôsb´Ä¶*§"\x15± Ù4²URâ¬$Â\x03\x15(2¬PL0\B-\b,¶&Ò\x0e\x15\x13áBeV«%¥)4À¥\x1adZ\x1a¥d#Hiáë. *ÊÒá.\x15n¦,di\fa'\x05Ct4\x18U\rÔ"¶ÚSIªØ,)»#TÅ4£\x04²¥ ÔaX©2l\x11m_ÁEv\x04ì(¬¨¾Å\x15"ÜQX¢(ü +)\x14\x1fТ²ïv¬ñài³ÊÇ\x0e\rVã\fnã5pc·ºeiZÃl4ÞSL³l\x13zNg8嬵ª¸lÞU³Y9¹Ä¨\x1a^[$Pªd¸i\x11\x14å\0âTÀL\x1cLmq[\x19³8Ýk¬ÞìË«FØÉk27åZo6®&7\Í )°dK\x03CN@d \x10(5Zo1»ÕÁ6Þé¥k\r°ÓxZ±²f\Mé9ã²Öªá³y\x04Ä4EHâT\r\r(DU2\4rÓyµmq1µÅlfZfrÌãuÝÝó\x7f 4Ú}\x1e\x0e TÓ 7[i³l}1Á³#ÿ\x14J^\x19*öL\x12À`ʬUPõX)[²\x13a%I¢Å 1&J¡Le*¡L¥\x17±eBÒF\x13)IlB²U¶Ö\x1c^Ó\r5k\x1cðÑ\x193\x06£Q¶Ö\x1a·0ÓV±Ç\r\x1aÑý ¯Ô¹6ÖÔ¹Uɱd£%ªÁÑ®ÆeÇ\r8àÓ\fÃ\x16X²¬1YefVTza\x1aL³,Ë\fÃ2òü)¢Ê±? \x7f¢¤Ôdò¨e\x16\x06GDjb\x04ºI¨5FëRÈ\bí:Ë')Ñ^[\x15S¤i[Ö+,íÿfÚiÊ\x7f¹ÅV\x06\x01è^[ym©IX!Xj0VUJÁ2¦+uÁ¨ÚÈ8W\x1c3&7\x16¸VpÌ\x06õ¤ÙÄp²\x0e\x15Ç\fÉů =4³Z:¼ «J^Õ`.ÅaÖzíVùLqÎJí bêG\x12UaPè]\x1d\fZ,hé)KûÇÕüEº»Ú¥hJ¬¢}×uSÀ d\x17ï^[«-còƩZ2h²\x03Xèçó½î]GE\x0e±\x03\x059=Ru\v\x182/iD¿\0ºæ8Ç\x13)®Z´¥³m.YÌYrÝZecy.ZÙ»3-eÅ ®ZjY¥³m.YÌ1Ãlðek±\x11º×`q:F\f\x06)10]û<C¹ÐíZåÌ9]\x15\\x12»¸:¯hŪ\x15UÉ:EÃzö¨8ªï sµ0t3yn6rTeY)F\fL\0·\x06:WÍTM"Y\x198UÁVU¥p%Êæ7Êã6W*Ì[MóuÊÞ©àâÍËQÀÌ<ÍMMÍÍIÇc1¶æög%´w£\x05TÉHwènkSr½VYªÃ[*&F+\x1d.Eà0e<Ò5P/*Ò\rJCv¨UØJ¬Ã1ðÒÛ*q£#sµµ¸Üâáf\x19km-²§\x1a278]J\x13½Å.ÔuE' Ã-\x1ar§DU®ëq-.Ýo9p^[·YmS w7dq¬pýÂÿxU{+ÿ\x7f£ý¯úÏú&Ö\x18M´ZWJ¿\x18Êjåôëÿ©ùM[ºßUÞ¿õü×eþ\aÿWí]þ¹²_ \x7fý¥öª¯Æ««þ\x0füWê ÿùÿ^[Ã\x18ÿ\x18VÇügUÓüZÜ©öÿWáRu]\x16¡Vªc\x1aÁª²ÞØÇyRX:1Wÿ\v°öX[pßv6÷a˱´º§\x02ô4½\x15\x13rz¼^Õ¡Ò\x12ÕT=<?Þ^[¡ìbí\x1fÍ3\x16fkLf3\aòÿS?àT94¶T4Ñѧw\a\x15q`#³åØâqHäô´¨è¦=ËÞ¼«£\f^ë\x0f»2*Æ&\f®ÉÊ]OÉÉÞ®§/§áêûy=wc\x1fñj÷[¸Þ1»QÙ©{º^â\x15ÞêØª\x1eX\x7f©ì /·±|:ª\x1d¡õqW<c\x1eæ</Ý]ÖõÛwu\x0eÌ\x0e§SAÿ;ªæðÊÐ wR+³£\x1e_ÅËBÿVI¿\fáêÃ0Ã\x17§µûx==+ºÇZe÷µ«/\a«¡\x04ü/6N¶?ÝWz¿è{^[aîáðù\x0f`Àð©e^ì«wâ~CâKËêþ\x03îÇÃäò»Ý½\x19V×µðï\x12L\vo§v$+ÃÜíxNÕKôèM+MJ©iw6ímècÓ<1ú\x1e¡4¾Ü¶£Â2DXÞæåîî1£Úu\x0fÈméá¥\x0fc»£ÝpüUÁ0=®¡Ë ÿuz«Ñ±ìü¼¯ èUO^[®ÌxeN\x04+òôðê!_.lRµxv~\x1f\x0f¡Òòá\x1e/§CÜëÑÍàè/Ofû³\x7f'j}\x060ÊpÆsû9|<¾½§Óà ^ª;ÃÙÔü\a!M:´Òü[²Ó´R^Ã\r¶ÇÇ,ÔõeѦﳳ\x17uc\x1c´96u\x7f£âÆ}ãÞÞfcCc\x1e¢E\x7fWt~\x1cUöÒ¸2\x0f»¢K§lc'±ÿijðyÎ35þGz²\x10¸<c\x18åèx«a·[áí|¿{Ìôìët\x18»®çZ¥L»^÷³ËæpðÇpÿºáôÛ©ì|\x1eoó~Âõ\x17`âJ/Ë»ÃÃl¹>ª÷"\x7f\x17Ë\x18å£MZhÔþ\x1f\x0f§\x0f\a½_Z±nü<ËEÃîÜÝYjíiîÿ\rÜgnXáÁêà_qôú/{²aåZ&/\x12vQÈIb4Y ,1k3*è«£¬§Ü\x1az|rÆÝ\x1f\x0fûÏìá¼½+ÕM/\x11ÐèÒÆ#nt_\x06\x16ØÒÓ\x03¨{ÊkÃ;:#ñvv*%Ù\x7f³ofª8wyªTû\x18W±·\x05ÐzWU%»?ÛF.ÇÑÝìì¿\x06\x7f°?O¡Ðÿmöü?ÊáàcàªjýèÐÜ4!\XV+øû}>GéËô}Þ\x1c¸yGòÂ<¥ö\x1aGS»jåøbÇèþÃà_Aåâª\x1d¦;½TOíîO½êÊeà¬?e¦Ûl\x18Æ\x16Þí\aÒ¡ô\x1e\f;\x0fufc;0ñi±õWbhÁÃßwdÄòù_\x0eô¤¼Ç.\f4;¾\x1a»§7Æ1XÇCJ=æ;±ê¦òÅù|OGÝá\x15&çACK\x1d\x18tÓ:¼;,Ye§-#\x1d íîn^\x1eÅTóV\x06é±a/Ì1+½ìu²íY9t\x1a£ó`Æ\x19p5iwÛ\x1f5bQ_Qø¤lë=ï.Ýs3;Ý\x1cÛÔû5taѧfÜM¼^\x06^[U\x15¦4â´Ð°?6Z¨iZ?\x11îú¼\x17n\x18ÓåûhêÛò\x0eG/j*[~úè_n\x1a,» U}=_B\x15îû\x0eóô\x1d.MÝtÄüº ¶=ÞSv«"´.¯áæû\x1d'o\x19Üí\x0e@÷tj¥¸÷t;<ºY¬Ö³ÝÝû~v]Ó\x02ñ%V 1¤ûs8®Î\x1c%Ù~\f&Xi%ÃLcú\x1aÆ\x18Òr÷Q¥G»f\x15a\x1eö\Ñå\x1dXõ'ÍBbhRéø½\x19nÆ4i£@Ñi(¬n¨®«ôl¶5É;æGFêOÌq5\x06\x062-1 ±ÌN\x18`c\f\x18±á¤Ò~\»«©¡ËÑ;Lv\x14¢i] ùc\x0eaTîû\0=ÀÙîm¦/&=Uº<cɦ!+¥¥)Õ\x19*\x7fõ0J^^[iø«Ãð\x13\x1eâá\x1e+Â÷\x19mKÞ\x15:\x14Á$ù*QÊ\x0e]!ìº1ðÀX²¨¦,L,±\x19K\x18 _ýÕ°Ûß\x19Xܹ¼c\fTð*ËïÜðUïw]Âë)̦](G ÝX-Q,NjÚÃ\x16UYL\x04<_ÜP\x05\x13*Êq9u%åÕ\x1aÆ?ýÃff\x065a9Æc1ݧ¤¢±"ø\x0fyîð\x1aI=Ô;¿gZ§ÄÑNì1iÛ1RȪ¾Ú]\x1d\x16ñèøuràÇ\x0eV\x11Ö®\a&¥0M¹iÛÎ\x0e^[lÛlbpn\x1e.\x19\x0fOs,22¬Táhø¦C\x1f«V¨²°ÊPÊ,a\x7fQ)Xô\a¸Åº¼Û½Àéy\x15+\x0e,Æ0F\v)M#@Âtv);2 \x7fl\x012\x061\x18jª\f¡\rtÖ¾uÆ3áÃJÊcºSS@Í\x03À£ÈàÑ ,¨¡Ý*Õ å Ü\x1dmGIR\\x189£\x04àêÒràs99«.m²tHò\fXÅ«;¶àQX°Ê¬e¦«*\Ôåw«G\x191c \x15ôV+f\x1c¥öT|0cµÜ»¢,bå´JúMÙ\SL^Q!Â_édU.ì\x1fÓàãÒ°Ó$Æ-\x1ahÓ\x05Q¦5bf\x1abjÄÉ 4ÃLZ24н)%¦Ì2\x15\x15T¼+Åfc,°Æ\x15öµ@¾-¦=º5W+\x15 áM4´ÂZ0)dË\f±ì¼<U\x15µÝ÷^[^®*òéÕú¥\x15c\fDW\v hB¾ÌS±Pab l¦\x0fbWôÓݺ¨_\x15Ò§²÷,<ælmÜ\x1eÑWj¿`÷tpb~\x7fWÐÑÀËNÛtp¦m¶Þ\x16êa¨%¸õ$ÓI\x18.ز⵼ÌÝTÆ©ì;Û\x1cWØx\x0fw`÷cJU.¶-\x13OccMÖ®4ÉÝá¥]㱫\x1cذÒÐp^[\x1c:\r¦Ë*a±©áµwmiï5B:<,KáÖ÷6\x1d\É\+â¬K\x12ªa UX\x18!\W\bá¥\x0e\x04+j¡_LzÓ4¾\x1c,¼\x0e!º®Lªcþæáø\\x04+FÝf*OJ'fàâWgA§Tʹ\x0fV*©Í+f* Íüi¦1\f|±O¢´M\x17ݸZ¤mÔ~,yQAêu\x0f/'\x15aÁô§i4=GqâÝSV ýMÙ\x1a®4ÚqT÷u\f\x06êèc,5N_\x15¶K)Á(Ýnà¶²Ác»rªZ¹eM\x0f»W\x17FG,¹aôÇwwk«ÃÕÞ²ÝâÚêèxKsNç\rÝUÃN^\x0eÕ¹Þ»;Ü\x0e§ åqpàÇ\x17W\x05ºËÃnæ1Xòît\x0e,èG\aSDLD<»ZØè\x11k\âÉ[¯\x11(pùKpå·Our=Êæõìeï3úÍ㪬Ob俬åW\x17ÆëÕñª]\x1dywS]´¨é®!\x10aë\x02 "6^[àê½ÖÎVZ½ÕzU¦×¡æ{8«Fâ®[\fVG¸ªU\rÞ^[t^Â\x15غ²-ºW'\x1aÌ®\v½ÃâQà\v^[~Ô6Úó]ÝY'gBìÆ1c<\x12¶éco\x06\x1c±¥¶=9«°érÛ\x03\x1d\x03gdëW\x06Ý\x11%¤´X/Lc\x0e³Ã¥'Ge`{®¦*àò!X\x1eáìGDrî]&\x1e\x03¢¡èJöwGQµË·W\x0e'£e\fcGéhèê!\\bWWtu»»µI\x0eÎ\x06®äwv¯ ^[LY\x18ÌeF2WXòȱhe\x12ʱØ9k&_â¬\v^[£àì©ðæÿ@âw4R+\vûµ]]\aÁì\x1e\x05Ç cT iOAh\b¶0îx\x11 àaX¥\x06I\x15[\´öbñ\x15b¾ã%nÀ\x17ÑaP«D+ý\x1c^^[\x14+ÝIc'\x01¡ý"Ô%Ø?Äá{¸=Ä+^[rþ1³Ljpw\x05ö)\x16Ñ\x13 «Ð;×Vãd]+\x1f\r->\x1dXA]Ë\x15Eb]#BÐâ(峏騯~©¥¥ìUMN.\a#ÐÇB:½:Î\fe\x17<\x19T'6Qc G2á2'hôUOËØBº\x04+\x18ÃÃï9f¦óg-½Ú`MÊ\x15Âr÷c\f2pbmó\x14w\x1eD+ñ*Kµ_+£\f1aÎFÆ/TjA}%i1=Ìic\x13\f\x04ie}³\x18z\x14VïsjþîAôGIÕ>\x18¬kL\fL\x7f úC\x06\x17.\x14ì]E<=FرÈÃ$c"Ë\x19e\x18c\v\x11 S\x18Y\f¦\x17#â¸U` má :ÌléVÛwú o¾a}\x05É/ðUOÐáHW#pªdöpÃ&F\x1a\rB½Ùñ\x14.)T¿J\x1etR¶ÊS+N /£Í ªN^\x1aw,u\f¬YY&,0²L²Úʪ\x01L0±¥/Ë\x0eµ^[\x14V\x15 ~MF¡\x13!\x13àQYK,bË\x056,%FIeÚÔ¹ Ê\x18É1(RÊË\x06\x06åIhj¿CÎxf_á\:\x15\x06(aU0öOq Å\x02èOOÞvÌŶ[Æ`¢·7i¥\x02¸&Ûhbþ-4ÄðUM\aGEÕ¶îìÔ\r1\x18Ë\x18YL1ÅeÀ©dÓI4,Gàòz©\x1d$B¹.®\x1c¬²K)`Ã\x18F1\fe]SN®¼]\vÍa;)ÛpÆ]WáWèÞò2*²ÐÄÌÌÆ?iEm»\rÚ\x18Ê4M. \vòÑR[\v°ôuzsT±\x1f\x18Æ\x1a%ÔÂp¥F#µÊêÜY+Vë°Ø*º\r·KDÐ¥µÂ¡Z±åó>Ü\x06é\x1e¬4\x18PÆ\auæ\aÃÞöm1à Z²\r\x1dÍ\x0e(©\x7f,aû\x10¬aB¿okå§z½ ©úl dò\x0fô`x*¦'ø>OþØp*§åGgËôûhæ¯.8¿¡Ø¦1X1i¡ò§;8Gíöh×¶\x1akLämÂi¦V'ìûZ¿AµOïlx»©ápböjO\x1fv1¶· ÛO¥<Z~1c=Ë/5\x14¿\x051ʱ.ÄmUCEÃ\x11üe\x02Û,6°>Pq5T©¦ï7\x13*É-\x18Ý@äå¡q2·\x15îÝ ^ÈCî«\x16U¿X¤&\x121a2\x19Iº»9\x10I»,²²Ëâ{·"µ>ç ë>\x1d-\fXc\x1a«^[Oày>"ú\x0fÞ,dM$ù¹_W-Úne¨Ó1Ó\x1fÆÚÉ2q §\x15Tv9\x10¬¢+-M\x11\x15©1ËU\x19?ÅOºª/ê®nEÀ\x1d\x19\r\x17Íí\x18û £Ù \x1f:Ê|Ev \¡\x1a^[L·lÒcWvk\rÙ\x1aµp^[Xjhîų +s,¡¶ìa26U¿mZÞ£Y3\x1d\x18Æ5ÍëZÕ¦\f·B iѲÐÂmÐÚÐn°ÔmŤ¸«\x10åÊÓbÐ\x13EHÑBjdhÜ\x112ËuêBdx\"K&Â\x14VÅ%¸¤°ÀþØlÇ3\x1c1*·T&\x1a²v%e]ÏX?Ábö\bV\x176¾[æã3¬ÌmÏVpäQ\9[j\x15i¹XÒÑl©-U§ÉÑÍËá³EC ÄÁ ÛÒNTÕ_(9\x17µN®hà§²º:\x17ÛU ò0b¨2Æ+\x11\x19o-6oyf¬JÑp´üÛ1pÕ[2F2\x1aT6ÕÃøí*\f&0=Þ\x1dÒû÷ütm¶Ú{ª©èôǫýá` |\x1e1J©·\x0fW³M>dRÑYJ¥Û\x18cöÓL4Êi;(ÛÒ½Nç¥c\x18b²*V\r´ú\x1d\x0e,e$¢<+ÓÓ½]Q§ÍH¦Óf\x14&ÇC\x1fÙ³ø\x1aU\fpÓR¤snõpáÄpÄ$5.Yc\x18\x12SrÛ"\x06(\x06\rM\x03\x1d¥(ÂX(¬¡Ôå§ÉÅàB½:¹W\x01§'ª½:ªà'½!^á\x17 þ?\x03¼\aê¸h42þ¦%¨Óiø²rc°r^[¸\x1d\x1d\x1c¸å²0>\x05ñ\aòêþ×ÜKYR&Ö\x06\x18×;Þ³33[i=í4â¬K\x11%Á¢\x1a§V\x18ì£ø[.eµÔǼíªOø~\x1f·ñô\x7fwð÷$§Ù@Åy/²\x1fKÍ?5ê¢\x1dEz¾WÅ÷Êâét\x02¼.ÐàòÆ\x14S\aÓð!Zj \x16*\x13Øéa2r$;¿¶B+¢\x060W¥Aw_\x03\x15ñWÎéW ªt«Cª\x10èWºµV2Fª*ÓE¢ªc\x18Æ4ÇöÀlÆÖ4Ò¡¤Ý£L&5½k\x1aÖz)')\fc¦¤1# (\x10\´_\x166,82Û\x06Ú6á¼Ìc»)¶Î^î\x132¿-íB\x1aÆ2 \x1dªèÇ{*¹\x0e\x02+°j1jÒÔ6Ôï[ªå10ÝY1qI\x197\x06Õ\v\aËN2c\x15w{.Ò¤ºE\x1fÙô KÁÔUN¢fxÆXÓ\x18(¬CÂB´ZSfà£(©rÝKyI+\0É"»N\º+\x14àÈÀm \x15Ñѱ$º9+f\x16\x06\x10\x17vOóòÆÇ{ËÔüT±\x16M\x17'7! ËÉ\x18\x16\x1e\v\x14L2£nLf2ËÕi\x02ÃG\x05«h\x17áÅÃlG\x162æ5@´?\bÓÐ\UÀ]\x18âï:&\x0fÅ¡Y_ÓU*Y1¦ªÒ°»tg^[ã5ì(¬´ÛC\x18ÆJÅQYc\x18úªTÆÛ\x16¢¶Ðb\x18,2TÊ^dzÂý´\x18T+ ÝD§GÚZòbÁÄÄ¢vqBò\x1f;o\rT¨ÊT«\x13ï\x1e\0/\x0f\x06Þ"EvÕ/vðÕJ»UQØ\x13æÙ°áÈ1\x18ê\x14ö© òÊ´Ñ\x18é\x16£FìLaþRÛ\x06&]6*§QX\\x18$(c\f]ª(å/\x14\x0e÷ΫÚàõZ\x1e\x177S\x1e31¿N\x02]Oc¦.Z̳íU\x15ý¶ì÷«±Á·2ú)\x17ºsWíìdu_³ÑµblB¶þÎ÷Í\bÙö¯Ûñ\x04Z\x0ft¢{ÃOv\x18ô2ýCbÊ{Õù«\x14e4Á¨2iR«E)jÕÚÛpÓ*Z\x16\f2Ó ü6ÕùÌÆÜð^üfÝ7rÍY¥IÂÙu8Q%ìnü\x18Æ.6ÙVN\x1a*¥Á1\x19F\r¸m\x0e.\x06©låù»%C,²ÆI,°¬c\x19f3&Xva"{±VU»JÆb`²Å¹\x10Õ¹\x10ÝE,Å*Y\x18ûcJÁXDXÊÁ\x04dùÌ֬ʸV+,\fd©\x18Æ,¶\x16Å\x15Õ w;pDúÁþ'árøe\x7foÈ wG \x16=¼,^ªô¯Mæ°°Ä\x02\x15ú¹»\x11^[\x1cVÜ1jí.jn2°Æ\f± âÐCP±\x18¥4è\r.\x06+\f`Ç\x16\8¯+ÅQ/Ó÷BLbÅàÃòÔn]\x05@¤å¥"²Ë\x18ÆQε¬iüj©S¬1D®Ôþ\x1f%b|4ÔO\v\x18åæÓÜß\x06mÂÓ&î\ageݧ%Të($+Øb!Z±ñcL7½o\x1a\x12g\x13\vMôßmµY£}¦ÕfVc,ÔÉB\x15næUÞá©Ë$ÈÄÂÆYqcOà ©.êì§À¢½IscªE Èy\v\x17E°ÇwÃ\x06¥´ý0YS\x1c¦«cÚ\x1cQÑÑãep¼Ê\x10ôÆYJ¥QRÉØ;$µ8n°r1ÞÊ {+\x05JøÌÇ &X°´äþ=UÌseJGºº7e8Q©\x03«\x03àSý\x18ró?.íRdQÞ®^\x0f¢N¢¥òþªªNâò]\x0f\vÀùpð^æ¥À l¶û«»Ì©,ÏEj®è\x7f\x1eÉàè\x17QO\x04ØÄ`cÃØ,WUÀnSiÂÒê\x18µeWÏLÍ1p\x18Z¸ªòGd%vU\x0e^âE¶:i¬Ì5\bÄ mP\r?w«j¸UhÃ\f\rØ2\x18ia&©4±1Ë\x188pÓlYJ\x16&#Ü\x17íÃG/ð\fVÓ©\x01\x04\aøùóÇJºZISãi)MCBX k3û0Q^¦F5c\x18÷bW»M4Âû«O.\x01µ[¹pÓö\x0eiº`ð÷\x1f!ÊîùÚ¼"Ëçvªò¤ þ¸OFàé~^ô þ1N·.\x1aLL\x19VÒ1(¬²\r\x16LÌÖÖfòU¡-<\x03Ùúiú}ªÕ(ʰ\x11Ð{4ÑFC -±:\bW±cêí÷ù·vêc\f4Á¦,2ecéîÒÝNÁO\x16\x1fF\x115j42À´pTáXÄa¦51/ú +\x17\x05?LTü\x1c4pÇ\x16\x14UOg÷\x14\x15$´ÆiÐÆ+\x18ÆDÁæcO&\vOÅKKF\x18c³GÐ'@é\x17Ú\x7fg8au0«\x14c\x1eE)ta\x1a¥¦Xi1«M\x1fõ¦-\x18XN¦1©2Ã\agÈ6n6iÖ}1ÿQEc\x1cµ\r40°c\x16!û]\x05\x12i³Ë\x19-TZ\x14\x01YVK/µ\x1da´´acDÕï{uqeû«$Cøb\x19a4B\x18»½¥\x12ÃõEK\x03û\fj%,m§µ¨DÛKHÕÃvÚVÜ,8}6ŶÊÓ\x03\f\fe*§\x17&ØÓl .\x1fN_\x0e\v)(Û\x04@\x0e\x06`$e@( \x18#UwàÆ©ª®îÅéY!VÅùI#áש{MO\x14õ9ÖPX67\x0e¸e>Ib\x12\x12K[ßN\x1d,\íÃ\ÖkX÷*§öÓÙÄè(,¥Úå{8\x05qÝÓXÌÍf³ZµÉÝk¢âó\x1c×\x02\x15¿«mW°t»t\ÕÍÄP\x13a;\x15Sb\x15«U8\x0e\f1Ë£cXàèaØ:=©Áv\x17XlݲìY6Sk\x03²Ë\x1df\x15S³µ=TÚæznº6G'CÔi\x1a#¸ÉEjÕ[q\x1a·aË\x15µÜ;R~\x14;»:½"ô÷}ÈÉ\x0fòc\x1fº\x14½¨JåÔsFâÁÑ©MRµ~4ʰYBVM©ÉÙòä«\x16Udvc\x19pÕÀh³Òn\f\r\b\x04u\x18á\x06ávK*¦b>1ºÕ\x06ÌjÆ[^[cLh6¬KF\x14Æc ÐmbÙn\b4\x10®¯¦+/ä¼²ÊdÁçYeÊ\x0eò(ÕìÉ^\x15eW¥cðÅØÊî§°}ªQÔB±UNäLw½6£Ù÷ra=\x18*ñ<¦\x1dØÄ*]ËÙlÆB\x1f\x17j\x05\x15ä÷îÆwu\x10¯ xWÄ¢^,»1ÂI/º¯Qá62@eÞË2Ë%â XÊ\x17\x12Ò¯\f;·\x1c·z¡KGHRÔ=U*cÙ8\x18c1ÕeTýæ`èý:4b¯\f>Âö!:8p®§T˨2¡þQ_訷HV\x0fâèÒ/Íù\x0eî\x15·²ð¡ù\x0f¹üUØ<\x03ò\fvB«áDhaB·\x1f/ê¶Z¤© Æ\x16"e\x10®\x11îµWæÉ'1«\x10bÅ\±µI¦0cÕ\6Úü\x13³WÃò.¬K» ~B/´:I\bêÊ\x01p\x19ESQBÊ)X0éVÍ\L\v\x01âý*Ç4W,0c\x18ÅJ0Çáà76[1\x1a\x1dW*v°â\x1c\x17ÁÞ§$ jÓ Ã\f`-5Evn§èBºç\x18ð§è¨pyªtLv\x10XÅ$(]"}U·fêqu'g'\x14Êþ±È#V\x1aii¦F%jÂXÁóT©ãôÄW¼8´\fz´¥S^[µTæw\x1e'èÉ:$íGÛG5c¡ËNÛ,\x1c+L4Òjª©`I%"\x06PmcT$71 ¥TöÓyo[Íc7¬Ó@Æ«ô0ùmËì4ÐÁµþËt\x01åù¾ßĽ¥$¾×y<?."C©_ Q]*>ÒâÏù É2Ì:s \v¨òü\x03À\x03\x13ÿ\x10\x102\x05ÿÿÿ\0¬÷¡ô \0\x14\0\x14\x12(HR@ªU\0U(P\x01 \x05\b Q!\x1ahÐi\0\0\x06@\x18\x01¦\x06\0\0d\x01\x1ahÐi\0\0\x06@\x15ùU\0\0È\0\x01¦\x18&©!4\x11©Bj©¨d6¦'\fJ!©¤z&A\r\r\0\x03ÔÐpEWÂ'ñ¯Ú\x13ÏòDþ0A\x03\bw`(¢(¢(¢)Rµ ý6Ú\x13÷¡?±&Èý û+õb&ÔN(ê'hOí ÿp\x116DîÊ'DOëDÖèQ;BtDýèÑ5\x14¸)vCÔ´¿ÙKD_® æÌ'¼'¼Wí^§\x10Ñ;Bx®Q8®á5O\bZDÞa5 í ÞԮн].°Bi¥G\x10a6m â\x13o0Âz¢aèI±Ý.ÒWêÊÙµcRÐ\x1d×Q°hbFyª^\aUÚÚé|=^[¿÷yßkvzoé¾®pÚtdÙÑ»£\x1c8¸¶·´¶³é{êEñP|ðDÂI~ÄM"\x7fDM"~j&èTL¢nMèQ4¢eDÚDÝ\x13j&7DÄM¤L¢mR7ÄJÚ\x13 TLªKhLª!2\x13!7×\x19¨Må\x1a¨+B2¬Õ\x1añ¨LÈLRZÙJÒò¬Ô&ÐDNª +Õ\x13\x10ªú"byDÉ*LIJü"e\x14®G^ÊÓ\x19%ݦ̩³ª\x17ò¡S DÄL¢dId&"f!2\x13*T¶¦úRÈû'äeRÒÑBtÅ*ÓËX¥ú\x14°\x14¿QK¢fFôª[ä'¡9DÑ\x13²&*l\r¡4¢tÈMEKðúI5 %»\x10\x1c:)\x13ÇÖ\x13Ì&ÀÞ <"b& \x04ñ 4¥\x7fPñ·y)k!9Q;¢wÔ&\x19 Ú\x13!7Êý\x11?©Sù½/KÒô¼ïxE\x7f\x05-ËCæRÁGç þpäÄ'ñ8¦"~£ùÂm ¨Má?\x13*¿¼'ë Á\x13ªe\x13ýÑ?GìÞ\x13Åz¢zUòDéJ¬\x05uýÙ ÷DýQ=(ÔL¢t,<;Q;B\x7fÕDÕ\x13÷Då\x13z^a=¡6øÊ'&êW \x13ÂTºÎ»²èõ¡=(>ð(¢j\x13¯\bJ¾UÀ&sá\x13OM?hM"t¢{Bt¢x¥K(\x11<ÈÚ\x13d-Ñ1\x13zÈOXNÐî*6\x14»á^qХ̥µBjRqB~U}(;Q;"uDÊ&¡6í\Q=¨¨ïDó Ì'jGhNÕÓÇzÆUÝ\x13NùUËò|Ý¡;Âk²&ôNQ:¢p¤O²&ÈVQelv*^¨!¦q`ªÉÐÈÅíTµ)b#ÔÕRìM\rMÉ4ø¢t¢zå\x15¥=2 é µ\x13û²ö`¢u®M*?Z\x13\x11>©*MQ2STL\x10dæN\x10\x17#EKÊ©NÅFOJ\x15]Ñ5\x14}¤Na9«Ö"lêót:×"¥ËxO\x10'Mê'ïJ\bÔå\x13Ò\x13ÖÜ\x13KÖ8ØqìT½ª\x15.ø!ÍRìaRÿ%L\x14º*X\x0f\x15\x13ðH¡=á2[)Kë õøDÒ\x13hOul®¨\x11:BqÖª©æÄ'\x0f)\x0e!>u\x13 ¦2ÌDÉKSY\x18ª&Æ%.\r¤³\x11Úâ\x13^[Q2¼Eªr\x13¥i\x13q\x15²'§\x0fb¥ê·7)hRÜ¥º¥k)Kr%V\r\a®ÑRàÙ#Ê'Z'u+(Mоða=d¥Õ\x13 ' ¥]Ô9DôO¥\x13êµBt¡<ê^Ìÿ\bò'º'â\x13tMHBwú)^^["mDÞÕ\v¢'0á8lÅ\x13!Z*^rW3\x03\x06\rN¥.\b]a>îiUÄ&"\x7f\x04LWø¡yX)|-e-ÎX [ \x0e\x1eÆ'Ö\x13(*Y óTb&ï¡\x13P ïPòû(N ÜÝ)x\x15,û\x05KÅRò»\x02|¡2J]Q5 Í4Ê'ÖRóà\x1dG!U0bRÀÞ)x!4ÁwóJt,ªCÌ&\x04ì*ü¡5 î¥}XÕTJ\x13½rDÚ\x13ª&Bd&Bd&TMUb_z^8Dì òj¼&Bd'UÊQ8îQ:HÂ|á=Q5 Òì¨Æ$6Y Ë(4RDÄ%6¬¢s^(OÂ\x16Èô©z×Í:¼aØÅ¾"ÞRÔÐ¥²à©e ÙUVQ1Ú\x13\x15Ì'¾á9\aÁ!K22¥Ò\x13jõDïWj¬}¡>°XLe çw)iäU}\x11:ÔLªY=\x06VL\x19Y\x18°d¦V%ªµM%a¬c)Ò\x13JµBe\x19\x1aÊ\x13\x0f0 º&¡=BnÚ\x13éÏ\x13í^Q7ÄO´&5 ïKBt`ï½e,(O5QX*\Âeo òz½(O5ò®ò&B`¤OA\x13¬&ªæµ é ª¯ÍíÆ¶ÓM6uÞª1\x13!=T¯w¡Uçµ O0Q8¢j¨í É\x13jû"f¡5"uDìÝDôêÞDé ¥)|«ÐE\x1eðýa>h±%ã)*èÄM\x17®5ü\x1a/rRï^"-§('4´&¨vâ⺢e\x13\x13¤&\x1aåK¥K¼ðC ±t\x1a)\Æð\x0e¥KYKD¥Á¹QïXÝ\x13!>!>Pd'Â'z&ø ½2\x13A>ª'¾\x11:BqU\x1d]{"vûå\x13U\x13¸sÊ'µU\x1dQ5õÄMÑ;"eB¼¢rõ"lé í â¼&èÏ v\x15,¯LId&J[×f¡<êËr&Èä×\x10øÔ²\x13¼& K\x112¢d'ÙÚ\x13JO \x03Ä·¹ª\Y<ª-HZ¥\x1c¡`©oPéV¡=¡6Ô&¡2ùR\x1e>\x1dªÈMWdMªRå\x13xNjá\x13(ZÑ<+¥B}*\x1c\04TºJYT»h0\x15Î ÒRÚP{'¥Ü©u àRæRÔ¥ âm)d¥ÐTºÑ1Na6ðDÞ\x13h\x17\x15 &)K/ODÉ\x13(MH¢n*^ôO¼'0xOTOHOjÞU\x1aÖµi¦¾é^$O0¼&ìL¬Hú{Pá\x13¡T÷¬}a:BrîÊï*Ï&ÔMË\x05-Ê:\x03KÛ)l2·° &d'OhO4Oµ\x13PBj\x13!;2ÓvÕ[ã*ÈL¦BvÖf2c&"m Å\x13hM¨\x02qDÕ\x13hMºÖÊ'ΡMá43)r,\x1c[ß\b¢uH(NµF«\v\x14½1VQÅR;¢s ÑJ2âUî*Ý\x13dNê&\adæÞ"â\x13*e\x13\x1d$,éBt*VîQ:»ÂsB}è\x1aDÈN\x04\H\Ô¥Õ\x13´¥Ôu!9ª]êé\x13AKy\x7f¥? ÷R8R¼Ñ=+á"püD_OU}á2zBvDü{×z\x13Þ)WJ&BuÈMÊY)ir!0)xD\x1aª\Y³)m*mµCÅRì3 \v!=\x11:5 µ é Ê$¯(T¥ïDÈ\x17ÂðÿâîH§ \x12\x18(3à ^ permalink raw reply [flat|nested] 106+ messages in thread
* [RFC PATCH] explicitly mark recursion count 2004-06-01 5:56 ` 4k stacks in 2.6 [worst offenders] Jörn Engel @ 2004-06-01 6:02 ` Jörn Engel 2004-06-01 12:20 ` Pavel Machek 2004-06-01 12:39 ` viro 0 siblings, 2 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-01 6:02 UTC (permalink / raw) To: Linus Torvalds, Andrew Morton Cc: Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Tue, 1 June 2004 07:56:16 +0200, Jörn Engel wrote: > > So effectively, it comes down to the recursive paths. Unless someone > comes up with a semantical parser that can figure out the maximum > number of iterations, we have to look at them manually. Linus, Andrew, would you accept patches like the one below? With such information and assuming that the comments will get maintained, it's relatively simple to unroll recursions and measure stack comsumption more accurately. Jörn -- ticks = jiffies; while (ticks == jiffies); ticks = jiffies; -- /usr/src/linux/init/main.c Add recursion markers to teach automated test tools how bad documented recursions really are. Currently, there is only a single such too that can use the information and there is always the danger of documentation and reality getting out of sync. But until there's a better tool... Currently, this patch also has a few cleanup bits included. The cleanups were helpful to figure out the depth of some recursions and could be useful on their own. If not, they are easily removed. arch/i386/kernel/apm.c | 4 ++ drivers/char/random.c | 6 ++++ drivers/ide/ide-tape.c | 33 +++++++++++++----------- drivers/ide/ide-timing.h | 60 ++++++++++++++++++-------------------------- drivers/isdn/i4l/isdn_tty.c | 5 +++ drivers/isdn/icn/icn.c | 5 +++ fs/block_dev.c | 5 +++ fs/quota_v2.c | 43 +++++++++++++++++++------------ kernel/signal.c | 7 +++++ kernel/sysctl.c | 10 +++++++ 10 files changed, 113 insertions(+), 65 deletions(-) --- linux-2.6.6stack/arch/i386/kernel/apm.c~recursion 2004-05-10 18:10:06.000000000 +0200 +++ linux-2.6.6stack/arch/i386/kernel/apm.c 2004-05-30 18:24:54.000000000 +0200 @@ -1058,6 +1058,10 @@ * monitor powerdown for us. */ +/** + * RECURSION: 2 + * STEP: apm_console_blank + */ static int apm_console_blank(int blank) { int error; --- linux-2.6.6stack/kernel/sysctl.c~recursion 2004-05-30 17:51:03.000000000 +0200 +++ linux-2.6.6stack/kernel/sysctl.c 2004-05-30 17:52:25.000000000 +0200 @@ -1188,6 +1188,11 @@ #ifdef CONFIG_PROC_FS /* Scan the sysctl entries in table and add them all into /proc */ + +/** + * RECURSION: 100 + * STEP: register_proc_table + */ static void register_proc_table(ctl_table * table, struct proc_dir_entry *root) { struct proc_dir_entry *de; @@ -1237,6 +1242,11 @@ /* * Unregister a /proc sysctl table and any subdirectories. */ + +/** + * RECURSION: 100 + * STEP: unregister_proc_table + */ static void unregister_proc_table(ctl_table * table, struct proc_dir_entry *root) { struct proc_dir_entry *de; --- linux-2.6.6stack/kernel/signal.c~recursion 2004-05-10 18:10:38.000000000 +0200 +++ linux-2.6.6stack/kernel/signal.c 2004-05-30 18:24:38.000000000 +0200 @@ -626,6 +626,13 @@ * actual continuing for SIGCONT, but not the actual stopping for stop * signals. The process stop is done as a signal action for SIG_DFL. */ + +/** + * RECURSION: 2 + * STEP: handle_stop_signal + * STEP: do_notify_parent_cldstop + * STEP: __group_send_sig_info + */ static void handle_stop_signal(int sig, struct task_struct *p) { struct task_struct *t; --- linux-2.6.6stack/fs/block_dev.c~recursion 2004-05-10 18:10:30.000000000 +0200 +++ linux-2.6.6stack/fs/block_dev.c 2004-05-31 17:20:53.000000000 +0200 @@ -547,6 +547,11 @@ } EXPORT_SYMBOL(bd_set_size); +/** + * RECURSION: 2 + * STEP: do_open + * STEP: blkdev_get + */ static int do_open(struct block_device *bdev, struct file *file) { struct module *owner = NULL; --- linux-2.6.6stack/fs/quota_v2.c~recursion 2004-05-10 18:10:32.000000000 +0200 +++ linux-2.6.6stack/fs/quota_v2.c 2004-05-30 18:36:23.000000000 +0200 @@ -352,6 +352,12 @@ } /* Insert reference to structure into the trie */ + +/** + * Recursion count equals V2_DQTREEDEPTH, keep both in sync + * RECURSION: 4 + * STEP: do_insert_tree + */ static int do_insert_tree(struct dquot *dquot, uint *treeblk, int depth) { struct file *filp = sb_dqopt(dquot->dq_sb)->files[dquot->dq_type]; @@ -369,12 +375,9 @@ *treeblk = ret; memset(buf, 0, V2_DQBLKSIZE); newact = 1; - } - else { - if ((ret = read_blk(filp, *treeblk, buf)) < 0) { - printk(KERN_ERR "VFS: Can't read tree quota block %u.\n", *treeblk); - goto out_buf; - } + } else if ((ret = read_blk(filp, *treeblk, buf)) < 0) { + printk(KERN_ERR "VFS: Can't read tree quota block %u.\n", *treeblk); + goto out_buf; } ref = (u32 *)buf; newblk = le32_to_cpu(ref[GETIDINDEX(dquot->dq_id, depth)]); @@ -389,14 +392,12 @@ } #endif newblk = find_free_dqentry(dquot, &ret); - } - else + } else ret = do_insert_tree(dquot, &newblk, depth+1); if (newson && ret >= 0) { ref[GETIDINDEX(dquot->dq_id, depth)] = cpu_to_le32(newblk); ret = write_blk(filp, *treeblk, buf); - } - else if (newact && ret < 0) + } else if (newact && ret < 0) put_free_dqblk(filp, dquot->dq_type, buf, *treeblk); out_buf: freedqbuf(buf); @@ -498,6 +499,12 @@ } /* Remove reference to dquot from tree */ + +/** + * Recursion count equals V2_DQTREEDEPTH, keep both in sync + * RECURSION: 4 + * STEP: remove_tree + */ static int remove_tree(struct dquot *dquot, uint *blk, int depth) { struct file *filp = sb_dqopt(dquot->dq_sb)->files[dquot->dq_type]; @@ -516,19 +523,17 @@ if (depth == V2_DQTREEDEPTH-1) { ret = free_dqentry(dquot, newblk); newblk = 0; - } - else + } else ret = remove_tree(dquot, &newblk, depth+1); if (ret >= 0 && !newblk) { int i; ref[GETIDINDEX(dquot->dq_id, depth)] = cpu_to_le32(0); - for (i = 0; i < V2_DQBLKSIZE && !buf[i]; i++); /* Block got empty? */ + for (i = 0; i < V2_DQBLKSIZE && !buf[i]; i++) + ; /* Block got empty? */ if (i == V2_DQBLKSIZE) { put_free_dqblk(filp, dquot->dq_type, buf, *blk); *blk = 0; - } - else - if ((ret = write_blk(filp, *blk, buf)) < 0) + } else if ((ret = write_blk(filp, *blk, buf)) < 0) printk(KERN_ERR "VFS: Can't write quota tree block %u.\n", *blk); } out_buf: @@ -584,6 +589,12 @@ } /* Find entry for given id in the tree */ + +/** + * Recursion count equals V2_DQTREEDEPTH, keep both in sync + * RECURSION: 4 + * STEP: find_tree_dqentry + */ static loff_t find_tree_dqentry(struct dquot *dquot, uint blk, int depth) { struct file *filp = sb_dqopt(dquot->dq_sb)->files[dquot->dq_type]; --- linux-2.6.6stack/drivers/char/random.c~recursion 2004-05-10 18:10:23.000000000 +0200 +++ linux-2.6.6stack/drivers/char/random.c 2004-05-30 18:48:55.000000000 +0200 @@ -1311,6 +1311,12 @@ * from the primary pool to the secondary extraction pool. We make * sure we pull enough for a 'catastrophic reseed'. */ + +/** + * RECURSION: 2 + * STEP: xfer_secondary_pool + * STEP: extract_entropy + */ static inline void xfer_secondary_pool(struct entropy_store *r, size_t nbytes, __u32 *tmp) { --- linux-2.6.6stack/drivers/ide/ide-timing.h~recursion 2004-01-09 07:59:26.000000000 +0100 +++ linux-2.6.6stack/drivers/ide/ide-timing.h 2004-05-31 16:56:23.000000000 +0200 @@ -208,63 +208,53 @@ return t; } +/** + * RECURSION: 2 + * STEP: ide_timing_compute + */ static int ide_timing_compute(ide_drive_t *drive, short speed, struct ide_timing *t, int T, int UT) { struct hd_driveid *id = drive->id; struct ide_timing *s, p; -/* - * Find the mode. - */ - - if (!(s = ide_timing_find_mode(speed))) + s = ide_timing_find_mode(speed); + if (!s) return -EINVAL; -/* - * If the drive is an EIDE drive, it can tell us it needs extended - * PIO/MWDMA cycle timing. - */ - - if (id && id->field_valid & 2) { /* EIDE drive */ - + /* If the drive is an EIDE drive, it can tell us it needs extended + * PIO/MWDMA cycle timing. + */ + if (id && (id->field_valid & 2)) { /* EIDE drive */ memset(&p, 0, sizeof(p)); switch (speed & XFER_MODE) { + case XFER_PIO: + if (speed <= XFER_PIO_2) + p.cycle = p.cyc8b = id->eide_pio; + else + p.cycle = p.cyc8b = id->eide_pio_iordy; + break; - case XFER_PIO: - if (speed <= XFER_PIO_2) p.cycle = p.cyc8b = id->eide_pio; - else p.cycle = p.cyc8b = id->eide_pio_iordy; - break; - - case XFER_MWDMA: - p.cycle = id->eide_dma_min; - break; + case XFER_MWDMA: + p.cycle = id->eide_dma_min; + break; } - ide_timing_merge(&p, t, t, IDE_TIMING_CYCLE | IDE_TIMING_CYC8B); } -/* - * Convert the timing to bus clock counts. - */ - + /* Convert the timing to bus clock counts. */ ide_timing_quantize(s, t, T, UT); -/* - * Even in DMA/UDMA modes we still use PIO access for IDENTIFY, S.M.A.R.T - * and some other commands. We have to ensure that the DMA cycle timing is - * slower/equal than the fastest PIO timing. - */ - + /* Even in DMA/UDMA modes we still use PIO access for IDENTIFY, + * S.M.A.R.T and some other commands. We have to ensure that the + * DMA cycle timing is slower/equal than the fastest PIO timing. + */ if ((speed & XFER_MODE) != XFER_PIO) { ide_timing_compute(drive, ide_find_best_mode(drive, XFER_PIO | XFER_EPIO), &p, T, UT); ide_timing_merge(&p, t, t, IDE_TIMING_ALL); } -/* - * Lenghten active & recovery time so that cycle time is correct. - */ - + /* Lenghten active & recovery time so that cycle time is correct. */ if (t->act8b + t->rec8b < t->cyc8b) { t->act8b += (t->cyc8b - (t->act8b + t->rec8b)) / 2; t->rec8b = t->cyc8b - t->act8b; --- linux-2.6.6stack/drivers/ide/ide-tape.c~recursion 2004-05-10 18:10:24.000000000 +0200 +++ linux-2.6.6stack/drivers/ide/ide-tape.c 2004-05-31 16:58:30.000000000 +0200 @@ -3653,6 +3653,11 @@ * the filemark is in our internal pipeline even if the tape doesn't * support spacing over filemarks in the reverse direction. */ + +/** + * RECURSION: 2 + * STEP: idetape_space_over_filemarks + */ static int idetape_space_over_filemarks (ide_drive_t *drive,short mt_op,int mt_count) { idetape_tape_t *tape = drive->driver_data; @@ -3711,21 +3716,21 @@ * Now we can issue the space command. */ switch (mt_op) { - case MTFSF: - case MTBSF: - idetape_create_space_cmd(&pc,mt_count-count,IDETAPE_SPACE_OVER_FILEMARK); - return (idetape_queue_pc_tail(drive, &pc)); - case MTFSFM: - case MTBSFM: - if (!tape->capabilities.sprev) - return (-EIO); - retval = idetape_space_over_filemarks(drive, MTFSF, mt_count-count); - if (retval) return (retval); - count = (MTBSFM == mt_op ? 1 : -1); - return (idetape_space_over_filemarks(drive, MTFSF, count)); - default: - printk(KERN_ERR "ide-tape: MTIO operation %d not supported\n",mt_op); + case MTFSF: + case MTBSF: + idetape_create_space_cmd(&pc,mt_count-count,IDETAPE_SPACE_OVER_FILEMARK); + return (idetape_queue_pc_tail(drive, &pc)); + case MTFSFM: + case MTBSFM: + if (!tape->capabilities.sprev) return (-EIO); + retval = idetape_space_over_filemarks(drive, MTFSF, mt_count-count); + if (retval) return (retval); + count = (MTBSFM == mt_op ? 1 : -1); + return (idetape_space_over_filemarks(drive, MTFSF, count)); + default: + printk(KERN_ERR "ide-tape: MTIO operation %d not supported\n",mt_op); + return (-EIO); } } --- linux-2.6.6stack/drivers/isdn/i4l/isdn_tty.c~recursion 2004-03-28 21:51:38.000000000 +0200 +++ linux-2.6.6stack/drivers/isdn/i4l/isdn_tty.c 2004-05-31 17:02:39.000000000 +0200 @@ -2519,6 +2519,11 @@ * For RING-message handle auto-ATA if register 0 != 0 */ +/** + * RECURSION: 2 + * STEP: isdn_tty_modem_result + * STEP: isdn_tty_cmd_ATA + */ static void isdn_tty_modem_result(int code, modem_info * info) { --- linux-2.6.6stack/drivers/isdn/icn/icn.c~recursion 2004-03-28 21:51:38.000000000 +0200 +++ linux-2.6.6stack/drivers/isdn/icn/icn.c 2004-05-31 17:03:55.000000000 +0200 @@ -1097,6 +1097,11 @@ /* * Delete card's pending timers, send STOP to linklevel */ + +/** + * RECURSION: 2 + * STEP: icn_stopcard + */ static void icn_stopcard(icn_card * card) { ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 6:02 ` [RFC PATCH] explicitly mark recursion count Jörn Engel @ 2004-06-01 12:20 ` Pavel Machek 2004-06-01 13:27 ` Jörn Engel 2004-06-01 19:29 ` Horst von Brand 2004-06-01 12:39 ` viro 1 sibling, 2 replies; 106+ messages in thread From: Pavel Machek @ 2004-06-01 12:20 UTC (permalink / raw) To: Jörn Engel Cc: Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel Hi! > > So effectively, it comes down to the recursive paths. Unless someone > > comes up with a semantical parser that can figure out the maximum > > number of iterations, we have to look at them manually. > > Linus, Andrew, would you accept patches like the one below? With such > information and assuming that the comments will get maintained, it's > relatively simple to unroll recursions and measure stack comsumption > more accurately. Perhaps some other format of comment should be introduced? Will not this interfere with linuxdoc? Pavel -- 934a471f20d6580d5aad759bf0d97ddc ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 12:20 ` Pavel Machek @ 2004-06-01 13:27 ` Jörn Engel 2004-06-01 13:32 ` Pavel Machek 2004-06-01 19:29 ` Horst von Brand 1 sibling, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-01 13:27 UTC (permalink / raw) To: Pavel Machek Cc: Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Tue, 1 June 2004 14:20:13 +0200, Pavel Machek wrote: > > Perhaps some other format of comment should be introduced? Will not > this interfere with linuxdoc? I'm open for suggestions. ;) Jörn -- Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. -- Rob Pike ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 13:27 ` Jörn Engel @ 2004-06-01 13:32 ` Pavel Machek 2004-06-01 13:37 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Pavel Machek @ 2004-06-01 13:32 UTC (permalink / raw) To: Jörn Engel Cc: Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel Hi! > > Perhaps some other format of comment should be introduced? Will not > > this interfere with linuxdoc? > > I'm open for suggestions. ;) /*! Recursion-count: 2 Whatever-else: 5 */ ? Pavel -- 934a471f20d6580d5aad759bf0d97ddc ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 13:32 ` Pavel Machek @ 2004-06-01 13:37 ` Jörn Engel 2004-06-01 19:48 ` Horst von Brand 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-01 13:37 UTC (permalink / raw) To: Pavel Machek Cc: Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Tue, 1 June 2004 15:32:29 +0200, Pavel Machek wrote: > > > > I'm open for suggestions. ;) > > /*! Recursion-count: 2 Whatever-else: 5 */ What I need is: 1. Recursion count 2. All functions involved in the recursion in the correct order (a calls b calls c calls d calls a, something like that). How do you pass 2? Jörn -- ticks = jiffies; while (ticks == jiffies); ticks = jiffies; -- /usr/src/linux/init/main.c ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 13:37 ` Jörn Engel @ 2004-06-01 19:48 ` Horst von Brand 0 siblings, 0 replies; 106+ messages in thread From: Horst von Brand @ 2004-06-01 19:48 UTC (permalink / raw) To: Jörn Engel Cc: Pavel Machek, Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel =?iso-8859-1?Q?J=F6rn?= Engel <joern@wohnheim.fh-wedel.de> said: [...] > What I need is: > 1. Recursion count How do you know the limit is enforced? By guessing? > 2. All functions involved in the recursion in the correct order (a > calls b calls c calls d calls a, something like that). But also b calls f calls g... Maintaining the full possible-call-graph is a _huge_ job, better done automatically (because somebody _will_screw up when doing it by hand). Plus the fun "structure spicked with all sorts of function pointers" object implementation in the kernel... note that your carefully maintained call graph/counts can be screwed up royally by any random third-party device driver or filesystem module. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 12:20 ` Pavel Machek 2004-06-01 13:27 ` Jörn Engel @ 2004-06-01 19:29 ` Horst von Brand 2004-06-01 19:58 ` Linus Torvalds 1 sibling, 1 reply; 106+ messages in thread From: Horst von Brand @ 2004-06-01 19:29 UTC (permalink / raw) To: Pavel Machek Cc: Jörn Engel, Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel Pavel Machek <pavel@suse.cz> said: > =?iso-8859-1?Q?J=F6rn?= Engel <joern@wohnheim.fh-wedel.de> said: > > > So effectively, it comes down to the recursive paths. Unless someone > > > comes up with a semantical parser that can figure out the maximum > > > number of iterations, we have to look at them manually. > > Linus, Andrew, would you accept patches like the one below? With such > > information and assuming that the comments will get maintained, it's > > relatively simple to unroll recursions and measure stack comsumption > > more accurately. > > Perhaps some other format of comment should be introduced? Will not > this interfere with linuxdoc? If the comment gets out of sync, you are toast. Too easy for that to happen, IMVHO. -- Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513 ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 19:29 ` Horst von Brand @ 2004-06-01 19:58 ` Linus Torvalds 2004-06-02 13:16 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Linus Torvalds @ 2004-06-01 19:58 UTC (permalink / raw) To: Horst von Brand Cc: Pavel Machek, Jörn Engel, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Tue, 1 Jun 2004, Horst von Brand wrote: > > If the comment gets out of sync, you are toast. Too easy for that to > happen, IMVHO. Yes. Recursion should be detectable automatically, the only thing you can't detect easily is the reason to _break_ recursion. So how about just having a simple loop finder, and then the only comment you need is a simple /* max recursion: N */ for any point in the loop. That still makes it interesting if one function is part of two loops, and is logically the place that breaks the recursion for one (or both - with different logic) of them. But does that actually happen? Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 19:58 ` Linus Torvalds @ 2004-06-02 13:16 ` Jörn Engel 2004-06-02 14:15 ` Linus Torvalds 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 13:16 UTC (permalink / raw) To: Linus Torvalds Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Tue, 1 June 2004 12:58:12 -0700, Linus Torvalds wrote: > On Tue, 1 Jun 2004, Horst von Brand wrote: > > > > If the comment gets out of sync, you are toast. Too easy for that to > > happen, IMVHO. > > Yes. > > Recursion should be detectable automatically, the only thing you can't > detect easily is the reason to _break_ recursion. Correct. My tool already detects recursions and prints warning, it just cannot make out the harmful ones and gives a warning for each. > So how about just having a simple loop finder, and then the only comment > you need is a simple /* max recursion: N */ for any point in the loop. That's what I basically want, at least for trivial recursions with only one function involved. For a->b->c->a type recursions, I also need to identify all involved functions in the correct order, that's where my ugly format comes from. RECURSION: 2 STEP: a STEP: b STEP: c This mean that the recursion from a to b to c and back can happen twice at most. Sure, the format is ugly. If anyone really cares I can change it into any other. But it gets the job done, so I don't care. > That still makes it interesting if one function is part of two loops, and > is logically the place that breaks the recursion for one (or both - with > different logic) of them. But does that actually happen? "Interesting" is the wrong word, really. Imo it doesn't make any sense to write such code and therefore I don't want to deal with it either. Print a warning and be done with it. See my output: WARNING: multiple recursions around check_sig() Jörn -- A victorious army first wins and then seeks battle. -- Sun Tzu ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 13:16 ` Jörn Engel @ 2004-06-02 14:15 ` Linus Torvalds 2004-06-02 14:27 ` Jörn Engel 2004-06-02 14:35 ` Davide Libenzi 0 siblings, 2 replies; 106+ messages in thread From: Linus Torvalds @ 2004-06-02 14:15 UTC (permalink / raw) To: Jörn Engel Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 Jun 2004, Jörn Engel wrote: > > For a->b->c->a type recursions, I also need to identify all involved > functions in the correct order, that's where my ugly format comes > from. Why? You really only need to know that _one_ of the entries break the recursion, and you need to know what the break depth is for that one entry. So for example, if "c" is annotated with "max recursion: 5", then you know that the above loop will recurse at most 5 times. I don't see why you need any other information. Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 14:15 ` Linus Torvalds @ 2004-06-02 14:27 ` Jörn Engel 2004-06-02 14:45 ` Linus Torvalds 2004-06-02 14:35 ` Davide Libenzi 1 sibling, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 14:27 UTC (permalink / raw) To: Linus Torvalds Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 June 2004 07:15:07 -0700, Linus Torvalds wrote: > On Wed, 2 Jun 2004, Jörn Engel wrote: > > > > For a->b->c->a type recursions, I also need to identify all involved > > functions in the correct order, that's where my ugly format comes > > from. > > Why? > > You really only need to know that _one_ of the entries break the > recursion, and you need to know what the break depth is for that one > entry. > > So for example, if "c" is annotated with "max recursion: 5", then you know > that the above loop will recurse at most 5 times. > > I don't see why you need any other information. Then you see something I don't see. For example there are quite a few recursions with some function like void foo(int depth) { if (!depth) { bar(1); } ... } bar will, maybe through several more functions call foo(1). How can you say that foo will beak this recursion after two rounds max? What if a changed bar decrements the depth value? The recursion will run for infinity now, won't it? And whoever changed bar doesn't have a clue that he now fucked up your comment to foo. I claim: There is no way to tell the depth of any recursion without looking at all involved functions. Prove me wrong, please. Jörn -- When in doubt, use brute force. -- Ken Thompson ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 14:27 ` Jörn Engel @ 2004-06-02 14:45 ` Linus Torvalds 2004-06-02 15:04 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Linus Torvalds @ 2004-06-02 14:45 UTC (permalink / raw) To: Jörn Engel Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 Jun 2004, Jörn Engel wrote: > > Then you see something I don't see. For example there are quite a few > recursions with some function like > > void foo(int depth) > { > if (!depth) { > bar(1); > } > ... > } > > bar will, maybe through several more functions call foo(1). > > How can you say that foo will beak this recursion after two rounds > max? The programmer had _better_ know that there is some upper limit. > I claim: > There is no way to tell the depth of any recursion without looking at > all involved functions. And I claim: recursion is illegal unless the programmer has some explicit recursion limiter. And if he has that recursion limiter in one of the functions, then he damn well better know it, and know the value it limits recursion to. Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 14:45 ` Linus Torvalds @ 2004-06-02 15:04 ` Jörn Engel 2004-06-02 15:12 ` Linus Torvalds 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 15:04 UTC (permalink / raw) To: Linus Torvalds Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 June 2004 07:45:10 -0700, Linus Torvalds wrote: > > The programmer had _better_ know that there is some upper limit. > > And I claim: recursion is illegal unless the programmer has some explicit > recursion limiter. And if he has that recursion limiter in one of the > functions, then he damn well better know it, and know the value it limits > recursion to. Can I read this as: Linus himself will use strong words to enforce all recursions in the kernel to be either removed or properly documented. In that case, you have 273 recursions to deal with. They are all in the data I attached a few posts back. Recursions would basically be in the same league as huge stack hogs, sounds good. Jörn -- Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. -- Doug MacIlroy ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 15:04 ` Jörn Engel @ 2004-06-02 15:12 ` Linus Torvalds 2004-06-02 15:27 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Linus Torvalds @ 2004-06-02 15:12 UTC (permalink / raw) To: Jörn Engel Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 Jun 2004, Jörn Engel wrote: > > Can I read this as: > Linus himself will use strong words to enforce all recursions in the > kernel to be either removed or properly documented. If we have a good detector that is reliable and easy to run, why not? It will take some time, but I think the problem so far has been that the recursion can be hard to see. Some "core" cases are well-known (memory allocations during memory allocation, and filename lookup), and they should be trivial to annotate. Knock wood. Others might be worse. > In that case, you have 273 recursions to deal with. They are all in > the data I attached a few posts back. Recursions would basically be > in the same league as huge stack hogs, sounds good. Yes. And with huge stack hogs, we've not exactly "fixed them all in a weekend", have we? But having a few people run the checking tools and nagging every once in a while ends up eventually fixing things. At least the most common ones. Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 15:12 ` Linus Torvalds @ 2004-06-02 15:27 ` Jörn Engel 2004-06-02 15:52 ` Linus Torvalds 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 15:27 UTC (permalink / raw) To: Linus Torvalds Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 June 2004 08:12:00 -0700, Linus Torvalds wrote: > On Wed, 2 Jun 2004, Jörn Engel wrote: > > > > Can I read this as: > > Linus himself will use strong words to enforce all recursions in the > > kernel to be either removed or properly documented. > > If we have a good detector that is reliable and easy to run, why not? Great! So the official format to document recursions is plain english for human readers? > It will take some time, but I think the problem so far has been that the > recursion can be hard to see. Some "core" cases are well-known (memory > allocations during memory allocation, and filename lookup), and they > should be trivial to annotate. Knock wood. Others might be worse. For sure. There are some functions with multiple recursions around them, real fun! :) > > In that case, you have 273 recursions to deal with. They are all in > > the data I attached a few posts back. Recursions would basically be > > in the same league as huge stack hogs, sounds good. > > Yes. And with huge stack hogs, we've not exactly "fixed them all in a > weekend", have we? But having a few people run the checking tools and > nagging every once in a while ends up eventually fixing things. At least > the most common ones. s/a few people/Jörn/ Legal reasons. I'll try to do this from time to time. Jörn -- To recognize individual spam features you have to try to get into the mind of the spammer, and frankly I want to spend as little time inside the minds of spammers as possible. -- Paul Graham ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 15:27 ` Jörn Engel @ 2004-06-02 15:52 ` Linus Torvalds 2004-06-02 16:17 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Linus Torvalds @ 2004-06-02 15:52 UTC (permalink / raw) To: Jörn Engel Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 Jun 2004, Jörn Engel wrote: > > > > If we have a good detector that is reliable and easy to run, why not? > > Great! So the official format to document recursions is plain english > for human readers? Actually, I'd suggest making it a preprocessor directive at the very top of the function itself. That way I can make sparse parse it too if I ever want to. So something like /* * This function is part of recursion, but we limit * the depth with the "depth" parameter to 5 */ int my_recursive_function(int depth, struct datastruct *arg) { declare_recursion_depth(5); ... where we could either just #define it to some no-op like #define declare_recursion_depth(x) \ extern void __dummy_function_never_used() or even do something fancier: #define declare_recursion_depth(x) \ static __init char recursion[] = __FILE__ ":" __FUNCTION__ "=" #x; which will generate a nice clean variable that shows up in "objdump" in all its glory, so that pretty much any tool can trivially parse it. And for something like sparse or other automated tools, we can trivially make it be something more appropriate, ie #define recursion_depth(x) \ __builtin_recursion_depth(x) or whatever checker-specific thing we want it to be. And other tools should be equally able to easily pick it up, either by just looking at the object file or re-defining the macro to suit themselves. I think the above syntax is both human-readable and "obviously parseable" in many trivial ways. Whaddaya think? Works for you? Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 15:52 ` Linus Torvalds @ 2004-06-02 16:17 ` Jörn Engel 2004-06-02 16:25 ` Linus Torvalds 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 16:17 UTC (permalink / raw) To: Linus Torvalds Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 June 2004 08:52:53 -0700, Linus Torvalds wrote: > > I think the above syntax is both human-readable and "obviously parseable" > in many trivial ways. Whaddaya think? Works for you? Works for me for trivial recursions (just one function involved. With a little more pain, it should work for basically everything. Only exception are multiple recursions around the same function. So unless you like to keep those suckers, I'm fine with it. Jörn -- The cheapest, fastest and most reliable components of a computer system are those that aren't there. -- Gordon Bell, DEC labratories ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 16:17 ` Jörn Engel @ 2004-06-02 16:25 ` Linus Torvalds 2004-06-02 17:17 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Linus Torvalds @ 2004-06-02 16:25 UTC (permalink / raw) To: Jörn Engel Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 Jun 2004, Jörn Engel wrote: > > Works for me for trivial recursions (just one function involved. With > a little more pain, it should work for basically everything. Only > exception are multiple recursions around the same function. So unless > you like to keep those suckers, I'm fine with it. Well, multiple recursion around the same function seems to be solvable two different ways: - "don't do that then". It really seems broken, but maybe there are really really good reasons _why_ it's not broken and why it happens. - make sure that the separate loops are broken in some _other_ place than in the function they share. A combination of the two may work well. I say "may", because maybe I'm wrong, and the condition is common and hard to avoid limiting in the shared function. I don't have your data (and I'm lazy, so quite frankly I'd much rather you do the analysis anyway ;). That said, I just don't see any sane alternatives to my "break in one place" thing. I believe that anything more complex that tries to explain the whole loop is just going to be a nightmare to maintain, and fragile as hell except for totally static legacy code that nobody touches any more. Linus ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 16:25 ` Linus Torvalds @ 2004-06-02 17:17 ` Jörn Engel 2004-06-02 17:32 ` Greg KH 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 17:17 UTC (permalink / raw) To: Linus Torvalds Cc: Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel, Greg KH On Wed, 2 June 2004 09:25:50 -0700, Linus Torvalds wrote: > > Well, multiple recursion around the same function seems to be solvable two > different ways: > - "don't do that then". It really seems broken, but maybe there are > really really good reasons _why_ it's not broken and why it happens. > - make sure that the separate loops are broken in some _other_ place than > in the function they share. > > A combination of the two may work well. > > I say "may", because maybe I'm wrong, and the condition is common and hard > to avoid limiting in the shared function. I don't have your data (and I'm > lazy, so quite frankly I'd much rather you do the analysis anyway ;). You *do* have my data, but I buy the lazy argument. ;) > That said, I just don't see any sane alternatives to my "break in one > place" thing. I believe that anything more complex that tries to explain > the whole loop is just going to be a nightmare to maintain, and fragile as > hell except for totally static legacy code that nobody touches any more. Amen, brother. Anything complicated will contain bugs tomorrow, if not already today. Let's see. Right now, we have two cases of multiple recursions: WARNING: multiple recursions around check_sig() WARNING: multiple recursions around usb_audio_recurseunit() check_sig() is bad code and can be cleaned up. Leaves usb_audio_recurseunit() as the only function in question, that one could actually be sane, although it looks rather interesting: WARNING: trivial recursion detected: 0 usb_audio_recurseunit WARNING: recursion detected: 16 usb_audio_selectorunit 0 usb_audio_recurseunit WARNING: multiple recursions around usb_audio_recurseunit() WARNING: recursion detected: 0 usb_audio_recurseunit 0 usb_audio_processingunit Greg, can you say whether this construct makes sense? Jörn -- Eighty percent of success is showing up. -- Woody Allen ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 17:17 ` Jörn Engel @ 2004-06-02 17:32 ` Greg KH 2004-06-02 17:46 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Greg KH @ 2004-06-02 17:32 UTC (permalink / raw) To: J?rn Engel Cc: Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, Jun 02, 2004 at 07:17:32PM +0200, J?rn Engel wrote: > > Leaves usb_audio_recurseunit() as the only function in question, that > one could actually be sane, although it looks rather interesting: > WARNING: trivial recursion detected: > 0 usb_audio_recurseunit > WARNING: recursion detected: > 16 usb_audio_selectorunit > 0 usb_audio_recurseunit > WARNING: multiple recursions around usb_audio_recurseunit() > WARNING: recursion detected: > 0 usb_audio_recurseunit > 0 usb_audio_processingunit > > Greg, can you say whether this construct makes sense? Well it's sane only if you think that USB descriptors can be sane :) Anyway, this loop will always terminate as we have a finite sized USB descriptor that this function is parsing. As to how many times we will recurse, I don't really know as I haven't spent much time looking into the different messed up USB audio devices out there on the market... Sorry I can't be of more help, but I don't think you need to worry about this function. thanks, greg k-h ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 17:32 ` Greg KH @ 2004-06-02 17:46 ` Jörn Engel 0 siblings, 0 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-02 17:46 UTC (permalink / raw) To: Greg KH Cc: Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 June 2004 10:32:00 -0700, Greg KH wrote: > On Wed, Jun 02, 2004 at 07:17:32PM +0200, J?rn Engel wrote: > > > > Leaves usb_audio_recurseunit() as the only function in question, that > > one could actually be sane, although it looks rather interesting: > > WARNING: trivial recursion detected: > > 0 usb_audio_recurseunit > > WARNING: recursion detected: > > 16 usb_audio_selectorunit > > 0 usb_audio_recurseunit > > WARNING: multiple recursions around usb_audio_recurseunit() > > WARNING: recursion detected: > > 0 usb_audio_recurseunit > > 0 usb_audio_processingunit > > > > Greg, can you say whether this construct makes sense? > > Well it's sane only if you think that USB descriptors can be sane :) > > Anyway, this loop will always terminate as we have a finite sized USB > descriptor that this function is parsing. As to how many times we will > recurse, I don't really know as I haven't spent much time looking into > the different messed up USB audio devices out there on the market... > > Sorry I can't be of more help, but I don't think you need to worry about > this function. That's ok. At least you can talk about that code without obvious disgust, which is a quality criterium in itself. Leaves exactly one multiply recursive function we might want to keep. So I just won't worry too much and ignore the warnings about it, fine. Jörn -- ticks = jiffies; while (ticks == jiffies); ticks = jiffies; -- /usr/src/linux/init/main.c ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 14:15 ` Linus Torvalds 2004-06-02 14:27 ` Jörn Engel @ 2004-06-02 14:35 ` Davide Libenzi 2004-06-02 18:20 ` Jörn Engel 1 sibling, 1 reply; 106+ messages in thread From: Davide Libenzi @ 2004-06-02 14:35 UTC (permalink / raw) To: Linus Torvalds Cc: Jörn Engel, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 Jun 2004, Linus Torvalds wrote: > On Wed, 2 Jun 2004, Jörn Engel wrote: > > > > For a->b->c->a type recursions, I also need to identify all involved > > functions in the correct order, that's where my ugly format comes > > from. > > Why? > > You really only need to know that _one_ of the entries break the > recursion, and you need to know what the break depth is for that one > entry. > > So for example, if "c" is annotated with "max recursion: 5", then you know > that the above loop will recurse at most 5 times. > > I don't see why you need any other information. Hmmm, I see more data to maintain to support a method that will never be even close to be perfect. How the thing works with callback triggered loops for example, where a function is not directly called, but instead is passed to another function (that maybe pass it to another function) that triggers the call. Or maybe it's another set of functions that might trigger the call (think about all *_operations for example). Eg, there's an evident loop possibility in epoll (triggered by callback'd wakeups) that is not in the list, and it's pretty hard to detect. Doing stack usage analisys from the source code is not easy, once you abbandon the trivial direct call scenario. - Davide ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 14:35 ` Davide Libenzi @ 2004-06-02 18:20 ` Jörn Engel 2004-06-02 18:37 ` Davide Libenzi 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 18:20 UTC (permalink / raw) To: Davide Libenzi Cc: Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Wed, 2 June 2004 07:35:39 -0700, Davide Libenzi wrote: > > Hmmm, I see more data to maintain to support a method that will never be > even close to be perfect. You get it wrong. This is mainly about Bad Code or Insufficient Documentation. In general, I want recursions to be removed, full stop. So there is not more data, but less. If the recursion is actually wanted, then those cases should either be so few and obvious that a single person can explain them all from memory. That, or things have to be written down somehow and unless you have a better suggestion, any format is better than nothing. Jörn -- Public Domain - Free as in Beer General Public - Free as in Speech BSD License - Free as in Enterprise Shared Source - Free as in "Work will make you..." ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 18:20 ` Jörn Engel @ 2004-06-02 18:37 ` Davide Libenzi 2004-06-02 18:58 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Davide Libenzi @ 2004-06-02 18:37 UTC (permalink / raw) To: Jörn Engel Cc: Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List On Wed, 2 Jun 2004, [iso-8859-1] Jörn Engel wrote: > On Wed, 2 June 2004 07:35:39 -0700, Davide Libenzi wrote: > > > > Hmmm, I see more data to maintain to support a method that will never be > > even close to be perfect. > > You get it wrong. This is mainly about Bad Code or Insufficient > Documentation. In general, I want recursions to be removed, full > stop. So there is not more data, but less. You're requesting to add and maintain data to feed a tool that catches only trivially visible recursion. I don't want to waste mine and your time explaining why your tool will never work, but if you want an hint, you can start thinking about all functions that sets/pass callbacks and/or sets operational functions. I don't know if you noticed that, but Linux is heavily function-pointer driven. Eg, one function setups a set of function pointers, and another 317 indirectly calls them. Having such comments, not only makes the maintainance heavier, but gives the false sense of safeness that once you drop that data in, you're protected against recursion. - Davide ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 18:37 ` Davide Libenzi @ 2004-06-02 18:58 ` Jörn Engel 2004-06-02 19:33 ` Valdis.Kletnieks ` (2 more replies) 0 siblings, 3 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-02 18:58 UTC (permalink / raw) To: Davide Libenzi Cc: Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List On Wed, 2 June 2004 11:37:50 -0700, Davide Libenzi wrote: > > You're requesting to add and maintain data to feed a tool that catches > only trivially visible recursion. I don't want to waste mine and your time > explaining why your tool will never work, but if you want an hint, you can > start thinking about all functions that sets/pass callbacks and/or sets > operational functions. I don't know if you noticed that, but Linux is > heavily function-pointer driven. Eg, one function setups a set of function > pointers, and another 317 indirectly calls them. Having such comments, not > only makes the maintainance heavier, but gives the false sense of safeness > that once you drop that data in, you're protected against recursion. Yeah, I know about the problems to generate a complete call graph. With function pointers, it is plain impossible to get it right in the most general case. Note the "in the most general case" part. You can get things right if you make some assumptions and those assumptions are actually valid. In my case the assumptions are: 1. all relevant function pointers are stuffed into some struct and 2. no casts are used to disguise function pointer as something else. If you stick with those rules, the resulting code is quite sane, which is much more important than any tools being usable. If the kernel doesn't stick to those rules for a good reason, I'd like to know about it, so I can adjust my tool. And if the kernel doesn't stick to those rules for no good reason, the code if broken and needs to be fixed. Is this sane? Jörn -- A victorious army first wins and then seeks battle. -- Sun Tzu ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 18:58 ` Jörn Engel @ 2004-06-02 19:33 ` Valdis.Kletnieks 2004-06-02 19:37 ` viro 2004-06-02 23:20 ` Davide Libenzi 2 siblings, 0 replies; 106+ messages in thread From: Valdis.Kletnieks @ 2004-06-02 19:33 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux Kernel Mailing List [-- Attachment #1: Type: text/plain, Size: 700 bytes --] On Wed, 02 Jun 2004 20:58:32 +0200, =?iso-8859-1?Q?J=F6rn?= Engel said: > Note the "in the most general case" part. You can get things right if > you make some assumptions and those assumptions are actually valid. > In my case the assumptions are: > 1. all relevant function pointers are stuffed into some struct and > 2. no casts are used to disguise function pointer as something else. That seems to be reasonable. And if we're aliasing, or shadowing, or casting function pointers to something else, or using a union to overlay it, that's just a time bomb waiting to explode. At the very least, such code should require a large "Danger! Here be large and nasty dragons!" marker. [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 18:58 ` Jörn Engel 2004-06-02 19:33 ` Valdis.Kletnieks @ 2004-06-02 19:37 ` viro 2004-06-02 19:45 ` Jörn Engel 2004-06-02 19:55 ` Valdis.Kletnieks 2004-06-02 23:20 ` Davide Libenzi 2 siblings, 2 replies; 106+ messages in thread From: viro @ 2004-06-02 19:37 UTC (permalink / raw) To: Jörn Engel Cc: Davide Libenzi, Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List On Wed, Jun 02, 2004 at 08:58:32PM +0200, Jörn Engel wrote: > Note the "in the most general case" part. You can get things right if > you make some assumptions and those assumptions are actually valid. > In my case the assumptions are: > 1. all relevant function pointers are stuffed into some struct and Wrong. They are often passed as arguments to generic helpers, without being ever put into any structures. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 19:37 ` viro @ 2004-06-02 19:45 ` Jörn Engel 2004-06-02 19:59 ` viro 2004-06-02 19:55 ` Valdis.Kletnieks 1 sibling, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-06-02 19:45 UTC (permalink / raw) To: viro Cc: Davide Libenzi, Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List On Wed, 2 June 2004 20:37:20 +0100, viro@parcelfarce.linux.theplanet.co.uk wrote: > On Wed, Jun 02, 2004 at 08:58:32PM +0200, Jörn Engel wrote: > > Note the "in the most general case" part. You can get things right if > > you make some assumptions and those assumptions are actually valid. > > In my case the assumptions are: > > 1. all relevant function pointers are stuffed into some struct and > > Wrong. They are often passed as arguments to generic helpers, without > being ever put into any structures. Ok. Would it be ok to use the following then? b1. Function pointer are passed as arguments to functions and b2. those pointer are called directly from the function, they are passed to. Either that or the previous two rules, renamed to a1 and a2. (Note that I care more about sane rules than what any random code in some dark corner happens to do right now.) Jörn -- A surrounded army must be given a way out. -- Sun Tzu ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 19:45 ` Jörn Engel @ 2004-06-02 19:59 ` viro 2004-06-03 6:55 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: viro @ 2004-06-02 19:59 UTC (permalink / raw) To: Jörn Engel Cc: Davide Libenzi, Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List On Wed, Jun 02, 2004 at 09:45:15PM +0200, Jörn Engel wrote: > On Wed, 2 June 2004 20:37:20 +0100, viro@parcelfarce.linux.theplanet.co.uk wrote: > > On Wed, Jun 02, 2004 at 08:58:32PM +0200, Jörn Engel wrote: > > > Note the "in the most general case" part. You can get things right if > > > you make some assumptions and those assumptions are actually valid. > > > In my case the assumptions are: > > > 1. all relevant function pointers are stuffed into some struct and > > > > Wrong. They are often passed as arguments to generic helpers, without > > being ever put into any structures. > > Ok. Would it be ok to use the following then? > > b1. Function pointer are passed as arguments to functions and > b2. those pointer are called directly from the function, they are > passed to. Again not guaranteed to be true - they can be (and often are) passed further. Moreover, they are also stored untyped in structures. Common pattern is foo.callback = f; foo.argument = p; iterate_over_blah(blah, &foo); Note that here f is the only thing that will see the value of p _and_ the only thing that cares about type of p. iterator itself doesn't care and can be used for different types. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 19:59 ` viro @ 2004-06-03 6:55 ` Jörn Engel 0 siblings, 0 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-03 6:55 UTC (permalink / raw) To: viro Cc: Davide Libenzi, Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List On Wed, 2 June 2004 20:59:44 +0100, viro@parcelfarce.linux.theplanet.co.uk wrote: > > > > Ok. Would it be ok to use the following then? > > > > b1. Function pointer are passed as arguments to functions and > > b2. those pointer are called directly from the function, they are > > passed to. > > Again not guaranteed to be true - they can be (and often are) passed further. Hmm. If that happens, I'm out of ideas for now. Cannot do more than give a warning. > Moreover, they are also stored untyped in structures. Common pattern > is > foo.callback = f; > foo.argument = p; > iterate_over_blah(blah, &foo); > > Note that here f is the only thing that will see the value of p _and_ the > only thing that cares about type of p. iterator itself doesn't care and > can be used for different types. Those cases I should already catch. If foo is of type "struct bar", "bar.callback" will be the function name for a pseudo-function. That function is called by iterate_over_blah and calls f. Unnamed struct get a name made up from the components of the struct, like ____FAKE.Name.Chip.stat.Regi.LILP.Opti.high.lowe->ProcessIMQEntry. Doesn't look pretty, but works. Jörn -- Time? What's that? Time is only worth what you do with it. -- Theo de Raadt ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 19:37 ` viro 2004-06-02 19:45 ` Jörn Engel @ 2004-06-02 19:55 ` Valdis.Kletnieks 1 sibling, 0 replies; 106+ messages in thread From: Valdis.Kletnieks @ 2004-06-02 19:55 UTC (permalink / raw) To: viro Cc: Jörn Engel, Davide Libenzi, Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List [-- Attachment #1: Type: text/plain, Size: 499 bytes --] On Wed, 02 Jun 2004 20:37:20 BST, viro@parcelfarce.linux.theplanet.co.uk said: > Wrong. They are often passed as arguments to generic helpers, without > being ever put into any structures. At least those can *hopefully* be automatically detected by looking at the function's definition. Are there any known cases where they're "passed through" from caller to generic_a to generic_b which ends up calling the function via pointer, or is it restricted to caller, generic_a, *(pointer)? [-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 18:58 ` Jörn Engel 2004-06-02 19:33 ` Valdis.Kletnieks 2004-06-02 19:37 ` viro @ 2004-06-02 23:20 ` Davide Libenzi 2004-06-03 7:29 ` Jörn Engel 2 siblings, 1 reply; 106+ messages in thread From: Davide Libenzi @ 2004-06-02 23:20 UTC (permalink / raw) To: Jörn Engel Cc: Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 1644 bytes --] On Wed, 2 Jun 2004, [iso-8859-1] Jörn Engel wrote: > Yeah, I know about the problems to generate a complete call graph. > With function pointers, it is plain impossible to get it right in the > most general case. > > Note the "in the most general case" part. You can get things right if > you make some assumptions and those assumptions are actually valid. > In my case the assumptions are: > 1. all relevant function pointers are stuffed into some struct and > 2. no casts are used to disguise function pointer as something else. > > If you stick with those rules, the resulting code is quite sane, which > is much more important than any tools being usable. If the kernel > doesn't stick to those rules for a good reason, I'd like to know about > it, so I can adjust my tool. And if the kernel doesn't stick to those > rules for no good reason, the code if broken and needs to be fixed. > > Is this sane? Think at file_operations as very simple example, and try to find out what is actually called when somewhere the code does f_op->*(). How would you build the graph down there. You'd have to record all the functions that have been assigned to such method throughout the code, and nest *all* their graph in place. And this will eventually trigger false positives. Similar thing with functions that accepts callbacks, either directly or inside structures. And this doesn't even start to take aliasing into account. Hunting stack hungry functions is very good, and having a tool that is maybe 60% precise in detecting loops is fine too. But requiring metadata to be maintained to support such tool is IMO wrong. - Davide ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-02 23:20 ` Davide Libenzi @ 2004-06-03 7:29 ` Jörn Engel 0 siblings, 0 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-03 7:29 UTC (permalink / raw) To: Davide Libenzi Cc: Linus Torvalds, Horst von Brand, Pavel Machek, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linux Kernel Mailing List On Wed, 2 June 2004 16:20:24 -0700, Davide Libenzi wrote: > > Think at file_operations as very simple example, and try to find out what > is actually called when somewhere the code does f_op->*(). How would you > build the graph down there. You'd have to record all the functions that > have been assigned to such method throughout the code, and nest *all* > their graph in place. And this will eventually trigger false positives. Bad example, I can deal with that just fine. :) Any code calling f_op->read must not break for any f_op->read. So it is perfectly sane to assume that all get called. I build the graph by turning f_op->read into a pseudo-function that calls all functions ever assigned to f_op->read. > Similar thing with functions that accepts callbacks, either directly or > inside structures. Those I only handle, if the callbacks are inside structures, true. Will fix it when I find the time for it. > And this doesn't even start to take aliasing into > account. What exactly do you call aliasing? Do you have an example? > Hunting stack hungry functions is very good, and having a tool > that is maybe 60% precise in detecting loops is fine too. But requiring > metadata to be maintained to support such tool is IMO wrong. Since Linus feels that said metadata is still useful without any tools, I'll just ignore this objection. ;) Still, some of the complaints were valid, thanks a bunch! My todo list has grown again. Jörn -- Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. -- Doug MacIlroy ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 6:02 ` [RFC PATCH] explicitly mark recursion count Jörn Engel 2004-06-01 12:20 ` Pavel Machek @ 2004-06-01 12:39 ` viro 2004-06-01 13:26 ` Jörn Engel 1 sibling, 1 reply; 106+ messages in thread From: viro @ 2004-06-01 12:39 UTC (permalink / raw) To: Jörn Engel Cc: Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Tue, Jun 01, 2004 at 08:02:05AM +0200, Jörn Engel wrote: > Add recursion markers to teach automated test tools how bad documented > recursions really are. Currently, there is only a single such too that > can use the information and there is always the danger of documentation > and reality getting out of sync. But until there's a better tool... > +/** > + * RECURSION: 100 > + * STEP: register_proc_table > + */ This is too ugly for words ;-/ Who will maintain that data, anyway? ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: [RFC PATCH] explicitly mark recursion count 2004-06-01 12:39 ` viro @ 2004-06-01 13:26 ` Jörn Engel 0 siblings, 0 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-01 13:26 UTC (permalink / raw) To: viro Cc: Linus Torvalds, Andrew Morton, Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, linux-kernel On Tue, 1 June 2004 13:39:22 +0100, viro@parcelfarce.linux.theplanet.co.uk wrote: > > > +/** > > + * RECURSION: 100 > > + * STEP: register_proc_table > > + */ > > This is too ugly for words ;-/ Who will maintain that data, anyway? What format do you propose? I don't care too much. Maintenance would get easier with less recursions, obviously. ;) I could hack up something that will generate digests from the function source code (through smatch or so) and put those digests into the comments. As long as they match, the comments remain valid. And that should get past lawyers, as I work on a different basis now. Jörn -- Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. -- Rob Pike ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 13:00 ` Jörn Engel 2004-05-26 13:05 ` Arjan van de Ven @ 2004-06-07 18:14 ` Timothy Miller 2004-06-08 6:26 ` Arjan van de Ven 1 sibling, 1 reply; 106+ messages in thread From: Timothy Miller @ 2004-06-07 18:14 UTC (permalink / raw) To: Jörn Engel Cc: Arjan van de Ven, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel Jörn Engel wrote: > But I'll shut up now and see if I can generate better data over the > weekend. -test11 still had fun stuff like 3k stack consumption over > some code paths in a pretty minimal kernel. Wonder what 2.6.6 will do > with allyesconfig. ;) That gave me an idea. Sometimes in chip design, we 'overconstrain' the logic synthesizer, because static timing analyzers often produce inaccurate results. Anyhow, what if we were to go to 4K stacks but in static code analysis, flag anything which uses more than 2K or even 1K? ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-06-07 18:14 ` 4k stacks in 2.6 Timothy Miller @ 2004-06-08 6:26 ` Arjan van de Ven 2004-06-08 8:45 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Arjan van de Ven @ 2004-06-08 6:26 UTC (permalink / raw) To: Timothy Miller Cc: Jörn Engel, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel [-- Attachment #1: Type: text/plain, Size: 396 bytes --] > That gave me an idea. Sometimes in chip design, we 'overconstrain' the > logic synthesizer, because static timing analyzers often produce > inaccurate results. Anyhow, what if we were to go to 4K stacks but in > static code analysis, flag anything which uses more than 2K or even 1K? the patch I sent to akpm went to 400 bytes actually, but yeah, even that already is debatable. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-06-08 6:26 ` Arjan van de Ven @ 2004-06-08 8:45 ` Jörn Engel 0 siblings, 0 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-08 8:45 UTC (permalink / raw) To: Arjan van de Ven Cc: Timothy Miller, Ingo Molnar, Andrea Arcangeli, Rik van Riel, Linus Torvalds, linux-kernel On Tue, 8 June 2004 08:26:25 +0200, Arjan van de Ven wrote: > > > That gave me an idea. Sometimes in chip design, we 'overconstrain' the > > logic synthesizer, because static timing analyzers often produce > > inaccurate results. Anyhow, what if we were to go to 4K stacks but in > > static code analysis, flag anything which uses more than 2K or even 1K? With 2.6.6, there are currently just a few non-recursive paths over 3k. 2k will give you a *lot* of output, but if you insist... ;) http://wh.fh-wedel.de/~joern/data.nointermezzo.cs2.2k.bz2 470k compressed, 65M uncompressed Feel free to send patches. > the patch I sent to akpm went to 400 bytes actually, but yeah, even that > already is debatable. 400 bytes? That is for a single function, I assume. Jörn -- Those who come seeking peace without a treaty are plotting. -- Sun Tzu ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 12:50 ` Jörn Engel 2004-05-26 12:53 ` Arjan van de Ven @ 2004-05-26 18:12 ` David S. Miller 2004-05-26 19:02 ` Matt Mackall 1 sibling, 1 reply; 106+ messages in thread From: David S. Miller @ 2004-05-26 18:12 UTC (permalink / raw) To: Jörn Engel; +Cc: mingo, andrea, riel, torvalds, arjanv, linux-kernel On Wed, 26 May 2004 14:50:14 +0200 Jörn Engel <joern@wohnheim.fh-wedel.de> wrote: > Change gcc to catch stack overflows before the fact and disallow > module load unless modules have those checks as well. That's easy, just enable profiling then implement a suitable _mcount that checks for stack overflow. I bet someone has done this already. For full coverage, some trap entry handler checks in entry.S would be necessary too of course. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 18:12 ` David S. Miller @ 2004-05-26 19:02 ` Matt Mackall 2004-05-26 19:25 ` Dave Jones 0 siblings, 1 reply; 106+ messages in thread From: Matt Mackall @ 2004-05-26 19:02 UTC (permalink / raw) To: David S. Miller Cc: J?rn Engel, mingo, andrea, riel, torvalds, arjanv, linux-kernel On Wed, May 26, 2004 at 11:12:22AM -0700, David S. Miller wrote: > On Wed, 26 May 2004 14:50:14 +0200 > J?rn Engel <joern@wohnheim.fh-wedel.de> wrote: > > > Change gcc to catch stack overflows before the fact and disallow > > module load unless modules have those checks as well. > > That's easy, just enable profiling then implement a suitable > _mcount that checks for stack overflow. I bet someone has done > this already. There was a patch floating around for this in the 2.2 era that I ported to 2.4 on one occassion. It won't tell you worst case though, just worst observed case. Sparse is probably not a bad place to put a real call chain stack analysis. -- Mathematics is the supreme nostalgia of our time. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 19:02 ` Matt Mackall @ 2004-05-26 19:25 ` Dave Jones 0 siblings, 0 replies; 106+ messages in thread From: Dave Jones @ 2004-05-26 19:25 UTC (permalink / raw) To: Matt Mackall Cc: David S. Miller, J?rn Engel, mingo, andrea, riel, torvalds, arjanv, linux-kernel On Wed, May 26, 2004 at 02:02:22PM -0500, Matt Mackall wrote: > There was a patch floating around for this in the 2.2 era that I > ported to 2.4 on one occassion. It won't tell you worst case though, > just worst observed case. > > Sparse is probably not a bad place to put a real call chain stack analysis. That won't measure any dynamic stack allocations that we're doing at runtime, nor will it test all n combinations of drivers, which is where most of the stack horrors have been found in recent times. Dave ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 20:10 ` Rik van Riel 2004-05-25 21:15 ` Andrea Arcangeli @ 2004-05-25 21:16 ` Andrew Morton 2004-05-25 21:48 ` Ingo Molnar 1 sibling, 1 reply; 106+ messages in thread From: Andrew Morton @ 2004-05-25 21:16 UTC (permalink / raw) To: Rik van Riel; +Cc: andrea, mingo, torvalds, phyprabab, linux-kernel Rik van Riel <riel@redhat.com> wrote: > > On Tue, 25 May 2004, Rik van Riel wrote: > > > The point is, people like to run bigger workloads on > > bigger systems. Otherwise they wouldn't bother buying > > those bigger systems. > > Btw, you're right about the VMAs. Looking through customer > stuff a bit more the more common issues are low memory being > eaten by dentry / inode cache - which you can't always reclaim > due to files being open, and don't always _want_ to reclaim > because that could well be a bigger performance hit than the > 4:4 split. I did some testing a year or two back with the normal zone wound down to a few hundred megs - filesytem benchmarks were *severely* impacted by the increased turnover rate of fs metadata pagecache and VFS caches. I forget the details, but it was "wow". > The primary impact of the dentry / inode cache using memory > isn't lowmem exhaustion, btw. It's lowmem fragmentation. > > Fragmentation causes fork trouble (gone with the 4k stacks) > and trouble for the network layer and kiobuf allocation, > which still do need higher order allocations. I'm suspecting we'll end up needing mempools (or something) of 1- and 2-order pages to support large-frame networking. I'm surprised there isn't more pressure to do something about this. Maybe people are increasing min_free_kbytes. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 21:16 ` 4g/4g for 2.6.6 Andrew Morton @ 2004-05-25 21:48 ` Ingo Molnar 2004-05-25 22:09 ` David S. Miller 0 siblings, 1 reply; 106+ messages in thread From: Ingo Molnar @ 2004-05-25 21:48 UTC (permalink / raw) To: Andrew Morton; +Cc: Rik van Riel, andrea, torvalds, phyprabab, linux-kernel * Andrew Morton <akpm@osdl.org> wrote: > > Btw, you're right about the VMAs. Looking through customer > > stuff a bit more the more common issues are low memory being > > eaten by dentry / inode cache - which you can't always reclaim > > due to files being open, and don't always _want_ to reclaim > > because that could well be a bigger performance hit than the > > 4:4 split. > > I did some testing a year or two back with the normal zone wound down > to a few hundred megs - filesytem benchmarks were *severely* impacted > by the increased turnover rate of fs metadata pagecache and VFS > caches. I forget the details, but it was "wow". and it's not only the normal workloads we know about - people really do lots of weird stuff with Linux (and we are happy that they use Linux and that Linux keeps chugging along), and people seem to prefer a 10% slowdown to a box that locks up or -ENOMEM's. I'm not trying to insert any unjustified fear, 3:1 can be ok with lots of RAM, but it's _clearly_ wishful thinking that 32 GB x86 will be OK with just 600 MB of lowmem. 600 MB of lowmem means a 1:80 lowmem to RAM ratio, which is insane. Yes, it will be OK with select workloads and applications. With 4:4 it's 3.4 GB lowmem and the ratio is down to a much saner 1:9, and boxes pushed against the wall keep up better. (but i wont attempt to convince Andrea - and i'm not at all unhappy that he is trying to fix 3:1 to be more usable on big boxes because those efforts also decrease the RAM footprint of the UP kernel, which is another sensitive area. These efforts also help sane 64-bit architectures, so it's a win-win situation even considering our disagreement wrt. 4:4.) > I'm suspecting we'll end up needing mempools (or something) of 1- and > 2-order pages to support large-frame networking. I'm surprised there > isn't more pressure to do something about this. Maybe people are > increasing min_free_kbytes. hm, 1.5K pretty much seems to be the standard. Plus large frames can be scatter-gathered via fragmented skbs. Seldom is there a need for a large skb to be linear. Ingo ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 21:48 ` Ingo Molnar @ 2004-05-25 22:09 ` David S. Miller 2004-05-25 22:20 ` Ingo Molnar 0 siblings, 1 reply; 106+ messages in thread From: David S. Miller @ 2004-05-25 22:09 UTC (permalink / raw) To: Ingo Molnar; +Cc: akpm, riel, andrea, torvalds, phyprabab, linux-kernel On Tue, 25 May 2004 23:48:17 +0200 Ingo Molnar <mingo@elte.hu> wrote: > hm, 1.5K pretty much seems to be the standard. Plus large frames can be > scatter-gathered via fragmented skbs. Seldom is there a need for a large > skb to be linear. Unfortunately TSO with non-sendfile apps makes huge 64K SKBs get built. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 22:09 ` David S. Miller @ 2004-05-25 22:20 ` Ingo Molnar 2004-05-25 23:10 ` David S. Miller 0 siblings, 1 reply; 106+ messages in thread From: Ingo Molnar @ 2004-05-25 22:20 UTC (permalink / raw) To: David S. Miller; +Cc: akpm, riel, andrea, torvalds, phyprabab, linux-kernel * David S. Miller <davem@redhat.com> wrote: > > hm, 1.5K pretty much seems to be the standard. Plus large frames can be > > scatter-gathered via fragmented skbs. Seldom is there a need for a large > > skb to be linear. > > Unfortunately TSO with non-sendfile apps makes huge 64K SKBs get > built. hm, shouldnt we disable TSO in this case - or is it a win even in this case? Ingo ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 22:20 ` Ingo Molnar @ 2004-05-25 23:10 ` David S. Miller 0 siblings, 0 replies; 106+ messages in thread From: David S. Miller @ 2004-05-25 23:10 UTC (permalink / raw) To: Ingo Molnar; +Cc: akpm, riel, andrea, torvalds, phyprabab, linux-kernel On Wed, 26 May 2004 00:20:32 +0200 Ingo Molnar <mingo@elte.hu> wrote: > > Unfortunately TSO with non-sendfile apps makes huge 64K SKBs get > > built. > > hm, shouldnt we disable TSO in this case - or is it a win even in this > case? It is a win even in this case. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-25 19:50 ` Rik van Riel 2004-05-25 20:10 ` Rik van Riel @ 2004-05-25 21:04 ` Bill Davidsen 1 sibling, 0 replies; 106+ messages in thread From: Bill Davidsen @ 2004-05-25 21:04 UTC (permalink / raw) To: Rik van Riel; +Cc: Andrea Arcangeli, Ingo Molnar, linux-kernel Rik van Riel wrote: > On Tue, 25 May 2004, Andrea Arcangeli wrote: > > >>Clearly by opening enough files or enough network sockets or enough vmas >>or similar, you can still run out of normal zone, even on a 2G system, >>but this is not the point or you would be shipping 4:4 on the 2G systems >>too, no? > > > The point is, people like to run bigger workloads on > bigger systems. Otherwise they wouldn't bother buying > those bigger systems. > Sure, and "bigger workloads" can mean a lot of small client processes talking to a threaded large process, like database or news, or it can be huge datasets, like image or multimedia processing. Unfortunately it seems that these all (ab)use the VM in various ways. x86 with lots of memory is likely to remain cost effective for years, not only because it not only allows more memory but needs more memory in most cases, but because vendors will hang on to their profit margins on 64 bit CPUs for that long. And for uses with many small client processes, the advantage of 64 bits is pretty small when you have 2MB processes. Given a bus which allows i/o to more memory, the benefits in performance are really hard to see. -- -bill davidsen (davidsen@tmr.com) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-23 19:43 4g/4g for 2.6.6 Phy Prabab 2004-05-23 20:32 ` Linus Torvalds @ 2004-05-24 1:33 ` Martin J. Bligh 2004-05-24 1:38 ` Phy Prabab 1 sibling, 1 reply; 106+ messages in thread From: Martin J. Bligh @ 2004-05-24 1:33 UTC (permalink / raw) To: Phy Prabab, linux-kernel > I have been researching the 4g patches for kernels. > Seems there was a rift between people over this. Is > there any plan to resume publishing 4g patches for > developing kernels? > > I am currently trying to get 4g to work with 2.6.6-mm5 > but of course running into issues,so any help on this > would be great! It's in -mjb tree - the update from 2.6.6-rc3 to 2.6.6 should be trivial. M. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4g/4g for 2.6.6 2004-05-24 1:33 ` Martin J. Bligh @ 2004-05-24 1:38 ` Phy Prabab 0 siblings, 0 replies; 106+ messages in thread From: Phy Prabab @ 2004-05-24 1:38 UTC (permalink / raw) To: Martin J. Bligh, linux-kernel [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 718 bytes --] Okay, I will work with that patch and see if I can it to apply. Thanks! Phy --- "Martin J. Bligh" <mbligh@aracnet.com> wrote: > > I have been researching the 4g patches for > kernels. > > Seems there was a rift between people over this. > Is > > there any plan to resume publishing 4g patches for > > developing kernels? > > > > I am currently trying to get 4g to work with > 2.6.6-mm5 > > but of course running into issues,so any help on > this > > would be great! > > It's in -mjb tree - the update from 2.6.6-rc3 to > 2.6.6 should be trivial. > > M. > __________________________________ Do you Yahoo!? Yahoo! Domains Claim yours for only $14.70/year http://smallbusiness.promotions.yahoo.com/offer ^ permalink raw reply [flat|nested] 106+ messages in thread
[parent not found: <1ZQpn-1Rx-1@gated-at.bofh.it>]
[parent not found: <1ZQz8-1Yh-15@gated-at.bofh.it>]
[parent not found: <1ZRFf-2Vt-3@gated-at.bofh.it>]
[parent not found: <203Zu-4aT-15@gated-at.bofh.it>]
* Re: 4k stacks in 2.6 [not found] ` <203Zu-4aT-15@gated-at.bofh.it> @ 2004-05-26 13:57 ` Andi Kleen 2004-05-26 18:17 ` hch 2004-05-26 20:39 ` Zwane Mwaikambo [not found] ` <206b3-5WN-33@gated-at.bofh.it> 1 sibling, 2 replies; 106+ messages in thread From: Andi Kleen @ 2004-05-26 13:57 UTC (permalink / raw) To: Ingo Molnar; +Cc: andrea, linux-kernel, arjanv Ingo Molnar <mingo@elte.hu> writes: > > do you realize that the 4K stacks feature also adds a separate softirq > and a separate hardirq stack? So the maximum footprint is 4K+4K+4K, with A nice combination would be 8K process stacks with separate irq stacks on i386. Any chance the CONFIGs for those two could be split? -Andi ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 13:57 ` 4k stacks in 2.6 Andi Kleen @ 2004-05-26 18:17 ` hch 2004-05-26 18:24 ` Andi Kleen 2004-05-26 20:39 ` Zwane Mwaikambo 1 sibling, 1 reply; 106+ messages in thread From: hch @ 2004-05-26 18:17 UTC (permalink / raw) To: Andi Kleen; +Cc: Ingo Molnar, andrea, linux-kernel, arjanv On Wed, May 26, 2004 at 03:57:05PM +0200, Andi Kleen wrote: > Ingo Molnar <mingo@elte.hu> writes: > > > > do you realize that the 4K stacks feature also adds a separate softirq > > and a separate hardirq stack? So the maximum footprint is 4K+4K+4K, with > > A nice combination would be 8K process stacks with separate irq stacks on > i386. > > Any chance the CONFIGs for those two could be split? Any reason not to enable interrupt stacks unconditionally and leave the stack size choice to the user? ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 18:17 ` hch @ 2004-05-26 18:24 ` Andi Kleen 0 siblings, 0 replies; 106+ messages in thread From: Andi Kleen @ 2004-05-26 18:24 UTC (permalink / raw) To: hch, Andi Kleen, Ingo Molnar, andrea, linux-kernel, arjanv On Wed, May 26, 2004 at 02:17:34PM -0400, hch@infradead.org wrote: > On Wed, May 26, 2004 at 03:57:05PM +0200, Andi Kleen wrote: > > Ingo Molnar <mingo@elte.hu> writes: > > > > > > do you realize that the 4K stacks feature also adds a separate softirq > > > and a separate hardirq stack? So the maximum footprint is 4K+4K+4K, with > > > > A nice combination would be 8K process stacks with separate irq stacks on > > i386. > > > > Any chance the CONFIGs for those two could be split? > > Any reason not to enable interrupt stacks unconditionally and leave > the stack size choice to the user? It will probably still break some other patches, like debuggers. Given that the kernel is supposed to be stable I would not change it unconditionally in 2.6. Maybe in 2.7. -Andi ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 13:57 ` 4k stacks in 2.6 Andi Kleen 2004-05-26 18:17 ` hch @ 2004-05-26 20:39 ` Zwane Mwaikambo 1 sibling, 0 replies; 106+ messages in thread From: Zwane Mwaikambo @ 2004-05-26 20:39 UTC (permalink / raw) To: Andi Kleen; +Cc: Ingo Molnar, andrea, linux-kernel, arjanv On Wed, 26 May 2004, Andi Kleen wrote: > Ingo Molnar <mingo@elte.hu> writes: > > > > do you realize that the 4K stacks feature also adds a separate softirq > > and a separate hardirq stack? So the maximum footprint is 4K+4K+4K, with > > A nice combination would be 8K process stacks with separate irq stacks on > i386. > > Any chance the CONFIGs for those two could be split? Couldn't this just be done with a THREAD_SIZE config option? ^ permalink raw reply [flat|nested] 106+ messages in thread
[parent not found: <206b3-5WN-33@gated-at.bofh.it>]
[parent not found: <20baw-1Lz-15@gated-at.bofh.it>]
* Re: 4k stacks in 2.6 [not found] ` <20baw-1Lz-15@gated-at.bofh.it> @ 2004-05-26 19:32 ` Andi Kleen 2004-05-27 11:27 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Andi Kleen @ 2004-05-26 19:32 UTC (permalink / raw) To: David S. Miller Cc: joern, mingo, andrea, riel, torvalds, arjanv, linux-kernel "David S. Miller" <davem@redhat.com> writes: >> Change gcc to catch stack overflows before the fact and disallow >> module load unless modules have those checks as well. It's impossible to do anything but panic, so it's not too helpful in practice. You can only do better for interrupts (not handle an interrupt when the stack is too low). > That's easy, just enable profiling then implement a suitable > _mcount that checks for stack overflow. I bet someone has done > this already. I did it for x86-64 a long time ago. Should be easy to port to i386 too. ftp://ftp.x86-64.org/pub/linux/debug/stackcheck-1 -Andi ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-26 19:32 ` Andi Kleen @ 2004-05-27 11:27 ` Jörn Engel 2004-05-27 13:49 ` Andrea Arcangeli 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-05-27 11:27 UTC (permalink / raw) To: Andi Kleen Cc: David S. Miller, mingo, andrea, riel, torvalds, arjanv, linux-kernel On Wed, 26 May 2004 21:32:32 +0200, Andi Kleen wrote: > "David S. Miller" <davem@redhat.com> writes: > > >> Change gcc to catch stack overflows before the fact and disallow > >> module load unless modules have those checks as well. > > It's impossible to do anything but panic, so it's not too helpful > in practice. Oh, panic is *very* helpful. Panic won't do random funny things, it will just stop the machine. If we got an immediate panic on any stack overflow, I would want 4k stacks right now. > > That's easy, just enable profiling then implement a suitable > > _mcount that checks for stack overflow. I bet someone has done > > this already. > > I did it for x86-64 a long time ago. Should be easy to port to i386 > too. > > ftp://ftp.x86-64.org/pub/linux/debug/stackcheck-1 Cool! If that is included, I don't have any objections against 4k stacks anymore. Jörn -- The cheapest, fastest and most reliable components of a computer system are those that aren't there. -- Gordon Bell, DEC labratories ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 11:27 ` Jörn Engel @ 2004-05-27 13:49 ` Andrea Arcangeli 2004-05-27 14:15 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-27 13:49 UTC (permalink / raw) To: Jörn Engel Cc: Andi Kleen, David S. Miller, mingo, riel, torvalds, arjanv, linux-kernel On Thu, May 27, 2004 at 01:27:05PM +0200, Jörn Engel wrote: > Cool! If that is included, I don't have any objections against 4k > stacks anymore. note that it will introduce an huge slowdown, there's no way to enable that in production. But for testing it's fine. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 13:49 ` Andrea Arcangeli @ 2004-05-27 14:15 ` Jörn Engel 2004-05-27 14:49 ` Andrea Arcangeli 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-05-27 14:15 UTC (permalink / raw) To: Andrea Arcangeli Cc: Andi Kleen, David S. Miller, mingo, riel, torvalds, arjanv, linux-kernel On Thu, 27 May 2004 15:49:50 +0200, Andrea Arcangeli wrote: > On Thu, May 27, 2004 at 01:27:05PM +0200, Jörn Engel wrote: > > Cool! If that is included, I don't have any objections against 4k > > stacks anymore. > > note that it will introduce an huge slowdown, there's no way to enable > that in production. But for testing it's fine. Would it be possible to add something short to the function preamble on x86 then? Similar to this code, maybe: if (!(stack_pointer & 0xe00)) /* less than 512 bytes left */ *NULL = 1; Not sure how this can be translated into short and fast x86 assembler, but if it is possible, I would really like to have it. Then all we have left to do is make sure no function ever uses more than 512 bytes. Famous last words, I know. Jörn -- Time? What's that? Time is only worth what you do with it. -- Theo de Raadt ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:15 ` Jörn Engel @ 2004-05-27 14:49 ` Andrea Arcangeli 2004-05-27 14:59 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Andrea Arcangeli @ 2004-05-27 14:49 UTC (permalink / raw) To: Jörn Engel Cc: Andi Kleen, David S. Miller, mingo, riel, torvalds, arjanv, linux-kernel On Thu, May 27, 2004 at 04:15:47PM +0200, Jörn Engel wrote: > On Thu, 27 May 2004 15:49:50 +0200, Andrea Arcangeli wrote: > > On Thu, May 27, 2004 at 01:27:05PM +0200, Jörn Engel wrote: > > > Cool! If that is included, I don't have any objections against 4k > > > stacks anymore. > > > > note that it will introduce an huge slowdown, there's no way to enable > > that in production. But for testing it's fine. > > Would it be possible to add something short to the function preamble > on x86 then? Similar to this code, maybe: > > if (!(stack_pointer & 0xe00)) /* less than 512 bytes left */ > *NULL = 1; > > Not sure how this can be translated into short and fast x86 assembler, > but if it is possible, I would really like to have it. Then all we > have left to do is make sure no function ever uses more than 512 > bytes. Famous last words, I know. If it would be _inlined_ it would be *much* faster, but it would likely be measurable anyways. Less measurable though. There's no way with gcc to inline the above in the preamble, one could hack gcc for it though (there's exactly an asm preable thing in gcc that is the one that is currently implemented as call mcount plus the register saving, chaning it to the above may be feasible, though it would need a new option in gcc) another nice thing to have (this one zerocost at runtime) would be a way to set a limit on the size of the local variables for each function. gcc knows that value very well, it's the sub it does on the stack pointer the first few asm instructions after the call. That would reduce the common mistakes. An equivalent script is the one from Keith Owens checking the vmlinux binary after compilation but I'm afraid people runs that one only after the fact. ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:49 ` Andrea Arcangeli @ 2004-05-27 14:59 ` Jörn Engel 2004-05-27 15:08 ` Keith Owens 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-05-27 14:59 UTC (permalink / raw) To: Andrea Arcangeli Cc: Andi Kleen, David S. Miller, mingo, riel, torvalds, arjanv, linux-kernel On Thu, 27 May 2004 16:49:16 +0200, Andrea Arcangeli wrote: > On Thu, May 27, 2004 at 04:15:47PM +0200, Jörn Engel wrote: > > > > Would it be possible to add something short to the function preamble > > on x86 then? Similar to this code, maybe: > > > > if (!(stack_pointer & 0xe00)) /* less than 512 bytes left */ > > *NULL = 1; > > > > Not sure how this can be translated into short and fast x86 assembler, > > but if it is possible, I would really like to have it. Then all we > > have left to do is make sure no function ever uses more than 512 > > bytes. Famous last words, I know. > > If it would be _inlined_ it would be *much* faster, but it would likely > be measurable anyways. Less measurable though. There's no way with gcc > to inline the above in the preamble, one could hack gcc for it though > (there's exactly an asm preable thing in gcc that is the one that is > currently implemented as call mcount plus the register saving, chaning > it to the above may be feasible, though it would need a new option in > gcc) It is on my list, although I care more about ppc32. Can anyone translate the above into assembler? > another nice thing to have (this one zerocost at runtime) would be a > way to set a limit on the size of the local variables for each function. > gcc knows that value very well, it's the sub it does on the stack > pointer the first few asm instructions after the call. That would > reduce the common mistakes. An equivalent script is the one from Keith > Owens checking the vmlinux binary after compilation but I'm afraid > people runs that one only after the fact. Plus the script is wrong sometimes. I have had trouble with sizes around 4G or 2G, and never found the time to really figure out what's going on. Might be an alloca thing that got misparsed somehow. Having the check in gcc should cause less surprises. Jörn -- It's not whether you win or lose, it's how you place the blame. -- unknown ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 14:59 ` Jörn Engel @ 2004-05-27 15:08 ` Keith Owens 2004-05-27 15:21 ` Jörn Engel 0 siblings, 1 reply; 106+ messages in thread From: Keith Owens @ 2004-05-27 15:08 UTC (permalink / raw) To: linux-kernel On Thu, 27 May 2004 16:59:35 +0200, =?iso-8859-1?Q?J=F6rn?= Engel <joern@wohnheim.fh-wedel.de> wrote: >On Thu, 27 May 2004 16:49:16 +0200, Andrea Arcangeli wrote: >> On Thu, May 27, 2004 at 04:15:47PM +0200, Jörn Engel wrote: >> An equivalent script is the one from Keith >> Owens checking the vmlinux binary after compilation but I'm afraid >> people runs that one only after the fact. > >Plus the script is wrong sometimes. I have had trouble with sizes >around 4G or 2G, and never found the time to really figure out what's >going on. Might be an alloca thing that got misparsed somehow. Some code results in negative adjustments to the stack size on exit, which look like 4G sizes. My script checks for those and ignores them. /^[89a-f].......$/d; ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 15:08 ` Keith Owens @ 2004-05-27 15:21 ` Jörn Engel 2004-05-27 15:34 ` Arjan van de Ven 0 siblings, 1 reply; 106+ messages in thread From: Jörn Engel @ 2004-05-27 15:21 UTC (permalink / raw) To: Keith Owens; +Cc: linux-kernel On Fri, 28 May 2004 01:08:02 +1000, Keith Owens wrote: > On Thu, 27 May 2004 16:59:35 +0200, > =?iso-8859-1?Q?J=F6rn?= Engel <joern@wohnheim.fh-wedel.de> wrote: > > > >Plus the script is wrong sometimes. I have had trouble with sizes > >around 4G or 2G, and never found the time to really figure out what's > >going on. Might be an alloca thing that got misparsed somehow. > > Some code results in negative adjustments to the stack size on exit, > which look like 4G sizes. My script checks for those and ignores them. > /^[89a-f].......$/d; Ok, looks as if only my script is wrong. Do you know what exactly causes such a negative adjustment? Jörn -- Optimizations always bust things, because all optimizations are, in the long haul, a form of cheating, and cheaters eventually get caught. -- Larry Wall ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 15:21 ` Jörn Engel @ 2004-05-27 15:34 ` Arjan van de Ven 2004-05-27 15:46 ` Jörn Engel 2004-06-01 5:25 ` Jörn Engel 0 siblings, 2 replies; 106+ messages in thread From: Arjan van de Ven @ 2004-05-27 15:34 UTC (permalink / raw) To: Jörn Engel; +Cc: Keith Owens, linux-kernel [-- Attachment #1: Type: text/plain, Size: 931 bytes --] On Thu, 2004-05-27 at 17:21, Jörn Engel wrote: > On Fri, 28 May 2004 01:08:02 +1000, Keith Owens wrote: > > On Thu, 27 May 2004 16:59:35 +0200, > > =?iso-8859-1?Q?J=F6rn?= Engel <joern@wohnheim.fh-wedel.de> wrote: > > > > > >Plus the script is wrong sometimes. I have had trouble with sizes > > >around 4G or 2G, and never found the time to really figure out what's > > >going on. Might be an alloca thing that got misparsed somehow. > > > > Some code results in negative adjustments to the stack size on exit, > > which look like 4G sizes. My script checks for those and ignores them. > > /^[89a-f].......$/d; > > Ok, looks as if only my script is wrong. Do you know what exactly > causes such a negative adjustment? you can write "add 100,%esp" as "sub -100, %esp" :) compilers seem to do that at times, probably some cpu model inside the compiler decides the later is better code in some cases :) [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 15:34 ` Arjan van de Ven @ 2004-05-27 15:46 ` Jörn Engel 2004-06-01 5:25 ` Jörn Engel 1 sibling, 0 replies; 106+ messages in thread From: Jörn Engel @ 2004-05-27 15:46 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Keith Owens, linux-kernel On Thu, 27 May 2004 17:34:26 +0200, Arjan van de Ven wrote: > > you can write "add 100,%esp" as "sub -100, %esp" :) > compilers seem to do that at times, probably some cpu model inside the > compiler decides the later is better code in some cases :) Makes sense (in a way). For x86 and ppc*, my script should be safe as a nice side effect: qr/^.*sub \$(0x$x{3,5}),\%esp$/o Anything above 5 digits is ignored. That also misses allocations above 1MB, but as long as human stupidity is finite... ;) Jörn -- ticks = jiffies; while (ticks == jiffies); ticks = jiffies; -- /usr/src/linux/init/main.c ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6 2004-05-27 15:34 ` Arjan van de Ven 2004-05-27 15:46 ` Jörn Engel @ 2004-06-01 5:25 ` Jörn Engel 1 sibling, 0 replies; 106+ messages in thread From: Jörn Engel @ 2004-06-01 5:25 UTC (permalink / raw) To: Arjan van de Ven; +Cc: Keith Owens, linux-kernel On Thu, 27 May 2004 17:34:26 +0200, Arjan van de Ven wrote: > > you can write "add 100,%esp" as "sub -100, %esp" :) > compilers seem to do that at times, probably some cpu model inside the > compiler decides the later is better code in some cases :) That and even worse things. sys_sendfile has a "sub $0x10,%esp" followed by an "add $0x20,%esp". Can you explain that one as well? 0x20 is the size of all automatic variables on i386. I have no idea what kind of trick gcc is playing there, but it appears to work which makes me only more curious. Jörn -- Simplicity is prerequisite for reliability. -- Edsger W. Dijkstra ^ permalink raw reply [flat|nested] 106+ messages in thread
* Re: 4k stacks in 2.6
@ 2004-05-26 15:17 Albert Cahalan
0 siblings, 0 replies; 106+ messages in thread
From: Albert Cahalan @ 2004-05-26 15:17 UTC (permalink / raw)
To: linux-kernel mailing list; +Cc: mingo, arjanv
Ingo Molnar writes:
> do you realize that the 4K stacks feature also adds
> a separate softirq and a separate hardirq stack?
> So the maximum footprint is 4K+4K+4K, with a clear
> and sane limit for each type of context, while the
> 2.4 kernel has 6.5K for all 3 contexts combined.
> (Also, in 2.4 irq contexts pretty much assumed that
> there's 2K of stack for them - leaving a de-facto 4K
> stack for the process and softirq contexts.) So in fact
> there is more space in 2.6 for all, and i dont really
> understand your fears.
Is that 4K per IRQ (total 64K to 1024K) or 4K total?
If it's total, then it's cheap to go with 32K.
The same goes for softirqs: 4K total, or per softirq?
^ permalink raw reply [flat|nested] 106+ messages in threadend of thread, other threads:[~2004-06-08 8:45 UTC | newest]
Thread overview: 106+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-05-23 19:43 4g/4g for 2.6.6 Phy Prabab
2004-05-23 20:32 ` Linus Torvalds
2004-05-23 20:51 ` Jeff Garzik
2004-05-24 1:55 ` Linus Torvalds
2004-05-24 2:19 ` Jeff Garzik
2004-05-24 2:33 ` Wim Coekaerts
2004-05-31 9:51 ` Pavel Machek
2004-05-24 3:30 ` Martin J. Bligh
2004-06-01 5:52 ` Eric W. Biederman
2004-05-23 21:55 ` Phy Prabab
2004-05-24 7:05 ` Arjan van de Ven
2004-05-24 7:11 ` Phy Prabab
2004-05-24 7:24 ` Arjan van de Ven
2004-05-24 7:27 ` Phy Prabab
2004-05-24 12:01 ` Dave Jones
2004-05-24 2:39 ` William Lee Irwin III
2004-05-24 8:25 ` Ingo Molnar
2004-05-24 12:48 ` Andrea Arcangeli
2004-05-25 19:15 ` Rik van Riel
2004-05-25 19:41 ` Andrea Arcangeli
2004-05-25 19:50 ` Rik van Riel
2004-05-25 20:10 ` Rik van Riel
2004-05-25 21:15 ` Andrea Arcangeli
2004-05-26 10:33 ` 4k stacks in 2.6 Ingo Molnar
2004-05-26 12:50 ` Jörn Engel
2004-05-26 12:53 ` Arjan van de Ven
2004-05-26 13:00 ` Jörn Engel
2004-05-26 13:05 ` Arjan van de Ven
2004-05-26 16:41 ` Jörn Engel
2004-05-27 12:45 ` Ingo Molnar
2004-05-27 13:59 ` Andrea Arcangeli
2004-05-27 14:03 ` Arjan van de Ven
2004-05-27 14:42 ` Andrea Arcangeli
2004-06-02 19:40 ` Bill Davidsen
2004-05-27 14:18 ` Brian Gerst
2004-05-27 14:50 ` Andrea Arcangeli
2004-05-27 14:55 ` Linus Torvalds
2004-05-27 15:39 ` Andrea Arcangeli
2004-05-27 18:31 ` Guy Sotomayor
2004-05-27 19:26 ` Brian Gerst
2004-06-01 5:56 ` 4k stacks in 2.6 [worst offenders] Jörn Engel
2004-06-01 6:02 ` [RFC PATCH] explicitly mark recursion count Jörn Engel
2004-06-01 12:20 ` Pavel Machek
2004-06-01 13:27 ` Jörn Engel
2004-06-01 13:32 ` Pavel Machek
2004-06-01 13:37 ` Jörn Engel
2004-06-01 19:48 ` Horst von Brand
2004-06-01 19:29 ` Horst von Brand
2004-06-01 19:58 ` Linus Torvalds
2004-06-02 13:16 ` Jörn Engel
2004-06-02 14:15 ` Linus Torvalds
2004-06-02 14:27 ` Jörn Engel
2004-06-02 14:45 ` Linus Torvalds
2004-06-02 15:04 ` Jörn Engel
2004-06-02 15:12 ` Linus Torvalds
2004-06-02 15:27 ` Jörn Engel
2004-06-02 15:52 ` Linus Torvalds
2004-06-02 16:17 ` Jörn Engel
2004-06-02 16:25 ` Linus Torvalds
2004-06-02 17:17 ` Jörn Engel
2004-06-02 17:32 ` Greg KH
2004-06-02 17:46 ` Jörn Engel
2004-06-02 14:35 ` Davide Libenzi
2004-06-02 18:20 ` Jörn Engel
2004-06-02 18:37 ` Davide Libenzi
2004-06-02 18:58 ` Jörn Engel
2004-06-02 19:33 ` Valdis.Kletnieks
2004-06-02 19:37 ` viro
2004-06-02 19:45 ` Jörn Engel
2004-06-02 19:59 ` viro
2004-06-03 6:55 ` Jörn Engel
2004-06-02 19:55 ` Valdis.Kletnieks
2004-06-02 23:20 ` Davide Libenzi
2004-06-03 7:29 ` Jörn Engel
2004-06-01 12:39 ` viro
2004-06-01 13:26 ` Jörn Engel
2004-06-07 18:14 ` 4k stacks in 2.6 Timothy Miller
2004-06-08 6:26 ` Arjan van de Ven
2004-06-08 8:45 ` Jörn Engel
2004-05-26 18:12 ` David S. Miller
2004-05-26 19:02 ` Matt Mackall
2004-05-26 19:25 ` Dave Jones
2004-05-25 21:16 ` 4g/4g for 2.6.6 Andrew Morton
2004-05-25 21:48 ` Ingo Molnar
2004-05-25 22:09 ` David S. Miller
2004-05-25 22:20 ` Ingo Molnar
2004-05-25 23:10 ` David S. Miller
2004-05-25 21:04 ` Bill Davidsen
2004-05-24 1:33 ` Martin J. Bligh
2004-05-24 1:38 ` Phy Prabab
[not found] <1ZQpn-1Rx-1@gated-at.bofh.it>
[not found] ` <1ZQz8-1Yh-15@gated-at.bofh.it>
[not found] ` <1ZRFf-2Vt-3@gated-at.bofh.it>
[not found] ` <203Zu-4aT-15@gated-at.bofh.it>
2004-05-26 13:57 ` 4k stacks in 2.6 Andi Kleen
2004-05-26 18:17 ` hch
2004-05-26 18:24 ` Andi Kleen
2004-05-26 20:39 ` Zwane Mwaikambo
[not found] ` <206b3-5WN-33@gated-at.bofh.it>
[not found] ` <20baw-1Lz-15@gated-at.bofh.it>
2004-05-26 19:32 ` Andi Kleen
2004-05-27 11:27 ` Jörn Engel
2004-05-27 13:49 ` Andrea Arcangeli
2004-05-27 14:15 ` Jörn Engel
2004-05-27 14:49 ` Andrea Arcangeli
2004-05-27 14:59 ` Jörn Engel
2004-05-27 15:08 ` Keith Owens
2004-05-27 15:21 ` Jörn Engel
2004-05-27 15:34 ` Arjan van de Ven
2004-05-27 15:46 ` Jörn Engel
2004-06-01 5:25 ` Jörn Engel
-- strict thread matches above, loose matches on Subject: below --
2004-05-26 15:17 Albert Cahalan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox