[patch] block highmem zero bounce v14

[patch] block highmem zero bounce v14

[Jens Axboe]

Hi,

I've uploaded a new version of the patch that allows direct I/O to
highmem memory without resorting to bouncing to lower memory. Changes
since last time include:

- Sync pci64-2.4.10p4 with davem (me)

- Fix megaraid virt_to_bus on pci_alloc_consistent (!) (Arjan)

- Fix ide-dma not doing the right thing with private bh submissions that
  aren't necessarily b_page based (me)

- Fix ide-scsi scatter-gather highmem (me)

- Fix IDE+SCSI highmem enable bug (me)

- Limit I/O highmem enable messages a bit (me)

It's against 2.4.10-pre9 and can be found right here:

*.kernel.org/pub/linux/kernel/people/axboe/patches/2.4.10-pre9/block-highmem-all-14

-- 
Jens Axboe

Re: [patch] block highmem zero bounce v14

[Linus Torvalds]

On Sun, 16 Sep 2001, Jens Axboe wrote:
>
> It's against 2.4.10-pre9 and can be found right here:
>
> *.kernel.org/pub/linux/kernel/people/axboe/patches/2.4.10-pre9/block-highmem-all-14

Jens, what's your feeling about the stability of these things, especially
wrt weird drivers?

Ie do you think this is really a 2.4.x thing, or early 2.5.x?

		Linus

Re: [patch] block highmem zero bounce v14

[Jens Axboe]

On Sun, Sep 16 2001, Linus Torvalds wrote:
> 
> On Sun, 16 Sep 2001, Jens Axboe wrote:
> >
> > It's against 2.4.10-pre9 and can be found right here:
> >
> > *.kernel.org/pub/linux/kernel/people/axboe/patches/2.4.10-pre9/block-highmem-all-14
> 
> Jens, what's your feeling about the stability of these things, especially
> wrt weird drivers?

One of the very first decisions I made wrt this patch was to make sure
that weird/old drivers could keep on working exactly the way they do now
and never have to worry about highmem stuff. That basically means
enabling the stuff on a per-driver basis after it's considered safe. The
can_dma_32 for SCSI and highmem for IDE flag serves that purpose. Stand
alone block drivers just use blk_queue_bounce_limit to enable highmem
I/O after blk_init_queue, if they don't they get highmem pages bounced
as they are used too.

That leaves drivers that are 'different', stuff like ide-scsi for
instance. I think I have most of these under control...

> Ie do you think this is really a 2.4.x thing, or early 2.5.x?

Most of it is really a cautious back port of the 2.5 stuff I've been
working on, and with the above considerations it is/was meant as a 2.4
thing :-)

-- 
Jens Axboe

Re: [patch] block highmem zero bounce v14

[Alan Cox]

> Most of it is really a cautious back port of the 2.5 stuff I've been
> working on, and with the above considerations it is/was meant as a 2.4
> thing :-)

So better deferred until 2.5, tried in 2.5 and backported to 2.4 IMHO

Alan

Re: [patch] block highmem zero bounce v14

[Jens Axboe]

On Mon, Sep 17 2001, Alan Cox wrote:
> > Most of it is really a cautious back port of the 2.5 stuff I've been
> > working on, and with the above considerations it is/was meant as a 2.4
> > thing :-)
> 
> So better deferred until 2.5, tried in 2.5 and backported to 2.4 IMHO

Maybe. At least the first thing I would like is for the pci64 patch to
be merged in 2.4. That should be very doable without risking breakage.
When that is done it's easier to see what the block-highmem patch does.
And I believe that we _can_ merge it in 2.4 without a 2.5 trial, it's
really not that intrusive.

-- 
Jens Axboe

Re: [patch] block highmem zero bounce v14

[Arjan van de Ven]

On Mon, Sep 17, 2001 at 12:00:12AM +0200, Jens Axboe wrote:
> On Sun, Sep 16 2001, Linus Torvalds wrote:
> > 
> > On Sun, 16 Sep 2001, Jens Axboe wrote:
> > >
> > > It's against 2.4.10-pre9 and can be found right here:
> > >
> > > *.kernel.org/pub/linux/kernel/people/axboe/patches/2.4.10-pre9/block-highmem-all-14
> > 
> > Jens, what's your feeling about the stability of these things, especially
> > wrt weird drivers?
> 
> One of the very first decisions I made wrt this patch was to make sure
> that weird/old drivers could keep on working exactly the way they do now
> and never have to worry about highmem stuff. 

unfortionatly, so far both megaraid and the 3ware driver broke. Megaraid is
easily fixable, but still. It shows that this patch is not without risk...

Re: [patch] block highmem zero bounce v14

[Alan Cox]

> > One of the very first decisions I made wrt this patch was to make sure
> > that weird/old drivers could keep on working exactly the way they do now
> > and never have to worry about highmem stuff. 
> 
> unfortionatly, so far both megaraid and the 3ware driver broke. Megaraid is
> easily fixable, but still. It shows that this patch is not without risk...

I am not sure the megaraid breakage is from that patch at the moment. There
is another 2.4.10pre change which seems to alter the request size limits for
the megaraid.  I don't know where this came from but it was not discussed
on the megaraid list or posted here by any megaraid folks. Given the 
megaraid firmware is umm "fragile" anything that changes what we feed it
really does want to go through the AMI/Dell people.

A secondary issue is that 64bit DMA is only correctly handled by certain
MegaRAID firmware. The checks for this are not correct until 1.1.17a-ac
in some cases (eg HP rebadged board with H.01.07 and H.01.08 firmware)

Alan

spurious interrupt with ac kernel but not with vanilla 2.4.9

[Luigi Genoni]

HI,

I noticed that on the abit kt7A MBs that i have, with via KT133A chipset,
after i installed the kernel 2.4.9-ac10 (ac11 and 12 as well),
at boot i get this message,

Sep 21 11:52:11 DarkStar kernel: ice 00:0b.0
Sep 21 11:52:11 DarkStar kernel: spurious 8259A interrupt: IRQ7.
 immediatelly before scsi adaptec detection and inizzializzation.

doing a cat /proc/interrupt I can read

ERR:	1

As many people I have parport on irq 7, and the parport simply works
anyway (tested with a printer).

I was thinking to an HW problem, but with vanilla 2.4.9 kernel I do not
get this message, and the code in arch/i386/kernel/i8259.c that should
detect this spurious interrup is just the same on those two kernels.
Changes are related to X86_IO_APIC, and I think this is the reason why
this spurious interrupt is detected, but I would like to know if i should
think I have some HW problem going on that stardard kernels do not detect.

bests
Luigi

Re: spurious interrupt with ac kernel but not with vanilla 2.4.9

[Alan Cox]

> I noticed that on the abit kt7A MBs that i have, with via KT133A chipset,
> after i installed the kernel 2.4.9-ac10 (ac11 and 12 as well),
> at boot i get this message,
> 
> Sep 21 11:52:11 DarkStar kernel: ice 00:0b.0
> Sep 21 11:52:11 DarkStar kernel: spurious 8259A interrupt: IRQ7.
>  immediatelly before scsi adaptec detection and inizzializzation.

Thats indicating an IRQ appeared and vanished. IRQ 7 is the IRQ that happens
to occur for this.

> I was thinking to an HW problem, but with vanilla 2.4.9 kernel I do not
> get this message, and the code in arch/i386/kernel/i8259.c that should
> detect this spurious interrup is just the same on those two kernels.
> Changes are related to X86_IO_APIC, and I think this is the reason why
> this spurious interrupt is detected, but I would like to know if i should
> think I have some HW problem going on that stardard kernels do not detect.

The APIC only applies to multiprocessor boxes unless you are building with
uniprocessor apic support. Build a non SMP kernel without apic support
and let me know what that does

Re: spurious interrupt with ac kernel but not with vanilla 2.4.9

[Luigi Genoni]

On Fri, 21 Sep 2001, Alan Cox wrote:

> > I noticed that on the abit kt7A MBs that i have, with via KT133A chipset,
> > after i installed the kernel 2.4.9-ac10 (ac11 and 12 as well),
> > at boot i get this message,
> >
> > Sep 21 11:52:11 DarkStar kernel: ice 00:0b.0
> > Sep 21 11:52:11 DarkStar kernel: spurious 8259A interrupt: IRQ7.
> >  immediatelly before scsi adaptec detection and inizzializzation.
>
> Thats indicating an IRQ appeared and vanished. IRQ 7 is the IRQ that happens
> to occur for this.
>
> > I was thinking to an HW problem, but with vanilla 2.4.9 kernel I do not
> > get this message, and the code in arch/i386/kernel/i8259.c that should
> > detect this spurious interrup is just the same on those two kernels.
> > Changes are related to X86_IO_APIC, and I think this is the reason why
> > this spurious interrupt is detected, but I would like to know if i should
> > think I have some HW problem going on that stardard kernels do not detect.
>
> The APIC only applies to multiprocessor boxes unless you are building with
> uniprocessor apic support. Build a non SMP kernel without apic support
> and let me know what that does

yes, i was using a non SMP kernel with both apic and io_apic support
enabled.

actally:

with APIC support
and
without IO_APIC support

I do not get this message again, but I am so sorry, because my processor
has integrated APIC support.

I just made another test on a dual Athlon 1200 Mhz per CPU on an MB with
AMD chipset, and here this problem does not appear.

Thanx

Luigi

Re: spurious interrupt with ac kernel but not with vanilla 2.4.9

[Alan Cox]

> > The APIC only applies to multiprocessor boxes unless you are building with
> > uniprocessor apic support. Build a non SMP kernel without apic support
> > and let me know what that does
> 
> yes, i was using a non SMP kernel with both apic and io_apic support
> enabled.

Ok.

> actally:
> 
> with APIC support
> and
> without IO_APIC support
> 
> I do not get this message again, but I am so sorry, because my processor
> has integrated APIC support.

Oh your configuration options should have worked. Its more a case of working
out now why the didnt. Knowing that it is uniprocessor apic triggered is a
help there

Re: spurious interrupt with ac kernel but not with vanilla 2.4.9

[Luigi Genoni]

On Fri, 21 Sep 2001, Alan Cox wrote:

>
> Oh your configuration options should have worked. Its more a case of working
> out now why the didnt. Knowing that it is uniprocessor apic triggered is a
> help there

OK, I am available for any kind of test. (this is my personal station, not
a production server).

Luigi

Re: spurious interrupt with ac kernel but not with vanilla 2.4.9

[John Levon]

On Fri, Sep 21, 2001 at 06:28:42PM +0100, Alan Cox wrote:

> > yes, i was using a non SMP kernel with both apic and io_apic support
> > enabled.
> 
> Ok.
> Oh your configuration options should have worked. Its more a case of working
> out now why the didnt. Knowing that it is uniprocessor apic triggered is a
> help there

I have been getting these spurious interrupt messages for a long time now with
the UP_APIC option enabled. Using interrupt-heavy things like catting logs on a vesafb
console are a pretty reliable way to trigger it.

This is on a bog-standard UP box with no IO-APIC.

It doesn't seem to affect stability at all.

regards
john

p.s. what's going on with the config options ? linus pre12 seems to get rid of the
APIC fake zero page thing and enable the local APIC on UP properly, but the config options
don't seem to have changed like in the ac tree

-- 
"If you're not part of the problem, you're part of the problem space." 

Re: [patch] block highmem zero bounce v14

[Jens Axboe]

On Fri, Sep 21 2001, Arjan van de Ven wrote:
> On Mon, Sep 17, 2001 at 12:00:12AM +0200, Jens Axboe wrote:
> > On Sun, Sep 16 2001, Linus Torvalds wrote:
> > > 
> > > On Sun, 16 Sep 2001, Jens Axboe wrote:
> > > >
> > > > It's against 2.4.10-pre9 and can be found right here:
> > > >
> > > > *.kernel.org/pub/linux/kernel/people/axboe/patches/2.4.10-pre9/block-highmem-all-14
> > > 
> > > Jens, what's your feeling about the stability of these things, especially
> > > wrt weird drivers?
> > 
> > One of the very first decisions I made wrt this patch was to make sure
> > that weird/old drivers could keep on working exactly the way they do now
> > and never have to worry about highmem stuff. 
> 
> unfortionatly, so far both megaraid and the 3ware driver broke. Megaraid is
> easily fixable, but still. It shows that this patch is not without risk...

megaraid broke because can_dma_32 was enabled by mistake.

jens

Re: [patch] block highmem zero bounce v14

[Arjan van de Ven]

On Sat, Sep 22, 2001 at 01:00:00PM +0200, Jens Axboe wrote:
> megaraid broke because can_dma_32 was enabled by mistake.

Nope; without that it was still bust.
Megaraid broke (and 3ware most likely as well) because  it had broken code
for the "only 1 scatter gather element" case....

Re: [patch] block highmem zero bounce v14

[Jens Axboe]

On Sat, Sep 22 2001, Arjan van de Ven wrote:
> On Sat, Sep 22, 2001 at 01:00:00PM +0200, Jens Axboe wrote:
> > megaraid broke because can_dma_32 was enabled by mistake.
> 
> Nope; without that it was still bust.
> Megaraid broke (and 3ware most likely as well) because  it had broken code
> for the "only 1 scatter gather element" case....

Ah ok, that's actually a case I hadn't counted on being buggy in
drivers. Typical, drive code sucks :-)

jens

Re: [patch] block highmem zero bounce v14

[Alan Cox]

> Nope; without that it was still bust.
> Megaraid broke (and 3ware most likely as well) because  it had broken code
> for the "only 1 scatter gather element" case....

Yet more evidence that it belongs in 2.5 first. Auditing every scsi driver
for that error (and I bet someone had it first and it was copied..) is
a big job

Re: [patch] block highmem zero bounce v14

[Jens Axboe]

On Sat, Sep 22 2001, Alan Cox wrote:
> > Nope; without that it was still bust.
> > Megaraid broke (and 3ware most likely as well) because  it had broken code
> > for the "only 1 scatter gather element" case....
> 
> Yet more evidence that it belongs in 2.5 first. Auditing every scsi driver
> for that error (and I bet someone had it first and it was copied..) is
> a big job

Somehow I knew you would say that, Alan. 

jens

Re: [patch] block highmem zero bounce v14

[Alan Cox]

> > Yet more evidence that it belongs in 2.5 first. Auditing every scsi driver
> > for that error (and I bet someone had it first and it was copied..) is
> > a big job
> 
> Somehow I knew you would say that, Alan. 

I spent a lot of my time debugging driver code, and if its in one driver,
its normally in ten. Look at the last serial driver fixup for example

Alan

Re: [patch] block highmem zero bounce v14

[Jens Axboe]

On Sat, Sep 22 2001, Alan Cox wrote:
> > > Yet more evidence that it belongs in 2.5 first. Auditing every scsi driver
> > > for that error (and I bet someone had it first and it was copied..) is
> > > a big job
> > 
> > Somehow I knew you would say that, Alan. 
> 
> I spent a lot of my time debugging driver code, and if its in one driver,
> its normally in ten. Look at the last serial driver fixup for example

That's true. However, as Arjan pointed out I can fix this up not to
rely on drivers working as-is for the new single entry sg behaviour.
This is already the case for regular block drivers.

Jens

Re: [patch] block highmem zero bounce v14

[Arnaldo Carvalho de Melo]

Em Sat, Sep 22, 2001 at 05:41:46PM +0100, Alan Cox escreveu:
> > Somehow I knew you would say that, Alan. 
> 
> I spent a lot of my time debugging driver code, and if its in one driver,
> its normally in ten. Look at the last serial driver fixup for example

I got some guys learning to hack the kernel, first task? write a
multiserial driver (new one for an old .br produced card by cyclades) that
doesn't uses BHs neither cli/sti, next task? You bet... 8) 2.5, were are
ya? come on... ;)

- Arnaldo

Re: [patch] block highmem zero bounce v14

[David S. Miller]

   From: Linus Torvalds <torvalds@transmeta.com>
   Date: Sun, 16 Sep 2001 14:48:32 -0700 (PDT)
   
   Jens, what's your feeling about the stability of these things, especially
   wrt weird drivers?
   
   Ie do you think this is really a 2.4.x thing, or early 2.5.x?

On my side of this work I feel that the 64-bit PCI dma infrastructure
by itself is a safe merge in 2.4.11 or something like that.

Later,
David S. Miller
davem@redhat.com

Re: [patch] block highmem zero bounce v14

[Jens Axboe]

On Thu, Sep 20 2001, David S. Miller wrote:
>    From: Linus Torvalds <torvalds@transmeta.com>
>    Date: Sun, 16 Sep 2001 14:48:32 -0700 (PDT)
>    
>    Jens, what's your feeling about the stability of these things, especially
>    wrt weird drivers?
>    
>    Ie do you think this is really a 2.4.x thing, or early 2.5.x?
> 
> On my side of this work I feel that the 64-bit PCI dma infrastructure
> by itself is a safe merge in 2.4.11 or something like that.

I agree, and seriously hope that Linus will merge the pci64 patch in the
first 2.4.11-pre's. Then we can start looking at the block highmem
patch.

-- 
Jens Axboe