[Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Rob Landley]

The -hda, -hdb, -hdc, and -hdd command line options for g3beige don't match 
the order the kernel assigns the drives.

The reason is that the  Linux kernel always initializes the cmd646 driver 
before the pmac driver, thus if there's a cmd646 it gets /dev/hda and 
/dev/hdb, and the pmac gets /dev/hdc and /dev/hdb.

If you only supply an -hda (and/or -hdb) with no -hdc or -hdd, then the cmd646 
driver never attaches to anything and only the pmac controller shows up, thus 
-hda and -hdb set /dev/hda and /dev/hdb.  But if you specify a -hdc it shows 
up as /dev/hda every time, and kicks the -hda entry to /dev/hdc.

Note that neither the kernel's CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST nor 
CONFIG_IDEPCI_PCIBUS_ORDER made any difference, because those affect multiple 
devices handled by the same driver, and this is a static driver initialization 
order issue.  When you statically link in both drivers, cmd64x always probes 
before pmac due to the above hardwired device order in the kernel, 100% 
reliable and deterministic.  It's hardwired, and you have to patch the kernel 
to change it. 

Here's a patch to the Linux kernel that changes the device probe order so the 
kernel behaves like g3beige is expecting it to:

--- a/drivers/ide/Makefile
+++ b/drivers/ide/Makefile
@@ -39,6 +39,7 @@
 obj-$(CONFIG_BLK_DEV_AMD74XX)          += amd74xx.o
 obj-$(CONFIG_BLK_DEV_ATIIXP)           += atiixp.o
 obj-$(CONFIG_BLK_DEV_CELLEB)           += scc_pata.o
+obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
 obj-$(CONFIG_BLK_DEV_CMD64X)           += cmd64x.o
 obj-$(CONFIG_BLK_DEV_CS5520)           += cs5520.o
 obj-$(CONFIG_BLK_DEV_CS5530)           += cs5530.o
@@ -76,8 +77,6 @@
 
 obj-$(CONFIG_BLK_DEV_CMD640)           += cmd640.o
 
-obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
-
 obj-$(CONFIG_IDE_H8300)                        += ide-h8300.o
 
 obj-$(CONFIG_IDE_GENERIC)              += ide-generic.o


The problem is, the kernel guys will never take that patch upstream because 
what they're currently doing isn't actually wrong.  Their behavior is 
consistent, the kernel's been probing the same devices in the same order since 
the 90's, and they don't really care what order things go in.

The problem is that the association between qemu's command line arguments and 
the devices they refer to is somewhat arbitrary.  On the other targets I've 
used (arm, mips, x86, and so on), the device QEMU initializes in response to 
"-hda" is the one the Linux kernel makes /dev/hda (or /dev/sda), and the one 
it intializes in response to "-hdc" is the one Linux makes /dev/hdc.  But in 
this case, they don't match up, and that's screwing up my same init/build 
script that works fine on all the other tarets.

Here's a patch to QEMU that makes those arguments intialize the devices the 
kernel expects them to.  This doesn't change where any of the hardware is on 
the board, just which command line arguments associate with which drives:

--- a/hw/ppc_oldworld.c
+++ b/hw/ppc_oldworld.c
@@ -346,16 +346,16 @@ static void ppc_heathrow_init (ram_addr_t ram_size,
         exit(1);
     }
 
-    /* First IDE channel is a MAC IDE on the MacIO bus */
-    hd[0] = drive_get(IF_IDE, 0, 0);
-    hd[1] = drive_get(IF_IDE, 0, 1);
+    /* Second IDE channel is a MAC IDE on the MacIO bus */
+    hd[0] = drive_get(IF_IDE, 1, 0);
+    hd[1] = drive_get(IF_IDE, 1, 1);
     dbdma = DBDMA_init(&dbdma_mem_index);
     ide_mem_index[0] = -1;
     ide_mem_index[1] = pmac_ide_init(hd, pic[0x0D], dbdma, 0x16, pic[0x02]);
 
-    /* Second IDE channel is a CMD646 on the PCI bus */
-    hd[0] = drive_get(IF_IDE, 1, 0);
-    hd[1] = drive_get(IF_IDE, 1, 1);
+    /* First IDE channel is a CMD646 on the PCI bus */
+    hd[0] = drive_get(IF_IDE, 0, 0);
+    hd[1] = drive_get(IF_IDE, 0, 1);
     hd[3] = hd[2] = NULL;
     pci_cmd646_ide_init(pci_bus, hd, 0);
 

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Alexander Graf]

On 13.02.2010, at 09:02, Rob Landley wrote:

> The -hda, -hdb, -hdc, and -hdd command line options for g3beige don't match 
> the order the kernel assigns the drives.
> 
> The reason is that the  Linux kernel always initializes the cmd646 driver 
> before the pmac driver, thus if there's a cmd646 it gets /dev/hda and 
> /dev/hdb, and the pmac gets /dev/hdc and /dev/hdb.
> 
> If you only supply an -hda (and/or -hdb) with no -hdc or -hdd, then the cmd646 
> driver never attaches to anything and only the pmac controller shows up, thus 
> -hda and -hdb set /dev/hda and /dev/hdb.  But if you specify a -hdc it shows 
> up as /dev/hda every time, and kicks the -hda entry to /dev/hdc.
> 
> Note that neither the kernel's CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST nor 
> CONFIG_IDEPCI_PCIBUS_ORDER made any difference, because those affect multiple 
> devices handled by the same driver, and this is a static driver initialization 
> order issue.  When you statically link in both drivers, cmd64x always probes 
> before pmac due to the above hardwired device order in the kernel, 100% 
> reliable and deterministic.  It's hardwired, and you have to patch the kernel 
> to change it. 
> 
> Here's a patch to the Linux kernel that changes the device probe order so the 
> kernel behaves like g3beige is expecting it to:
> 
> --- a/drivers/ide/Makefile
> +++ b/drivers/ide/Makefile
> @@ -39,6 +39,7 @@
> obj-$(CONFIG_BLK_DEV_AMD74XX)          += amd74xx.o
> obj-$(CONFIG_BLK_DEV_ATIIXP)           += atiixp.o
> obj-$(CONFIG_BLK_DEV_CELLEB)           += scc_pata.o
> +obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> obj-$(CONFIG_BLK_DEV_CMD64X)           += cmd64x.o
> obj-$(CONFIG_BLK_DEV_CS5520)           += cs5520.o
> obj-$(CONFIG_BLK_DEV_CS5530)           += cs5530.o
> @@ -76,8 +77,6 @@
> 
> obj-$(CONFIG_BLK_DEV_CMD640)           += cmd640.o
> 
> -obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> -
> obj-$(CONFIG_IDE_H8300)                        += ide-h8300.o
> 
> obj-$(CONFIG_IDE_GENERIC)              += ide-generic.o
> 
> 
> The problem is, the kernel guys will never take that patch upstream because 
> what they're currently doing isn't actually wrong.  Their behavior is 
> consistent, the kernel's been probing the same devices in the same order since 
> the 90's, and they don't really care what order things go in.
> 
> The problem is that the association between qemu's command line arguments and 
> the devices they refer to is somewhat arbitrary.  On the other targets I've 
> used (arm, mips, x86, and so on), the device QEMU initializes in response to 
> "-hda" is the one the Linux kernel makes /dev/hda (or /dev/sda), and the one 
> it intializes in response to "-hdc" is the one Linux makes /dev/hdc.  But in 
> this case, they don't match up, and that's screwing up my same init/build 
> script that works fine on all the other tarets.
> 
> Here's a patch to QEMU that makes those arguments intialize the devices the 
> kernel expects them to.  This doesn't change where any of the hardware is on 
> the board, just which command line arguments associate with which drives:

This is wrong. On my OpenSUSE 11.1 guest the devices come up in correct order. They also do so on Aurelien's Debian images (IIRC). I guess it mostly works fine when using modules instead of compiled in drivers.

Please find a real G3 beige and see what's different on it. I'd bet the real difference is that all 4 devices are attached to MacIO. But from what I remember DBDMA with cd-roms wasn't considered stable, hence the use of cmd64x on the second channel.


Alex

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Aurelien Jarno]

On Sat, Feb 13, 2010 at 11:28:44AM +0100, Alexander Graf wrote:
> 
> On 13.02.2010, at 09:02, Rob Landley wrote:
> 
> > The -hda, -hdb, -hdc, and -hdd command line options for g3beige don't match 
> > the order the kernel assigns the drives.
> > 
> > The reason is that the  Linux kernel always initializes the cmd646 driver 
> > before the pmac driver, thus if there's a cmd646 it gets /dev/hda and 
> > /dev/hdb, and the pmac gets /dev/hdc and /dev/hdb.
> > 
> > If you only supply an -hda (and/or -hdb) with no -hdc or -hdd, then the cmd646 
> > driver never attaches to anything and only the pmac controller shows up, thus 
> > -hda and -hdb set /dev/hda and /dev/hdb.  But if you specify a -hdc it shows 
> > up as /dev/hda every time, and kicks the -hda entry to /dev/hdc.
> > 
> > Note that neither the kernel's CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST nor 
> > CONFIG_IDEPCI_PCIBUS_ORDER made any difference, because those affect multiple 
> > devices handled by the same driver, and this is a static driver initialization 
> > order issue.  When you statically link in both drivers, cmd64x always probes 
> > before pmac due to the above hardwired device order in the kernel, 100% 
> > reliable and deterministic.  It's hardwired, and you have to patch the kernel 
> > to change it. 
> > 
> > Here's a patch to the Linux kernel that changes the device probe order so the 
> > kernel behaves like g3beige is expecting it to:
> > 
> > --- a/drivers/ide/Makefile
> > +++ b/drivers/ide/Makefile
> > @@ -39,6 +39,7 @@
> > obj-$(CONFIG_BLK_DEV_AMD74XX)          += amd74xx.o
> > obj-$(CONFIG_BLK_DEV_ATIIXP)           += atiixp.o
> > obj-$(CONFIG_BLK_DEV_CELLEB)           += scc_pata.o
> > +obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> > obj-$(CONFIG_BLK_DEV_CMD64X)           += cmd64x.o
> > obj-$(CONFIG_BLK_DEV_CS5520)           += cs5520.o
> > obj-$(CONFIG_BLK_DEV_CS5530)           += cs5530.o
> > @@ -76,8 +77,6 @@
> > 
> > obj-$(CONFIG_BLK_DEV_CMD640)           += cmd640.o
> > 
> > -obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> > -
> > obj-$(CONFIG_IDE_H8300)                        += ide-h8300.o
> > 
> > obj-$(CONFIG_IDE_GENERIC)              += ide-generic.o
> > 
> > 
> > The problem is, the kernel guys will never take that patch upstream because 
> > what they're currently doing isn't actually wrong.  Their behavior is 
> > consistent, the kernel's been probing the same devices in the same order since 
> > the 90's, and they don't really care what order things go in.
> > 
> > The problem is that the association between qemu's command line arguments and 
> > the devices they refer to is somewhat arbitrary.  On the other targets I've 
> > used (arm, mips, x86, and so on), the device QEMU initializes in response to 
> > "-hda" is the one the Linux kernel makes /dev/hda (or /dev/sda), and the one 
> > it intializes in response to "-hdc" is the one Linux makes /dev/hdc.  But in 
> > this case, they don't match up, and that's screwing up my same init/build 
> > script that works fine on all the other tarets.
> > 
> > Here's a patch to QEMU that makes those arguments intialize the devices the 
> > kernel expects them to.  This doesn't change where any of the hardware is on 
> > the board, just which command line arguments associate with which drives:
> 
> This is wrong. On my OpenSUSE 11.1 guest the devices come up in correct order. They also do so on Aurelien's Debian images (IIRC). I guess it mostly works fine when using modules instead of compiled in drivers.
> 
> Please find a real G3 beige and see what's different on it. I'd bet the real difference is that all 4 devices are attached to MacIO. But from what I remember DBDMA with cd-roms wasn't considered stable, hence the use of cmd64x on the second channel.
> 

Exactly, that's the issue to fix here, make DBDMA work with CD-ROM so we
can get rid of the cmd64x controller.

-- 
Aurelien Jarno	                        GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Alexander Graf]

On 13.02.2010, at 12:58, Aurelien Jarno wrote:

> On Sat, Feb 13, 2010 at 11:28:44AM +0100, Alexander Graf wrote:
>> 
>> On 13.02.2010, at 09:02, Rob Landley wrote:
>> 
>>> The -hda, -hdb, -hdc, and -hdd command line options for g3beige don't match 
>>> the order the kernel assigns the drives.
>>> 
>>> The reason is that the  Linux kernel always initializes the cmd646 driver 
>>> before the pmac driver, thus if there's a cmd646 it gets /dev/hda and 
>>> /dev/hdb, and the pmac gets /dev/hdc and /dev/hdb.
>>> 
>>> If you only supply an -hda (and/or -hdb) with no -hdc or -hdd, then the cmd646 
>>> driver never attaches to anything and only the pmac controller shows up, thus 
>>> -hda and -hdb set /dev/hda and /dev/hdb.  But if you specify a -hdc it shows 
>>> up as /dev/hda every time, and kicks the -hda entry to /dev/hdc.
>>> 
>>> Note that neither the kernel's CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST nor 
>>> CONFIG_IDEPCI_PCIBUS_ORDER made any difference, because those affect multiple 
>>> devices handled by the same driver, and this is a static driver initialization 
>>> order issue.  When you statically link in both drivers, cmd64x always probes 
>>> before pmac due to the above hardwired device order in the kernel, 100% 
>>> reliable and deterministic.  It's hardwired, and you have to patch the kernel 
>>> to change it. 
>>> 
>>> Here's a patch to the Linux kernel that changes the device probe order so the 
>>> kernel behaves like g3beige is expecting it to:
>>> 
>>> --- a/drivers/ide/Makefile
>>> +++ b/drivers/ide/Makefile
>>> @@ -39,6 +39,7 @@
>>> obj-$(CONFIG_BLK_DEV_AMD74XX)          += amd74xx.o
>>> obj-$(CONFIG_BLK_DEV_ATIIXP)           += atiixp.o
>>> obj-$(CONFIG_BLK_DEV_CELLEB)           += scc_pata.o
>>> +obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
>>> obj-$(CONFIG_BLK_DEV_CMD64X)           += cmd64x.o
>>> obj-$(CONFIG_BLK_DEV_CS5520)           += cs5520.o
>>> obj-$(CONFIG_BLK_DEV_CS5530)           += cs5530.o
>>> @@ -76,8 +77,6 @@
>>> 
>>> obj-$(CONFIG_BLK_DEV_CMD640)           += cmd640.o
>>> 
>>> -obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
>>> -
>>> obj-$(CONFIG_IDE_H8300)                        += ide-h8300.o
>>> 
>>> obj-$(CONFIG_IDE_GENERIC)              += ide-generic.o
>>> 
>>> 
>>> The problem is, the kernel guys will never take that patch upstream because 
>>> what they're currently doing isn't actually wrong.  Their behavior is 
>>> consistent, the kernel's been probing the same devices in the same order since 
>>> the 90's, and they don't really care what order things go in.
>>> 
>>> The problem is that the association between qemu's command line arguments and 
>>> the devices they refer to is somewhat arbitrary.  On the other targets I've 
>>> used (arm, mips, x86, and so on), the device QEMU initializes in response to 
>>> "-hda" is the one the Linux kernel makes /dev/hda (or /dev/sda), and the one 
>>> it intializes in response to "-hdc" is the one Linux makes /dev/hdc.  But in 
>>> this case, they don't match up, and that's screwing up my same init/build 
>>> script that works fine on all the other tarets.
>>> 
>>> Here's a patch to QEMU that makes those arguments intialize the devices the 
>>> kernel expects them to.  This doesn't change where any of the hardware is on 
>>> the board, just which command line arguments associate with which drives:
>> 
>> This is wrong. On my OpenSUSE 11.1 guest the devices come up in correct order. They also do so on Aurelien's Debian images (IIRC). I guess it mostly works fine when using modules instead of compiled in drivers.
>> 
>> Please find a real G3 beige and see what's different on it. I'd bet the real difference is that all 4 devices are attached to MacIO. But from what I remember DBDMA with cd-roms wasn't considered stable, hence the use of cmd64x on the second channel.
>> 
> 
> Exactly, that's the issue to fix here, make DBDMA work with CD-ROM so we
> can get rid of the cmd64x controller.

Speaking of which - in my PPC64 enabling series I use MacIO for all 4 IDE devices. At least with recent kernels, Linux just detects DMA being broken on the CD-ROM and doesn't use it.

Alex

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Aurelien Jarno]

On Sat, Feb 13, 2010 at 01:04:03PM +0100, Alexander Graf wrote:
> 
> On 13.02.2010, at 12:58, Aurelien Jarno wrote:
> 
> > On Sat, Feb 13, 2010 at 11:28:44AM +0100, Alexander Graf wrote:
> >> 
> >> On 13.02.2010, at 09:02, Rob Landley wrote:
> >> 
> >>> The -hda, -hdb, -hdc, and -hdd command line options for g3beige don't match 
> >>> the order the kernel assigns the drives.
> >>> 
> >>> The reason is that the  Linux kernel always initializes the cmd646 driver 
> >>> before the pmac driver, thus if there's a cmd646 it gets /dev/hda and 
> >>> /dev/hdb, and the pmac gets /dev/hdc and /dev/hdb.
> >>> 
> >>> If you only supply an -hda (and/or -hdb) with no -hdc or -hdd, then the cmd646 
> >>> driver never attaches to anything and only the pmac controller shows up, thus 
> >>> -hda and -hdb set /dev/hda and /dev/hdb.  But if you specify a -hdc it shows 
> >>> up as /dev/hda every time, and kicks the -hda entry to /dev/hdc.
> >>> 
> >>> Note that neither the kernel's CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST nor 
> >>> CONFIG_IDEPCI_PCIBUS_ORDER made any difference, because those affect multiple 
> >>> devices handled by the same driver, and this is a static driver initialization 
> >>> order issue.  When you statically link in both drivers, cmd64x always probes 
> >>> before pmac due to the above hardwired device order in the kernel, 100% 
> >>> reliable and deterministic.  It's hardwired, and you have to patch the kernel 
> >>> to change it. 
> >>> 
> >>> Here's a patch to the Linux kernel that changes the device probe order so the 
> >>> kernel behaves like g3beige is expecting it to:
> >>> 
> >>> --- a/drivers/ide/Makefile
> >>> +++ b/drivers/ide/Makefile
> >>> @@ -39,6 +39,7 @@
> >>> obj-$(CONFIG_BLK_DEV_AMD74XX)          += amd74xx.o
> >>> obj-$(CONFIG_BLK_DEV_ATIIXP)           += atiixp.o
> >>> obj-$(CONFIG_BLK_DEV_CELLEB)           += scc_pata.o
> >>> +obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> >>> obj-$(CONFIG_BLK_DEV_CMD64X)           += cmd64x.o
> >>> obj-$(CONFIG_BLK_DEV_CS5520)           += cs5520.o
> >>> obj-$(CONFIG_BLK_DEV_CS5530)           += cs5530.o
> >>> @@ -76,8 +77,6 @@
> >>> 
> >>> obj-$(CONFIG_BLK_DEV_CMD640)           += cmd640.o
> >>> 
> >>> -obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> >>> -
> >>> obj-$(CONFIG_IDE_H8300)                        += ide-h8300.o
> >>> 
> >>> obj-$(CONFIG_IDE_GENERIC)              += ide-generic.o
> >>> 
> >>> 
> >>> The problem is, the kernel guys will never take that patch upstream because 
> >>> what they're currently doing isn't actually wrong.  Their behavior is 
> >>> consistent, the kernel's been probing the same devices in the same order since 
> >>> the 90's, and they don't really care what order things go in.
> >>> 
> >>> The problem is that the association between qemu's command line arguments and 
> >>> the devices they refer to is somewhat arbitrary.  On the other targets I've 
> >>> used (arm, mips, x86, and so on), the device QEMU initializes in response to 
> >>> "-hda" is the one the Linux kernel makes /dev/hda (or /dev/sda), and the one 
> >>> it intializes in response to "-hdc" is the one Linux makes /dev/hdc.  But in 
> >>> this case, they don't match up, and that's screwing up my same init/build 
> >>> script that works fine on all the other tarets.
> >>> 
> >>> Here's a patch to QEMU that makes those arguments intialize the devices the 
> >>> kernel expects them to.  This doesn't change where any of the hardware is on 
> >>> the board, just which command line arguments associate with which drives:
> >> 
> >> This is wrong. On my OpenSUSE 11.1 guest the devices come up in correct order. They also do so on Aurelien's Debian images (IIRC). I guess it mostly works fine when using modules instead of compiled in drivers.
> >> 
> >> Please find a real G3 beige and see what's different on it. I'd bet the real difference is that all 4 devices are attached to MacIO. But from what I remember DBDMA with cd-roms wasn't considered stable, hence the use of cmd64x on the second channel.
> >> 
> > 
> > Exactly, that's the issue to fix here, make DBDMA work with CD-ROM so we
> > can get rid of the cmd64x controller.
> 
> Speaking of which - in my PPC64 enabling series I use MacIO for all 4 IDE devices. At least with recent kernels, Linux just detects DMA being broken on the CD-ROM and doesn't use it.
> 

Same on PPC32, except that when DMA is not used, the VM freeze after a
few accesses to the drive.

-- 
Aurelien Jarno	                        GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Rob Landley]

On Saturday 13 February 2010 04:28:44 Alexander Graf wrote:
> On 13.02.2010, at 09:02, Rob Landley wrote:
> > The -hda, -hdb, -hdc, and -hdd command line options for g3beige don't
> > match the order the kernel assigns the drives.
> >
> > The reason is that the  Linux kernel always initializes the cmd646 driver
> > before the pmac driver, thus if there's a cmd646 it gets /dev/hda and
> > /dev/hdb, and the pmac gets /dev/hdc and /dev/hdb.
> >
> > If you only supply an -hda (and/or -hdb) with no -hdc or -hdd, then the
> > cmd646 driver never attaches to anything and only the pmac controller
> > shows up, thus -hda and -hdb set /dev/hda and /dev/hdb.  But if you
> > specify a -hdc it shows up as /dev/hda every time, and kicks the -hda
> > entry to /dev/hdc.
> >
> > Note that neither the kernel's CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST nor
> > CONFIG_IDEPCI_PCIBUS_ORDER made any difference, because those affect
> > multiple devices handled by the same driver, and this is a static driver
> > initialization order issue.  When you statically link in both drivers,
> > cmd64x always probes before pmac due to the above hardwired device order
> > in the kernel, 100% reliable and deterministic.  It's hardwired, and you
> > have to patch the kernel to change it.
> >
> > Here's a patch to the Linux kernel that changes the device probe order so
> > the kernel behaves like g3beige is expecting it to:
> >
> > --- a/drivers/ide/Makefile
> > +++ b/drivers/ide/Makefile
> > @@ -39,6 +39,7 @@
> > obj-$(CONFIG_BLK_DEV_AMD74XX)          += amd74xx.o
> > obj-$(CONFIG_BLK_DEV_ATIIXP)           += atiixp.o
> > obj-$(CONFIG_BLK_DEV_CELLEB)           += scc_pata.o
> > +obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> > obj-$(CONFIG_BLK_DEV_CMD64X)           += cmd64x.o
> > obj-$(CONFIG_BLK_DEV_CS5520)           += cs5520.o
> > obj-$(CONFIG_BLK_DEV_CS5530)           += cs5530.o
> > @@ -76,8 +77,6 @@
> >
> > obj-$(CONFIG_BLK_DEV_CMD640)           += cmd640.o
> >
> > -obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
> > -
> > obj-$(CONFIG_IDE_H8300)                        += ide-h8300.o
> >
> > obj-$(CONFIG_IDE_GENERIC)              += ide-generic.o
> >
> >
> > The problem is, the kernel guys will never take that patch upstream
> > because what they're currently doing isn't actually wrong.  Their
> > behavior is consistent, the kernel's been probing the same devices in the
> > same order since the 90's, and they don't really care what order things
> > go in.
> >
> > The problem is that the association between qemu's command line arguments
> > and the devices they refer to is somewhat arbitrary.  On the other
> > targets I've used (arm, mips, x86, and so on), the device QEMU
> > initializes in response to "-hda" is the one the Linux kernel makes
> > /dev/hda (or /dev/sda), and the one it intializes in response to "-hdc"
> > is the one Linux makes /dev/hdc.  But in this case, they don't match up,
> > and that's screwing up my same init/build script that works fine on all
> > the other tarets.
> >
> > Here's a patch to QEMU that makes those arguments intialize the devices
> > the kernel expects them to.  This doesn't change where any of the
> > hardware is on the board, just which command line arguments associate
> > with which drives:
>
> This is wrong. On my OpenSUSE 11.1 guest the devices come up in correct
> order. They also do so on Aurelien's Debian images (IIRC). I guess it
> mostly works fine when using modules instead of compiled in drivers.

When using modules the devices come up in the order the modules get inserted.  
The first module inserted allocates /dev/hda and /dev/hdb, the second one 
inserted allocates /dev/hdc and /dev/hdd.  Also, udev can remap the suckers 
arbitrarily if it wants to.

Of course this assumes you're using initramfs and a modular kernel, and not 
all embedded systems want those extra layers.

In fact take that to its logical conclusion and there's no reason for qemu to 
have separate -hda, -hdb, -hdc, and -hdd.  It can just have "-disk image1.sqf 
-disk image2.ext3 -disk image3.sqf" and let udev look for the magic in-band 
signalling inside each image to determine what to mount via uuid.

> Please find a real G3 beige and see what's different on it.

It's not a question of hardware.  That's why I showed the kernel patch that 
can also reliably change this behavior.  It's a question of:

A) The Linux kernel's device ordering being based on driver initialization 
order when dealing with different types of controllers, which in the case of a 
non-modular kernel is hardwired and always has been.  (The first patch shows 
changing that order within the kernel.)

B) The qemu command line options associating with different devices than the 
Linux kernel is doing.  The one -hda initializes is not the one the Linux 
kernel uses as -hda when both sets of drivers are statically linked into the 
kernel.  (The second patch changes that ordering within qemu.  It does NOT 
alter where any devices are, it just alters which command line option goes 
with which device.)

> I'd bet the
> real difference is that all 4 devices are attached to MacIO. But from what
> I remember DBDMA with cd-roms wasn't considered stable, hence the use of
> cmd64x on the second channel.

My use case is attaching three disk images to the system, not a CD or DVD 
image, so I'd be happy if all 3 went through one controller.  The /dev/hda is 
a generic squashfs root filesystem image containing basic development tools.  
The /dev/hdb is a writeable 2 gigabyte scratch disk.  The /dev/hdc is another 
squashfs containing build scripts and pre-extracted source tarballs.

I pondered merging them into a single partitioned image to work around this 
issue, but they do very different things and have good reason to be orthogonal.  
The hdb images are the only writeable ones, the other two are read only.  The 
hdc image is portable across architectures (it's just read-only source code 
and shell scripts) while hda isn't, so I actually have arm, mips, and x86 all 
building the same set of binaries sharing a single hdc file, but each with its 
own hda and hdb images...

I also pondered writing a different qemu launch script that called qemu with "-
hdc root.sqf -hdd workspace.ext3 -hda build.sqf" but this is the only platform 
I'd need to do that for.  All the others work correctly with the same set of 
arguments (with a couple $VARIABLES in there).  This would need a special 
case, because the g3beige behavior is different than the other platforms.

> Alex

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Rob Landley]

On Saturday 13 February 2010 06:04:03 Alexander Graf wrote:
> > Exactly, that's the issue to fix here, make DBDMA work with CD-ROM so we
> > can get rid of the cmd64x controller.
>
> Speaking of which - in my PPC64 enabling series I use MacIO for all 4 IDE
> devices. At least with recent kernels, Linux just detects DMA being broken
> on the CD-ROM and doesn't use it.

Could you point me at a copy of that patch?  That sounds like it would be a 
decent solution for me, I'm happy to test it if it helps push it upstream.

> Alex

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Alexander Graf]

Am Sat 13 Feb 2010 07:27:21 PM CET schrieb Rob Landley <rob@landley.net>:

> On Saturday 13 February 2010 04:28:44 Alexander Graf wrote:
>> On 13.02.2010, at 09:02, Rob Landley wrote:
>> > The -hda, -hdb, -hdc, and -hdd command line options for g3beige don't
>> > match the order the kernel assigns the drives.
>> >
>> > The reason is that the  Linux kernel always initializes the cmd646 driver
>> > before the pmac driver, thus if there's a cmd646 it gets /dev/hda and
>> > /dev/hdb, and the pmac gets /dev/hdc and /dev/hdb.
>> >
>> > If you only supply an -hda (and/or -hdb) with no -hdc or -hdd, then the
>> > cmd646 driver never attaches to anything and only the pmac controller
>> > shows up, thus -hda and -hdb set /dev/hda and /dev/hdb.  But if you
>> > specify a -hdc it shows up as /dev/hda every time, and kicks the -hda
>> > entry to /dev/hdc.
>> >
>> > Note that neither the kernel's CONFIG_BLK_DEV_IDE_PMAC_ATA100FIRST nor
>> > CONFIG_IDEPCI_PCIBUS_ORDER made any difference, because those affect
>> > multiple devices handled by the same driver, and this is a static driver
>> > initialization order issue.  When you statically link in both drivers,
>> > cmd64x always probes before pmac due to the above hardwired device order
>> > in the kernel, 100% reliable and deterministic.  It's hardwired, and you
>> > have to patch the kernel to change it.
>> >
>> > Here's a patch to the Linux kernel that changes the device probe order so
>> > the kernel behaves like g3beige is expecting it to:
>> >
>> > --- a/drivers/ide/Makefile
>> > +++ b/drivers/ide/Makefile
>> > @@ -39,6 +39,7 @@
>> > obj-$(CONFIG_BLK_DEV_AMD74XX)          += amd74xx.o
>> > obj-$(CONFIG_BLK_DEV_ATIIXP)           += atiixp.o
>> > obj-$(CONFIG_BLK_DEV_CELLEB)           += scc_pata.o
>> > +obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
>> > obj-$(CONFIG_BLK_DEV_CMD64X)           += cmd64x.o
>> > obj-$(CONFIG_BLK_DEV_CS5520)           += cs5520.o
>> > obj-$(CONFIG_BLK_DEV_CS5530)           += cs5530.o
>> > @@ -76,8 +77,6 @@
>> >
>> > obj-$(CONFIG_BLK_DEV_CMD640)           += cmd640.o
>> >
>> > -obj-$(CONFIG_BLK_DEV_IDE_PMAC)         += pmac.o
>> > -
>> > obj-$(CONFIG_IDE_H8300)                        += ide-h8300.o
>> >
>> > obj-$(CONFIG_IDE_GENERIC)              += ide-generic.o
>> >
>> >
>> > The problem is, the kernel guys will never take that patch upstream
>> > because what they're currently doing isn't actually wrong.  Their
>> > behavior is consistent, the kernel's been probing the same devices in the
>> > same order since the 90's, and they don't really care what order things
>> > go in.
>> >
>> > The problem is that the association between qemu's command line arguments
>> > and the devices they refer to is somewhat arbitrary.  On the other
>> > targets I've used (arm, mips, x86, and so on), the device QEMU
>> > initializes in response to "-hda" is the one the Linux kernel makes
>> > /dev/hda (or /dev/sda), and the one it intializes in response to "-hdc"
>> > is the one Linux makes /dev/hdc.  But in this case, they don't match up,
>> > and that's screwing up my same init/build script that works fine on all
>> > the other tarets.
>> >
>> > Here's a patch to QEMU that makes those arguments intialize the devices
>> > the kernel expects them to.  This doesn't change where any of the
>> > hardware is on the board, just which command line arguments associate
>> > with which drives:
>>
>> This is wrong. On my OpenSUSE 11.1 guest the devices come up in correct
>> order. They also do so on Aurelien's Debian images (IIRC). I guess it
>> mostly works fine when using modules instead of compiled in drivers.
>
> When using modules the devices come up in the order the modules get inserted.
> The first module inserted allocates /dev/hda and /dev/hdb, the second one
> inserted allocates /dev/hdc and /dev/hdd.  Also, udev can remap the suckers
> arbitrarily if it wants to.
>
> Of course this assumes you're using initramfs and a modular kernel, and not
> all embedded systems want those extra layers.
>
> In fact take that to its logical conclusion and there's no reason for qemu to
> have separate -hda, -hdb, -hdc, and -hdd.  It can just have "-disk image1.sqf
> -disk image2.ext3 -disk image3.sqf" and let udev look for the magic in-band
> signalling inside each image to determine what to mount via uuid.

We're trying to emulate real world hardware here, not something PV'ish  
where Linux and Qemu go hand in hand and cooperate. If you want that,  
use -disk if=virtio :-).

>
>> Please find a real G3 beige and see what's different on it.
>
> It's not a question of hardware.  That's why I showed the kernel patch that
> can also reliably change this behavior.  It's a question of:
>
> A) The Linux kernel's device ordering being based on driver initialization
> order when dealing with different types of controllers, which in the  
>  case of a
> non-modular kernel is hardwired and always has been.  (The first patch shows
> changing that order within the kernel.)

The basic point is that the Linux kernel doesn't guarantee any  
ordering of device node names when there's more than one controller in  
the system. That's why you're screwed in this configuration - we have  
2 IDE controllers each of which serving one channel w/ two devices.

>
> B) The qemu command line options associating with different devices than the
> Linux kernel is doing.  The one -hda initializes is not the one the Linux
> kernel uses as -hda when both sets of drivers are statically linked into the
> kernel.  (The second patch changes that ordering within qemu.  It does NOT
> alter where any devices are, it just alters which command line option goes
> with which device.)

I think you're missing the point. The only reason the cmd64x is used  
is because the CD-ROM failed on MacIO. If you now change the order,  
you end up with the CD-ROM being on MacIO. You could just as well have  
gotten rid of the cmd64x altogether, making it only one IDE controller  
in the system and thus reliably detected.

>
>> I'd bet the
>> real difference is that all 4 devices are attached to MacIO. But from what
>> I remember DBDMA with cd-roms wasn't considered stable, hence the use of
>> cmd64x on the second channel.
>
> My use case is attaching three disk images to the system, not a CD or DVD
> image, so I'd be happy if all 3 went through one controller.  The /dev/hda is
> a generic squashfs root filesystem image containing basic development tools.
> The /dev/hdb is a writeable 2 gigabyte scratch disk.  The /dev/hdc is another
> squashfs containing build scripts and pre-extracted source tarballs.

Unfortunately there are more users of Qemu than you :-). And  
installing from a virtual CD-ROM is a pretty big use case.


Alex

Re: [Qemu-devel] qemu-system-ppc -m g3beige -hda is setting /dev/hdc on Linux.

[Alexander Graf]

Am Sat 13 Feb 2010 07:29:33 PM CET schrieb Rob Landley <rob@landley.net>:

> On Saturday 13 February 2010 06:04:03 Alexander Graf wrote:
>> > Exactly, that's the issue to fix here, make DBDMA work with CD-ROM so we
>> > can get rid of the cmd64x controller.
>>
>> Speaking of which - in my PPC64 enabling series I use MacIO for all 4 IDE
>> devices. At least with recent kernels, Linux just detects DMA being broken
>> on the CD-ROM and doesn't use it.
>
> Could you point me at a copy of that patch?  That sounds like it would be a
> decent solution for me, I'm happy to test it if it helps push it upstream.

git://repo.or.cz/qemu/agraf.git

But I don't think testing would help pushing it upstream.


Alex