From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gleb Natapov Subject: Re: [PATCHv4 15/15] Pass boot device list to firmware. Date: Thu, 18 Nov 2010 14:37:08 +0200 Message-ID: <20101118123708.GN7948@redhat.com> References: <20101116141112.GS7948@redhat.com> <20101116190246.GA27851@redhat.com> <20101118101827.GC7948@redhat.com> <20101118113831.GD31261@redhat.com> <20101118114504.GI7948@redhat.com> <20101118115230.GE31261@redhat.com> <20101118121605.GK7948@redhat.com> <20101118122320.GC31987@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Blue Swirl , qemu-devel@nongnu.org, kvm@vger.kernel.org, armbru@redhat.com, alex.williamson@redhat.com, kevin@koconnor.net To: "Michael S. Tsirkin" Return-path: Received: from mx1.redhat.com ([209.132.183.28]:62765 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751130Ab0KRMhR convert rfc822-to-8bit (ORCPT ); Thu, 18 Nov 2010 07:37:17 -0500 Content-Disposition: inline In-Reply-To: <20101118122320.GC31987@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On Thu, Nov 18, 2010 at 02:23:20PM +0200, Michael S. Tsirkin wrote: > On Thu, Nov 18, 2010 at 02:16:05PM +0200, Gleb Natapov wrote: > > On Thu, Nov 18, 2010 at 01:52:30PM +0200, Michael S. Tsirkin wrote: > > > On Thu, Nov 18, 2010 at 01:45:04PM +0200, Gleb Natapov wrote: > > > > On Thu, Nov 18, 2010 at 01:38:31PM +0200, Michael S. Tsirkin wr= ote: > > > > > On Thu, Nov 18, 2010 at 12:18:27PM +0200, Gleb Natapov wrote: > > > > > > On Wed, Nov 17, 2010 at 09:54:27PM +0000, Blue Swirl wrote: > > > > > > > 2010/11/16 Gleb Natapov : > > > > > > > > On Tue, Nov 16, 2010 at 06:30:19PM +0000, Blue Swirl wr= ote: > > > > > > > >> >> Perhaps the FW path should use device class names i= f no name is specified. > > > > > > > >> > What do you mean by "device class name". We can do s= omething like this: > > > > > > > >> > if (dev->child_bus.lh_first) > > > > > > > >> > =9A =9A =9A =9Areturn dev->child_bus.lh_first->info-= >name; > > > > > > > >> > > > > > > > > >> > i.e if there is child bus use its bus name as fw nam= e. This will make > > > > > > > >> > all pci devices to have "pci" as fw name automatical= ly. The problem is > > > > > > > >> > that theoretically same device can provide different= buses. > > > > > > > >> > > > > > > > >> I meant PCI class name, like "display" for display con= trollers, > > > > > > > >> "network" for NICs etc. > > > > > > > >> > > > > > > > > That is what my pci bus related patch is doing already. > > > > > > > > > > > > > > > >> >> I'll try Sparc32 to see how this fits there. > > > > > > > >> > > > > > > > >> Except bootindex is not implemented for SCSI. > > > > > > > > Will look into adding it. > > > > > > >=20 > > > > > > > Thanks. The bootindex on Sparc32 looks like this: > > > > > > > bootindex /esp@0000000078800000/disk@1,0 > > > > > > > /ethernet@ffffffffffffffff/ethernet-phy@0 > > > > > > >=20 > > > > > > For arches other then x86 there is a lot of work left to be= done :) > > > > > > For starter exotic sparc buses should get their own get_fw_= dev_path() > > > > > > implementation. > > > > > >=20 > > > > > > > I don't think I got Lance setup right. > > > > > > >=20 > > > > > > > OF paths for the devices would be: > > > > > > > /iommu@0,10000000/sbus@0,10001000/espdma@5,8400000/esp@5,= 8800000/sd@1,0 > > > > > > > /iommu@0,10000000/sbus@0,10001000/ledma@5,8400010/le@5,8c= 00000 > > > > > > If qdev hierarchy does not correspond to real HW there is n= o much we can > > > > > > do expect for fixing qdev. > > > > >=20 > > > > > That's bad. This raises a concern: if these paths expose qde= v > > > > > internals, any attempt to fix this will break migration. > > > > >=20 > > > > The path expose internal HW hierarchy. It is designed to do so.= Qdev > > > > designed to do the same: describe HW hierarchy. If qdev fails t= o do so it > > > > is broken. > > >=20 > > > Yes. But since you use qdev to build up the path, a broken > > > qdev will give you a broken path. > > >=20 > > Qdev bug. Fix it like any other bug. The nice is that when you comp= are > > device path produced by qdev and real HW you can see when qdev is w= rong. > >=20 > > > > I do not see connection to migration at all since the path is > > > > not used in migration code. > > >=20 > > > The connection is that if we pass the list with path 1 which you = define > > > as broken to BIOS, then migrate to a machine with an updated qemu > > > which has a correct path, BIOS won't be able to complete the boot= =2E > > You solve it like you solve all such issue with -M machine type. >=20 > So that's unavoidable if we think paths are correct. > But if we know they are wrong, we are better off > correcting them first IMO. >=20 They are correct for x86. My patch set does not even tries to cover all HW. If sparc want to use them to it better be fixed. Or if there is eno= ugh info in the path to determine device it may choose to use it as is. > > But the problem exists only if migration happens in a short window > > between start of the boot process and BIOS reading boot order strin= g. > > After reboot new qemu should have new BIOS. >=20 > That makes it even more nasty, doesn't it? No. >=20 > > > Right? Same in reverse direction. > > Reverse direction is not and never was supported. > >=20 > > > As solution could be a fuzzy matching > > > of paths that wiull let us recover. > > >=20 > > Firmware can try its best of course, but nothing is guarantied. >=20 > No I mean qemu could do matching fuzzily. > This way if we get a path from the old BIOS we can > survive. Qemu does not take paths from BIOS so I don't know what are you talking about here. >=20 > > > > > > >=20 > > > > > > > The logic for ESP is that ESP (registers at 0x78800000, s= lot offset > > > > > > > 0x880000) is handled by the DMA controller (registers at = 0x78400000, > > > > > > > slot offset 0x840000), they are in a SBus slot #5, and SB= us (registers > > > > > > > at 0x10001000) is in turn handled by IOMMU (registers at = 0x10000000). > > > > > > > Lance should be handled the same way. > > > > > > >=20 > > > > > > > This hierarchy is partly known by QEMU because DMA access= es use this > > > > > > > flow, but not otherwise. There is no concept of SBus slot= s, DMA talks > > > > > > > to IOMMU directly. Though in this case both ESP, Lance an= d their DMA > > > > > > > controllers are on board devices in a MACIO chip. It may = be possible > > > > > > > to add the hierarchy information at each stage. > > > > > > >=20 > > > > > > > It should also be possible for BIOS to determine the devi= ce just from > > > > > > > the physical address if we ignored OF compatibility. > > > > > > It would be nice to be OF compatible at least at some level= =2E Of course OF > > > > > > spec is not strict enough to have two different implementat= ions produce > > > > > > exactly same device path that can be compared by strcpy. C= an we apply > > > > > > the series now? At least for x86 it provides useful paths a= nd work can > > > > > > be continue for other arches by interested parties. > > > > > >=20 > > > > > > -- > > > > > > Gleb. > > > > >=20 > > > > > Something I only now realized is that we commit > > > > > to never changing the paths for any architecture > > > > > that supports migration. > > > > >=20 > > > > No connection to migration whatsoever. > > >=20 > > > It just seems silly to use different paths for the same thing. > > >=20 > > > Besides the connection above, I was hoping to use these paths > > > for section names in migration. If we can't guarantee they are > > > stable, we'll have to roll our own, and if we do this, > > > with stability guarantees required for migration format, > > > maybe use it for other things like BIOS as well? > > >=20 > > It doesn't matter what do you use for migration purposes as long as= it > > depend on qdev hierarchy it will have problem when qdev hierarchy > > changes and if it doesn't you can't produce unique names reliably. > >=20 > > -- > > Gleb. >=20 > We can, it's not like OF is the only way to enumerate. > We could have driver-specific paths for example, exactly like we > currently have. I.e. paths don't have to be globally unique because > each driver has it's own domain. >=20 What we have currently is not even close to be correct. It happens to work since it is implemented only for one bus type and we can have only one of this bus right now. And of course it is not suitable for passing boot path to FW since it does not describe path to device. > It seems cleaner to use an existing spec but we must figure out > how it will not become a support issue. >=20 If you think you can figure out how to describe device path (or even give globally unique name to device) and to not depend on internal qdev implementation go ahead and do that for migration. -- Gleb.