Xen on ARM DT based boot protocol

Xen on ARM DT based boot protocol

[Stefano Stabellini]

Hi Ian,
Andre was pointing out to me today that the current boot protocol has a
problem:


chosen {
              bootargs = "dom0_mem=128M console=com1,vga";
              xen,dom0-bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
};


because we are using bootargs for Xen arguments, the bootloader cannot
just pass the same two lines to Xen on ARM and to Linux on ARM.
It would make their life easier if we had instead:


chosen {
              xen,xen-bootarg = "dom0_mem=128M console=com1,vga";
              bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
};

Are we still on time to make this change on 4.3?

- Stefano

Re: Xen on ARM DT based boot protocol

[George Dunlap]

On 23/05/13 13:39, Stefano Stabellini wrote:
> Hi Ian,
> Andre was pointing out to me today that the current boot protocol has a
> problem:
>
>
> chosen {
>                bootargs = "dom0_mem=128M console=com1,vga";
>                xen,dom0-bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> };
>
>
> because we are using bootargs for Xen arguments, the bootloader cannot
> just pass the same two lines to Xen on ARM and to Linux on ARM.
> It would make their life easier if we had instead:
>
>
> chosen {
>                xen,xen-bootarg = "dom0_mem=128M console=com1,vga";
>                bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> };
>
> Are we still on time to make this change on 4.3?

Is this just an ARM-specific thing, or would it affect x86 as well?

The state of the ARM port when we do the release is up to you guys, so 
I'm fine with whatever.  ARM is still tech-preview anyway, right?  No 
promises (yet) that the interface won't change?  So whichever interface 
you chose, if it turns out to be the wrong one, you can change it for 
4.4 if necessary.

  -George

Re: Xen on ARM DT based boot protocol

[Ian Campbell]

On Thu, 2013-05-23 at 13:39 +0100, Stefano Stabellini wrote:
> Hi Ian,
> Andre was pointing out to me today that the current boot protocol has a
> problem:
> 
> 
> chosen {
>               bootargs = "dom0_mem=128M console=com1,vga";
>               xen,dom0-bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> };
> 
> 
> because we are using bootargs for Xen arguments, the bootloader cannot
> just pass the same two lines to Xen on ARM and to Linux on ARM.
> It would make their life easier if we had instead:
> 
> 
> chosen {
>               xen,xen-bootarg = "dom0_mem=128M console=com1,vga";
>               bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> };

You mean that instead of 
	if (xen /*or whatever*/) {
		add("bootargs", xencmd);
		add("xen,dom0-bootargs", lcmd);
	} else
		add("xen,xen-bootargs", xencmd);
		add("bootargs", lcmd);
	}
they can have:
	add("bootargs", lcmd);
	if (xen /*or whatever*/) {
		add("xen,xen-bootargs", xencmd);
	}
? Seems like the bootloader needs to know about Xen either way?

Regardless I suppose it's a harmless enough change, although it has
knock on affects on the dom0 dtb creation stuff etc so it might be
useful to see a proposed patch.

What about docs/misc/arm/device-tree/booting.txt though? Or more
generally the fact that the xen,dom0-bootargs "protocol" is really just
a quick hack and not the final thing we actually want bootloaders to
implement (we really want multiboot2 or something).

> Are we still on time to make this change on 4.3?

I think if we think it is definitely the right thing to do we can/should
just do it.

Ian.

Re: Xen on ARM DT based boot protocol

[David Vrabel]

On 23/05/13 13:39, Stefano Stabellini wrote:
> Hi Ian,
> Andre was pointing out to me today that the current boot protocol has a
> problem:
> 
> 
> chosen {
>               bootargs = "dom0_mem=128M console=com1,vga";
>               xen,dom0-bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> };
> 
> 
> because we are using bootargs for Xen arguments, the bootloader cannot
> just pass the same two lines to Xen on ARM and to Linux on ARM.

I think it is unlikely the command line arguments for Linux are the same
for bare metal and dom0.  Even in this example here you have a
Xen-specific option (earlyprintk=xenboot).

> It would make their life easier if we had instead:
> 
> 
> chosen {
>               xen,xen-bootarg = "dom0_mem=128M console=com1,vga";
>               bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> };

David

Re: Xen on ARM DT based boot protocol

[Stefano Stabellini]

On Thu, 23 May 2013, David Vrabel wrote:
> On 23/05/13 13:39, Stefano Stabellini wrote:
> > Hi Ian,
> > Andre was pointing out to me today that the current boot protocol has a
> > problem:
> > 
> > 
> > chosen {
> >               bootargs = "dom0_mem=128M console=com1,vga";
> >               xen,dom0-bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> > };
> > 
> > 
> > because we are using bootargs for Xen arguments, the bootloader cannot
> > just pass the same two lines to Xen on ARM and to Linux on ARM.
> 
> I think it is unlikely the command line arguments for Linux are the same
> for bare metal and dom0.  Even in this example here you have a
> Xen-specific option (earlyprintk=xenboot).
 
Actually that is not true, in fact in the example above the only
difference is earlyprintk=xenboot that is just a debug option.

Re: Xen on ARM DT based boot protocol

[Andre Przywara]

On 05/23/2013 11:04 PM, Stefano Stabellini wrote:

Hi,

> On Thu, 23 May 2013, David Vrabel wrote:
>> On 23/05/13 13:39, Stefano Stabellini wrote:
>>> Hi Ian,
>>> Andre was pointing out to me today that the current boot protocol has a
>>> problem:
>>>
>>>
>>> chosen {
>>>                bootargs = "dom0_mem=128M console=com1,vga";
>>>                xen,dom0-bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
>>> };
>>>
>>>
>>> because we are using bootargs for Xen arguments, the bootloader cannot
>>> just pass the same two lines to Xen on ARM and to Linux on ARM.
>>
>> I think it is unlikely the command line arguments for Linux are the same
>> for bare metal and dom0.  Even in this example here you have a
>> Xen-specific option (earlyprintk=xenboot).
>
> Actually that is not true, in fact in the example above the only
> difference is earlyprintk=xenboot that is just a debug option.

Interestingly Linux/ARM does not seem to expect a parameter to 
earlyprintk, the actual device to use is hardcoded in 
arch/arm/include/debug/<mach>.S and earlyprintk is just a switch. So 
earlyprintk=xen is broken anyway ;-)

But using bootargs for Dom0 could just be a fallback, so what about:

if xen,xen-bootargs is provided: use it for Xen, else use "";
if xen,dom0-bootargs is provided: use it for Dom0, else
	if bootargs is provided: use that for Dom0
write Dom0 cmdline into chosen/bootargs in Dom0's dtb

And is there a need for the "xen," prefix? I thought that this syntax is 
only needed for the compatible property, no?

So eventually there could be one dtb to rule them all:
chosen {
	xen,xen-bootargs = "...";
	xen,dom0-bootargs = "...";
	bootargs = "...";
}
My concern is not so much about the bootloader's understanding of the 
dtb, but more about the vendor provided one, which one would expect to 
be in flash ROM, for instance. We can provide only one here.

Will send a patch tomorrow.

Regards,
Andre.

Re: Xen on ARM DT based boot protocol

[Ian Campbell]

On Fri, 2013-05-24 at 00:27 +0200, Andre Przywara wrote:
> On 05/23/2013 11:04 PM, Stefano Stabellini wrote:
> 
> Hi,
> 
> > On Thu, 23 May 2013, David Vrabel wrote:
> >> On 23/05/13 13:39, Stefano Stabellini wrote:
> >>> Hi Ian,
> >>> Andre was pointing out to me today that the current boot protocol has a
> >>> problem:
> >>>
> >>>
> >>> chosen {
> >>>                bootargs = "dom0_mem=128M console=com1,vga";
> >>>                xen,dom0-bootargs = "earlyprintk=xenboot console=ttyAMA1 root=/dev/mmcblk0 debug rw init=/bin/bash";
> >>> };
> >>>
> >>>
> >>> because we are using bootargs for Xen arguments, the bootloader cannot
> >>> just pass the same two lines to Xen on ARM and to Linux on ARM.
> >>
> >> I think it is unlikely the command line arguments for Linux are the same
> >> for bare metal and dom0.  Even in this example here you have a
> >> Xen-specific option (earlyprintk=xenboot).
> >
> > Actually that is not true, in fact in the example above the only
> > difference is earlyprintk=xenboot that is just a debug option.
> 
> Interestingly Linux/ARM does not seem to expect a parameter to 
> earlyprintk, the actual device to use is hardcoded in 
> arch/arm/include/debug/<mach>.S and earlyprintk is just a switch. So 
> earlyprintk=xen is broken anyway ;-)
> 
> But using bootargs for Dom0 could just be a fallback, so what about:
> 
> if xen,xen-bootargs is provided: use it for Xen, else use "";
> if xen,dom0-bootargs is provided: use it for Dom0, else
> 	if bootargs is provided: use that for Dom0
> write Dom0 cmdline into chosen/bootargs in Dom0's dtb
> 
> And is there a need for the "xen," prefix? I thought that this syntax is 
> only needed for the compatible property, no?
> 
> So eventually there could be one dtb to rule them all:
> chosen {
> 	xen,xen-bootargs = "...";
> 	xen,dom0-bootargs = "...";
> 	bootargs = "...";
> }

This makes sense, I think, assuming we want to continue with this scheme
at all.

> My concern is not so much about the bootloader's understanding of the 
> dtb, but more about the vendor provided one, which one would expect to 
> be in flash ROM, for instance. We can provide only one here.

The chosen nodes, even if they were provided in the vendor DTB, are
generally overwritten by the bootloader I think.

> Will send a patch tomorrow.
> 
> Regards,
> Andre.
>