Re: [Qemu-devel] [qemu-s390x] [RFC 14/15] s390-bios: Support booting from real dasd device

[Cornelia Huck <cohuck@redhat.com>]

On Wed, 18 Jul 2018 12:55:51 +0200
Halil Pasic <pasic@linux.ibm.com> wrote:

> On 07/18/2018 09:40 AM, Cornelia Huck wrote:
> > On Tue, 17 Jul 2018 22:43:27 +0200
> > David Hildenbrand <david@redhat.com> wrote:
> >   
> >> On 05.07.2018 19:25, Jason J. Herne wrote:  
> >   
> >>> +*****************************************
> >>> +***** How this all pertains to Qemu *****
> >>> +*****************************************
> >>> +
> >>> +In theory we should merely have to do the following to IPL/boot a guest
> >>> +operating system from a DASD device:
> >>> +
> >>> +1. Place a "Read IPL" ccw into memory location 0x0 with chaining bit on.
> >>> +2. Execute channel program at 0x0.
> >>> +3. LPSW 0x0.
> >>> +
> >>> +However, our emulation of the machine's channel program logic is missing one key
> >>> +feature that is required for this process to work: non-prefetch of ccw data.
> >>> +
> >>> +When we start a channel program we pass the channel subsystem parameters via an
> >>> +ORB (Operation Request Block). One of those parameters is a prefetch bit. If the
> >>> +bit is on then Qemu is allowed to read the entire channel program from guest
> >>> +memory before it starts executing it. This means that any channel commands that
> >>> +read additional channel commands will not work as expected because the newly
> >>> +read commands will only exist in guest memory and NOT within Qemu's channel
> >>> +subsystem memory. Qemu's channel subsystem's implementation currently requires
> >>> +this bit to be on for all channel programs. This is a problem because the IPL
> >>> +process consists of transferring control from the "Read IPL" ccw immediately to
> >>> +the IPL1 channel program that was read by "Read IPL".
> >>> ++  
> >>
> >> I have way too little insight into channel devices and how QEMU
> >> implements them, however I wonder what hinders us from implementing
> >> support for !prefetch in QEMU?
> >>
> >> What you tailored here seems impressive :) Just want to know what the
> >> technical background of this prefetch thingy in QEMU is.  
> > 
> > This has to do with how vfio-ccw processes and translates channel
> > programs.
> > 
> > Currently, we hand over the chain of channel commands to the kernel to
> > be translated (guest->host addresses) and to execute ssch on the real
> > subchannel. However, this requires sending the channel program over in
> > one go, which makes it impossible for the guest to modify an in-flight
> > channel program (there are tricks like putting a suspend marker on a
> > channel command and moving that marker forward as you go which make it
> > possible to know that a channel command has not yet been processed;
> > IIRC the lcs driver in Linux does that, or at least used to do that).
> > Our implementation currently does not accommodate that (the Linux dasd
> > driver does not use that feature). It's not impossible to implement it,
> > but it would require some effort (and I don't think anybody currently
> > has spare time for that...)  
> 
> 
> I disagree, IMHO we can not implement generic support for !prefetch in
> vfio-ccw with what we have at our disposal at the abstraction level we
> are currently working on. (If we were to abandon using IO instructions
> in the host but rely on lower level protocols like FC it may be possible,
> I don't want to make a statement about that).
> 
> The problem is the address translation. If the channel program reads
> something that is going to be used as a ccw by the same (e.g. CCW-IPL)
> the address within that ccw that was read is a guest-physical address.
> 
> We however need to translate the addresses in the guest ccws (and in the
> (m)idaw's) too before these get executed as a part of a host channel
> program. And because the address of the ccws themselves needs to be
> 31 bit addressable in host physical we actually copy the channel program
> form guest memory to suitable host memory in the vfio-ccw driver.
> 
> So to translate the new stuff we would actually have to stop the channel
> program and resubmit the rest (either by suspend+resume or by break
> chaining+ssch). The problem with that an execution of a channel program
> that is composed of four ccws A,B,C,D and an execution of a channel
> programs composed of ccws A,B immediately followed by and execution
> of a channel program composed of the ccws C,D is not the same. I.e. it
> is not generally safe to break a chain of ccws.

Exploiting suspending would have been my idea. Probably combined with a
new interface that fetches ccw-by-ccw.

But I don't think it makes sense to spend time thinking about this
right now.

> 
> If you still think it's possible to implement !prefetch, could you please
> explain your idea.
> 
> Regards,
> Halil
>