Re: xen-vbd interface (segment size expansion) - FreeBSD host have it.

Re: xen-vbd interface (segment size expansion) - FreeBSD host have it.

[Konrad Rzeszutek Wilk]

On Mon, Aug 20, 2012 at 06:56:02PM +0000, Justin Gibbs wrote:
> Ronghui,
> 
> It has been a while since I've actively worked on the blkif stuff.  ...
> 
> That said, I'm happy to help in whatever ways I can to help get the blkif interface sorted out.  I see several steps that should be taken:
> 
> 1) Fix the Xen and QEMU builds so that QEMU keeps its own copy of the Xen interface version.  This will allow interfaces to rev safely and in a coordinated fashion (i.e. update interface in Xen, then add support for the new interface in QEMU upstream).
> 
> 2) Complete support in drivers for the existing blkif interface so that you get maximum performance against systems using the existing multi-page extensions.
> 
> 3) Do something to allow for larger and more numerous requests.
> 
> On point 3, my approach was to try to perturb the existing protocol as little as possible in the hopes that other implementations could quickly be enhanced to support the feature.  However, there is a lot of ugliness in the existing blkif interface.  I can certainly understand the desires of some to just replace blkif with a blkif2.
> 
> What are your current plans in this area?  How can I be of assistance?

Hey Justin and Ronghui,

Note: I've expanded the email thread to include xen-devel.

I was wondering how the protocol you developed works when it comes
to migration to a host that does not support the new features?

Specifically how do deal with a guest which tries to replay in progress I/Os
that do not fit within the old-segment size (11)?

Re: xen-vbd interface (segment size expansion) - FreeBSD host have it.

[Duan, Ronghui]

Hi, I am back from a long holiday. Sorry for the delay replay.
> I was wondering how the protocol you developed works when it comes to
> migration to a host that does not support the new features?
>
For my part, when VM migrate to a host which do not support larger segments, frontend will fall back to use original protocol with old-segments 11.

> Specifically how do deal with a guest which tries to replay in progress I/Os
> that do not fit within the old-segment size (11)?
The in-progress bio which have larger segments > 11 will receive an io error. I do not find a better way to handle it yet.

> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> Sent: Tuesday, September 25, 2012 11:26 PM
> To: Justin Gibbs; xen-devel@lists.xensource.com
> Cc: Duan, Ronghui
> Subject: Re: xen-vbd interface (segment size expansion) - FreeBSD host have
> it.
> 
> On Mon, Aug 20, 2012 at 06:56:02PM +0000, Justin Gibbs wrote:
> > Ronghui,
> >
> > It has been a while since I've actively worked on the blkif stuff.  ...
> >
> > That said, I'm happy to help in whatever ways I can to help get the blkif
> interface sorted out.  I see several steps that should be taken:
> >
> > 1) Fix the Xen and QEMU builds so that QEMU keeps its own copy of the
> Xen interface version.  This will allow interfaces to rev safely and in a
> coordinated fashion (i.e. update interface in Xen, then add support for the
> new interface in QEMU upstream).
> >
> > 2) Complete support in drivers for the existing blkif interface so that you
> get maximum performance against systems using the existing multi-page
> extensions.
> >
> > 3) Do something to allow for larger and more numerous requests.
> >
> > On point 3, my approach was to try to perturb the existing protocol as little
> as possible in the hopes that other implementations could quickly be
> enhanced to support the feature.  However, there is a lot of ugliness in the
> existing blkif interface.  I can certainly understand the desires of some to just
> replace blkif with a blkif2.
> >
> > What are your current plans in this area?  How can I be of assistance?
> 
> Hey Justin and Ronghui,
> 
> Note: I've expanded the email thread to include xen-devel.
> 
> I was wondering how the protocol you developed works when it comes to
> migration to a host that does not support the new features?
> 
> Specifically how do deal with a guest which tries to replay in progress I/Os
> that do not fit within the old-segment size (11)?

Re: xen-vbd interface (segment size expansion) - FreeBSD host have it.

[Konrad Rzeszutek Wilk]

On Thu, Oct 18, 2012 at 01:59:36AM +0000, Duan, Ronghui wrote:
> Hi, I am back from a long holiday. Sorry for the delay replay.
> > I was wondering how the protocol you developed works when it comes to
> > migration to a host that does not support the new features?
> >
> For my part, when VM migrate to a host which do not support larger segments, frontend will fall back to use original protocol with old-segments 11.
> 
> > Specifically how do deal with a guest which tries to replay in progress I/Os
> > that do not fit within the old-segment size (11)?
> The in-progress bio which have larger segments > 11 will receive an io error. I do not find a better way to handle it yet.

Justin, how did you guys handle it in FreeBSD? Or is it dependent on
the backends always supporting these large segments?

thanks!
> 
> > -----Original Message-----
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > Sent: Tuesday, September 25, 2012 11:26 PM
> > To: Justin Gibbs; xen-devel@lists.xensource.com
> > Cc: Duan, Ronghui
> > Subject: Re: xen-vbd interface (segment size expansion) - FreeBSD host have
> > it.
> > 
> > On Mon, Aug 20, 2012 at 06:56:02PM +0000, Justin Gibbs wrote:
> > > Ronghui,
> > >
> > > It has been a while since I've actively worked on the blkif stuff.  ...
> > >
> > > That said, I'm happy to help in whatever ways I can to help get the blkif
> > interface sorted out.  I see several steps that should be taken:
> > >
> > > 1) Fix the Xen and QEMU builds so that QEMU keeps its own copy of the
> > Xen interface version.  This will allow interfaces to rev safely and in a
> > coordinated fashion (i.e. update interface in Xen, then add support for the
> > new interface in QEMU upstream).
> > >
> > > 2) Complete support in drivers for the existing blkif interface so that you
> > get maximum performance against systems using the existing multi-page
> > extensions.
> > >
> > > 3) Do something to allow for larger and more numerous requests.
> > >
> > > On point 3, my approach was to try to perturb the existing protocol as little
> > as possible in the hopes that other implementations could quickly be
> > enhanced to support the feature.  However, there is a lot of ugliness in the
> > existing blkif interface.  I can certainly understand the desires of some to just
> > replace blkif with a blkif2.
> > >
> > > What are your current plans in this area?  How can I be of assistance?
> > 
> > Hey Justin and Ronghui,
> > 
> > Note: I've expanded the email thread to include xen-devel.
> > 
> > I was wondering how the protocol you developed works when it comes to
> > migration to a host that does not support the new features?
> > 
> > Specifically how do deal with a guest which tries to replay in progress I/Os
> > that do not fit within the old-segment size (11)?

Re: xen-vbd interface (segment size expansion) - FreeBSD host have it.

[Justin Gibbs]

On Oct 18, 2012, at 8:50 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
 wrote:

> On Thu, Oct 18, 2012 at 01:59:36AM +0000, Duan, Ronghui wrote:
>> Hi, I am back from a long holiday. Sorry for the delay replay.
>>> I was wondering how the protocol you developed works when it comes to
>>> migration to a host that does not support the new features?
>>> 
>> For my part, when VM migrate to a host which do not support larger segments, frontend will fall back to use original protocol with old-segments 11.
>> 
>>> Specifically how do deal with a guest which tries to replay in progress I/Os
>>> that do not fit within the old-segment size (11)?
>> The in-progress bio which have larger segments > 11 will receive an io error. I do not find a better way to handle it yet.
> 
> Justin, how did you guys handle it in FreeBSD? Or is it dependent on
> the backends always supporting these large segments?

The current API forces I/Os larger than 44k to get chopped up in
order to transit blkif.  The ability to negotiate a larger blkif
request size just means that you can sometimes reduce the amount
of "scatter-gather" work performed by the driver.  You must still
deal with the fact that a logical I/O submitted to blkfront may
need to be broken up, or may not completely fit within the size of
the negotiated ring.  Upon reconnect, the newly negotiated parameters
are in effect and reissued I/Os are subject to the rules that apply
to that connection.

I'd have to go review the FreeBSD blkfront driver to see if it
handles correctly all of the issues that arise with a ring that
shrinks (e.g. it may assume that an I/O will fit within an empty
ring), but there is certainly no technical reason that these issues
can't be addressed.

--
Justin

Re: xen-vbd interface (segment size expansion) - FreeBSD host have it.

[Konrad Rzeszutek Wilk]

On Thu, Oct 18, 2012 at 03:34:27PM +0000, Justin Gibbs wrote:
> On Oct 18, 2012, at 8:50 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>  wrote:
> 
> > On Thu, Oct 18, 2012 at 01:59:36AM +0000, Duan, Ronghui wrote:
> >> Hi, I am back from a long holiday. Sorry for the delay replay.
> >>> I was wondering how the protocol you developed works when it comes to
> >>> migration to a host that does not support the new features?
> >>> 
> >> For my part, when VM migrate to a host which do not support larger segments, frontend will fall back to use original protocol with old-segments 11.
> >> 
> >>> Specifically how do deal with a guest which tries to replay in progress I/Os
> >>> that do not fit within the old-segment size (11)?
> >> The in-progress bio which have larger segments > 11 will receive an io error. I do not find a better way to handle it yet.
> > 
> > Justin, how did you guys handle it in FreeBSD? Or is it dependent on
> > the backends always supporting these large segments?
> 
> The current API forces I/Os larger than 44k to get chopped up in
> order to transit blkif.  The ability to negotiate a larger blkif
> request size just means that you can sometimes reduce the amount
> of "scatter-gather" work performed by the driver.  You must still
> deal with the fact that a logical I/O submitted to blkfront may
> need to be broken up, or may not completely fit within the size of
> the negotiated ring.  Upon reconnect, the newly negotiated parameters
> are in effect and reissued I/Os are subject to the rules that apply
> to that connection.
> I'd have to go review the FreeBSD blkfront driver to see if it
> handles correctly all of the issues that arise with a ring that
> shrinks (e.g. it may assume that an I/O will fit within an empty
> ring), but there is certainly no technical reason that these issues
> can't be addressed.

OK, so the one issue I keep on circling back to is there an benefit
of doing a seperate ring for segments - as opposed of re-using the existing
ring and just shrinking/expanding the number of segments?

I do like re-using the ring and just shrinking/expanding it b/c it
ends up being all neatly tucked inside the existing ring.


> 
> --
> Justin