Driver Architecture Changes

Driver Architecture Changes

[Ewan Mellor]

I have committed a moderately large change to the device driver infrastructure
in Xen.  This change takes large pieces of code from Rusty Russell's skeleton
driver, but with this work refactored so that it is available to all
Xenbus-based split drivers.  This change and it's relation to Rusty's work is
discussed in an email that will follow on immediately.

My intention is that we solidify the driver layer around this architecture as
we head for a Xen 3.0 release.  A number of subtle bugs have been addressed by
this change, and the code is now shorter and simpler, which should make it
easier for new drivers to be written too.

I have ported the net and block drivers to this, but not blktap, tpmfront, or
tpmback.  I shall be making a first pass over those three drivers very
shortly, but I would appreciate it if those people who are interested in those
drivers could help with testing and tweaking, as obviously I don't understand
them quite as well as the major drivers.

Cheers,

Ewan.

RE: Driver Architecture Changes

[Magenheimer, Dan (HP Labs Fort Collins)]

Hmmm... It appears that this has completely broken the
block drivers for ia64.   Since it is such a massive change,
I'd imagine it will be difficult to track down the problem(s).
I had rather hoped that we were done with this kind of
massive change pre-3.0.

With Xen/ia64 block devices working in the main tree, I was
going to ask if it might make sense for some basic Xen/ia64
tests to be part of the pre-commit regression test, but alas
I guess I am too late.

Ewan, would you be willing to work with Kevin to get your
re-architected driver infrastructure working on Xen/ia64?
Hopefully it will be easier than I fear but utilizing your direct
expertise with the new architecture, it will be much easier
than Kevin figuring it out from scratch.

Until this is all working again, it won't make much sense to
merge xen-ia64-unstable into xen-unstable, and there are
some fairly critical bug fixes already waiting.

Thanks,
Dan

> From: Ewan Mellor <ewan@xensource.com>
> Subject: [Xen-devel] Driver Architecture Changes
> To: Xen-devel <xen-devel@lists.xensource.com>
> Message-ID: <20051115001511.GA28109@leeni.uk.xensource.com>
> Content-Type: text/plain; charset=us-ascii
> 
> I have committed a moderately large change to the device 
> driver infrastructure
> in Xen.  This change takes large pieces of code from Rusty 
> Russell's skeleton
> driver, but with this work refactored so that it is available to all
> Xenbus-based split drivers.  This change and it's relation to 
> Rusty's work is
> discussed in an email that will follow on immediately.
> 
> My intention is that we solidify the driver layer around this 
> architecture as
> we head for a Xen 3.0 release.  A number of subtle bugs have 
> been addressed by
> this change, and the code is now shorter and simpler, which 
> should make it
> easier for new drivers to be written too.
> 
> I have ported the net and block drivers to this, but not 
> blktap, tpmfront, or
> tpmback.  I shall be making a first pass over those three drivers very
> shortly, but I would appreciate it if those people who are 
> interested in those
> drivers could help with testing and tweaking, as obviously I 
> don't understand
> them quite as well as the major drivers.

Re: Driver Architecture Changes

[Ewan Mellor]

On Tue, Nov 15, 2005 at 07:25:10AM -0800, Magenheimer, Dan (HP Labs Fort Collins) wrote:

> Ewan, would you be willing to work with Kevin to get your
> re-architected driver infrastructure working on Xen/ia64?

Certainly.  Let me know what you need.

Ewan.

RE: Driver Architecture Changes

[Magenheimer, Dan (HP Labs Fort Collins)]

Thanks!  But false alarm... when I recloned, rebuilt, rebooted and
tried again, it seems to work!  I think I had some inconsistencies
because I had some hg problems yesterday.

That said, it still might be a good idea to bring up the ia64
boxes that arrived at Cambridge last month and incorporate them
into some regression testing.

Sorry for any inconvenience!
Dan

> -----Original Message-----
> From: Ewan Mellor [mailto:ewan@xensource.com] 
> Sent: Tuesday, November 15, 2005 8:34 AM
> To: Magenheimer, Dan (HP Labs Fort Collins)
> Cc: Tian, Kevin; xen-devel@lists.xensource.com; 
> xen-ia64-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Driver Architecture Changes
> 
> On Tue, Nov 15, 2005 at 07:25:10AM -0800, Magenheimer, Dan 
> (HP Labs Fort Collins) wrote:
> 
> > Ewan, would you be willing to work with Kevin to get your
> > re-architected driver infrastructure working on Xen/ia64?
> 
> Certainly.  Let me know what you need.
> 
> Ewan.
> 

Re: Driver Architecture Changes

[Stefan Berger]

[-- Attachment #1.1: Type: text/plain, Size: 791 bytes --]

Hello Ewan,

  I am following the changes with the TPM driver pair. I was wondering 
what causes the 'add' action to be invoked on the hotplug scripts and why 
would one not see a remove action if a previous 'add' action was 
encountered and it had finished successfully.

Regards,
   Stefan


xen-devel-bounces@lists.xensource.com wrote on 11/15/2005 10:33:35 AM:

> On Tue, Nov 15, 2005 at 07:25:10AM -0800, Magenheimer, Dan (HP Labs 
> Fort Collins) wrote:
> 
> > Ewan, would you be willing to work with Kevin to get your
> > re-architected driver infrastructure working on Xen/ia64?
> 
> Certainly.  Let me know what you need.
> 
> Ewan.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

[-- Attachment #1.2: Type: text/html, Size: 1138 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Driver Architecture Changes

[Ewan Mellor]

On Wed, Nov 16, 2005 at 04:40:34PM -0500, Stefan Berger wrote:

> Hello Ewan,
> 
>   I am following the changes with the TPM driver pair. I was wondering
> what causes the 'add' action to be invoked on the hotplug scripts and why
> would one not see a remove action if a previous 'add' action was
> encountered and it had finished successfully.

The add action occurs when the device is registered with the kernel
(xenbus_probe.c:xenbus_probe_node), and you should get a remove when the
device is no longer referenced.  Once the frontend driver has flushed through
and closed down, it switches to state Closed, which is seen in the backend
driver's otherend_changed function (blkback/xenbus.c:frontend_changed for
example).  This then calls device_unregister, which ought to be enough to
ensure that no-one is referencing the device, and so the kernel will clean up
the sysfs entries etc and you will get a remove hotplug event.

If that's not happening, then there may be a bug which needs further
investigation.

HTH,

Ewan.

Re: Driver Architecture Changes

[Stefan Berger]

[-- Attachment #1.1: Type: text/plain, Size: 1460 bytes --]

Ewan Mellor <ewan@xensource.com> wrote on 11/17/2005 12:26:45 PM:

> On Wed, Nov 16, 2005 at 04:40:34PM -0500, Stefan Berger wrote:
> 
> > Hello Ewan,
> > 
> >   I am following the changes with the TPM driver pair. I was wondering
> > what causes the 'add' action to be invoked on the hotplug scripts and 
why
> > would one not see a remove action if a previous 'add' action was
> > encountered and it had finished successfully.
> 
> The add action occurs when the device is registered with the kernel
> (xenbus_probe.c:xenbus_probe_node), and you should get a remove when the
> device is no longer referenced.  Once the frontend driver has flushed 
through
> and closed down, it switches to state Closed, which is seen in the 
backend
> driver's otherend_changed function (blkback/xenbus.c:frontend_changed 
for
> example).  This then calls device_unregister, which ought to be enough 
to
> ensure that no-one is referencing the device, and so the kernel will 
clean up
> the sysfs entries etc and you will get a remove hotplug event.

The mistake I had made was not to switch to the Closed state. So now I am 
seeing the remove as well. Thanks.

Another question: I see that the .suspend function is not used in the 
frontends anymore, but the structure still has that member. Is the suspend 
member about to be removed?

> 
> If that's not happening, then there may be a bug which needs further
> investigation.

Was my bug.

  Stefan

> 
> HTH,
> 
> Ewan.

[-- Attachment #1.2: Type: text/html, Size: 1919 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Driver Architecture Changes

[Ewan Mellor]

On Thu, Nov 17, 2005 at 02:51:38PM -0500, Stefan Berger wrote:

> Another question: I see that the .suspend function is not used in the
> frontends anymore, but the structure still has that member. Is the suspend
> member about to be removed?

The block and net drivers don't need this at the moment, but it seems like
some driver may need it some time, so no, we won't be removing it.  It does no
harm to support it, and it may be useful to someone.

Ewan.

Re: Driver Architecture Changes

[Muli Ben-Yehuda]

On Thu, Nov 17, 2005 at 09:37:37PM +0000, Ewan Mellor wrote:
> On Thu, Nov 17, 2005 at 02:51:38PM -0500, Stefan Berger wrote:
> 
> > Another question: I see that the .suspend function is not used in the
> > frontends anymore, but the structure still has that member. Is the suspend
> > member about to be removed?
> 
> The block and net drivers don't need this at the moment, but it seems like
> some driver may need it some time, so no, we won't be removing it.  It does no
> harm to support it, and it may be useful to someone.

Is there any driver in the tree that needs it? if not, the Linux way
would be to remove it until such a driver makes it into the tree and
then add it. It's a very effective policy for keeping bloat at bay.

Cheers,
Muli
-- 
Muli Ben-Yehuda
http://www.mulix.org | http://mulix.livejournal.com/

Re: Driver Architecture Changes

[Murillo Fernandes Bernardes]

On Thursday 17 November 2005 15:26, Ewan Mellor wrote:
> On Wed, Nov 16, 2005 at 04:40:34PM -0500, Stefan Berger wrote:
> > Hello Ewan,
> >
> >   I am following the changes with the TPM driver pair. I was wondering
> > what causes the 'add' action to be invoked on the hotplug scripts and why
> > would one not see a remove action if a previous 'add' action was
> > encountered and it had finished successfully.
>

I am looking network drivers. Hotplug scripts expects the offline action too, 
but that does not occur. Is that correct? 

All other actions (add, online, remove) are ok.

If offline is not needed anymore I think we should remove that from hotplug 
scripts. I can do that.

-- 
Murillo Fernandes Bernardes

Re: Driver Architecture Changes

[Ewan Mellor]

On Fri, Nov 18, 2005 at 02:47:59PM -0200, Murillo Fernandes Bernardes wrote:

> On Thursday 17 November 2005 15:26, Ewan Mellor wrote:
> > On Wed, Nov 16, 2005 at 04:40:34PM -0500, Stefan Berger wrote:
> > > Hello Ewan,
> > >
> > >   I am following the changes with the TPM driver pair. I was wondering
> > > what causes the 'add' action to be invoked on the hotplug scripts and why
> > > would one not see a remove action if a previous 'add' action was
> > > encountered and it had finished successfully.
> >
> 
> I am looking network drivers. Hotplug scripts expects the offline action too, 
> but that does not occur. Is that correct? 

No, that's not correct.  If we're losing the offline event, then that's a bug.

Ewan.

Re: Driver Architecture Changes

[Stefan Berger]

[-- Attachment #1.1: Type: text/plain, Size: 998 bytes --]

xen-devel-bounces@lists.xensource.com wrote on 11/18/2005 03:10:55 AM:

> On Thu, Nov 17, 2005 at 09:37:37PM +0000, Ewan Mellor wrote:
> > On Thu, Nov 17, 2005 at 02:51:38PM -0500, Stefan Berger wrote:
> > 
> > > Another question: I see that the .suspend function is not used in 
the
> > > frontends anymore, but the structure still has that member. Is the 
suspend
> > > member about to be removed?
> > 
> > The block and net drivers don't need this at the moment, but it seems 
like
> > some driver may need it some time, so no, we won't be removing it.It 
does no
> > harm to support it, and it may be useful to someone.
> 
> Is there any driver in the tree that needs it? if not, the Linux way
> would be to remove it until such a driver makes it into the tree and
> then add it. It's a very effective policy for keeping bloat at bay.

The TPM driver does need it. I was wondering about the future of the 
method since the network and block device drivers had used it before.

Cheers,
  Stefan

[-- Attachment #1.2: Type: text/html, Size: 1281 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel