First release of GPL PV drivers for Windows

First release of GPL PV drivers for Windows

[James Harper]

This is the first release of my GPL'd PV drivers for Windows. Only block
device drivers so far. Also only really useful to anyone who knows
something about windows driver development.

My test environment is Xen 3.1.0 and Windows 2003 sp2. YMMV.

You can download the source zip file at
http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip. No binary release
yet.

Please do not in any way consider it ready for production use.
Definitely do not use them on a production DomU. While I have never seen
them cause a problem with the hypervisor or Dom0, I would also be very
wary of using them on a production physical machine too. There is still
at least one race condition in there that hangs Windows after a fairly
short time (eg after a few minutes of doing a copy of
C:\Windows\System32).

I haven't got to probing device sizes yet, it only supports a single
device of 1048756 * 512 bytes (eg ~500mb), so make your virtual device
this size. The drivers currently do not enumerate existing block
devices, so if you want to use them you'll have to attach a device at
runtime after installing the PV drivers, eg:

xm block-attach virtdemo phy:/dev/vg00/virt-virtdemo-1 /dev/hdc w

Then your block device will show up under Windows.

To build, you'll need the Windows WDF 6000 (not the later one which is
Longhorn only I think) which you can get from Microsoft. It's not small.
You'll also need the xen/common/public folder from the xen source tree,
which needs to be in common/include/public in the Windows PV drivers
source tree. I've been doing checked build's and using Dbgview from
SysInternals for testing. Run 'build' from both the xenpci and xenvbd
folders, then copy everything from target up to your test server.

I've probably omitted some vitally important step from the above, so
please let me know if you need any assistance compiling.

To install the drivers, find the Xen PCI driver (probably an Unknown
Device in Device Manager) and install from the directory where you put
the compiled drivers. After the PCI driver installs, everything under
/device will be enumerated (eg vbd, vif, etc), again as 'Unknown
Device'. There is only currently a driver for vbd. Install that the same
way.

Once you've done that, attach the block device with the above 'xm
block-attach' command, and a disk should show up. Partition and format
the disk with the Disk Management thing in Computer Management under
windows. If you try and add a second disk it will blow up.

The source code really needs a tidy up. The Windows DDK stuff has it's
own coding style which I've imitated in places, and then xen has it's
own too, so this code is mostly one, the other, both, or neither. I'll
get to that as soon as I get the major bugs out.

Known limitations (eg things that prevent it actually being useful):
. A race condition or something like that is causing a hang.
. Only supports a single block device
. Doesn't unload properly
. Sometimes doesn't attach properly (haven't looked into this, but it
sometimes doesn't notice me attaching block devices...)
. Block device must be exactly 1048756 * 512 bytes in size
. No power management
. Almost certainly won't support migration or suspend/resume.
. Does not automatically enumerate existing block devices
. Does not support booting from a PV block device (due to the above, and
probably for other reasons)
. Probably really poor performance - no optimisation done at all yet. It
crashes too soon to be able to run any testing on at this point.
. I'm not sure that I've implemented the block device in a way that
Microsoft would approve of (eg maybe not guaranteed to work under newer
windows operating systems)... maybe it should be a scsi miniport driver,
except those have other requirements that don't mesh well with xen.
. Probably lots of other things too.

Once I've sorted out a few of the above issues I'll put together a
binary release.

Future plans:
. Virtual IRQ's... not sure if I can do this under Windows... everything
is just a port event handler currently.
. Better resource allocation to drivers (eg the PCI enumerator should be
able to hand out IRQ's and address space to the lower drivers, which
would make the scsi miniport driver do-able).
. proper shutdown (need a user space driver to do this)
. network adapters (vif)
. virtual scsi (eg a front end for the scsi passthrough stuff)
. balloon drivers (this should actually be pretty easy)

Enjoy!

James

PS. Does anyone have any objections to discussing these drivers on the
xen-devel mailing until I can set up a windows specific list somewhere?

Re: First release of GPL PV drivers for Windows

[Atsushi SAKAI]

Hi,

Where is the exact address?
http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip.

Following error is appeared in my IE (same as Firefox)

HTTP Error 404 - File or directory not found.
Internet Information Services (IIS)

Thanks
Atsushi SAKAI


"James Harper" <james.harper@bendigoit.com.au> wrote:
 
> You can download the source zip file at
> http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip. No binary release
> yet.
> 

RE: First release of GPL PV drivers for Windows

[James Harper]

Looking at the logs on the webserver, someone is trying to download it
with the '.' appended after the zip file. Perhaps that is you?

Try http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip

If that doesn't work, I will email it to you privately.

Thanks

James

> -----Original Message-----
> From: Atsushi SAKAI [mailto:sakaia@jp.fujitsu.com]
> Sent: Wednesday, 31 October 2007 17:01
> To: James Harper
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] First release of GPL PV drivers for Windows
> 
> Hi,
> 
> Where is the exact address?
> http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip.
> 
> Following error is appeared in my IE (same as Firefox)
> 
> HTTP Error 404 - File or directory not found.
> Internet Information Services (IIS)
> 
> Thanks
> Atsushi SAKAI
> 
> 
> "James Harper" <james.harper@bendigoit.com.au> wrote:
> 
> > You can download the source zip file at
> > http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip. No binary release
> > yet.
> >
> 

Re: First release of GPL PV drivers for Windows

[Atsushi SAKAI]

Hi, James

Thanks it works!

Thanks
Atsushi SAKAI

"James Harper" <james.harper@bendigoit.com.au> wrote:

> Looking at the logs on the webserver, someone is trying to download it
> with the '.' appended after the zip file. Perhaps that is you?
> 
> Try http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip
> 
> If that doesn't work, I will email it to you privately.
> 
> Thanks
> 
> James
> 
> > -----Original Message-----
> > From: Atsushi SAKAI [mailto:sakaia@jp.fujitsu.com]
> > Sent: Wednesday, 31 October 2007 17:01
> > To: James Harper
> > Cc: xen-devel@lists.xensource.com
> > Subject: Re: [Xen-devel] First release of GPL PV drivers for Windows
> > 
> > Hi,
> > 
> > Where is the exact address?
> > http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip.
> > 
> > Following error is appeared in my IE (same as Firefox)
> > 
> > HTTP Error 404 - File or directory not found.
> > Internet Information Services (IIS)
> > 
> > Thanks
> > Atsushi SAKAI
> > 
> > 
> > "James Harper" <james.harper@bendigoit.com.au> wrote:
> > 
> > > You can download the source zip file at
> > > http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip. No binary release
> > > yet.
> > >
> > 
> 

Re: First release of GPL PV drivers for Windows

[Andy Grover]

James Harper wrote:
> This is the first release of my GPL'd PV drivers for Windows. Only block
> device drivers so far. Also only really useful to anyone who knows
> something about windows driver development.

Great to see this, James!

> My test environment is Xen 3.1.0 and Windows 2003 sp2. YMMV.
> 
> You can download the source zip file at
> http://www.meadowcourt.org/WindowsXenPV-0.0.1.zip. No binary release
> yet.

Yeah I'd think a binary release could wait until everything is stable.

> To build, you'll need the Windows WDF 6000 (not the later one which is
> Longhorn only I think) which you can get from Microsoft. It's not small.
> You'll also need the xen/common/public folder from the xen source tree,
> which needs to be in common/include/public in the Windows PV drivers
> source tree. I've been doing checked build's and using Dbgview from
> SysInternals for testing. Run 'build' from both the xenpci and xenvbd
> folders, then copy everything from target up to your test server.
> 
> I've probably omitted some vitally important step from the above, so
> please let me know if you need any assistance compiling.

I don't see a xen/common/public folder in the xen-3.1-testing hg tree.
Did you mean xen/include/public?

BTW I have remote debugging with WinDBG working, which really helps --
follow the directions here (and my comment at the bottom)
http://www.evilbitz.com/2006/12/13/howto-debugging-a-remote-windows-hvm-under-xen-using-windbg/

> The source code really needs a tidy up. The Windows DDK stuff has it's
> own coding style which I've imitated in places, and then xen has it's
> own too, so this code is mostly one, the other, both, or neither. I'll
> get to that as soon as I get the major bugs out.

Yeah I don't know if there is really a good solution to this...

I'd like to start submitting patches very soon (as soon as I get it
working :) so it would be very nice to have it under SCM somewhere as
part of a hg clone of xen-unstable? Or in xen-unstable itself. That
would resolve any include path issues, I think.

Also, now that there are alpha GPLed Win PV drivers, this is a good
chance for those who already have Win PV drivers they can't GPL
(XenSource, Virtual Iron and Novell, due to basing their versions on the
DDK sample code) to help out with improvements to this codebase.

Regards -- Andy

RE: Re: First release of GPL PV drivers for Windows

[James Harper]

> I don't see a xen/common/public folder in the xen-3.1-testing hg tree.
> Did you mean xen/include/public?

Yes.

> BTW I have remote debugging with WinDBG working, which really helps --
> follow the directions here (and my comment at the bottom)
>
http://www.evilbitz.com/2006/12/13/howto-debugging-a-remote-windows-hvm-
> under-xen-using-windbg/

Cool. I'll check that out. I've done a big shuffle around of the block
driver code to fix a few problems, and I'm still getting hangs, but I
think dbgview from sysinternals is dropping kernel messages so I think
I'm missing out on where the action is, as the logs I am getting don't
tell me that there is anything out of whack. Of course there could be a
simple off-by-one error or something somewhere.

> 
> > The source code really needs a tidy up. The Windows DDK stuff has
it's
> > own coding style which I've imitated in places, and then xen has
it's
> > own too, so this code is mostly one, the other, both, or neither.
I'll
> > get to that as soon as I get the major bugs out.
> 
> Yeah I don't know if there is really a good solution to this...
> 
> I'd like to start submitting patches very soon (as soon as I get it
> working :) so it would be very nice to have it under SCM somewhere as
> part of a hg clone of xen-unstable? Or in xen-unstable itself. That
> would resolve any include path issues, I think.

Yes. I'm thinking of hosting my own in the short term if I can't find
another simple solution.

Thanks

James

Re: Re: First release of GPL PV drivers for Windows

[Grabber]

[-- Attachment #1.1: Type: text/plain, Size: 1936 bytes --]

James Harper,

What you think about create a site to your project?


Regards,
Luiz Vitor Martinez Cardoso aka Grabber.

On Nov 1, 2007 10:16 PM, James Harper <james.harper@bendigoit.com.au> wrote:

>
> > I don't see a xen/common/public folder in the xen-3.1-testing hg tree.
> > Did you mean xen/include/public?
>
> Yes.
>
> > BTW I have remote debugging with WinDBG working, which really helps --
> > follow the directions here (and my comment at the bottom)
> >
> http://www.evilbitz.com/2006/12/13/howto-debugging-a-remote-windows-hvm-
> > under-xen-using-windbg/
>
> Cool. I'll check that out. I've done a big shuffle around of the block
> driver code to fix a few problems, and I'm still getting hangs, but I
> think dbgview from sysinternals is dropping kernel messages so I think
> I'm missing out on where the action is, as the logs I am getting don't
> tell me that there is anything out of whack. Of course there could be a
> simple off-by-one error or something somewhere.
>
> >
> > > The source code really needs a tidy up. The Windows DDK stuff has
> it's
> > > own coding style which I've imitated in places, and then xen has
> it's
> > > own too, so this code is mostly one, the other, both, or neither.
> I'll
> > > get to that as soon as I get the major bugs out.
> >
> > Yeah I don't know if there is really a good solution to this...
> >
> > I'd like to start submitting patches very soon (as soon as I get it
> > working :) so it would be very nice to have it under SCM somewhere as
> > part of a hg clone of xen-unstable? Or in xen-unstable itself. That
> > would resolve any include path issues, I think.
>
> Yes. I'm thinking of hosting my own in the short term if I can't find
> another simple solution.
>
> Thanks
>
> James
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>



-- 
Atenciosamente,
Luiz Vitor.

[-- Attachment #1.2: Type: text/html, Size: 2784 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Re: First release of GPL PV drivers for Windows

[James Harper]

> BTW I have remote debugging with WinDBG working, which really helps --
> follow the directions here (and my comment at the bottom)
>
http://www.evilbitz.com/2006/12/13/howto-debugging-a-remote-windows-hvm-
> under-xen-using-windbg/
> 

That appears to work okay... the only thing is it's really really slow
(I'm dumping out >50mb of dbgprint logs). Does the baud rate you specify
actually make any difference?

Also, I can't seem to use the kdbgctrl command to increase the buffer
for the dbgprint messages. It just tells me access denied all the
time...

Thanks

James

Re: First release of GPL PV drivers for Windows

[Andy Grover]

James Harper wrote:
>> BTW I have remote debugging with WinDBG working, which really helps --
>> follow the directions here (and my comment at the bottom)
>>
> http://www.evilbitz.com/2006/12/13/howto-debugging-a-remote-windows-hvm-
>> under-xen-using-windbg/
> That appears to work okay... the only thing is it's really really slow
> (I'm dumping out >50mb of dbgprint logs). Does the baud rate you specify
> actually make any difference?

Not afaik. Yeah it's not super-fast, for some reason. It's been fast
enough for me, I don't know where the bottleneck is. It can be useful
for some things, but obviously there are cases where debugprints are better.

(One cool feature is ".kdfiles" -- you can have it load the driver
you're working on from your dev machine. This removes the need to reboot
simply to copy a new version of the driver over.)

> Also, I can't seem to use the kdbgctrl command to increase the buffer
> for the dbgprint messages. It just tells me access denied all the
> time...

Don't know, haven't used.

Regards -- Andy