XEN CD --> USB stick

XEN CD --> USB stick

[Eivind]

Hello all, 
(cc'd Jared, the author of XenCD)

I have moreover a general question regarding the construction of XenCD.

As you probably know, XEN does not yet support USB. Although, with a
new BIOS I am able to boot the XEN cd from my usb stick using the same
image - a pure cp -Ra xencd/* /mnt/usb/.

Now, since I cannot mount a usb stick to the /dev/sda* it becomes more
complicated to start other example domains as available on the CD
(there is no /boot/<kernel>).

Reverse engineering the image by taking the user space ram disk, unzip
it and mount it through a loopback device gives me access to the
filesystem which I can modify. However, the disk is too small so I
need to use 'dd if=/dev/zero of=...' to make another image which I
copy the entire existing FS to, compress it and then try to boot from
it.

This does not work - kernel panic. Could someone please get me some
instruction on how this work?

Thanks in advance
- Eivind


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: XEN CD --> USB stick

[Mark Williamson]

> As you probably know, XEN does not yet support USB. Although, with a
> new BIOS I am able to boot the XEN cd from my usb stick using the same
> image - a pure cp -Ra xencd/* /mnt/usb/.

It does support USB - you need to select it in your kernel config, as for 
vanilla Linux.

Cheers,
Mark

> Now, since I cannot mount a usb stick to the /dev/sda* it becomes more
> complicated to start other example domains as available on the CD
> (there is no /boot/<kernel>).
>
> Reverse engineering the image by taking the user space ram disk, unzip
> it and mount it through a loopback device gives me access to the
> filesystem which I can modify. However, the disk is too small so I
> need to use 'dd if=/dev/zero of=...' to make another image which I
> copy the entire existing FS to, compress it and then try to boot from
> it.
>
> This does not work - kernel panic. Could someone please get me some
> instruction on how this work?
>
> Thanks in advance
> - Eivind
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

RE: XEN CD --> USB stick

[Ian Pratt]

> > As you probably know, XEN does not yet support USB. Although, with a
> > new BIOS I am able to boot the XEN cd from my usb stick 
> using the same
> > image - a pure cp -Ra xencd/* /mnt/usb/.
> 
> It does support USB - you need to select it in your kernel 
> config, as for 
> vanilla Linux.

Do you regard the USB stuff as ready for prime time? Should we be
enabling it in the default config?

Ian


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: XEN CD --> USB stick

[Eivind]

 Well, I finally got the USB working!

But I am still stuck on creating a duplicate filesystem image that
comes with XenCD, i.e. /boot/xen0-init. From what I have experienced,
it is a multiboot module image.

After hours of googling I cannot figure out how I would make an image
multiboot... (a replica of the /boot/xen0-init).

Any ideas is greatly appreciated.

Best, 
- Eivind


On Mon, 28 Feb 2005 21:07:41 -0000, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> 
> > > As you probably know, XEN does not yet support USB. Although, with a
> > > new BIOS I am able to boot the XEN cd from my usb stick
> > using the same
> > > image - a pure cp -Ra xencd/* /mnt/usb/.
> >
> > It does support USB - you need to select it in your kernel
> > config, as for
> > vanilla Linux.
> 
> Do you regard the USB stuff as ready for prime time? Should we be
> enabling it in the default config?
> 
> Ian
>


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: XEN CD --> USB stick

[Mark Williamson]

> Do you regard the USB stuff as ready for prime time? Should we be
> enabling it in the default config?

The Linux USB stack seems to work OK.  I never convinced myself that the 
default Linux HCD drivers really understood pseudophysical memory but it 
doesn't seem to have bitten me in practice *touch wood*

usbfront / usbback need more development to support all sorts of random 
devices, so although it's stable and functional for me right now I don't 
think this should be enabled by default.

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

RE: XEN CD --> USB stick

[Ian Pratt]

> The Linux USB stack seems to work OK.  I never convinced 
> myself that the 
> default Linux HCD drivers really understood pseudophysical 
> memory but it 
> doesn't seem to have bitten me in practice *touch wood*

All the pseudophysical memory issues shoud be taken care of within the
Linux macros. It's only if the driver is buggy that its an issue.
 
Please could you checkin a patch that enables what you believe works. 

> usbfront / usbback need more development to support all sorts 
> of random 
> devices, so although it's stable and functional for me right 
> now I don't 
> think this should be enabled by default.

OK, thanks, that's good to know. 
Has usbfront/back been forwarded ported to work on 2.6 yet?
Is there stuff we could do to make it more stable, perhaps restricting
the set of devices it attempts to virtualize?

Thanks,
Ian


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

RE: XEN CD --> USB stick

[Harry Butterworth]

On Tue, 2005-03-01 at 08:37 +0000, Ian Pratt wrote:

> Has usbfront/back been forwarded ported to work on 2.6 yet?

No, not yet, I'm still plugging away at this.

Harry.



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: XEN CD --> USB stick

[Mark Williamson]

> > The Linux USB stack seems to work OK.  I never convinced
> > myself that the
> > default Linux HCD drivers really understood pseudophysical
> > memory but it
> > doesn't seem to have bitten me in practice *touch wood*
>
> All the pseudophysical memory issues shoud be taken care of within the
> Linux macros. It's only if the driver is buggy that its an issue.

Take a look at 2.6/drivers/usb/host/uhci-hcd.c::uhci_submit_common.  It 
doesn't do the conversions required to cope with pseudophysical memory.  As 
far as I can tell, USB client drivers cheerfully pass kmalloc'd memory down 
to this function.

As far as I'm concerned, this is still a worry.

> > usbfront / usbback need more development to support all sorts
> > of random
> > devices, so although it's stable and functional for me right
> > now I don't
> > think this should be enabled by default.
>
> OK, thanks, that's good to know.
> Has usbfront/back been forwarded ported to work on 2.6 yet?
> Is there stuff we could do to make it more stable, perhaps restricting
> the set of devices it attempts to virtualize?

Harry's being doing some good work on the 2.6 port but it's not completed yet.

Devices that don't virtualise well just don't work, it doesn't impact 
stability.  I doubt there's any way of identifying these devices other than 
by trying them out (depends whether the manufacturer put a stupid chip in the 
device).

Eventually we will support basically everything but that'll require faking out 
much more in software, which is probably not worth doing yet.

Bottom line: people who want to play with USB virtualisation should be able to 
do so successfully but should bear in mind it's experimental.

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: XEN CD --> USB stick

[Nivedita Singhvi]

Mark Williamson wrote:

> Harry's being doing some good work on the 2.6 port but it's not completed yet.

One of the reasons we consider this important is because
the cdrom drive on our Bladecenter platforms is a USB
device, and we would like to resolve the various issues
required to use it from the guest OS. This not only
covers the USB support in some form, but also issues
with removable media, installing software on the guest
domains directly from CD, etc. There are currently
workarounds for some of these requirements, but I'd
really like to see 2.6 USB working and stable.

> Devices that don't virtualise well just don't work, it doesn't impact 
> stability.  I doubt there's any way of identifying these devices other than 
> by trying them out (depends whether the manufacturer put a stupid chip in the 
> device).

We have some testing on our roadmap, but given the sheer
range of USB devices and issues related to them, it will
be a matter of having people test their own hw. To that
end, it will help to have some code in the tree for
people to play around with earlier rather than later,
because it will help stabilize the code to get a larger
group of people testing it..

> Bottom line: people who want to play with USB virtualisation should be able to 
> do so successfully but should bear in mind it's experimental.

So I did like Ian's idea of putting whatever's working, no
matter how minimal (with a LOT of caveats and warnings),
into the tree. You and Harry are the best judge of when it's
ready to go in, of course, but I would appreciate an early
push :)..

thanks,
Nivedita



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: XEN CD --> USB stick

[Mark Williamson]

> One of the reasons we consider this important is because
> the cdrom drive on our Bladecenter platforms is a USB
> device, and we would like to resolve the various issues
> required to use it from the guest OS. This not only
> covers the USB support in some form, but also issues
> with removable media, installing software on the guest
> domains directly from CD, etc. There are currently
> workarounds for some of these requirements, but I'd
> really like to see 2.6 USB working and stable.

Ah, I see.  I'm hopeful that CD ROM drives will require relatively little 
"magic" to make them work correctly under virtualisation.

Of course, the other way to go from your PoV would be to add some extra 
functionality to the VBD driver to better support CDs.  That would probably 
be useful for other purposes (e.g. exposing ISOs as CD drives) that people 
want to do.

Do you mean that this USB CDROM drive is actually integrated into the blade 
enclosure?  How is it shared among blades?  Or is it just that the standard 
means of installing software is to plug in a USB CDROM (that's what we used 
to do on our blades).

> We have some testing on our roadmap, but given the sheer
> range of USB devices and issues related to them, it will
> be a matter of having people test their own hw.

Yes, that's my feeling also.  I think that the additional emulation of 
control-plane stuff required to support the "problem" devices will be common 
to most (all?) troublesome devices.

> To that 
> end, it will help to have some code in the tree for
> people to play around with earlier rather than later,
> because it will help stabilize the code to get a larger
> group of people testing it..

I agree.

> > Bottom line: people who want to play with USB virtualisation should be
> > able to do so successfully but should bear in mind it's experimental.
>
> So I did like Ian's idea of putting whatever's working, no
> matter how minimal (with a LOT of caveats and warnings),
> into the tree. You and Harry are the best judge of when it's
> ready to go in, of course, but I would appreciate an early
> push :)..

I'd certainly be in favour of pushing support into the tree as soon as 
something's working reliably for us.  I'd like to support the "awkward" 
devices later on.

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

RE: XEN CD --> USB stick

[Ian Pratt]

> Take a look at 
> 2.6/drivers/usb/host/uhci-hcd.c::uhci_submit_common.  It 
> doesn't do the conversions required to cope with 
> pseudophysical memory.  As 
> far as I can tell, USB client drivers cheerfully pass 
> kmalloc'd memory down to this function.

Sounds like this should be changed to dma_alloc_coherent.

Ian



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id\x14396&op=click

Re: status of dom0 USB and USB virtualisation (was: XEN CD --> USB stick)

[Mark Williamson]

Replying to my own post... ;-)

> Do you mean that this USB CDROM drive is actually integrated into the blade
> enclosure?  How is it shared among blades? 

It looks to me like it's presented to the blades as a USB device that goes 
away when it's assigned to another blade.  Cute!

USB virtualisation would allow you to do exactly the same thing for virtual 
servers in a blade as you do for blades in an enclosure - neat.  I see where 
you're coming from now.

Cheers,
Mark

> > Or is it just that the standard  
> means of installing software is to plug in a USB CDROM (that's what we used
> to do on our blades).
>
> > We have some testing on our roadmap, but given the sheer
> > range of USB devices and issues related to them, it will
> > be a matter of having people test their own hw.
>
> Yes, that's my feeling also.  I think that the additional emulation of
> control-plane stuff required to support the "problem" devices will be
> common to most (all?) troublesome devices.
>
> > To that
> > end, it will help to have some code in the tree for
> > people to play around with earlier rather than later,
> > because it will help stabilize the code to get a larger
> > group of people testing it..
>
> I agree.
>
> > > Bottom line: people who want to play with USB virtualisation should be
> > > able to do so successfully but should bear in mind it's experimental.
> >
> > So I did like Ian's idea of putting whatever's working, no
> > matter how minimal (with a LOT of caveats and warnings),
> > into the tree. You and Harry are the best judge of when it's
> > ready to go in, of course, but I would appreciate an early
> > push :)..
>
> I'd certainly be in favour of pushing support into the tree as soon as
> something's working reliably for us.  I'd like to support the "awkward"
> devices later on.
>
> Cheers,
> Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: status of dom0 USB and USB virtualisation

[Nivedita Singhvi]

Mark Williamson wrote:

> Replying to my own post... ;-)

A good thing you did, because I failed to receive your
original post! :) So is the mailing list being a bit wacky(?)..

> USB virtualisation would allow you to do exactly the same thing for virtual 
> servers in a blade as you do for blades in an enclosure - neat.  I see where 
> you're coming from now.

Yes, exactly.

>>I'd certainly be in favour of pushing support into the tree as soon as
>>something's working reliably for us.  I'd like to support the "awkward"
>>devices later on.

Cool :). I figured we could put up a list of known-to-work
devices or something of the sort with input from the list/users
if that would be useful...

thanks,
Nivedita




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: status of dom0 USB and USB virtualisation

[Mark Williamson]

> A good thing you did, because I failed to receive your
> original post! :) So is the mailing list being a bit wacky(?)..

Sourceforge acts up a bit occasionally - one symptom is that e-mails arrive 
out of order (with some messages being delayed for significant lengths of 
time).

> >>I'd certainly be in favour of pushing support into the tree as soon as
> >>something's working reliably for us.  I'd like to support the "awkward"
> >>devices later on.
>
> Cool :). I figured we could put up a list of known-to-work
> devices or something of the sort with input from the list/users
> if that would be useful...

I agree, although perhaps a blacklist of unfriendly devices would be better?  
This sort of thing could arguably go on the Wiki once it's up and running.

Cheers,
Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: status of dom0 USB and USB virtualisation

[Eivind]

Hello All, 

So if I understand this right, I cannot use my USB flash device with
domain 0 as of yet? (I have enabled it in the kernel configuration
following all the necessary steps of the Linux USB-FAQ, but did not
have success mounting usbfs on /proc/bus/usb, and so on...). The same
problem is with XenCD as well.

Is this true? - please let me know.

Sincerely,
-Eivind


On Tue, 01 Mar 2005 18:34:56 -0800, Nivedita Singhvi <niv@us.ibm.com> wrote:
> Mark Williamson wrote:
> 
> > Replying to my own post... ;-)
> 
> A good thing you did, because I failed to receive your
> original post! :) So is the mailing list being a bit wacky(?)..
> 
> > USB virtualisation would allow you to do exactly the same thing for virtual
> > servers in a blade as you do for blades in an enclosure - neat.  I see where
> > you're coming from now.
> 
> Yes, exactly.
> 
> >>I'd certainly be in favour of pushing support into the tree as soon as
> >>something's working reliably for us.  I'd like to support the "awkward"
> >>devices later on.
> 
> Cool :). I figured we could put up a list of known-to-work
> devices or something of the sort with input from the list/users
> if that would be useful...
> 
> thanks,
> Nivedita
> 
>


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: status of dom0 USB and USB virtualisation

[Nivedita Singhvi]

Eivind wrote:
> Hello All, 
> 
> So if I understand this right, I cannot use my USB flash device with
> domain 0 as of yet? (I have enabled it in the kernel configuration
> following all the necessary steps of the Linux USB-FAQ, but did not
> have success mounting usbfs on /proc/bus/usb, and so on...). The same
> problem is with XenCD as well.
> 
> Is this true? - please let me know.

Not sure why you would have a problem on
domain0 - those should be the native drivers.

And if you are using the 2.4 kernel, then guest
domains also have access to virtualized usb
devices. 2.6 kernels are not supported at this
time..

thanks,
Nivedita


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click