Software RAID on Windows using Embedded Linux?

Software RAID on Windows using Embedded Linux?

[Ewan Grantham]

I know, this is borderline, but figure this is the group of folks who
will know. I do a lot of audio and video stuff for myself and my
family. I also have a rather unusual networking setup. Long story
short, when I try to run Linux as my primary OS, I usually end up
reinstalling Windows after a couple weeks because there are still
holes in what I can do. That isn't the fault of Linux as much as folks
who write device drivers or have video codecs that require DirectShow.

However, the big thing I really miss from Linux that keeps me trying
to find a way to convert is the support for Software RAID 5.

It occured to me yesterday that perhaps the trick would be to use QEMU
to run Knoppix or Damn Small Linux under Windows, and then setup a
RAID 5 array under one of those. Not to mention then having access to
Linux for some other fun stuff.

I'm not sure if that's even possible, and if it is, how much trouble I
would have moving files around to and from the RAID array if it's
setup that way. So I'm wondering if anyone on the list has ever tried
this?

TIA,
Ewan

Re: Software RAID on Windows using Embedded Linux?

[Molle Bestefich]

Ewan Grantham wrote:
> I know, this is borderline, but figure this is the group of folks who
> will know. I do a lot of audio and video stuff for myself and my
> family. I also have a rather unusual networking setup. Long story
> short, when I try to run Linux as my primary OS, I usually end up
> reinstalling Windows after a couple weeks because there are still
> holes in what I can do. That isn't the fault of Linux as much as folks
> who write device drivers or have video codecs that require DirectShow.
> 
> However, the big thing I really miss from Linux that keeps me trying
> to find a way to convert is the support for Software RAID 5.
> 
> It occured to me yesterday that perhaps the trick would be to use QEMU
> to run Knoppix or Damn Small Linux under Windows, and then setup a
> RAID 5 array under one of those. Not to mention then having access to
> Linux for some other fun stuff.
> 
> I'm not sure if that's even possible, and if it is, how much trouble I
> would have moving files around to and from the RAID array if it's
> setup that way. So I'm wondering if anyone on the list has ever tried
> this?

Not too much trouble, you should be able to just setup Samba in the
Knoppix/DSL and access your files through QEMU's network emulation.

It will be damn slow, however :-).

Perhaps buy a NAS device like Linksys NSLU2?
It's cheap and has low power consumption - but you'll need to hack it
to add disks (via USB).

http://peter.korsgaard.com/articles/debian-nslu2.php
 or
http://www.tomsnetworking.com/Sections-article85-page3.php

explains how to get into the Linux guts of the box.

Re: Software RAID on Windows using Embedded Linux?

[Ewan Grantham]

On 7/24/05, Molle Bestefich <molle.bestefich@gmail.com> wrote:
> Ewan Grantham wrote:
> > I know, this is borderline, but figure this is the group of folks who
> > will know. I do a lot of audio and video stuff for myself and my
> > family. I also have a rather unusual networking setup. Long story
> > short, when I try to run Linux as my primary OS, I usually end up
> > reinstalling Windows after a couple weeks because there are still
> > holes in what I can do. That isn't the fault of Linux as much as folks
> > who write device drivers or have video codecs that require DirectShow.
> >
> > However, the big thing I really miss from Linux that keeps me trying
> > to find a way to convert is the support for Software RAID 5.
> >
> > It occured to me yesterday that perhaps the trick would be to use QEMU
> > to run Knoppix or Damn Small Linux under Windows, and then setup a
> > RAID 5 array under one of those. Not to mention then having access to
> > Linux for some other fun stuff.
> >
> > I'm not sure if that's even possible, and if it is, how much trouble I
> > would have moving files around to and from the RAID array if it's
> > setup that way. So I'm wondering if anyone on the list has ever tried
> > this?
> 
> Not too much trouble, you should be able to just setup Samba in the
> Knoppix/DSL and access your files through QEMU's network emulation.
> 
> It will be damn slow, however :-).
> 
> Perhaps buy a NAS device like Linksys NSLU2?
> It's cheap and has low power consumption - but you'll need to hack it
> to add disks (via USB).
> 
> http://peter.korsgaard.com/articles/debian-nslu2.php
>  or
> http://www.tomsnetworking.com/Sections-article85-page3.php
> 
> explains how to get into the Linux guts of the box.

Interesting idea. However, I note that it is rather slow as well -
since you're then accessing disks over 100 Mbps rather than at full
USB2. And it appears you can only attach 2 USB disks to it.

Which makes me wonder just how slow you mean when you say that the
QEMU version would be slow? Slow compared to native access? Slow
compared to the NSLU2 solution discussed above? Or slow compared to a
snail being attacked by one of those Hawaiian Catepillars? :-)

Thanks,
Ewan

Re: Software RAID on Windows using Embedded Linux?

[Ewan Grantham]

One other idea I've had (and y'all can tell me how much I'm showing my
lack of knowledge in this area) is to try recompiling the mdadm and
other tools using cygwin. But I assume that this is an idea that's
been floated before and didn't work since I haven't seen any signs of
this having been done before, and it seems like a pretty obvious one
if it were easy to do.

Thanks again,
Ewan

Re: Software RAID on Windows using Embedded Linux?

[Tyler]

mdadm is just the tool to manipulate (start, stop, create, view) a raid 
device and its components.. the MD (md) driver is part of the linux 
kernel (or a kernel module), and is the one doing the actual raid work 
(XOR, striping, etc.).

Regards,
Tyler.

Ewan Grantham wrote:

>One other idea I've had (and y'all can tell me how much I'm showing my
>lack of knowledge in this area) is to try recompiling the mdadm and
>other tools using cygwin. But I assume that this is an idea that's
>been floated before and didn't work since I haven't seen any signs of
>this having been done before, and it seems like a pretty obvious one
>if it were easy to do.
>
>Thanks again,
>Ewan
>-
>To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>  
>

Re: Software RAID on Windows using Embedded Linux?

[Molle Bestefich]

On 7/24/05, Ewan Grantham <ewan.grantham@gmail.com> wrote:
> On 7/24/05, Molle Bestefich <molle.bestefich@gmail.com> wrote:
> > Ewan Grantham wrote:
> > > I know, this is borderline, but figure this is the group of folks who
> > > will know. I do a lot of audio and video stuff for myself and my
> > > family. I also have a rather unusual networking setup. Long story
> > > short, when I try to run Linux as my primary OS, I usually end up
> > > reinstalling Windows after a couple weeks because there are still
> > > holes in what I can do. That isn't the fault of Linux as much as folks
> > > who write device drivers or have video codecs that require DirectShow.
> > >
> > > However, the big thing I really miss from Linux that keeps me trying
> > > to find a way to convert is the support for Software RAID 5.
> > >
> > > It occured to me yesterday that perhaps the trick would be to use QEMU
> > > to run Knoppix or Damn Small Linux under Windows, and then setup a
> > > RAID 5 array under one of those. Not to mention then having access to
> > > Linux for some other fun stuff.
> > >
> > > I'm not sure if that's even possible, and if it is, how much trouble I
> > > would have moving files around to and from the RAID array if it's
> > > setup that way. So I'm wondering if anyone on the list has ever tried
> > > this?
> >
> > Not too much trouble, you should be able to just setup Samba in the
> > Knoppix/DSL and access your files through QEMU's network emulation.
> >
> > It will be damn slow, however :-).
> >
> > Perhaps buy a NAS device like Linksys NSLU2?
> > It's cheap and has low power consumption - but you'll need to hack it
> > to add disks (via USB).
> >
> > http://peter.korsgaard.com/articles/debian-nslu2.php
> >  or
> > http://www.tomsnetworking.com/Sections-article85-page3.php
> >
> > explains how to get into the Linux guts of the box.
> 
> Interesting idea. However, I note that it is rather slow as well -
> since you're then accessing disks over 100 Mbps rather than at full
> USB2. And it appears you can only attach 2 USB disks to it.

You should be able to hack it to accept more than 2 disks..
You're right about the 100 mbps network interface..

> Which makes me wonder just how slow you mean when you say that the
> QEMU version would be slow? Slow compared to native access? Slow
> compared to the NSLU2 solution discussed above? Or slow compared to a
> snail being attacked by one of those Hawaiian Catepillars? :-)

"Slow" as in a passing tortoise would take that system by surprise =)..

Based on QEMU being a CPU emulator, not a virtualization system, thus
penalizing performance with a factor 10 or so.

Actually, just checked their web site, and they now have a
non-open-source alternative that uses virtualization, so if you're
going to use that it just might work ok.  If you're willing to live
with the fact that MD will at times suck up some CPU.  Looking forward
to hearing whether you can live with the solution or not (if you go
that way) :-).

Re: Software RAID on Windows using Embedded Linux?

[Ewan Grantham]

Well, as I have been investigating how I would set things up with QEMU
and the Accelerator, I got pointed to this wiki posting for coLinux:
http://wiki.colinux.org/cgi-bin/raid_solution

So am now thinking that might be worth looking into. Though I'd also
love to hear from anyone who has already tried this.

Thanks again for the time and assistance,
Ewan

Re: Software RAID on Windows using Embedded Linux?

[Ewan Grantham]

Well, the past couple of days have been quite a learning experience for me...

To begin with, and to show I'm not totally pig headed, I tried the
Tom's Hardware approach. Turns out that it doesn't work in multiple
ways in my situation. Specifically:

- Dynamic Disks aren't supported in XP Home, not even with the hacked files

- While Dynamic Disks are supported in Pro and Server, you can't make
firewire or USB-2 disks dynamic in any version of windows. There is a
registry hack to make firewire disks capable of being made dynamic in
Win XP, but there is no such hack for USB-2

IOW, in my situation, these answers aren't an option.

So I got coLinux setup with samba and raid. Does it work? Yes. Is it
fast enough? Well, that depends on your needs. For just plain copying
files around, it's about 1/3 the speed of a native copy for large
files between external disks. For playing back music, this is fast
enough. For playing back video, it isn't.

Which brings me back to the question - would I see any better
performance with QEMU and the preliminary version of the accelerator
for Windows, or am I just going to have to accept that I need another
machine if I want a RAID array? Not that I expect anyone else here has
tried this, but figured y'all might find this to be useful info. Of
course if someone who's already more familiar with QEMU did want to
try this and let me know if it works... :-)

Thanks again,
Ewan