AOE

AOE

[James Harper]

If anyone is wanting to experiment with SAN technology, but finding
iSCSI too fiddly, have a look at AOE (ATA over Ethernet). There is a
software AoE server, and the kernel client is already in 2.6.11.

I use AOE root on dom0, and on the block devices exported to the other
domains (eg dom0 exports AOE devices, the other domains don't do AOE
themselves). This allows for domain migration to work nicely.

Have a look at: http://aoetools.sourceforge.net/ (tools + software
server)
And: http://www.coraid.com/ (hardware AOE server)

I can post some details for getting Debian root on AOE if anyone is
interested.

James

Re: AOE

[Keir Fraser]

On 22 May 2005, at 05:51, James Harper wrote:

> If anyone is wanting to experiment with SAN technology, but finding
> iSCSI too fiddly, have a look at AOE (ATA over Ethernet). There is a
> software AoE server, and the kernel client is already in 2.6.11.
>
> I use AOE root on dom0, and on the block devices exported to the other
> domains (eg dom0 exports AOE devices, the other domains don't do AOE
> themselves). This allows for domain migration to work nicely.

I'm surprised that the protocol packets don't include an end-to-end 
checksum. If the packets pass through a switch that occasionally 
garbles packets then you could end up committing erroneous sectors to 
disc.

On the other hand, if this is a significant problem then NFS/iscsi IP 
checksumming may not save your data either, since the IP checksum is so 
weak. For large volumes of data you care about the integrity of, you 
probably really want an end-to-end CRC.

  -- Keir

Re: AOE

[Keir Fraser]

On 22 May 2005, at 08:48, Keir Fraser wrote:

> I'm surprised that the protocol packets don't include an end-to-end 
> checksum. If the packets pass through a switch that occasionally 
> garbles packets then you could end up committing erroneous sectors to 
> disc.
>
> On the other hand, if this is a significant problem then NFS/iscsi IP 
> checksumming may not save your data either, since the IP checksum is 
> so weak. For large volumes of data you care about the integrity of, 
> you probably really want an end-to-end CRC.

Oops. The Ethernet CRC is end-to-end on a LAN, even if it's switched. 
:-)

  -- Keir

Re: AOE

[Christian Limpach]

On 5/22/05, James Harper <james.harper@bendigoit.com.au> wrote:
> If anyone is wanting to experiment with SAN technology, but finding
> iSCSI too fiddly, have a look at AOE (ATA over Ethernet). There is a
> software AoE server, and the kernel client is already in 2.6.11.
> 
> I use AOE root on dom0, and on the block devices exported to the other
> domains (eg dom0 exports AOE devices, the other domains don't do AOE
> themselves). This allows for domain migration to work nicely.

Have you ever tried gnbd?  How does this compare to gnbd?  We've used
gnbd in exactly the same setup.

    christian

RE: AOE

[James Harper]

I've tried nbd, and possibly enbd (can't remember if I got that working
or not). At the time I was using a server which had performed okay as
Windows 2000 Server for some time. When running Linux though, it crashed
all the time. The problem turned out to be a bad stick of memory, but by
then I'd been through a few attempts of iSCSI and nbd and had cursed
their unreliability (obviously, it turned out to be the server itself in
the end).

Assuming a working AoE server on the same L2 network, you simply need to
have an 'up' network adapter, and load the AoE module. You don't even
need a TCP/IP stack running. Apart from MAC filtering, there is no
security, so it should really be on its own Ethernet segment, or VLAN. 

HTH

James

> -----Original Message-----
> From: Christian Limpach [mailto:christian.limpach@gmail.com]
> Sent: Sunday, 22 May 2005 18:30
> To: James Harper
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] AOE
> 
> On 5/22/05, James Harper <james.harper@bendigoit.com.au> wrote:
> > If anyone is wanting to experiment with SAN technology, but finding
> > iSCSI too fiddly, have a look at AOE (ATA over Ethernet). There is a
> > software AoE server, and the kernel client is already in 2.6.11.
> >
> > I use AOE root on dom0, and on the block devices exported to the
other
> > domains (eg dom0 exports AOE devices, the other domains don't do AOE
> > themselves). This allows for domain migration to work nicely.
> 
> Have you ever tried gnbd?  How does this compare to gnbd?  We've used
> gnbd in exactly the same setup.
> 
>     christian