Re: NBD (vs. iSCSI vs. EATA vs...) (fwd)

Re: NBD (vs. iSCSI vs. EATA vs...) (fwd)

[Guennadi Liakhovetski]

Hi

3 my emails of 26.05 didn't make it through to the list, although James 
did receive at least one of them. I'll try again, sorry, if they do come 
doubled in the end.

Thanks
Guennadi
---
Guennadi Liakhovetski


---------- Forwarded message ----------
Date: Thu, 26 May 2005 01:41:54 +0200 (CEST)
From: Guennadi Liakhovetski <lyakh@poirot.grange>
To: James Bottomley <James.Bottomley@SteelEye.com>
Cc: SCSI Mailing List <linux-scsi@vger.kernel.org>, Paul.Clements@steeleye.com
Subject: Re: NBD (vs. iSCSI vs. EATA vs...)

(Dropped most recepients of the original "ata over ethernet question" 
discussion, including lkml, from CC)

Hi

On Thu, 12 May 2005, James Bottomley wrote:
> However, there is room for improvement in nbd, notably the handling of
> packet commands, which looks to be eminently doable in the current
> infrastructure (this would basically make nbd a replicator for the linux
> block system, and would probably necessitate some client side changes to
> achieve).  If you have any thoughts in this direction, you could drop an
> email to the maintainer.

Ok, I came to a stage, when I start a server and a client, and then on the 
client side I can do open / close cycles, which would be respectively 
passed to the server, thus not keeping the server file(s) busy all the 
time. This allows, e.g., to export a CD-ROM, mount / umount it, eject it 
(not yet over nbd), insert a new one, mount, etc. Also, open errors are 
passed back, so, you get a nice "no medium" error on the client side 
without a CD.

IIUC, the suggestion from James to implement ATAPI translates in 
user-space terms to implementing respective ioctl's, because this is what 
nbd gets, and this is what nbd-server has to reproduce.

So, the work is not finished, at least some ioctl's are needed. I had to 
modify the kernel driver, and both the server and the client apps. No 
backward compatibility, sorry:-( What I'd like to do now is to ask - is 
there an interest in such changes to be integrated in the kernel / 
nbd-package. If there is one - what are your wishes? If we get that far, 
what should I post here - only the kernel patch, and user-space separately 
to sf (I think?), or both?

(see also my another post today to linux-scsi)

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: NBD (vs. iSCSI vs. EATA vs...) (fwd)

[Guennadi Liakhovetski]

The second one. The reply from James to the first one did come to the 
list. So, just resending this one. Would be nice to hear some comments.

Thanks
Guennadi
---
Guennadi Liakhovetski


---------- Forwarded message ----------
Date: Thu, 26 May 2005 19:48:59 +0200 (CEST)
From: Guennadi Liakhovetski <lyakh@poirot.grange>
To: James Bottomley <James.Bottomley@SteelEye.com>
Cc: SCSI Mailing List <linux-scsi@vger.kernel.org>,
    Paul Clements <Paul.Clements@SteelEye.com>
Subject: Re: NBD (vs. iSCSI vs. EATA vs...)

(I wonder why my emails still haven't appeared on linux-scsi?)

On Wed, 25 May 2005, James Bottomley wrote:

> On Thu, 2005-05-26 at 01:41 +0200, Guennadi Liakhovetski wrote:
> > IIUC, the suggestion from James to implement ATAPI translates in 
> > user-space terms to implementing respective ioctl's, because this is what 
> > nbd gets, and this is what nbd-server has to reproduce.
> 
> Actually, no, that's not what I was thinking.
> 
> nbd currently only understands the block layer REQ_CMD (which are basic
> READ/WRITE commands) if it understood REQ_BLOCK_PC it would probably be
> sufficient a transport for sending the packet commands that are
> necessary to drive a remote device.

Aha, I see...

> The idea being that you could then do
> 
> cdrom - nbd <--\/\-->remote - ide/scsi - CD-ROM device
> 
> to drive an actual cdrom.  There are some pieces missing in this vision
> (like how to attach cdrom to nbd) but it should be doable in principle.

You need the cdrom driver to convert user requests to cdrom commands, 
which with your scheme would be collected by nbd from the block queue, 
passed to nbd-server to be translated back to user-space requests...

Well, doesn't this mean, that you would have to look, if the server 
exports a SCSI cdrom, nbd would have to add a SCSI bus to the system and a 
cdrom on it, respectively for ide. So, you would just get a new srX / sdX 
/ hdX / loopX? Or you would have to write a new cdrom driver for the "nbd 
bus", or teach one of existing to work over nbd (ide or scsi?). Wouldn't 
either of these 3 possibilities be too complicated? The third one could be 
not too difficult, but you would have to choose a victim - one of ide or 
scsi cdrom drivers. And what do you do with discs?...

OTOH, if you just pass all user-space requests to the server and just pass 
back return code, you wouldn't need high-level drivers on the host side. 
You would miss anything, comming not from userspace - any non REQ_CMD 
requests from filesystems. But those we could just add separately - for 
that you don't need cdrom / sd / hd drivers, right?

So, just pass ioctl()'s directly and convert the rest (filesystem) block 
requests separately - wouldn't it be easier and avoid any modifications to 
other drivers?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: NBD (vs. iSCSI vs. EATA vs...) (fwd)

[James Bottomley]

On Sat, 2005-05-28 at 10:32 +0200, Guennadi Liakhovetski wrote:
> So, just pass ioctl()'s directly and convert the rest (filesystem)
> block 
> requests separately - wouldn't it be easier and avoid any
> modifications to 
> other drivers?

Before you go any further, I suggest you think about what an ioctl is
and why we're going through a process of trying to get rid of them in
the kernel:  They're structured streams of information, where the
structure is architecture and 64/32 bit dependent (look at all the
issues over compat ioctls).  Inherently these things will be incredibly
nasty to convert to an architecture neutral network protocol, and you'll
have to provide a translator to and from every possible ioctl...they're
inherently unsuitable things for building a protocol on top of.  On the
other hand, the current nbd protocol is block request based, which is
why I suggested looking at that instead.

James

Re: NBD (vs. iSCSI vs. EATA vs...) (fwd)

[Guennadi Liakhovetski]

On Sat, 28 May 2005, James Bottomley wrote:

> On Sat, 2005-05-28 at 10:32 +0200, Guennadi Liakhovetski wrote:
> > So, just pass ioctl()'s directly and convert the rest (filesystem)
> > block 
> > requests separately - wouldn't it be easier and avoid any
> > modifications to 
> > other drivers?
> 
> Before you go any further, I suggest you think about what an ioctl is
> and why we're going through a process of trying to get rid of them in
> the kernel:  They're structured streams of information, where the
> structure is architecture and 64/32 bit dependent (look at all the
> issues over compat ioctls).  Inherently these things will be incredibly
> nasty to convert to an architecture neutral network protocol, and you'll
> have to provide a translator to and from every possible ioctl...they're
> inherently unsuitable things for building a protocol on top of.  On the
> other hand, the current nbd protocol is block request based, which is
> why I suggested looking at that instead.

Yeah... Well, whatever way it goes further, I think, 1 thing we do have to 
do before implementing that functionality - moving open / close in 
nbd-server from start / stop to be triggered by the client - right? And 
the flags and the return code should be passed accordingly. That's exactly 
what I've done so far.

As for the ioctls vs. REQ_BLOCK_PC... Well, I am open. I just thought: 
either we convert ioctls to ioctls, or block commands to ioctls, in any 
case ioctls are what we have to issue on the server side. But I'll rather 
trust you than myself on this:-) So, if no other opinions pop up in this 
discussion, I might try to play in that direction a bit. Will see.

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: NBD (vs. iSCSI vs. EATA vs...) (fwd)

[Bryan Henderson]

>I suggest you think about what an ioctl is
>and why we're going through a process of trying to get rid of them in
>the kernel:  They're structured streams of information, where the
>structure is architecture and 64/32 bit dependent (look at all the
>issues over compat ioctls).

I think that's a poor characterization of an ioctl.  _Some_ ioctls have an 
architecture and 64/32 bit dependent structure, but it isn't inherent in 
an ioctl.  In fact, most ioctls don't have that problem.

What an ioctl is is something that's defined for a special case (e.g. a 
certain device type).  The lesson from the compat ioctl mess is that it 
hurts to have special cases.  Plenty of system calls are as 
architecture-dependent as any ioctl, but since they aren't defined 
separately in every device driver, they're easier to update for new 
architectures.

Incidentally, I'm reserving judgment on whether the disadvantages of 
ioctls outweigh their advantages.

--
Bryan Henderson                     IBM Almaden Research Center
San Jose CA                         Filesystems 

Re: NBD (vs. iSCSI vs. EATA vs...) (fwd)

[Guennadi Liakhovetski]

Hi all

On Sun, 29 May 2005, Guennadi Liakhovetski wrote:

> On Sat, 28 May 2005, James Bottomley wrote:
> 
> > Before you go any further, I suggest you think about what an ioctl is
> > and why we're going through a process of trying to get rid of them in
> > the kernel:  They're structured streams of information, where the
> > structure is architecture and 64/32 bit dependent (look at all the
> > issues over compat ioctls).  Inherently these things will be incredibly

I haven't given up, I just have little time:-)

Let me just see, if my current understanding is nearer to what was 
suggested by James B.:

Get ide-cd (ATAPI command generator) and cdrom (ioctl translator) drivers 
work with nbd by emulating an ATAPI CD-ROM. You would have to explain 
users why they need the ide cdrom driver even when they are connecting to 
a quite different target...

I would add - make this a configuration option, since not all (embedded) 
systems might want to compile and load ide-cd and cdrom drivers just to be 
able to use nbd, if they only need a network-exported hd.

Still a question - when to do open / close on the server: on start / stop 
(disconnect) - not nice, from open / close on client, on lockdoor() 
GPCMD_PREVENT_ALLOW_MEDIUM_REMOVAL commands? My guess would still be from 
client open / close, because then you can pass flags / return code. Not 
sure how easy it is to do this 64bit clean?

Thanks
Guennadi
---
Guennadi Liakhovetski