iSCSI back-end design

[Jeff Garzik <jeff@garzik.org>]

The iSCSI target daemon now serves data persistently stored in an mmap'd 
file, as an alternative to RAM storage that exists for the lifetime of 
the daemon.

It has been verified to work with Linux and Microsoft iSCSI initiators 
(== iSCSI clients), appearing as a regular SCSI disk on your system.  It 
answers all the standard SCSI commands, issued by the kernel's SCSI 
subsystem, or userspace SCSI utilities like sg3_utils.

Usage is quite trivial:

	$ dd if=/dev/zero of=/tmp/target0.data bs=1M count=2000
	$ ./itd -f /tmp/target0.data

Then direct your iSCSI initiator (via iscsiadm on Linux) to the portal 
(== IPv4 or IPv6 address) where itd is listening.

Now, on to thinking about a Hail back-end for itd:

One of the recent additions to SCSI (http://www.t10.org/) is "thin 
provisioning", which enables easy management of SCSI devices such as 
flash storage or large arrays, where LBAs (hardware sectors) exist in 
one of two states:

	- mapped: present and have valid user data
	- unmapped: does not contain valid user data

Previously, all LBAs on all SCSI disks were assumed to be mapped.

The ability to unmap sectors, and know which sectors are unmapped, have 
clear advantages to flash-based storage hardware, which may use this 
information to erase or re-use unmapped flash cels.

Similarly, large SCSI arrays -- or Project Hail software -- may make use 
of unmapped sectors by avoiding allocation of storage.  As it applies to 
itd, I was thinking that our SCSI device may be implemented internally 
as an array of 256kB pages, some of which are filled with data, and some 
of which are not:

[0] NULL
[1] NULL
[2] chunkd object 1
[3] chunkd object 12398
[4] NULL
[5] chunkd object 22
[6] ...

The array itself is replicated across multiple chunkd nodes.  The 
superblock, containing information on where to find the array, is stored 
in CLD, updated once every 30-60 seconds, or upon a SYNCHRONIZE CACHE 
command.

The downside to large hardware sectors, of sizes such as 256k, is that 
itd would need to perform a read-modify-write cycle for <256k writes. 
This is mitigated by a large cache, but is nonetheless a factor.  We 
could easily choose a page size of 4k or even 512 bytes, but that, in 
turn, increases the array management overhead.

New SCSI commands related to thin provisioning are already implemented 
in Linux, and are documented here: 
http://www.t10.org/cgi-bin/ac.pl?t=f&f=sbc3r22.pdf

	Jeff