[ANNOUNCE] TCM/LIO v4.0.0-rc4 for 2.6.36-rc4

[ANNOUNCE] TCM/LIO v4.0.0-rc4 for 2.6.36-rc4

[Nicholas A. Bellinger]

Greetings all,

It is my great pleasure to announce that TCM/LIO v4.0.0-rc4 for v2.6.36-rc4 has
been tagged and pushed into lio-core-2.6.git/lio-4.0.  The last weeks have been
really very busy and many gracious thanks go out to all of the individuals who
made comments and helped bring together this v4.0.0-rc4 release.  A list of the
major changes includes:

	*) Updates + bugfixes for tcm_fc (Joe Eykholt patches)
	*) Drop host_lock around struct scsi_cmnd->scsi_done() in tcm_loop (Vasu Dev assist)
	*) Add Extended INQUIRY Data VPD Page
	*) Numerous target_core_base.h cleanups (Konrad Rzeszutek Wilk assist)
	*) Add support for struct target_core_fabric_ops->new_cmd_map() optimization
	*) Add support for struct se_task->task_sg[] chaining for HW target mode
	   pci_map_sg() and friends.
	*) Convert msleep() to cpu_relax() usage in several tight loops.
	   (Konrad Rzeszutek Wilk assist)
	*) Add optimization to convert TCM Core and tcm_loop, lio-target and tcm_fc
           to use struct se_cmd descriptors from within fabric module dependent I/O
	   descriptors.  (Joe Eykholt assist)
	*) Convert lio-target to use libcrypto crc32c
	*) Add support to LIO-Target for SSE v4.2 (Nehalem) CRC32C offload instruction
	   (Mike Christie assist)
	*) Add support to LIO-Target for per iSCSI connection CPU scheduling affinity
	   (Phillip Reisner assist)
	*) Convert TCM Core to use include/linux/dma-mapping.h:enum dma_data_direction
	   (Boaz Harrosh assist)
	*) Add support for BIDI Commands and XDWRITE_READ_10 emulation (Boaz Harrosh assist)
	*) Add support for native 32-bit CDBs VARIABLE_LENGTH_CMD support and
           XDWRITE_READ_32 emulation (James Bottomley and Martin Petersen assist)

The shortlog and diffstat have been attached below for reference.

And a final thanks to the folks who contributed minor patches and/or comments, but where
not mentioned above.

Thank you!

--nab

Joe Eykholt (4):
  tcm_fc: handle new ft_recv interface from libfc
  tcm_fc: add missing method for tpg_check_prod_mode_write_protect
  tcm_fc: se_sess->se_acl not set
  tcm_fc: fix cmd leak

Nicholas Bellinger (63):
  tcm/pscsi: Convert struct bio->bi_rw assignment to use REQ_WRITE
  tcm/fileio: Bump version to v4.0
  tcm: Remove copyright info from includes
  tcm: Update copyright information
  tcm_fc: Fix build failure with gcc 4.3.2
  tcm_fc: fix misuse of kunmap_atomic()
  tcm_fc: Change version to 0.3
  tcm_loop: Drop host_lock around struct scsi_cmnd->scsi_done()
  tcm: Add support for Extended INQUIRY Data VPD Page
  tcm: Remove legacy struct se_lun->lun_type usage
  tcm: Remove legacy DEV_STATUS_THR_* defines
  tcm: Convert TCM base defines to enums, round 1
  tcm: Convert NAF_DYNAMIC_NODE_ACL to struct
    se_node_acl->dynamic_node_acl bitfield
  tcm: Convert MAP_SG_KMAP to struct se_map_sg->sg_kmap_active bitfield
  tcm: Convert TCM base defines to enums, round 2:
  tcm: Convert DEF_PR_REGISTERED to struct
    se_dev_entry->def_pr_registered bitfield
  tcm: Convert PRF_ISID_PRESENT_AT_REG to struct
    t10_pr_registration->isid_present_at_reg bitfield
  tcm: Convert struct se_cmd->scsi_sense_reason enum and add TCM_*
    prefix
  tcm: Add comments for t10_alua_index_t, t10_task_attr_index_t and
    t10_reservations_index_t
  tcm/pr: Add proper comment for struct
    t10_reservation_template->pr_res_holder
  tcm/alua: Move (*t10_*)() callers into struct t10_reservation_ops
  tcm: Cleanups for struct se_transport_task usage
  tcm: Add support for TRANSPORT_NEW_CMD_MAP
  tcm: Add support for struct target_core_fabric_ops->task_sg_chaining
  tcm/ramdisk: Add special case for TFO->task_sg_chaining=1 with
    RAMDISK_DR
  tcm: Convert struct se_node_acl->device_list_lock to spin_*lock_irq()
  tcm: Convert struct se_device->dev_status_lock to spin_*lock_irq()
  tcm_loop: Convert to use new optimized TFO->new_cmd_map() caller
  tcm_loop: Convert to using root_device_register() and
    root_device_unregister()
  tcm_loop: Use IS_ERR() conditional for root_device_register()
  tcm: Move remaining structure member related defs into related
    structures
  tcm: Add enum names and comments to target_core_base.h
  tcm/alua: Cleanups for target_core_alua.c
  tcm: Convert to cpu_relax() for atomic_read() in tight loops
  tcm: Add support for fabric module provided struct se_cmd descriptors
  tcm_loop: Convert to pre-allocated struct se_cmd descriptors
  lio-target: Convert to use pre-allocated struct se_cmd descriptors
  tcm_fc: Convert to use pre-allocated struct se_cmd descriptors
  lio-target: Convert to use libcrypto crc32c
  lio-target: Add support for Intel Nehalem crc32-intel libcrypto
    offload
  lio-target: Convert to use bitmap for handling thread_id allocation
  lio-target: Add support for per iSCSI connection CPU scheduling
    affinity
  tcm: Convert SE_DIRECTION_* to use enum dma_data_direction
  tcm_loop: Convert to use enum dma_data_direction
  tcm_fc: Convert to use enum dma_data_direction
  lio-target: Convert ISCSI_* data_direction macros to enum
    dma_data_direction
  tcm: Convert handful of target_core_base.h structure members to enums
  tcm: Convert to use include/scsi/scsi_cmnd.h:MAX_COMMAND_SIZE
  tcm: Update TRANSPORT_SENSE_BUFFER define comment
  tcm: Fix TRANSPORT_NEW_CMD_MAP failure case
  tcm_loop: Fix tcm_loop_new_cmd_map() failure return values
  tcm_loop: Fix TMR release patch OOPS
  tcm: Add support for BIDI operation and XDWRITE_READ_10 emulation
  tcm_loop: Add DMA_BIDIRECTIONAL CDB support
  tcm_loop: Remove BUGs() from I_T Nexus fallback code
  tcm: Remove DMA_BIDIRECTIONAL for BIDI-COMMANDS
  tcm_loop: Remove DMA_BIDIRECTIONAL usage for BIDI-COMMANDS
  tcm: Rename transport_xor_callback() to transport_complete_callback()
  lio-target: Enable crc32c_x86_offload=1 by default
  lio-target: Fix comment for TA_CRC32C_X86_OFFLOAD
  tcm: Add native 32-byte CDB support
  tcm: Add VARIABLE_LENGTH_CMD support w/ XDWRITE_READ_32 emulation
  tcm_loop: Enable 32-byte CDB support in virtual SCSI LLD

 drivers/target/lio-target/Kbuild                  |    1 -
 drivers/target/lio-target/iscsi_crc.c             |  171 ----
 drivers/target/lio-target/iscsi_crc.h             |    9 -
 drivers/target/lio-target/iscsi_target.c          |  453 ++++++++---
 drivers/target/lio-target/iscsi_target.h          |    1 +
 drivers/target/lio-target/iscsi_target_configfs.c |    6 +
 drivers/target/lio-target/iscsi_target_core.h     |   40 +-
 drivers/target/lio-target/iscsi_target_erl0.c     |    4 +-
 drivers/target/lio-target/iscsi_target_erl1.c     |   15 +-
 drivers/target/lio-target/iscsi_target_erl2.c     |    3 +-
 drivers/target/lio-target/iscsi_target_login.c    |   99 +++-
 drivers/target/lio-target/iscsi_target_login.h    |    1 +
 drivers/target/lio-target/iscsi_target_mib.c      |    6 +-
 drivers/target/lio-target/iscsi_target_nego.c     |   18 +-
 drivers/target/lio-target/iscsi_target_tmr.c      |   36 +-
 drivers/target/lio-target/iscsi_target_tpg.c      |   19 +
 drivers/target/lio-target/iscsi_target_tpg.h      |    1 +
 drivers/target/lio-target/iscsi_target_util.c     |   56 +-
 drivers/target/lio-target/iscsi_thread_queue.c    |   58 ++-
 drivers/target/lio-target/iscsi_thread_queue.h    |    5 +
 drivers/target/target_core_alua.c                 |  133 ++--
 drivers/target/target_core_alua.h                 |   14 +-
 drivers/target/target_core_configfs.c             |    8 +-
 drivers/target/target_core_device.c               |  133 ++--
 drivers/target/target_core_fabric_configfs.c      |    8 +-
 drivers/target/target_core_fabric_lib.c           |    5 +-
 drivers/target/target_core_file.c                 |    6 +-
 drivers/target/target_core_file.h                 |   34 +-
 drivers/target/target_core_hba.c                  |    4 +-
 drivers/target/target_core_hba.h                  |   29 -
 drivers/target/target_core_iblock.c               |    8 +-
 drivers/target/target_core_iblock.h               |   29 +-
 drivers/target/target_core_mib.c                  |   38 +-
 drivers/target/target_core_mib.h                  |   26 -
 drivers/target/target_core_pr.c                   |   53 +-
 drivers/target/target_core_pscsi.c                |   21 +-
 drivers/target/target_core_pscsi.h                |   32 +-
 drivers/target/target_core_rd.c                   |   55 +-
 drivers/target/target_core_rd.h                   |   32 +-
 drivers/target/target_core_scdb.c                 |   42 +-
 drivers/target/target_core_scdb.h                 |   31 +-
 drivers/target/target_core_stgt.c                 |    4 +-
 drivers/target/target_core_stgt.h                 |   30 +-
 drivers/target/target_core_tmr.c                  |    4 +-
 drivers/target/target_core_tpg.c                  |   64 +-
 drivers/target/target_core_transport.c            |  909 ++++++++++++++++-----
 drivers/target/target_core_ua.c                   |   24 +-
 drivers/target/tcm_fc/tcm_fc.h                    |    8 +-
 drivers/target/tcm_fc/tfc_cmd.c                   |  137 ++--
 drivers/target/tcm_fc/tfc_conf.c                  |   13 +-
 drivers/target/tcm_fc/tfc_io.c                    |   29 +-
 drivers/target/tcm_fc/tfc_sess.c                  |   15 +-
 drivers/target/tcm_loop/tcm_loop_configfs.c       |   35 +-
 drivers/target/tcm_loop/tcm_loop_core.h           |   12 +-
 drivers/target/tcm_loop/tcm_loop_fabric.c         |   91 +--
 drivers/target/tcm_loop/tcm_loop_fabric.h         |    2 -
 drivers/target/tcm_loop/tcm_loop_fabric_scsi.c    |  243 +++----
 drivers/target/tcm_loop/tcm_loop_fabric_scsi.h    |    2 +-
 include/target/target_core_base.h                 |  475 ++++++-----
 include/target/target_core_configfs.h             |   27 -
 include/target/target_core_device.h               |   28 -
 include/target/target_core_fabric_ops.h           |   12 +
 include/target/target_core_tpg.h                  |   34 +-
 include/target/target_core_transport.h            |   47 +-
 64 files changed, 2200 insertions(+), 1788 deletions(-)

Re: [ANNOUNCE] TCM/LIO v4.0.0-rc4 for 2.6.36-rc4

[FUJITA Tomonori]

On Fri, 17 Sep 2010 13:22:10 -0700
"Nicholas A. Bellinger" <nab@linux-iscsi.org> wrote:

> It is my great pleasure to announce that TCM/LIO v4.0.0-rc4 for v2.6.36-rc4 has
> been tagged and pushed into lio-core-2.6.git/lio-4.0.  The last weeks have been
> really very busy and many gracious thanks go out to all of the individuals who
> made comments and helped bring together this v4.0.0-rc4 release.  A list of the
> major changes includes:

Have you addressed all the non-SCSI obstacles to mainline inclusion?
Sorry, I can't track TCM thread but I thought that you got some sysfs
issues at least?

If so, can you tell me where I can find reviewable patchset? I really
hope that you drop stuff from the patchset that isn't a must for
target support (e.g. queuecommand lock stuff). If you keep adding
something new to tha patchset, it's really difficult to review it.

Re: [ANNOUNCE] TCM/LIO v4.0.0-rc4 for 2.6.36-rc4

[Nicholas A. Bellinger]

On Sat, 2010-09-18 at 13:00 +0900, FUJITA Tomonori wrote:
> On Fri, 17 Sep 2010 13:22:10 -0700
> "Nicholas A. Bellinger" <nab@linux-iscsi.org> wrote:
> 
> > It is my great pleasure to announce that TCM/LIO v4.0.0-rc4 for v2.6.36-rc4 has
> > been tagged and pushed into lio-core-2.6.git/lio-4.0.  The last weeks have been
> > really very busy and many gracious thanks go out to all of the individuals who
> > made comments and helped bring together this v4.0.0-rc4 release.  A list of the
> > major changes includes:
> 
> Have you addressed all the non-SCSI obstacles to mainline inclusion?
> Sorry, I can't track TCM thread but I thought that you got some sysfs
> issues at least?

So the only remaining item outside of drivers/target/ code that needs to
be addressed for .37 is the symlink referencing counting bit.  This is
the case where a struct config_group with symlinks that use a
destination that is in a struct config_group *above* where the source
link struct config_group lives, and currently makes it impossible to
signal (from theconfigfs consumer perspective) the destination symlink
that the parent struct config_group is being dropped with rmdir().

Joel and I have been discussing potential resoultions here:

http://marc.info/?l=linux-fsdevel&m=128414860505481&w=2

but we have yet to come to a resolution, and the ->check_link() patch is
still in use in lio-core-2.6.git/lio-4.0 code.

The possibilities that have been discussed so far include:

*) Include the original struct config_item_operations->check_link()
patch and require the configfs customer to be aware of this case when
fs/configfs/symlink.c:configfs_unlink().  At this point this still works
OK for me, but I do acknowledge that making the configfs consumer aware
of this issue is really not the proper right long term solution.

*) Use refcounting for the symlinks inside of the configfs consumer.  I
asked Joel about how to do this in his last response, but I have not yet
heard back from him.   Joel, would you be so kind as to elaborate on
what you meant by this..?

*) Make fs/configfs/symlink.c code handle this specific "destination
link outside of parent struct config_group" symlink reference count case
internally instead of requiring configfs consumers be aware of the
issue.

At this point I am still leaning towards #3 (with Joel's blessing) as
the cleanest  long term solution, but I am still happy to persue #2 for
fabric independent configfs handlers target_core_fabric_configfs.c code
for the .37 merge.  I am still open suggestions (again, with Joel's
blessing) about how to get this item properly resolved.

> 
> If so, can you tell me where I can find reviewable patchset?

So the RFC cuts have been going into a seperate tree here:

http://git.kernel.org/?p=linux/kernel/git/nab/lio-4.0.git;a=summary

Mike has requested an RFC v2 for him to review TCM Core and TCM_Loop
v4.0.0-rc4 changes.  So I will be respinning this tree with the latest
changes from lio-core-2.6.git/lio-4.0 and re-posting in the next days.

> I really hope that you drop stuff from the patchset that isn't a must for
> target support (e.g. queuecommand lock stuff). If you keep adding
> something new to tha patchset, it's really difficult to review it.
> --

Indeed, I plan to keep the drop-host_lock stuff a logically seperate item,
and the TCM_Loop ->queuecommand() caller in the upcoming RFC v2 patches
will still contain the original host_lock unlock() -> do_work() -> lock()
optimization in use by many mainline LLDs today.  I will plan to send a
seperate patch to James depending how/when the drop-host_lock stuff is merged.

Many thanks for your comments Tomo-san!

--nab

Re: [ANNOUNCE] TCM/LIO v4.0.0-rc4 for 2.6.36-rc4

[Boaz Harrosh]

On 09/18/2010 10:32 PM, Nicholas A. Bellinger wrote:
> On Sat, 2010-09-18 at 13:00 +0900, FUJITA Tomonori wrote:
> So the RFC cuts have been going into a seperate tree here:
> 
> http://git.kernel.org/?p=linux/kernel/git/nab/lio-4.0.git;a=summary
> 

This is the two weeks ago stuff, right. Where is the lio-core-2.6.git/ ?

> Mike has requested an RFC v2 for him to review TCM Core and TCM_Loop
> v4.0.0-rc4 changes.  So I will be respinning this tree with the latest
> changes from lio-core-2.6.git/lio-4.0 and re-posting in the next days.
> 

Me to. And a git tree and a git web, as well. I'll postpone any review
to that time.

Thanks
Boaz

Re: [ANNOUNCE] TCM/LIO v4.0.0-rc4 for 2.6.36-rc4

[Nicholas A. Bellinger]

On Sun, 2010-09-19 at 15:55 +0200, Boaz Harrosh wrote:
> On 09/18/2010 10:32 PM, Nicholas A. Bellinger wrote:
> > On Sat, 2010-09-18 at 13:00 +0900, FUJITA Tomonori wrote:
> > So the RFC cuts have been going into a seperate tree here:
> > 
> > http://git.kernel.org/?p=linux/kernel/git/nab/lio-4.0.git;a=summary
> > 
> 
> This is the two weeks ago stuff, right. Where is the lio-core-2.6.git/ ?
> 
> > Mike has requested an RFC v2 for him to review TCM Core and TCM_Loop
> > v4.0.0-rc4 changes.  So I will be respinning this tree with the latest
> > changes from lio-core-2.6.git/lio-4.0 and re-posting in the next days.
> > 
> 
> Me to. And a git tree and a git web, as well. I'll postpone any review
> to that time.

Hi Boaz,

So as mentioned earlier, the main upstream branch where all of the
review changes have been moving through is located at
lio-core-2.6.git/lio-4.0:

http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=shortlog;h=refs/heads/lio-4.0

I will be doing a RFCv2 posting for TCM Core and TCM_Loop v4.0.0-rc4
with the very latest changes this week for Mike, Tomo-san and yourself
to further review.  This series will be in the usual one-file-per-commit
format and will be pushed these into a rfcv2-for-37 branch against
v2.6.36-rc4 in the seperate pristine lio-4.0.git tree mentioned above.

Also just FYI, the workflow I plan to use post-merge for TCM is to
continue to merge all changes in via the lio-core-2.6.git tree, and send
out pull requests for patches destined to mainline via a seperate
pristine tree + branches.

Best,

--nab