Re: Inhibit auto-attach of scsi disks ?

Re: Inhibit auto-attach of scsi disks ?

[Bryan Henderson]

I agree that Linux would be cleaner if auto-attach were optional.  I
hesitate to call that an artistic preference -- I think of it as
scientific.  Unattached is a valid, meaningful state, more primitive than
attached, so it should be possible for a device to come up to just that
state.

But when we speak of inhibiting partition table reading, I think that's
another (missing) feature:  I should be able to control whether Linux
considers there to be partitions on my disk or not (and change the fact
dynamically).  Note that just because the first sector contains something
that appears to be a partition table doesn't mean it is.  Then I can
imagine bringing up a disk in 3 stages: 1) initialize the SCSI low level
driver and identify the device at that level; 2) attach the device as a
SCSI disk; 3) create the partition devices (read the partition table).

Re: Inhibit auto-attach of scsi disks ?

[Alan Cox]

On Wed, 2002-10-02 at 19:10, Bryan Henderson wrote:
> But when we speak of inhibiting partition table reading, I think that's
> another (missing) feature:  I should be able to control whether Linux
> considers there to be partitions on my disk or not (and change the fact

Why do you want to. Linux always offers you the entire disk anyway. 

Re: Inhibit auto-attach of scsi disks ?

[Rogier Wolff]

On Wed, Oct 02, 2002 at 08:51:39PM +0100, Alan Cox wrote:
> On Wed, 2002-10-02 at 19:10, Bryan Henderson wrote:
> > But when we speak of inhibiting partition table reading, I think that's
> > another (missing) feature:  I should be able to control whether Linux
> > considers there to be partitions on my disk or not (and change the fact
> 
> Why do you want to. Linux always offers you the entire disk anyway. 

Andries has argued this before, but I encountered a real-life example
today.

Today I was asked to recover data from a device where reading one
of the "bad" blocks would cause an immediate "lockup" of the device:
It would report ALL future blocks as bad as well. Only a power-cycle
could revert it to reporting other blocks as "working". 

In this case, the partition table luckily didn't reference any of the
bad blocks. But if it did, it would have made the recoverable 99% of
the device unaccessable. 

		Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* The Worlds Ecosystem is a stable system. Stable systems may experience *
* excursions from the stable situation. We are currenly in such an       * 
* excursion: The stable situation does not include humans. ***************

Re: Inhibit auto-attach of scsi disks ?

[Doug Ledford]

On Wed, Oct 02, 2002 at 11:30:45PM +0200, Rogier Wolff wrote:
> On Wed, Oct 02, 2002 at 08:51:39PM +0100, Alan Cox wrote:
> > On Wed, 2002-10-02 at 19:10, Bryan Henderson wrote:
> > > But when we speak of inhibiting partition table reading, I think that's
> > > another (missing) feature:  I should be able to control whether Linux
> > > considers there to be partitions on my disk or not (and change the fact
> > 
> > Why do you want to. Linux always offers you the entire disk anyway. 
> 
> Andries has argued this before, but I encountered a real-life example
> today.
> 
> Today I was asked to recover data from a device where reading one
> of the "bad" blocks would cause an immediate "lockup" of the device:
> It would report ALL future blocks as bad as well. Only a power-cycle
> could revert it to reporting other blocks as "working". 
> 
> In this case, the partition table luckily didn't reference any of the
> bad blocks. But if it did, it would have made the recoverable 99% of
> the device unaccessable. 

There's also the additional case of certain drives which may be reserved 
by another initiator in a multi-initiator state which will then return an 
I/O error for attempts to read the partition table (this can also happen 
when certain drives, such as EMC type stuff (I'm not actually implying 
EMC, because I can't remember the brand I saw this on), where the INQUIRY 
data doesn't indicate that the device is actually offline, but it really 
is, so attempts to read from it generate errors).  It's not real urgent 
stuff, but there are a few reasons to not read the partition table nor the 
geometry or capacity of the device until you actually want to use it (for 
example catch the first attempt to open any partition on a device and if 
we haven't read the partition table yet then do so and then determine if 
the open succeeds, something like that).

-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: Inhibit auto-attach of scsi disks ?

[Rogier Wolff]

On Wed, Oct 02, 2002 at 05:44:40PM -0400, Doug Ledford wrote:
> example catch the first attempt to open any partition on a device and if 
> we haven't read the partition table yet then do so and then determine if 
> the open succeeds, something like that).

FYI, I think I saw the kernel doing this already today, albeit, not 
completely correct: It only scanned for partitions once I opened  
/dev/sda, for which knowing the partition table is irrelevant. 
usb-storage.

				Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* The Worlds Ecosystem is a stable system. Stable systems may experience *
* excursions from the stable situation. We are currenly in such an       * 
* excursion: The stable situation does not include humans. ***************

Re: Inhibit auto-attach of scsi disks ?

[Scott Merritt]

On Wed, 2 Oct 2002 11:10:20 -0700
"Bryan Henderson" <hbryan@us.ibm.com> wrote:
 
> I agree that Linux would be cleaner if auto-attach were optional ...

> But when we speak of inhibiting partition table reading, I think that's
> another (missing) feature:  I should be able to control whether Linux
> considers there to be partitions on my disk or not ...

Can you envision an "elegant" way in which these preferences could be communicated to the scsi disk drivers ? - including the case where you have only a scsi disk as root ?  Perhaps a kernel command line option together with some type of "noauto" option in /etc/fstab ???  There's probably a much better place, but I'm not "clever" enough to come up with it ... :)

Best regards, Scott.

Re: Inhibit auto-attach of scsi disks ?

[Rogier Wolff]

On Wed, Oct 02, 2002 at 11:10:20AM -0700, Bryan Henderson wrote:
> imagine bringing up a disk in 3 stages: 1) initialize the SCSI low level
> driver and identify the device at that level; 2) attach the device as a
> SCSI disk; 3) create the partition devices (read the partition table).

I fully agree! The Unix philosophy is that you can easily make a 
"script" or something that "does the one extra thing". But if 
something automatically does something for you, then you can't
make a script that omits that step that shouldn't have happened
in the first place. 

This is the power of Unix: as a user you get to make unexpected
combinations that the designers didn't think of. 

In this case, we have some history to take along. For example,
if we'd require a userlevel program to initiate partition table
reading, we'd make a whole lot of systems that do "root=/dev/sda3"
non-bootable. That is not an option. 

Personally, I think lowlevel drivers should NOT bother with 
partitions. We should have a "partition driver", which will
pass the requests it gets on to a lowlevel driver with an 
offset. This is the conceptual level. For performance reasons, 
we'll end up doing something in ll_rw_block that does:

	if (major == partition_major) {
		block += partitions[minor].block_offset;
		major = partitions[minor].major;
		minor = partitions[minor].minor;
	}	

just 3 instructions extra, but it relieves ALL lower level drivers
of having to deal with partitions. 

Then we'd have a "ide disk driver" which sanely supports 256 
IDE drives on one major. We also get a driver which supports a 
total of 256 partitions, in a dense way. Currently we are having
trouble because SOME people have 16 partitions, and SOME people 
have more than 16 disks. We currently then have to allocate 
16 disks * 16 partitions = 256 partitions which are seldomly
used, but individual users hit one or the other limitation
quite easily. 

This is what I think SHOULD eventually be implemented. We'll 
need to solve some compatiblity issues for the intervening period. 

One way to ensure compatiblity would be to make all "currently 
partitioned" devices one of those magic partitioner devices, 
and reroute to the NEW real, dense driver by a compatibility 
module which only has to do some initializations. 

				Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* The Worlds Ecosystem is a stable system. Stable systems may experience *
* excursions from the stable situation. We are currenly in such an       * 
* excursion: The stable situation does not include humans. ***************

Re: Inhibit auto-attach of scsi disks ?

[Bryan Henderson]

>We should have a "partition driver", which will
>pass the requests it gets on to a lowlevel driver with an 
>offset.

That sounds exactly right to me.  Very much like the raw device driver.

But this driver should not read the partition table.  A user space program 
is perfectly capable of reading the partition table, and it can use ioctls 
to create partition devices and bind them to base devices.  Even 
overlapping if the user is serious.  This opens up a whole wide world of 
partitioning schemes.

Root filesystems are always a pain, but every SCSI root filesystem system 
I've seen (which is not many) uses an initial ramdisk and a fair amount of 
intelligence to load the proper SCSI adapter driver.  The initrd level 
system could easily establish the partitions too.

And a special case hack for simple systems whose original root filesystem 
really is on a SCSI partition would be OK by me.  It would be just like 
the hack that figures out which filesystem driver to use for that 
partition. (For those who don't know -- the kernel simply tries to mount 
with every bound-in filesystem driver it has until one succeeds.  And it 
has a special mount option that says "don't get excited if you're unable 
to do this mount -- I'm just fishing.")

Re: Inhibit auto-attach of scsi disks ?

[Bryan Henderson]

>>I should be able to control whether Linux
>> considers there to be partitions on my disk or not 
>
>Why do you want to?  Linux always offers you the entire disk anyway.

Others have given arcane examples of where this is useful, but I am coming 
at it from a philosophical point of view.  It's great that Linux can 
magically detect and make available partitions on the disk, but some of us 
like our systems to work with as little magic as possible.  It makes the 
system easier to understand, easier to diagnose when it doesn't work, and 
more flexible.

It's clear to me that the partitions are conceptually a layer above basic 
disks, so I'd like to see the layers separable.

Re: [PATCH] first cut at fixing unable to requeue with no outstanding commands

[Patrick Mansfield]

On Mon, Sep 30, 2002 at 08:38:49PM -0400, James Bottomley wrote:
> andmike@us.ibm.com said:
> > Are we guaranteed that blk_run_queues will be called for all types of
> > I/O? 
> 
> Pretty much, it looks like.  It's mainly triggered by I/O stuff in fs, 
> including the buffer functions, so I think the assumption that we will be 
> called eventually is good.
> 
> James

What about applications and devices that only send one IO at a time?
They could still hang.

We have: tape (st or osst), sg usage, partitioning, scanning, direct
use of the block device (i.e. dd if=/dev/sda), and probably others.

-- Patrick Mansfield

Re: [PATCH] first cut at fixing unable to requeue with no outstanding commands

[James Bottomley]

patmans@us.ibm.com said:
> What about applications and devices that only send one IO at a time?
> They could still hang.

> We have: tape (st or osst), sg usage, partitioning, scanning, direct
> use of the block device (i.e. dd if=/dev/sda), and probably others. 

I don't believe so.

Unplugging is a global thing.  The request function for a plugged queue will 
always be run, that's a guarantee, so using this approach, the queue will 
stall but never hang forever.

I think for a rejection of a command with none outstanding, a stall is what 
you want (give the host/device time to recover from whatever the problem is).  
The length of the stall will be dependent on I/O pressure in the system, which 
is also roughly what you want: we can afford to give a device quite a while to 
recover if we're not desparate to get I/O to it. We can also handle BUSY 
returns this way too...

James

Inhibit auto-attach of scsi disks ?

[Scott Merritt]

> patmans@us.ibm.com said:
> use of the block device (i.e. dd if=/dev/sda), and probably others. 

I've noticed (at least in LK 2.4.19) that the scsi system will automatically "attach" any scsi disk-type device that it finds while scanning the scsi bus for new devices.  This includes issuing read capacity commands and attempting to read and parse that device's partition table.

I was wondering if there has been any discussion of a method/option by which a user could inhibit the automatic attachment of disk units ?  This might be desired if the users intent was to reformat, repartition, or perform other low-level activities.

Thanks, Scott.

Re: Inhibit auto-attach of scsi disks ?

[Alan Cox]

You can do all those things while it is attached as a disk device.

Alan

Re: Inhibit auto-attach of scsi disks ?

[Scott Merritt]

Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> You can do all those things while it is attached as a disk device.

Yes - my concerns are perhaps more "artistic" (and possibly misguided).  If I am mounting a disk for low level maintenance, it may not have a valid partition table and I may not appreciate the Syslog warnings related to the partition table.  Furthermore, all though I haven't investigated, it seems like it might be tricky to determine that I have altered the partition table and that /dev/sdb6 is no longer a valid partition.  To me, it just seemed that giving the user/administrator some way to control/inhibit the auto-attachment would be a "cleaner" solution to the problem - but that's just one man's opinion ... :)

Regards, Scott.

Re: Inhibit auto-attach of scsi disks ?

[Doug Ledford]

On Tue, Oct 01, 2002 at 09:49:37PM -0400, Scott Merritt wrote:
> 
> Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > You can do all those things while it is attached as a disk device.
> 
> Yes - my concerns are perhaps more "artistic" (and possibly misguided).  If I am mounting a disk for low level maintenance, it may not have a valid partition table and I may not appreciate the Syslog warnings related to the partition table.  Furthermore, all though I haven't investigated, it seems like it might be tricky to determine that I have altered the partition table and that /dev/sdb6 is no longer a valid partition.  To me, it just seemed that giving the user/administrator some way to control/inhibit the auto-attachment would be a "cleaner" solution to the problem - but that's just one man's opinion ... :)

You are confusing auto attachment with auto mounting.  When you do the low 
level things you suggest, the disk should not be *mounted* but it *must* 
be attached (or else there won't be a scsi device for you to open and tell 
to format itself for instance).  Same goes for fdisk partition operations, 
the disk shouldn't be mounted, but it must be attached for fdisk to be 
able to write to it.  Then, when the partition tables are rewritten, fdisk 
tells the kernel to reread the tables from disk, which the kernel happily 
does as long as no one currently has any of the previous partitions still 
mounted.

-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: Inhibit auto-attach of scsi disks ?

[Scott Merritt]

On Tue, 1 Oct 2002 21:58:38 -0400
Doug Ledford <dledford@redhat.com> wrote:

> You are confusing auto attachment with auto mounting.  When you do the low 
> level things you suggest, the disk should not be *mounted* but it *must* 
> be attached (or else there won't be a scsi device for you to open and tell 
> to format itself for instance).

Yes, I would certainly agree that the scsi disk must first be "attached".  However, I'm not entirely clear on the need to *always* read/parse the partition table as an integral part of "attaching" the scsi disk.

Assuming that I understand you correctly, I think this is simply a slightly different "artistic" view.  Under the assumption that an "early" read of the partition table does no "damage", there is clearly no compelling reason to change.  About the best argument I could come up with is some type of hypothetical disk device that would appreciate a firmware/parameter download from the "main" filesystem before being forced to perform it's first data operation.  An admittedly tenuous example ... :)

Best regards, Scott.

Re: Inhibit auto-attach of scsi disks ?

[Alan Cox]

You need it attached to write the partition table.