generating a Linux WWN?

generating a Linux WWN?

[Jeff Garzik]

Is there an accepted way to generate a SAS address, when the adapter 
does not supply one (NVRAM invalid or missing, etc.)?

Unless somebody complains, I was just planning to use 
get_random_bytes().  But maybe Linux has an IEEE id we can use, to make 
the practice a bit more legitimate and avoid stomping on others?  Or an 
IEEE id specifically designed for generated-address purposes?

	Jeff

Re: generating a Linux WWN?

[Michael Reed]

Jeff Garzik wrote:
> Is there an accepted way to generate a SAS address, when the adapter
> does not supply one (NVRAM invalid or missing, etc.)?
> 
> Unless somebody complains, I was just planning to use

I'm not complaining.  I'm circulating this for discussion with our
SAS hardware provider and internally.  Give us a little time to
mull this over.

Thanks,
 Mike


> get_random_bytes().  But maybe Linux has an IEEE id we can use, to make
> the practice a bit more legitimate and avoid stomping on others?  Or an
> IEEE id specifically designed for generated-address purposes?
> 
>     Jeff
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: generating a Linux WWN?

[James Smart]

Uh, although this may work very well for small constrained configs, as one
who debugs larger environments (and things always grow or get connected in
ways you don't expect), depending on a random number for uniqueness makes
me very unsettled. Debugging that small-percentage potential, when/if it
does occur can be an absolute nightmare with all kinds of weird behavior.
Rarely will you solve it from the context of a single server. There are
lots of ways to cheat, but I believe they all have to be based on some
small part that is guaranteed to be unique (like the IEEE id's).

-- james s

Jeff Garzik wrote:
> Is there an accepted way to generate a SAS address, when the adapter 
> does not supply one (NVRAM invalid or missing, etc.)?
> 
> Unless somebody complains, I was just planning to use 
> get_random_bytes().  But maybe Linux has an IEEE id we can use, to make 
> the practice a bit more legitimate and avoid stomping on others?  Or an 
> IEEE id specifically designed for generated-address purposes?
> 
>     Jeff
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

RE: generating a Linux WWN?

[Patrick_Boyd]

Unfortunately, it looks like IEEE doesn't have any OID's registered for
Linux or other reserved areas
(http://standards.ieee.org/regauth/oui/oui.txt). However, it does look
like they go in order... so maybe if you used an OID of 0xFFFFFF you
could at least guarantee that you didn't conflict with any company's SAS
WWNs.

Patrick Boyd

James Smart wrote:
>Uh, although this may work very well for small constrained configs, as
one
>who debugs larger environments (and things always grow or get connected
in
>ways you don't expect), depending on a random number for uniqueness
makes
>me very unsettled. Debugging that small-percentage potential, when/if
it
>does occur can be an absolute nightmare with all kinds of weird
behavior.
>Rarely will you solve it from the context of a single server. There are
>lots of ways to cheat, but I believe they all have to be based on some
>small part that is guaranteed to be unique (like the IEEE id's).
>
>-- james s
>
>Jeff Garzik wrote:
>> Is there an accepted way to generate a SAS address, when the adapter 
>> does not supply one (NVRAM invalid or missing, etc.)?
>> 
>> Unless somebody complains, I was just planning to use 
>> get_random_bytes().  But maybe Linux has an IEEE id we can use, to
make 
>> the practice a bit more legitimate and avoid stomping on others?  Or
an 
>> IEEE id specifically designed for generated-address purposes?
>> 
>>     Jeff
>> 
>> 
>> 
>> -
>> To unsubscribe from this list: send the line "unsubscribe linux-scsi"
in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>-
>To unsubscribe from this list: send the line "unsubscribe linux-scsi"
in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: generating a Linux WWN?

[Matthew Wilcox]

On Thu, Sep 27, 2007 at 09:16:13AM -0500, Patrick_Boyd@Dell.com wrote:
> Unfortunately, it looks like IEEE doesn't have any OID's registered for
> Linux or other reserved areas
> (http://standards.ieee.org/regauth/oui/oui.txt). However, it does look
> like they go in order... so maybe if you used an OID of 0xFFFFFF you
> could at least guarantee that you didn't conflict with any company's SAS
> WWNs.

It's something that happens frequently enough that we should come up
with a proper way of handling this.  I heard a story of someone at HP
taking an old computer, reading the MAC address from the motherboard,
then snapping the board in half.  I suppose if you're going to use a MAC
address from a 10Mbit ethernet card for a SAS WWN, there's no chance of
conflict, but still ... 3com might choose to do the same thing, and then
we're in trouble.

I don't have a good solution for WWN assignment.  Even if we get a
24-bit OID assignment for 'software use' or something, how do we control
the use withi the SAN to be sure we get no overlapping WWNs?

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: generating a Linux WWN?

[Michael Reed]

Matthew Wilcox wrote:
> On Thu, Sep 27, 2007 at 09:16:13AM -0500, Patrick_Boyd@Dell.com wrote:
>> > Unfortunately, it looks like IEEE doesn't have any OID's registered for
>> > Linux or other reserved areas
>> > (http://standards.ieee.org/regauth/oui/oui.txt). However, it does look
>> > like they go in order... so maybe if you used an OID of 0xFFFFFF you
>> > could at least guarantee that you didn't conflict with any company's SAS
>> > WWNs.
> 
> It's something that happens frequently enough that we should come up
> with a proper way of handling this.  I heard a story of someone at HP
> taking an old computer, reading the MAC address from the motherboard,
> then snapping the board in half.  I suppose if you're going to use a MAC
> address from a 10Mbit ethernet card for a SAS WWN, there's no chance of
> conflict, but still ... 3com might choose to do the same thing, and then
> we're in trouble.

Record the WWN of your SAS / FC port so that if/when it goes missing you
can put it back?  Have spares on site?

> 
> I don't have a good solution for WWN assignment.  Even if we get a
> 24-bit OID assignment for 'software use' or something, how do we control
> the use withi the SAN to be sure we get no overlapping WWNs?

Your point here is that overriding a WWN assignment is asking for trouble.


Mike

Re: generating a Linux WWN?

[Michael Reed]

Michael Reed wrote:
> 
> Matthew Wilcox wrote:
>> On Thu, Sep 27, 2007 at 09:16:13AM -0500, Patrick_Boyd@Dell.com wrote:
>>>> Unfortunately, it looks like IEEE doesn't have any OID's registered for
>>>> Linux or other reserved areas
>>>> (http://standards.ieee.org/regauth/oui/oui.txt). However, it does look
>>>> like they go in order... so maybe if you used an OID of 0xFFFFFF you
>>>> could at least guarantee that you didn't conflict with any company's SAS
>>>> WWNs.
>> It's something that happens frequently enough that we should come up
>> with a proper way of handling this.  I heard a story of someone at HP
>> taking an old computer, reading the MAC address from the motherboard,
>> then snapping the board in half.  I suppose if you're going to use a MAC
>> address from a 10Mbit ethernet card for a SAS WWN, there's no chance of
>> conflict, but still ... 3com might choose to do the same thing, and then
>> we're in trouble.
> 
> Record the WWN of your SAS / FC port so that if/when it goes missing you
> can put it back?  Have spares on site?

Our vendor recommends fixing the board, not applying a band-aid.  A random
WWN will potentially not work as the board firmware does some level of
sanity checking.  YMMV.

Mike

> 
>> I don't have a good solution for WWN assignment.  Even if we get a
>> 24-bit OID assignment for 'software use' or something, how do we control
>> the use withi the SAN to be sure we get no overlapping WWNs?
> 
> Your point here is that overriding a WWN assignment is asking for trouble.
> 
> 
> Mike
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: generating a Linux WWN?

[Jeff Garzik]

Michael Reed wrote:
> Record the WWN of your SAS / FC port so that if/when it goes missing you
> can put it back?  Have spares on site?

Linux:  not just for million dollar data centers any more :)

(sorry, couldn't resist :))

More seriously, SAS will be making an increasing appearance on commodity 
hardware, through the new SAS/SATA chips that are just starting to make 
their appearance (like the chips I'm working on).  And Marvell and 
Broadcom are big guys.  Just wait until the tiny Taiwanese shops start 
cranking out SAS/SATA chips.

With that, you'll get all the joys of low-cost volume production, and 
the user field foibles that come with it.

	Jeff

Re: generating a Linux WWN?

[Jeff Garzik]

Matthew Wilcox wrote:
> On Thu, Sep 27, 2007 at 09:16:13AM -0500, Patrick_Boyd@Dell.com wrote:
>> Unfortunately, it looks like IEEE doesn't have any OID's registered for
>> Linux or other reserved areas
>> (http://standards.ieee.org/regauth/oui/oui.txt). However, it does look
>> like they go in order... so maybe if you used an OID of 0xFFFFFF you
>> could at least guarantee that you didn't conflict with any company's SAS
>> WWNs.
> 
> It's something that happens frequently enough that we should come up
> with a proper way of handling this.  I heard a story of someone at HP
> taking an old computer, reading the MAC address from the motherboard,
> then snapping the board in half.  I suppose if you're going to use a MAC
> address from a 10Mbit ethernet card for a SAS WWN, there's no chance of
> conflict, but still ... 3com might choose to do the same thing, and then
> we're in trouble.
> 
> I don't have a good solution for WWN assignment.  Even if we get a
> 24-bit OID assignment for 'software use' or something, how do we control
> the use withi the SAN to be sure we get no overlapping WWNs?

To a perfectionist, there are no good answers besides "try your best."

The popular solutions I've seen are:

1) Random bytes, or fixed id + random bytes.  Even on a worldwide net, 
collisions are highly unlikely.

2) [pseudo-]hash of reasonably static properties of your platform.

3) Admin makes a site-specific decision, blending cases #1 and #2 with 
other schemes such as site-defined numbering.

In the ethernet world, all the drivers permit programmable MAC 
addresses, and many generate a random MAC if one is unavailable through 
other means.  In practice this is deployed rarely, though things like 
bonding and seamless failover are making much more use of programmable 
MAC addresses than ever before.

For case #2, one might consider creating a WWN for an ATA device that 
lacks one (they often do, even with a defined WWN field), by hashing the 
vendor id, firmware rev, and media serial number together.  But even 
with such hueristics, persistence and uniqueness are difficult to recreate.

With either case 1 or 2 -- both have seen success in the field -- you 
are essentially relying on luck after a certain point.  And that's life.

	Jeff

Re: generating a Linux WWN?

[Andi Kleen]

> The popular solutions I've seen are:
> 
> 1) Random bytes, or fixed id + random bytes.  Even on a worldwide net, 
> collisions are highly unlikely.

The problem is that the random bytes are not necessarily random; especially
not at boot:

get_random_bytes gets its randomness from the entropy pool. The pool starts with the 
time of the day and some always the same data and then gets feed by time 
stamps from a few interrupts. This doesn't include network interrupts
(or rather only a small handfull of network drivers set the flag -- but
at early boot you don't have any network anyways), but is primarily keyboard/mouse 
and storage interrupts. On headless systems typically only storage interrupts.

Now do you see the problem with using get_random_bytes() to get your
storage working? In many cases the time of the day in the pool
will prevent the worst, but if you have a cluster where all boxes boot 
at the same time it's quite likely your early randomness will
be all the same.(ok cluster boots are usually staggered, but you
still have a large number of systems booting at the same time) 

The whole thing would probably work if you could mix something 
machine unique into the entropy pool at boot (e.g. a network
mac address); but Linux can't do that do to boot strap ordering
issues.

I agree with the people who say this is a highly dangerous
thing to do. It is much better to fail clearly that to create a hard
to debug subtle problem in a storage network.

If you have prototype hardware that doesn't supply an unique ID then
I would suggest you add some module parameter to allow choosing a unique 
(or even random) one. But don't make it default.

-Andi

Re: generating a Linux WWN?

[Jeff Garzik]

James Smart wrote:
> Uh, although this may work very well for small constrained configs, as one
> who debugs larger environments (and things always grow or get connected in
> ways you don't expect), depending on a random number for uniqueness makes
> me very unsettled. Debugging that small-percentage potential, when/if it
> does occur can be an absolute nightmare with all kinds of weird behavior.
> Rarely will you solve it from the context of a single server. There are
> lots of ways to cheat, but I believe they all have to be based on some
> small part that is guaranteed to be unique (like the IEEE id's).

That's the ideal world, sure :)

But the reality is, just like with ethernet MAC addresses, you don't 
always have one when you "supposed to" have one.

There are obvious limitations and problems arising from a generated WWN 
-- particularly for an HBA, where serial numbers and other unique 
identifiers are scarce.  But that doesn't wish the missing-WWN problem 
away.  Decades of networking experience have taught us a few things :)

As an aside, we also need a way to override the HBA's WWN, just like we 
do with ethernet MAC address.

	Jeff

Re: generating a Linux WWN?

[Michael Reed]

Jeff Garzik wrote:
> James Smart wrote:
>> Uh, although this may work very well for small constrained configs, as one
>> who debugs larger environments (and things always grow or get connected in
>> ways you don't expect), depending on a random number for uniqueness makes
>> me very unsettled. Debugging that small-percentage potential, when/if it
>> does occur can be an absolute nightmare with all kinds of weird behavior.
>> Rarely will you solve it from the context of a single server. There are
>> lots of ways to cheat, but I believe they all have to be based on some
>> small part that is guaranteed to be unique (like the IEEE id's).
> 
> That's the ideal world, sure  :) 
> 
> But the reality is, just like with ethernet MAC addresses, you don't always have one when you "supposed to" have one.
> 

I'm curious about the problem you're trying to solve.  Generally a missing WWN,
as having a unique WWN should be a product requirement, is an indicator of
a problem.  Shouldn't the proper solution be to have the card repaired?
Arbitrarily assigning a WWN could mask or introduce other problems, in particular
with regard to storage connectivity.

> There are obvious limitations and problems arising from a generated WWN -- particularly for an HBA, where serial numbers and other unique identifiers are scarce.  But that doesn't wish the missing-WWN problem away.  Decades of networking experience have taught us a few things  :) 
> 
> As an aside, we also need a way to override the HBA's WWN, just like we do with ethernet MAC address.

We do?  Perhaps if it's corrupt and you know the correct value.  But, in that case
shouldn't the card be replaced?  When would you need to override the default WWN?

Linux: it's not just for desktops anymore.  :)

Mike


> 
>     Jeff
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: generating a Linux WWN?

[Douglas Gilbert]

Michael Reed wrote:
> 
> Jeff Garzik wrote:
>> James Smart wrote:
>>> Uh, although this may work very well for small constrained configs, as one
>>> who debugs larger environments (and things always grow or get connected in
>>> ways you don't expect), depending on a random number for uniqueness makes
>>> me very unsettled. Debugging that small-percentage potential, when/if it
>>> does occur can be an absolute nightmare with all kinds of weird behavior.
>>> Rarely will you solve it from the context of a single server. There are
>>> lots of ways to cheat, but I believe they all have to be based on some
>>> small part that is guaranteed to be unique (like the IEEE id's).
>> That's the ideal world, sure  :) 
>>
>> But the reality is, just like with ethernet MAC addresses, you don't always have one when you "supposed to" have one.
>>
> 
> I'm curious about the problem you're trying to solve.  Generally a missing WWN,
> as having a unique WWN should be a product requirement, is an indicator of
> a problem.  Shouldn't the proper solution be to have the card repaired?
> Arbitrarily assigning a WWN could mask or introduce other problems, in particular
> with regard to storage connectivity.

I have come across two types of problems:
  a) the scsi_debug driver looks like one or more logical units
     behind one or more dual ported SAS devices. All generated
     SAS address (for target ports, target devices and lus)
     should be distinct and preferably world wide unique.
     [My scsi_ses driver has a similar problem.]
  b) SAS HBAs that can run in target mode don't have enough
     unique SAS addresses allocated. A SAS initiator (the HBA's
     normal mode) only needs a couple a SAS addresses (e.g.
     3: one for the 4 internal phys, another for the 4
     external phys and 1 for the SAS initiator _device_
     (but I have never seen that done properly)). When you turn
     the SAS HBA around as a target (or initiator/target for
     bonus points) then each logical unit connected to or
     simulated by the SAS target needs a unique SAS address.

Years ago I asked for linux to get a IEEE OID. In the absence
of that perhaps we should use 0xffffff. FreeBSD, OpenBSD etc
face a similar problem.

Doug Gilbert

>> There are obvious limitations and problems arising from a generated WWN -- particularly for an HBA, where serial numbers and other unique identifiers are scarce.  But that doesn't wish the missing-WWN problem away.  Decades of networking experience have taught us a few things  :) 
>>
>> As an aside, we also need a way to override the HBA's WWN, just like we do with ethernet MAC address.
> 
> We do?  Perhaps if it's corrupt and you know the correct value.  But, in that case
> shouldn't the card be replaced?  When would you need to override the default WWN?
> 
> Linux: it's not just for desktops anymore.  :)
> 
> Mike
> 
> 
>>     Jeff

Re: generating a Linux WWN?

[Jeff Garzik]

Douglas Gilbert wrote:
> I have come across two types of problems:
>   a) the scsi_debug driver looks like one or more logical units
>      behind one or more dual ported SAS devices. All generated
>      SAS address (for target ports, target devices and lus)
>      should be distinct and preferably world wide unique.
>      [My scsi_ses driver has a similar problem.]
>   b) SAS HBAs that can run in target mode don't have enough
>      unique SAS addresses allocated. A SAS initiator (the HBA's
>      normal mode) only needs a couple a SAS addresses (e.g.
>      3: one for the 4 internal phys, another for the 4
>      external phys and 1 for the SAS initiator _device_
>      (but I have never seen that done properly)). When you turn
>      the SAS HBA around as a target (or initiator/target for
>      bonus points) then each logical unit connected to or
>      simulated by the SAS target needs a unique SAS address.
> 
> Years ago I asked for linux to get a IEEE OID. In the absence
> of that perhaps we should use 0xffffff. FreeBSD, OpenBSD etc
> face a similar problem.


In absence of a better solution, I would be quite happy to start a 
precedent of using a high identifier -- but not 0xffffff.

Maybe we could start using 0xFE for open source operating systems. 
FE-00-00 for Linux, FE-00-01 for OpenBSD, etc.

Once it catches on, assign IANA the task of managing the FE-XX-XX 
numberspace.  The IEEE yearly fee alone for an identifier makes it 
unrealistic for open source operating systems, otherwise.

	Jeff

Re: generating a Linux WWN?

[Jeff Garzik]

Michael Reed wrote:
> I'm curious about the problem you're trying to solve.  Generally a missing WWN,

I'm writing two SAS HBA drivers, Marvell 6440 and Broadcom 8603/HT1100, 
and must deal with this scenario.


> as having a unique WWN should be a product requirement, is an indicator of
> a problem.  Shouldn't the proper solution be to have the card repaired?
> Arbitrarily assigning a WWN could mask or introduce other problems, in particular
> with regard to storage connectivity.

If you are an admin suffering from the /common/ problem of missing or 
invalid NVRAM, would you rather wait for card repair or get it going 
immediately?

The standard Linux policy for unique ids like ethernet MAC has been, for 
the 98% case where you have a useful MAC from the hardware, of course 
use it.  For the 2% case, you give the admin solutions for supplying a 
useful MAC.

Mirroring decades of ethernet NIC history, the HBA silicon and the 
source of the unique id are most often vastly different.  Normally it is 
a different chip on the same board, but sometimes the unique id source 
is not on the same board at all.

For those reasons and more, Linux users have a LONG history of running 
into situations where, for one reason or another, their silicon is fine 
but their unique id is not.  I've heard them all:
	- hw from government auction, had NVRAM intentionally wiped
	- hw resale w/ NVRAM wiped by paranoid corporate IT staff
	- hw with NVRAM chip missing
	- old hw out of warranty
	- old dev kit (they rarely ship with useful NVRAM data)
	- and plenty more

We have to behave sanely in the 2% case.  "refuse to operate" is just 
not an option, when this problem case is so historically clear, and so 
easy to work around.

Linux drivers have to continue operating long after support contracts 
run out, long after chip EOL, and sometimes, long after companies go out 
of business.

	Jeff

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> Michael Reed wrote:
> 
> > as having a unique WWN should be a product requirement, is an indicator of
> > a problem.  Shouldn't the proper solution be to have the card repaired?
> > Arbitrarily assigning a WWN could mask or introduce other problems, in particular
> > with regard to storage connectivity.

This is absolutely correct.

> If you are an admin suffering from the /common/ problem of missing or 
> invalid NVRAM, would you rather wait for card repair or get it going 
> immediately?

If you have a SAS HA, and it is missing its NVRAM (erased, etc), then
this is something catastrophic, and you have much bigger problems to solve
than just to "generate" a WWN.

It is extremely, extremely *uncommon* for SAS HA to be missing their
manufacturing sector/NVRAM, for products out in the field.  Internally
bound HAs, for testing, validation, etc, may be missing their MS/NVRAM,
but those products are never supposed to leave the testing/validation
process, without the "packager" writing their MS/NVRAM into the device.

> The standard Linux policy for unique ids like ethernet MAC has been, for 
> the 98% case where you have a useful MAC from the hardware, of course 
> use it.  For the 2% case, you give the admin solutions for supplying a 
> useful MAC.

So the same rule of principle applies to SAS HAs, to quote:
   "you give the admin solutions for supplying a useful WWN [was: MAC]"

If the SAS HA does NOT have a WWN, then you'll just complain in the
kernel log, and not start it up, unless the admin has provided one
via kernel/module parameter.

In the case where a WWN is missing, it is essential that userspace
provides a WWN to the driver, possibly via a kernel/module parameter,
and then the driver will use that WWN to assign to the HA.

But you MUST NOT establish policy by "generating" it internally in the
kernel.  I know it would be "cool", but it is unsound.  So instead,
establish mechanism.

> Mirroring decades of ethernet NIC history, the HBA silicon and the 
> source of the unique id are most often vastly different.  Normally it is 
> a different chip on the same board, but sometimes the unique id source 
> is not on the same board at all.
> 
> For those reasons and more, Linux users have a LONG history of running 
> into situations where, for one reason or another, their silicon is fine 
> but their unique id is not.  I've heard them all:
> 	- hw from government auction, had NVRAM intentionally wiped
> 	- hw resale w/ NVRAM wiped by paranoid corporate IT staff
> 	- hw with NVRAM chip missing
> 	- old hw out of warranty
> 	- old dev kit (they rarely ship with useful NVRAM data)
> 	- and plenty more
> 
> We have to behave sanely in the 2% case.  "refuse to operate" is just 
> not an option, when this problem case is so historically clear, and so 
> easy to work around.
> 
> Linux drivers have to continue operating long after support contracts 
> run out, long after chip EOL, and sometimes, long after companies go out 
> of business.

This still does not justify "let's generate it in the kernel".
If the MS/NVRAM is missing and thus there is no hw WWN, then it
should be left to the admin to provide one via a kernel/module option.

   Luben

Re: generating a Linux WWN?

[Jeff Garzik]

Luben Tuikov wrote:
> This still does not justify "let's generate it in the kernel".

Once an admin specifies to use an alternate WWN provision, the method 
used to obtain that WWN is an implementation detail and irrelevant.

	Jeff

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> Luben Tuikov wrote:
> > This still does not justify "let's generate it in the kernel".
> 
> Once an admin specifies to use an alternate WWN provision, the method 
> used to obtain that WWN is an implementation detail and irrelevant.

But you're arguing that the kernel should generate it to "help out"
SAS drivers with HW that has had it's MS/NVRAM erased or corrupted.

The kernel has no business generating SAS WWN.

   Luben

Re: generating a Linux WWN?

[Jeff Garzik]

Luben Tuikov wrote:
> --- Jeff Garzik <jeff@garzik.org> wrote:
>> Luben Tuikov wrote:
>>> This still does not justify "let's generate it in the kernel".
>> Once an admin specifies to use an alternate WWN provision, the method 
>> used to obtain that WWN is an implementation detail and irrelevant.
> 
> But you're arguing that the kernel should generate it to "help out"
> SAS drivers with HW that has had it's MS/NVRAM erased or corrupted.
> 
> The kernel has no business generating SAS WWN.

Perhaps you missed the English text "once an admin specifies" in the 
quoted email.

	Jeff

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> Luben Tuikov wrote:
> > --- Jeff Garzik <jeff@garzik.org> wrote:
> >> Luben Tuikov wrote:
> >>> This still does not justify "let's generate it in the kernel".
> >> Once an admin specifies to use an alternate WWN provision, the method 
> >> used to obtain that WWN is an implementation detail and irrelevant.
> > 
> > But you're arguing that the kernel should generate it to "help out"
> > SAS drivers with HW that has had it's MS/NVRAM erased or corrupted.
> > 
> > The kernel has no business generating SAS WWN.
> 
> Perhaps you missed the English text "once an admin specifies" in the 
> quoted email.

Was your "once an admin specifies" reply to my email
supposed to be "Oh, I see, generating a SAS WWN in the kernel
is not that good an idea after all.  I'll implement it
as a kernel/module option." ?

    Luben

Re: generating a Linux WWN?

[Jeff Garzik]

Luben Tuikov wrote:
> --- Jeff Garzik <jeff@garzik.org> wrote:
>> Luben Tuikov wrote:
>>> --- Jeff Garzik <jeff@garzik.org> wrote:
>>>> Luben Tuikov wrote:
>>>>> This still does not justify "let's generate it in the kernel".
>>>> Once an admin specifies to use an alternate WWN provision, the method 
>>>> used to obtain that WWN is an implementation detail and irrelevant.
>>> But you're arguing that the kernel should generate it to "help out"
>>> SAS drivers with HW that has had it's MS/NVRAM erased or corrupted.
>>>
>>> The kernel has no business generating SAS WWN.
>> Perhaps you missed the English text "once an admin specifies" in the 
>> quoted email.
> 
> Was your "once an admin specifies" reply to my email
> supposed to be "Oh, I see, generating a SAS WWN in the kernel
> is not that good an idea after all.  I'll implement it
> as a kernel/module option." ?

The admin will have the option to auto-generate a WWN, should they so 
desire.

	Jeff

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> Luben Tuikov wrote:
> > --- Jeff Garzik <jeff@garzik.org> wrote:
> >> Luben Tuikov wrote:
> >>> --- Jeff Garzik <jeff@garzik.org> wrote:
> >>>> Luben Tuikov wrote:
> >>>>> This still does not justify "let's generate it in the kernel".
> >>>> Once an admin specifies to use an alternate WWN provision, the method 
> >>>> used to obtain that WWN is an implementation detail and irrelevant.
> >>> But you're arguing that the kernel should generate it to "help out"
> >>> SAS drivers with HW that has had it's MS/NVRAM erased or corrupted.
> >>>
> >>> The kernel has no business generating SAS WWN.
> >> Perhaps you missed the English text "once an admin specifies" in the 
> >> quoted email.
> > 
> > Was your "once an admin specifies" reply to my email
> > supposed to be "Oh, I see, generating a SAS WWN in the kernel
> > is not that good an idea after all.  I'll implement it
> > as a kernel/module option." ?
> 
> The admin will have the option to auto-generate a WWN, should they so 
> desire.

What does that mean?  "auto-generate"?  By whom?
What if they don't "auto-generate" (whatever that means)?

Re: generating a Linux WWN?

[Jeff Garzik]

Luben Tuikov wrote:
> --- Jeff Garzik <jeff@garzik.org> wrote:
>> The admin will have the option to auto-generate a WWN, should they so 
>> desire.

> What does that mean?  "auto-generate"?  By whom?

The admin tells the kernel module to auto-generate a WWN, and the kernel 
module happily obliges.

The generation algorithm is whatever makes people happy.  I would 
probably pick a fixed prefix like 0x6C 0x69 0x63 ("lin"), something that 
doesn't conflict with IEEE org ids for a long time to come.  Then, 
get_random_bytes() or hash some useful machine characteristics for the 
rest of the bytes.


> What if they don't "auto-generate" (whatever that means)?

Well, you either have a WWN or you don't :)

	Jeff

Re: generating a Linux WWN?

[Douglas Gilbert]

Jeff Garzik wrote:
> Luben Tuikov wrote:
>> --- Jeff Garzik <jeff@garzik.org> wrote:
>>> The admin will have the option to auto-generate a WWN, should they so
>>> desire.
> 
>> What does that mean?  "auto-generate"?  By whom?
> 
> The admin tells the kernel module to auto-generate a WWN, and the kernel
> module happily obliges.
> 
> The generation algorithm is whatever makes people happy.  I would
> probably pick a fixed prefix like 0x6C 0x69 0x63 ("lin"), something that
> doesn't conflict with IEEE org ids for a long time to come.  Then,
> get_random_bytes() or hash some useful machine characteristics for the
> rest of the bytes.
> 
> 
>> What if they don't "auto-generate" (whatever that means)?
> 
> Well, you either have a WWN or you don't :)

Jeff,
It is not quite black and white. As I tried to point
out, when a SAS HBA is run in target mode, there is
a good chance there aren't _enough_ unique SAS addresses
allocated by the manufacturer.

There should be naa-5 addresses for the ports, it is
the target device address and 0 or more lu addresses that
may need to be generated.


Doug Gilbert

Re: generating a Linux WWN?

[Jeff Garzik]

Douglas Gilbert wrote:
> Jeff Garzik wrote:
>> Luben Tuikov wrote:
>>> --- Jeff Garzik <jeff@garzik.org> wrote:
>>>> The admin will have the option to auto-generate a WWN, should they so
>>>> desire.
>>> What does that mean?  "auto-generate"?  By whom?
>> The admin tells the kernel module to auto-generate a WWN, and the kernel
>> module happily obliges.
>>
>> The generation algorithm is whatever makes people happy.  I would
>> probably pick a fixed prefix like 0x6C 0x69 0x63 ("lin"), something that
>> doesn't conflict with IEEE org ids for a long time to come.  Then,
>> get_random_bytes() or hash some useful machine characteristics for the
>> rest of the bytes.
>>
>>
>>> What if they don't "auto-generate" (whatever that means)?
>> Well, you either have a WWN or you don't :)
> 
> Jeff,
> It is not quite black and white. As I tried to point
> out, when a SAS HBA is run in target mode, there is
> a good chance there aren't _enough_ unique SAS addresses
> allocated by the manufacturer.
> 
> There should be naa-5 addresses for the ports, it is
> the target device address and 0 or more lu addresses that
> may need to be generated.

Oh quite agreed ... I was definitely talking about the initiator.

The thread has come full circle again :)  I would definitely like to 
figure out a standard way to generate new SAS addresses.

	Jeff

Re: generating a Linux WWN?

[Matthew Jacob]

>
> The generation algorithm is whatever makes people happy.  I would
> probably pick a fixed prefix like 0x6C 0x69 0x63 ("lin"), something that
> doesn't conflict with IEEE org ids for a long time to come.  Then,
> get_random_bytes() or hash some useful machine characteristics for the
> rest of the bytes.

The "probably" here is kind of amusing.

Look, Linux development has the moolah (look at all the money IBM and
SGI and others have shoveled in)- why don't you get a Linux block of
OUIs from the IEEE and set up an RSS-like feed so the Open Source
community can get WWNs with that OUI and a incrementing serial number
as needed- that would be a *real* service. I've thought about doing
this, but I'd have to fork over at least 1700$USD to get an official
IEEE OUI and over the last ten years I've already spent far too much
of my own money on open source support.

Heck, maybe even the IEEE would comp you on OUIs.

Re: generating a Linux WWN?

[Jeff Garzik]

Matthew Jacob wrote:
>> The generation algorithm is whatever makes people happy.  I would
>> probably pick a fixed prefix like 0x6C 0x69 0x63 ("lin"), something that
>> doesn't conflict with IEEE org ids for a long time to come.  Then,
>> get_random_bytes() or hash some useful machine characteristics for the
>> rest of the bytes.
> 
> The "probably" here is kind of amusing.
> 
> Look, Linux development has the moolah (look at all the money IBM and
> SGI and others have shoveled in)- why don't you get a Linux block of
> OUIs from the IEEE and set up an RSS-like feed so the Open Source
> community can get WWNs with that OUI and a incrementing serial number
> as needed- that would be a *real* service. I've thought about doing
> this, but I'd have to fork over at least 1700$USD to get an official
> IEEE OUI and over the last ten years I've already spent far too much
> of my own money on open source support.
> 
> Heck, maybe even the IEEE would comp you on OUIs.

Certainly, I would love for this to happen!  Although my employer (Red 
Hat), SGI, IBM or whomever might be willing to tackle this, it might be 
more appropriate for the Linux Foundation.

Anybody wanna volunteer to pursue this?

	Jeff

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> Matthew Jacob wrote:
> >> The generation algorithm is whatever makes people happy.  I would
> >> probably pick a fixed prefix like 0x6C 0x69 0x63 ("lin"), something that
> >> doesn't conflict with IEEE org ids for a long time to come.  Then,
> >> get_random_bytes() or hash some useful machine characteristics for the
> >> rest of the bytes.
> > 
> > The "probably" here is kind of amusing.
> > 
> > Look, Linux development has the moolah (look at all the money IBM and
> > SGI and others have shoveled in)- why don't you get a Linux block of
> > OUIs from the IEEE and set up an RSS-like feed so the Open Source
> > community can get WWNs with that OUI and a incrementing serial number
> > as needed- that would be a *real* service. I've thought about doing
> > this, but I'd have to fork over at least 1700$USD to get an official
> > IEEE OUI and over the last ten years I've already spent far too much
> > of my own money on open source support.
> > 
> > Heck, maybe even the IEEE would comp you on OUIs.
> 
> Certainly, I would love for this to happen!  Although my employer (Red 
> Hat), SGI, IBM or whomever might be willing to tackle this, it might be 
> more appropriate for the Linux Foundation.
> 
> Anybody wanna volunteer to pursue this?

SAS WWN have nothing to do with Linux.

The kernel should not generate SAS WWN and assign them to SAS initiator
ports.  SAS WWNs should be given to SAS initiator ports and SAS target
ports by a storage management application which maintains and configures
the storage domain, zoning, permissions, connectivity, etc.

What is needed is persistence, regardless of reboots of what OS
is running on the host CPU/system.  SAS WWNs are properties of
the SAS target/initiator port, they are not properties of when
the host system was booted, or what OS is running on it.

Think about it this way: you don't change your name every morning
you wake up and MAC addresses are not generated by the kernel for
a network device which has none.  Those are persistent across
reboots or type of host OS.

   Luben

Re: generating a Linux WWN?

[Jeff Garzik]

Luben Tuikov wrote:
> What is needed is persistence, regardless of reboots of what OS
> is running on the host CPU/system.  SAS WWNs are properties of
> the SAS target/initiator port, they are not properties of when
> the host system was booted, or what OS is running on it.


As part of the previously described schemes, the admin is certainly 
allowed to choose an option (manual WWN specification) that enables an 
external system of persistance.

But such a system should not be a /requirement/, when all an admin might 
want to do is simply get their card working.

The admin knows whats best for their site, and may choose.

	Jeff

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> Luben Tuikov wrote:
> > What is needed is persistence, regardless of reboots of what OS
> > is running on the host CPU/system.  SAS WWNs are properties of
> > the SAS target/initiator port, they are not properties of when
> > the host system was booted, or what OS is running on it.
> 
> 
> As part of the previously described schemes, the admin is certainly 
> allowed to choose an option (manual WWN specification) that enables an 
> external system of persistance.
> 
> But such a system should not be a /requirement/, when all an admin might 
> want to do is simply get their card working.

It *is* a requirement, in fact at all levels of the product.

How would you query the storage domain (at boot time mind you) that
no such WWN exists, and/or will not exist when another storage component
is connected to or comes online, if you generated it in the kernel?

SAS ports not having a WWN as part of the MS/NVRAM is an *exception*,
and a bad one at that.  The driver should just complain and not bring
up the HA.

Your "want to get their card working" way of view is very
simplistic to justify generating and assigning SAS WWN in the kernel.
This is the job of the manufacturer/packager, not the host OS.

    Luben


> 
> The admin knows whats best for their site, and may choose.
> 
> 	Jeff

Re: generating a Linux WWN?

[David Miller]

From: Luben Tuikov <ltuikov@yahoo.com>
Date: Wed, 3 Oct 2007 15:08:48 -0700 (PDT)

> Your "want to get their card working" way of view is very
> simplistic to justify generating and assigning SAS WWN in the kernel.
> This is the job of the manufacturer/packager, not the host OS.

When you are thousands of miles away from the data center and lose all
of your storage and therefore can't boot correctly because the WWN
info is corrupted, you won't have this unbelievably fascist attitude
about this problem.

Give people an _OPTION_!

This is about as anti-social as when the Intel folks refused to
themselves put in a driver option to try to use an eepro100 card even
if the EEPROM was corrupted and had a bad checksum.

For the person who hits this, it's a big issue to have a way to still
try to bring things up.

If you don't provide this, you want people to suffer more than
necessary when something goes wrong, and that by definition makes you
an asshole.

Re: generating a Linux WWN?

[Luben Tuikov]

--- David Miller <davem@davemloft.net> wrote:
> From: Luben Tuikov <ltuikov@yahoo.com>
> Date: Wed, 3 Oct 2007 15:08:48 -0700 (PDT)
> 
> > Your "want to get their card working" way of view is very
> > simplistic to justify generating and assigning SAS WWN in the kernel.
> > This is the job of the manufacturer/packager, not the host OS.
> 
> When you are thousands of miles away from the data center and lose all
> of your storage and therefore can't boot correctly because the WWN
> info is corrupted, you won't have this unbelievably fascist attitude
> about this problem.
> 
> Give people an _OPTION_!
> 
> This is about as anti-social as when the Intel folks refused to
> themselves put in a driver option to try to use an eepro100 card even
> if the EEPROM was corrupted and had a bad checksum.
> 
> For the person who hits this, it's a big issue to have a way to still
> try to bring things up.
> 
> If you don't provide this, you want people to suffer more than
> necessary when something goes wrong, and that by definition makes you
> an asshole.

No, not all.

If the MS/NVRAM is corrupted (becomes corrupted) then it is better
for the admin to see that the system wouldn't boot or that the storage
domain is not visible (if the system doesn't boot from it), and provide
a sample WWN at kernel boot line or module load time, just to
connect to the storage network.

But having a WWN generator in the kernel, although not terribly
difficult to write, makes it possible to create an inconsistent
storage domain.  It is that possibility which troubles me,
due to the intention of SAS WWNs.

I guess much of this argument doesn't really matter, since
most SAS HA would really be a ROC, and as thus much more
would need to happen before the HA is useable, than just
having a MS/NVRAM with the WWN.

Here is what "modinfo aic94xx.ko" (the version I maintain)
says:

parm:           sas_addr_str:
        Provide a SAS address for a single host adapter.
        This is for testing only! You should not need to provide this!

    Luben

Re: generating a Linux WWN?

[Matthew Jacob]

> But having a WWN generator in the kernel, although not terribly
> difficult to write, makes it possible to create an inconsistent
> storage domain.  It is that possibility which troubles me,
> due to the intention of SAS WWNs.

But one should make sure that you *do* create a consistent storage
domain. If you can seed the system with a couple dozen free WWNs for
use when/if needed, why not?

People are content to believe that UUID generation is sufficiently
unique, and for the bit space it occupies that's probably a reasonable
clain. But you can't jam that into the smaller bit space WWNs.

For smaller bit spaces like 64 bit WWNs, you cannot, as you correctly
point out, generate reliably unique numbers all by youself. The
closest you could probably come is using a type 5 WWN with a known
single OUI and then use "seconds since January 1 2007" as a serial
number in the 36 bit VID space that gives you about 8 years before it
wraps- the likely collision rate here would be pretty low.

A much better choice is to get real stamped serial number WWNs. I also
hold with some of the other folks on this discussion that some of this
is policy that the admin should be allowed to choose. After they've
segmented the company's main fabric by choosing unwisely and
forgetting to practice safe zoning we'll choose to buy them a drink or
two.

Re: generating a Linux WWN?

[Jeff Garzik]

Matthew Jacob wrote:
> A much better choice is to get real stamped serial number WWNs. I also
> hold with some of the other folks on this discussion that some of this
> is policy that the admin should be allowed to choose. After they've
> segmented the company's main fabric by choosing unwisely and
> forgetting to practice safe zoning we'll choose to buy them a drink or
> two.

hehe :)

Let's not forget, too, that this entire thread is talking about the 1% case.

The common case everybody agrees on -- the manufacturer should provide a 
unique WWN for each board etc.  That's the 99% case.  And just like 
ethernet NICs, the system works as intended /most of the time/.

	Jeff

Re: generating a Linux WWN?

[Matthew Wilcox]

On Wed, Oct 03, 2007 at 08:23:14PM -0700, Matthew Jacob wrote:
> For smaller bit spaces like 64 bit WWNs, you cannot, as you correctly
> point out, generate reliably unique numbers all by youself. The
> closest you could probably come is using a type 5 WWN with a known
> single OUI and then use "seconds since January 1 2007" as a serial
> number in the 36 bit VID space that gives you about 8 years before it
> wraps- the likely collision rate here would be pretty low.

Unless you generate more than one WWN per second.  We could enforce that
by have an msleep(1000) in the WWN generator function ... which might
discourage people from using it except when they really had to ;-)
Of course, if you turn on two computers in the same zone at the same
time, you're still screwed.

-- 
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: generating a Linux WWN?

[James Bottomley]

On Wed, 2007-10-03 at 15:17 -0700, David Miller wrote:
> From: Luben Tuikov <ltuikov@yahoo.com>
> Date: Wed, 3 Oct 2007 15:08:48 -0700 (PDT)
> 
> > Your "want to get their card working" way of view is very
> > simplistic to justify generating and assigning SAS WWN in the kernel.
> > This is the job of the manufacturer/packager, not the host OS.
> 
> When you are thousands of miles away from the data center and lose all
> of your storage and therefore can't boot correctly because the WWN
> info is corrupted, you won't have this unbelievably fascist attitude
> about this problem.
> 
> Give people an _OPTION_!
> 
> This is about as anti-social as when the Intel folks refused to
> themselves put in a driver option to try to use an eepro100 card even
> if the EEPROM was corrupted and had a bad checksum.
> 
> For the person who hits this, it's a big issue to have a way to still
> try to bring things up.
> 
> If you don't provide this, you want people to suffer more than
> necessary when something goes wrong, and that by definition makes you
> an asshole.

The tenor of this debate is becoming slightly heated, so it needs to
cool down a notch.

For the record, what the current in-kernel aic94xx driver does for this
case is allow a parameter override to specify the WWN in the case where
the card burned in one has gone missing or is corrupt.  I think this is
the correctly balanced approach to the problem.

For the record, I pretty much agree completely with Luben's position on
this:  I want to allow the user a method to correct any problem (i.e.
supply a WWN override).  However, I also want them to think about the
issue before they do this.  What I really don't want is the driver
silently correcting what it thinks to be a defective WWN and generating
its own replacement because that's a recipe for disaster on a SAN (SANs
are a lot less robust than networks to duplicate MAC addresses: because
of the way expander routing is done, you can possibly collapse the
entire SAN with duplicate WWNs).

So, the procedure for a SAS card should be:

     1. Obtain the WWN from a device specific storage method (like
        flash)
     2. Replace this with a passed in command line parameter if one
        exists
     3. refuse to attach if there's still no valid WWN and give an
        explicit method saying what the problem is (and possibly how to
        fix it)

James

Re: generating a Linux WWN?

[David Miller]

From: James Bottomley <James.Bottomley@SteelEye.com>
Date: Fri, 05 Oct 2007 18:09:06 -0400

> For the record, what the current in-kernel aic94xx driver does for this
> case is allow a parameter override to specify the WWN in the case where
> the card burned in one has gone missing or is corrupt.  I think this is
> the correctly balanced approach to the problem.

Let's then provide a global and consistent knob for the user to set in
this situation, instead of a different one per-driver.

auto_wwn=1 or somthing like that

Re: generating a Linux WWN?

[James Bottomley]

On Fri, 2007-10-05 at 15:11 -0700, David Miller wrote:
> From: James Bottomley <James.Bottomley@SteelEye.com>
> Date: Fri, 05 Oct 2007 18:09:06 -0400
> 
> > For the record, what the current in-kernel aic94xx driver does for this
> > case is allow a parameter override to specify the WWN in the case where
> > the card burned in one has gone missing or is corrupt.  I think this is
> > the correctly balanced approach to the problem.
> 
> Let's then provide a global and consistent knob for the user to set in
> this situation, instead of a different one per-driver.

Agreed; we can shift the parameter to the transport class so it's
consistent for all SAS drivers

> auto_wwn=1 or somthing like that

I'd far prefer

override_wwn = <fully specific WWN>

since I assume auto_wwn means get the kernel to generate one?

James

Re: generating a Linux WWN?

[David Miller]

From: James Bottomley <James.Bottomley@SteelEye.com>
Date: Fri, 05 Oct 2007 18:14:48 -0400

> On Fri, 2007-10-05 at 15:11 -0700, David Miller wrote:
> > auto_wwn=1 or somthing like that
> 
> I'd far prefer
> 
> override_wwn = <fully specific WWN>
> 
> since I assume auto_wwn means get the kernel to generate one?

I think providing both possibilities (kernel auto-generated and user
specified) is appropriate.

Re: generating a Linux WWN?

[Jeff Garzik]

David Miller wrote:
> I think providing both possibilities (kernel auto-generated and user
> specified) is appropriate.

And that's been my plan from day one...  which is remarkably a lot like 
the behavior of several net drivers.  :)

* attempt to read WWN during module load
* admin may optionally choose to manually specify OR auto-generate a WWN
* fail, if no WWN [stating the obvious]

Regards,

	Jeff

Re: generating a Linux WWN?

[David Miller]

From: Jeff Garzik <jeff@garzik.org>
Date: Fri, 05 Oct 2007 18:41:40 -0400

> And that's been my plan from day one...  which is remarkably a lot like 
> the behavior of several net drivers.  :)
> 
> * attempt to read WWN during module load
> * admin may optionally choose to manually specify OR auto-generate a WWN
> * fail, if no WWN [stating the obvious]

And like with networking cases, we have situations where a platform
might be able to assist.

There are several systems that provide a "system MAC address"
to use when you can't obtain one for a specific device.  This
valus is provided via a firmware property.

Similarly, we could ask an interface that goes to platform to
ask them for a WWN.

We have code which does this in some of the qlogic FC drivers
already.

Re: generating a Linux WWN?

[Jeff Garzik]

David Miller wrote:
> From: Jeff Garzik <jeff@garzik.org>
> Date: Fri, 05 Oct 2007 18:41:40 -0400
> 
>> And that's been my plan from day one...  which is remarkably a lot like 
>> the behavior of several net drivers.  :)
>>
>> * attempt to read WWN during module load
>> * admin may optionally choose to manually specify OR auto-generate a WWN
>> * fail, if no WWN [stating the obvious]
> 
> And like with networking cases, we have situations where a platform
> might be able to assist.
> 
> There are several systems that provide a "system MAC address"
> to use when you can't obtain one for a specific device.  This
> valus is provided via a firmware property.
> 
> Similarly, we could ask an interface that goes to platform to
> ask them for a WWN.

Good point -- I bet I will run into this when I get a Broadcom HT-1100 
test system, which is the on-board version of the board supported by my 
new 'broadsas' driver.

	Jeff

Re: generating a Linux WWN?

[James Bottomley]

On Fri, 2007-10-05 at 15:17 -0700, David Miller wrote:
> From: James Bottomley <James.Bottomley@SteelEye.com>
> Date: Fri, 05 Oct 2007 18:14:48 -0400
> 
> > On Fri, 2007-10-05 at 15:11 -0700, David Miller wrote:
> > > auto_wwn=1 or somthing like that
> > 
> > I'd far prefer
> > 
> > override_wwn = <fully specific WWN>
> > 
> > since I assume auto_wwn means get the kernel to generate one?
> 
> I think providing both possibilities (kernel auto-generated and user
> specified) is appropriate.

My problem with auto generated is that it's provably impossible to
generate globally unique numbers for WWNs without some internal source
of uniqueness (I know sparcs have this in their serial number, but most
PCs unfortunately don't).

I know the auto generated number can be statistically reasonably unique,
but sysadmins are lazy people.  If they run into this problem, they'll
take the knob with the on/off switch rather than the think about the
problem and specify the full WWN; and then, being busy people, they'll
forget about it as "problem solved".  When they do this, statistically
(and probably years later) there will be a cluster reboot where the
entire SAN simply collapses and no-one knows why ... the poor SAN
administrator will likely spend weeks working out the problem is.

I'd rather not give an interface which seems like a magic bullet but
which is a potential time bomb in a cluster (particularly when it has
such a long fuse).

James

Re: generating a Linux WWN?

[Jeff Garzik]

James Bottomley wrote:
> My problem with auto generated is that it's provably impossible to
> generate globally unique numbers for WWNs without some internal source
> of uniqueness (I know sparcs have this in their serial number, but most
> PCs unfortunately don't).
> 
> I know the auto generated number can be statistically reasonably unique,
> but sysadmins are lazy people.  If they run into this problem, they'll
> take the knob with the on/off switch rather than the think about the
> problem and specify the full WWN; and then, being busy people, they'll
> forget about it as "problem solved".  When they do this, statistically
> (and probably years later) there will be a cluster reboot where the
> entire SAN simply collapses and no-one knows why ... the poor SAN
> administrator will likely spend weeks working out the problem is.

Why, if we give lazy administrators root access, that's all they'll use, 
and they will just think "problem solved" until a serious security issue 
arises that takes down the cluster.

See how silly and un-Linux that logic is?  In Linux, the admin has the 
power to make stupid decisions -- or to make informed decisions that 
disagree your rigid "an admin should never do that" line of thought. 
It's their hardware.

You're also using the 1% case of a 1% case of a 1% case to argue against 
a feature that is useful in making things Just Work(tm).

	Jeff

Re: generating a Linux WWN?

[James Bottomley]

On Sat, 2007-10-06 at 10:36 -0400, Jeff Garzik wrote:
> James Bottomley wrote:
> > My problem with auto generated is that it's provably impossible to
> > generate globally unique numbers for WWNs without some internal source
> > of uniqueness (I know sparcs have this in their serial number, but most
> > PCs unfortunately don't).
> > 
> > I know the auto generated number can be statistically reasonably unique,
> > but sysadmins are lazy people.  If they run into this problem, they'll
> > take the knob with the on/off switch rather than the think about the
> > problem and specify the full WWN; and then, being busy people, they'll
> > forget about it as "problem solved".  When they do this, statistically
> > (and probably years later) there will be a cluster reboot where the
> > entire SAN simply collapses and no-one knows why ... the poor SAN
> > administrator will likely spend weeks working out the problem is.
> 
> Why, if we give lazy administrators root access, that's all they'll use, 
> and they will just think "problem solved" until a serious security issue 
> arises that takes down the cluster.
> 
> See how silly and un-Linux that logic is?  In Linux, the admin has the 
> power to make stupid decisions -- or to make informed decisions that 
> disagree your rigid "an admin should never do that" line of thought. 
> It's their hardware.
> 
> You're also using the 1% case of a 1% case of a 1% case to argue against 
> a feature that is useful in making things Just Work(tm).

So your use case for this feature is a savvy system admin whose going to
turn it on for a one time boot while he figures out what the correct WWN
override is and who will then immediately set the system up to override
properly on its next boot?  That's what seems to me to be the unlikely
scenario.

I know when I'm under the gun to solve a problem I'll do the first thing
I find that actually works.  So I'd rather only expose an interface that
allows them to get it right.

If you remember Rusty's guide to interfaces, this is a level 14 easy to
misuse interface: "The obvious use is wrong"; since the obvious use is
to put it in module parameters and have the problem go away (for
now ...).  Actually, I could be harsher and say it's level 17 "There's
no correct use" because statistically every time you use it, you expose
yourself to potential duplicate WWNs.

James

Re: generating a Linux WWN?

[Jeff Garzik]

James Bottomley wrote:
> If you remember Rusty's guide to interfaces, this is a level 14 easy to
> misuse interface: "The obvious use is wrong"; since the obvious use is
> to put it in module parameters and have the problem go away (for
> now ...).  Actually, I could be harsher and say it's level 17 "There's
> no correct use" because statistically every time you use it, you expose
> yourself to potential duplicate WWNs.


Now that you have said "there's no correct use" you have managed to 
logic yourself into silly-land.

That is utterly specious logic when duplicate WWNs are quite unlikely, 
and furthermore -- as demonstrated by use in network drivers -- use of 
the feature itself is not the common case.

Field experience directly contradicts this entire line of reasoning.

	Jeff

Re: generating a Linux WWN?

[James Bottomley]

On Sat, 2007-10-06 at 11:23 -0400, Jeff Garzik wrote:
> James Bottomley wrote:
> > If you remember Rusty's guide to interfaces, this is a level 14 easy to
> > misuse interface: "The obvious use is wrong"; since the obvious use is
> > to put it in module parameters and have the problem go away (for
> > now ...).  Actually, I could be harsher and say it's level 17 "There's
> > no correct use" because statistically every time you use it, you expose
> > yourself to potential duplicate WWNs.
> 
> 
> Now that you have said "there's no correct use" you have managed to 
> logic yourself into silly-land.
> 
> That is utterly specious logic when duplicate WWNs are quite unlikely, 
> and furthermore -- as demonstrated by use in network drivers -- use of 
> the feature itself is not the common case.
> 
> Field experience directly contradicts this entire line of reasoning.

OK, give me your use case.

James

Re: generating a Linux WWN?

[Jeff Garzik]

James Bottomley wrote:
> On Sat, 2007-10-06 at 11:23 -0400, Jeff Garzik wrote:
>> James Bottomley wrote:
>>> If you remember Rusty's guide to interfaces, this is a level 14 easy to
>>> misuse interface: "The obvious use is wrong"; since the obvious use is
>>> to put it in module parameters and have the problem go away (for
>>> now ...).  Actually, I could be harsher and say it's level 17 "There's
>>> no correct use" because statistically every time you use it, you expose
>>> yourself to potential duplicate WWNs.
>>
>> Now that you have said "there's no correct use" you have managed to 
>> logic yourself into silly-land.
>>
>> That is utterly specious logic when duplicate WWNs are quite unlikely, 
>> and furthermore -- as demonstrated by use in network drivers -- use of 
>> the feature itself is not the common case.
>>
>> Field experience directly contradicts this entire line of reasoning.
> 
> OK, give me your use case.

I already have, in this thread.

	Jeff

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> James Bottomley wrote:
> > If you remember Rusty's guide to interfaces, this is a level 14 easy to
> > misuse interface: "The obvious use is wrong"; since the obvious use is
> > to put it in module parameters and have the problem go away (for
> > now ...).  Actually, I could be harsher and say it's level 17 "There's
> > no correct use" because statistically every time you use it, you expose
> > yourself to potential duplicate WWNs.
> 
> 
> Now that you have said "there's no correct use" you have managed to 
> logic yourself into silly-land.
> 
> That is utterly specious logic when duplicate WWNs are quite unlikely, 
> and furthermore -- as demonstrated by use in network drivers -- use of 
> the feature itself is not the common case.
> 
> Field experience directly contradicts this entire line of reasoning.

You have SAS SAN's field experience?

   Luben

Re: generating a Linux WWN?

[David Miller]

From: James Bottomley <James.Bottomley@SteelEye.com>
Date: Sat, 06 Oct 2007 10:04:55 -0500

> If you remember Rusty's guide to interfaces, this is a level 14 easy to
> misuse interface: "The obvious use is wrong"; since the obvious use is
> to put it in module parameters and have the problem go away (for
> now ...).  Actually, I could be harsher and say it's level 17 "There's
> no correct use" because statistically every time you use it, you expose
> yourself to potential duplicate WWNs.

Every time you use GIT you expose yourself to potential duplicate SHAs
which will corrupt the tree.

This argument borders on lunacy, give it up :-)

Re: generating a Linux WWN?

[Michael Reed]

David Miller wrote:
> From: James Bottomley <James.Bottomley@SteelEye.com>
> Date: Sat, 06 Oct 2007 10:04:55 -0500
> 
>> If you remember Rusty's guide to interfaces, this is a level 14 easy to
>> misuse interface: "The obvious use is wrong"; since the obvious use is
>> to put it in module parameters and have the problem go away (for
>> now ...).  Actually, I could be harsher and say it's level 17 "There's
>> no correct use" because statistically every time you use it, you expose
>> yourself to potential duplicate WWNs.
> 
> Every time you use GIT you expose yourself to potential duplicate SHAs
> which will corrupt the tree.
> 
> This argument borders on lunacy, give it up :-)

As long as two systems on the same SAN are quite unlikely to wind up with
the same WWN.  override_wwn=5000000000000001 comes to mind.  Two months
later, another admin does the same thing to another system.  As Jeff has
pointed out, there really should be some method in place that allows for
some randomness in the wwn.  (Other than fixing the broken hardware. :)
Handing the admins the rope doesn't mean we want to see it around their necks!

And, while I'm thinking about it, whatever implementation is chosen, doesn't
it need to allow for the targeting of a particular sas host adapter and
perhaps even a particular sas port on the adapter?

Mike

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:

> James Bottomley wrote:
> > My problem with auto generated is that it's provably impossible to
> > generate globally unique numbers for WWNs without some internal source
> > of uniqueness (I know sparcs have this in their serial number, but most
> > PCs unfortunately don't).
> > 
> > I know the auto generated number can be statistically reasonably unique,
> > but sysadmins are lazy people.  If they run into this problem, they'll
> > take the knob with the on/off switch rather than the think about the
> > problem and specify the full WWN; and then, being busy people, they'll
> > forget about it as "problem solved".  When they do this, statistically
> > (and probably years later) there will be a cluster reboot where the
> > entire SAN simply collapses and no-one knows why ... the poor SAN
> > administrator will likely spend weeks working out the problem is.
> 
> Why, if we give lazy administrators root access, that's all they'll use, 
> and they will just think "problem solved" until a serious security issue 
> arises that takes down the cluster.
> 
> See how silly and un-Linux that logic is?  In Linux, the admin has the 
> power to make stupid decisions -- or to make informed decisions that 
> disagree your rigid "an admin should never do that" line of thought. 
> It's their hardware.
> 
> You're also using the 1% case of a 1% case of a 1% case to argue against 
> a feature that is useful in making things Just Work(tm).

What he's arguing about is the _capability_ for a node in a SAN
to *(auto)generate* WWN and assign it to itself (at every
reboot, etc).  This is considered a _rogue_ node and *no* SAN
architect or admin will tolerate such a node.

The problem here is NOT the _capability_ to assign a WWN.
Admin does have the capability to assign (if missing) and/or
override (if present) a WWN currently with aic94xx driver.
This is fine.

The problem is "(auto)generate".  Think of SANs.

   Luben

Re: generating a Linux WWN?

[David Miller]

From: James Bottomley <James.Bottomley@SteelEye.com>
Date: Sat, 06 Oct 2007 09:11:10 -0500

> My problem with auto generated is that it's provably impossible to
> generate globally unique numbers for WWNs without some internal source
> of uniqueness (I know sparcs have this in their serial number, but most
> PCs unfortunately don't).

If you have at least one ethernet NIC, you have at least one
globally unique MAC address, which is 6 bytes of uniqueness to
draw from.

I'm sure there are countless other possibilities.

I think the SAN issue is way overstated.  Sure it's real, but
so are SHA conflicts in the kernel GIT repository.

Re: generating a Linux WWN?

[Luben Tuikov]

--- James Bottomley <James.Bottomley@SteelEye.com> wrote:
> 
> For the record, what the current in-kernel aic94xx driver does for this
> case is allow a parameter override to specify the WWN in the case where
> the card burned in one has gone missing or is corrupt.  I think this is
> the correctly balanced approach to the problem.
> 
> For the record, I pretty much agree completely with Luben's position on
> this:  I want to allow the user a method to correct any problem (i.e.
> supply a WWN override).  However, I also want them to think about the
> issue before they do this.  What I really don't want is the driver
> silently correcting what it thinks to be a defective WWN and generating
> its own replacement because that's a recipe for disaster on a SAN (SANs
> are a lot less robust than networks to duplicate MAC addresses: because
> of the way expander routing is done, you can possibly collapse the
> entire SAN with duplicate WWNs).
> 
> So, the procedure for a SAS card should be:
> 
>      1. Obtain the WWN from a device specific storage method (like
>         flash)
>      2. Replace this with a passed in command line parameter if one
>         exists
>      3. refuse to attach if there's still no valid WWN and give an
>         explicit method saying what the problem is (and possibly how to
>         fix it)

That is absolutely correct.

If a node in a SAN doesn't come up, the SAN admin will inspect
the kernel log and find out that the driver failed to attach
due to a missing WWN.  This is a grave condition.

In enterprise, if one component is compromised, then the whole
system should fail and be brought to admin's attention.

The admin may decide to replace this card with a working one,
or ask the Storage Manager to generate a temporary WWN (which could
be the next machine over), or if the admin themselves maintain
the SAN, they can generate a temporary WWN.

But it is imperative that the admin immediately knows
of the fact that a node in the SAN has experienced a problem
and cannot read its WWN.

Alternatively, if the SAN is zoned, then the _rogue_ node
who autogenerated its WWN will be rejected access to the SAN.
A node which autogenerates WWNs is considered a security
risk and a rogue node.

   Luben

Re: generating a Linux WWN?

[Luben Tuikov]

--- Jeff Garzik <jeff@garzik.org> wrote:
> Luben Tuikov wrote:
> > --- Jeff Garzik <jeff@garzik.org> wrote:
> >> Luben Tuikov wrote:
> >>> --- Jeff Garzik <jeff@garzik.org> wrote:
> >>>> Luben Tuikov wrote:
> >>>>> This still does not justify "let's generate it in the kernel".
> >>>> Once an admin specifies to use an alternate WWN provision, the method 
> >>>> used to obtain that WWN is an implementation detail and irrelevant.
> >>> But you're arguing that the kernel should generate it to "help out"
> >>> SAS drivers with HW that has had it's MS/NVRAM erased or corrupted.
> >>>
> >>> The kernel has no business generating SAS WWN.
> >> Perhaps you missed the English text "once an admin specifies" in the 
> >> quoted email.
> > 
> > Was your "once an admin specifies" reply to my email
> > supposed to be "Oh, I see, generating a SAS WWN in the kernel
> > is not that good an idea after all.  I'll implement it
> > as a kernel/module option." ?
> 
> The admin will have the option to auto-generate a WWN, should they so 
> desire.

Do you mean:
  "The admin will have the option to SPECIFY(SET) a WWN, should they so 
   desire."
OR do you mean:
  "The admin will have the option to HAVE THE KERNEL auto-generate a WWN,
   should they so desire."

Re: generating a Linux WWN?

[Jeff Garzik]

Luben Tuikov wrote:
> Do you mean:
>   "The admin will have the option to SPECIFY(SET) a WWN, should they so 
>    desire."
> OR do you mean:
>   "The admin will have the option to HAVE THE KERNEL auto-generate a WWN,
>    should they so desire."

Both.  It is up to admin policy to decide if they wish to use the 
available board's WWN (if present/valid), manually apportion WWNs, or 
whether the kernel's generation algo is fine with them.

	Jeff

Re: generating a Linux WWN?

[Michael Reed]

Jeff Garzik wrote:
> Luben Tuikov wrote:
>> Do you mean:
>>   "The admin will have the option to SPECIFY(SET) a WWN, should they
>> so    desire."
>> OR do you mean:
>>   "The admin will have the option to HAVE THE KERNEL auto-generate a WWN,
>>    should they so desire."
> 
> Both.  It is up to admin policy to decide if they wish to use the
> available board's WWN (if present/valid), manually apportion WWNs, or
> whether the kernel's generation algo is fine with them.

Could the reassignment of a board's WWN present any security issues?
A board which is allowed to access only some targets is now allowed to
access others.  I would assume that most sites that care about security
would have other measures in place to guard against this, but it's still
worth pondering.

Mike

> 
>     Jeff
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: generating a Linux WWN?

[Jeff Garzik]

Michael Reed wrote:
> 
> Jeff Garzik wrote:
>> Luben Tuikov wrote:
>>> Do you mean:
>>>   "The admin will have the option to SPECIFY(SET) a WWN, should they
>>> so    desire."
>>> OR do you mean:
>>>   "The admin will have the option to HAVE THE KERNEL auto-generate a WWN,
>>>    should they so desire."
>> Both.  It is up to admin policy to decide if they wish to use the
>> available board's WWN (if present/valid), manually apportion WWNs, or
>> whether the kernel's generation algo is fine with them.
> 
> Could the reassignment of a board's WWN present any security issues?
> A board which is allowed to access only some targets is now allowed to
> access others.  I would assume that most sites that care about security
> would have other measures in place to guard against this, but it's still
> worth pondering.

Well, at that point the admin has made a conscious decision to deviate 
from the norm, either for site-specific reasons or for 
getting-his-hardware-working reasons.  So we assume the admin who made a 
proactive step in this direction knows what he is doing.

Just like the rest of Linux, we just give you the rope...  :)

	Jeff