Re: SES Enclosure Management.

[Jeff Johnson <jeff.johnson@aeoncomputing.com>]

On 2/14/12 12:58 PM, Joe Landman wrote:
> On 02/14/2012 03:53 PM, Robert Woodworth wrote:
>> On 02/14/2012 01:42 PM, Joe Landman wrote:
>>> On 02/14/2012 03:31 PM, NeilBrown wrote:
>>>> On Tue, 14 Feb 2012 10:30:37 -0700 Robert Woodworth
>>>> <robertjwoodworth@gmail.com> wrote:
>>>>
>>>>> Has anyone ever thought of integrating SES managed enclosures into 
>>>>> the
>>>>> kernel RAID system? I briefly looked through the archives and have
>>>>> not found anything on the topic.
>>>>>
>>>>> Some HW based RAID controllers do this flawlessly now, there is no
>>>>> reason why the kernel RAID cannot also. (LSI MegaRAID)
>>>>>
>>>>> 1) When a drive is part if a managed enclosure, the RAID system 
>>>>> should
>>>>> address it by location instead of by enumerated device node. The SES
>>>>> device in the enclosure can map the physical slot to a physical 
>>>>> drive.
>>>>> The RAID admin (mdamd) should be able to add/fail/identify devices
>>>>> based on slot.
>>>>
>>>> Does this just mean that the admin should using names in
>>>> /dev/disk/by-path/
>>>> rather than /dev/sdXX to address devices? What can md or mdadm do to
>>>> help?
>>>>
>>>
>>> Not sure on the SES (or SGPIO side), but one of the things we've been
>>> doing has been to create a file with disk placement "coordinates", so
>>> as to map serial number and device to physical location.
>>>
>>
>> With real SES managed enclosures, you issue a SCSI command to read SES
>> Page1 and Page2 to get the details about the drives in any given slot.
>> This currently works fine in Linux with the sg_utils3 package. From the
>> command line, 'sg_ses -p 2 /dev/sgXX` where the device is the SES 
>> device.
>>
>> Take a look at your systems, if you see a device at
>> /sys/class/enclosure/XXXX/ then you have a managed enclosure attached.
>>
>
> Got it.  Thanks.  Will look and see.  Should be pretty straight 
> forward to do this.
>
>>
>>>>>
>>>>> 2) If the RAID system fails a drive, it should notify the SES
>>>>> management and turn on the fail bit and the fail LED.
>>>>
>>>> "mdadm --monitor" will run a script on drive failure. This could 
>>>> easily
>>>> notify the SES management.
>>>
>>> Yes, we are using this now for notifications and logging.
>>>
>>>>
>>>> So maybe all we need here is a script to plug in to mdadm... Would
>>>> you like
>>>> to write one?
>>>>
>>>
>>> Just need a "standard" SES (or SGPIO) mechanism to hook into, and we
>>> should be able to support this. Right now we have to work through HBA
>>> scripts.
>> A true managed enclosure has nothing to do with the HBA. A managed
>> enclosure provides a device on the SCSI bus and you exclusively
>> communicate with that device regardless of the HBA. Most HW RAIDs (LSI
>> MegaRAID) will hide the SES device exactly like they hide the physical
>> disks.
>>
>
> Ok.  Let me look to see if we can do this.  If so, we should be able 
> to help contribute some scripts.
I've been doing this for a while. In SES the various elements (slot, 
power supplies, fans, etc) are named.  At least if your JBOD vendor has 
their **** together. There is no need to decipher arrays of hex values 
and modify bits to assert control. Once you know the names of your 
elements you can address them by name for status and control (fault LED, 
power on/off, temps, etc).

--Jeff

-- 
------------------------------
Jeff Johnson
Manager
Aeon Computing

jeff.johnson@aeoncomputing.com
www.aeoncomputing.com
t: 858-412-3810 x101   f: 858-412-3845
m: 619-204-9061

4905 Morena Boulevard, Suite 1313 - San Diego, CA 92117