Areca arcmsr kernel integration

Areca arcmsr kernel integration

[Theodore Bullock]

It would be really nice to see the arcmsr driver integrated into the
mainline kernel in the nearish future.

Ideally, the driver would be available before the next major
distribution is published so that the hardware can be officially
supported by the distribution developers eg. Novell, Red Hat, Canonical
... etc.

After a brief review of upcoming schedules, Fedora Core looks to be
updated next in October with the last test release currently scheduled
for October. After that it will probably be SuSE.

We have a good number of workstations with this hardware here, but
support isn't available from our distribution.

Checking the recent posts it seems that there are two outstanding issues

> - PAE (cast of dma_addr_t to unsigned long) issues. 
> - SYNCHRONIZE_CACHE is ignored.  This is wrong.  The sync cache in the

> shutdown notifier isn't sufficient.

That said, we have been using an older version of the driver off the
Areca website for some time now with no major issues (other than kernel
update problems). 

Is it possible to get the driver integrated and fix these problems
after?

Ted Bullock
Nortel Networks

Re: Areca arcmsr kernel integration

[Greg KH]

On Fri, Jul 28, 2006 at 02:07:12PM -0400, Theodore Bullock wrote:
> It would be really nice to see the arcmsr driver integrated into the
> mainline kernel in the nearish future.
> 
> Ideally, the driver would be available before the next major
> distribution is published so that the hardware can be officially
> supported by the distribution developers eg. Novell, Red Hat, Canonical
> ... etc.
> 
> After a brief review of upcoming schedules, Fedora Core looks to be
> updated next in October with the last test release currently scheduled
> for October. After that it will probably be SuSE.
> 
> We have a good number of workstations with this hardware here, but
> support isn't available from our distribution.
> 
> Checking the recent posts it seems that there are two outstanding issues
> 
> > - PAE (cast of dma_addr_t to unsigned long) issues. 
> > - SYNCHRONIZE_CACHE is ignored.  This is wrong.  The sync cache in the
> 
> > shutdown notifier isn't sufficient.
> 
> That said, we have been using an older version of the driver off the
> Areca website for some time now with no major issues (other than kernel
> update problems). 
> 
> Is it possible to get the driver integrated and fix these problems
> after?

Why not fix them up and submit the corrected driver?  If no one wants to
put the effort into fixing these issues now, why would they do the work
later?

thanks,

greg k-h

Re: Areca arcmsr kernel integration

[Bron Gondwana]

(adding Rob and Dax to CC and erich as primary recipient)

On Fri, Jul 28, 2006 at 01:27:49PM -0700, Greg KH wrote:
> On Fri, Jul 28, 2006 at 02:07:12PM -0400, Theodore Bullock wrote:
> > It would be really nice to see the arcmsr driver integrated into the
> > mainline kernel in the nearish future.
> > 
> > Ideally, the driver would be available before the next major
> > distribution is published so that the hardware can be officially
> > supported by the distribution developers eg. Novell, Red Hat, Canonical
> > ... etc.
> > 
> > After a brief review of upcoming schedules, Fedora Core looks to be
> > updated next in October with the last test release currently scheduled
> > for October. After that it will probably be SuSE.
> > 
> > We have a good number of workstations with this hardware here, but
> > support isn't available from our distribution.
> > 
> > Checking the recent posts it seems that there are two outstanding issues
> > 
> > > - PAE (cast of dma_addr_t to unsigned long) issues. 
> > > - SYNCHRONIZE_CACHE is ignored.  This is wrong.  The sync cache in the
> > 
> > > shutdown notifier isn't sufficient.
> > 
> > That said, we have been using an older version of the driver off the
> > Areca website for some time now with no major issues (other than kernel
> > update problems). 
> > 
> > Is it possible to get the driver integrated and fix these problems
> > after?
> 
> Why not fix them up and submit the corrected driver?  If no one wants to
> put the effort into fixing these issues now, why would they do the work
> later?
> 
> thanks,
> 
> greg k-h

Hi erich and other interested parties,

I agree with Theodore that it would be a great idea to get the arcmsr
driver and I'm sure we'd all be happy to help you with whatever you need
to make this happen.

Is any of the technical hardware documentation available to us so we can
help you with the driver design?  I'm certainly happy to help with
testing and/or documentation, and I have some C programming skills,
though I haven't done any serious kernel programming.

Regards,

Bron.

Re: Areca arcmsr kernel integration

[Robert Mueller]

>> > Checking the recent posts it seems that there are two outstanding 
>> > issues
>> >
>> > > - PAE (cast of dma_addr_t to unsigned long) issues.
>> > > - SYNCHRONIZE_CACHE is ignored.  This is wrong.  The sync cache in 
>> > > the
>> >
>> > > shutdown notifier isn't sufficient.
>> >
>> > That said, we have been using an older version of the driver off the
>> > Areca website for some time now with no major issues (other than kernel
>> > update problems).
>> >
>> > Is it possible to get the driver integrated and fix these problems
>> > after?
>>
>> Why not fix them up and submit the corrected driver?  If no one wants to
>> put the effort into fixing these issues now, why would they do the work
>> later?

I believe these final issues have already been fixed:

---
http://marc.theaimsgroup.com/?l=linux-scsi&m=115226175822438&w=2

From: Erich Chen <erich@areca.com.tw>

  1- fix sysfs has more than one value per file
  2- PAE issues (cast of dma_addr_t to unsigned long)
  3- unblock SYNCHRONIZE_CACHE

Signed-off-by: Erich Chen erich@areca.com.tw
---

And another one after that that Andrew brought up:

http://marc.theaimsgroup.com/?l=linux-scsi&m=115259088411283&w=2

I see that all of this was in 2.6.18-rc1-mm2

---
http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.18-rc1/2.6.18-rc1-mm2/announce.txt

 SCSI fixes.

-drivers-scsi-arcmsr-cleanups.patch
-areca-raid-linux-scsi-driver-update7.patch
-areca-raid-linux-scsi-driver-update7-fix.patch

 Folded into areca-raid-linux-scsi-driver.patch

+areca-sysfs-fix.patch

 Fix breakage due to sysfs API changes.
---

But I don't see anything in 2.6.18-rc2-mm1 or 2.6.18-rc2

http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.18-rc2/2.6.18-rc2-mm1/announce.txt
http://kernel.org/pub/linux/kernel/v2.6/testing/ChangeLog-2.6.18-rc2

Rob

RE: Areca arcmsr kernel integration

[Theodore Bullock]

>I believe these final issues have already been fixed:

Ok, so how does this go from here into the mainline kernel?

Re: Areca arcmsr kernel integration

[Andrew Morton]

On Mon, 31 Jul 2006 17:32:07 -0400
"Theodore Bullock" <tbullock@nortel.com> wrote:

> 
> >I believe these final issues have already been fixed:
> 
> Ok, so how does this go from here into the mainline kernel?

James has moved the driver into the scsi-misc tree, so I assume he has
2.6.19 plans for it.

Re: Areca arcmsr kernel integration

[James Bottomley]

On Mon, 2006-07-31 at 20:03 -0700, Andrew Morton wrote:
> > Ok, so how does this go from here into the mainline kernel?
> 
> James has moved the driver into the scsi-misc tree, so I assume he has
> 2.6.19 plans for it.

Yes, that's the usual path for scsi-misc.

James

Re: Areca arcmsr kernel integration for 2.6.18?

[Dax Kelson]

On Wed, 2006-08-02 at 10:53 -0400, James Bottomley wrote:
> On Mon, 2006-07-31 at 20:03 -0700, Andrew Morton wrote:
> > > Ok, so how does this go from here into the mainline kernel?
> > 
> > James has moved the driver into the scsi-misc tree, so I assume he has
> > 2.6.19 plans for it.
> 
> Yes, that's the usual path for scsi-misc.
> 
> James

It would be great if the arcmsr driver could be included in 2.6.18 so it
can make into all the new distro releases that will be happening the
last 3-4 months of the year.

It is completely self contained and it isn't changing any existing code
(ergo it can't break anything) so I believe there is quite a bit of
precedence for "late" inclusion in 2.6.18?

There are lots of users of this hardware (myself included) that would be
very appreciative.

Dax Kelson

Re: Areca arcmsr kernel integration for 2.6.18?

[Adrian Bunk]

On Wed, Aug 23, 2006 at 05:25:51PM -0600, Dax Kelson wrote:
> On Wed, 2006-08-02 at 10:53 -0400, James Bottomley wrote:
> > On Mon, 2006-07-31 at 20:03 -0700, Andrew Morton wrote:
> > > > Ok, so how does this go from here into the mainline kernel?
> > > 
> > > James has moved the driver into the scsi-misc tree, so I assume he has
> > > 2.6.19 plans for it.
> > 
> > Yes, that's the usual path for scsi-misc.
> > 
> > James
> 
> It would be great if the arcmsr driver could be included in 2.6.18 so it
> can make into all the new distro releases that will be happening the
> last 3-4 months of the year.
> 
> It is completely self contained and it isn't changing any existing code
> (ergo it can't break anything) so I believe there is quite a bit of
> precedence for "late" inclusion in 2.6.18?
>...

It would for sure create a bad example for other people trying to get
code merged outside the merge window...

> Dax Kelson

cu
Adrian

-- 

    Gentoo kernels are 42 times more popular than SUSE kernels among
    KLive users  (a service by SUSE contractor Andrea Arcangeli that
    gathers data about kernels from many users worldwide).

       There are three kinds of lies: Lies, Damn Lies, and Statistics.
                                                    Benjamin Disraeli

Re: Areca arcmsr kernel integration for 2.6.18?

[Greg KH]

On Wed, Aug 23, 2006 at 05:25:51PM -0600, Dax Kelson wrote:
> On Wed, 2006-08-02 at 10:53 -0400, James Bottomley wrote:
> > On Mon, 2006-07-31 at 20:03 -0700, Andrew Morton wrote:
> > > > Ok, so how does this go from here into the mainline kernel?
> > > 
> > > James has moved the driver into the scsi-misc tree, so I assume he has
> > > 2.6.19 plans for it.
> > 
> > Yes, that's the usual path for scsi-misc.
> > 
> > James
> 
> It would be great if the arcmsr driver could be included in 2.6.18 so it
> can make into all the new distro releases that will be happening the
> last 3-4 months of the year.

What distros would that be?  And how do you know that they are going to
freeze their kernels at 2.6.18?

> It is completely self contained and it isn't changing any existing code
> (ergo it can't break anything) so I believe there is quite a bit of
> precedence for "late" inclusion in 2.6.18?

Then it can easily be bundled as a "kernel module package" for those
same distros if this is the case :)

thanks,

greg k-h

Re: Areca arcmsr kernel integration for 2.6.18?

[Dax Kelson]

On Wed, 2006-08-23 at 20:42 -0700, Greg KH wrote:
> On Wed, Aug 23, 2006 at 05:25:51PM -0600, Dax Kelson wrote:
> > On Wed, 2006-08-02 at 10:53 -0400, James Bottomley wrote:
> > > On Mon, 2006-07-31 at 20:03 -0700, Andrew Morton wrote:
> > > > > Ok, so how does this go from here into the mainline kernel?
> > > > 
> > > > James has moved the driver into the scsi-misc tree, so I assume he has
> > > > 2.6.19 plans for it.
> > > 
> > > Yes, that's the usual path for scsi-misc.
> > > 
> > > James
> > 
> > It would be great if the arcmsr driver could be included in 2.6.18 so it
> > can make into all the new distro releases that will be happening the
> > last 3-4 months of the year.
> 
> What distros would that be?  And how do you know that they are going to
> freeze their kernels at 2.6.18?

Well, I don't know for sure, but thankfully most distro development is
pretty transparent.

The current Fedora Core 6 development (and consequently RHEL5 and
CentOS5) is using 2.6.18-rc kernels (actually as of yesterday, your git
tree).

The current plan is for Debian Etch to freeze on October 18th with a
release in December. There is a good possibility they'll move from
2.6.17 to .18.

Dax Kelson

Re: Areca arcmsr kernel integration for 2.6.18?

[Arjan van de Ven]

On Wed, 2006-08-23 at 23:56 -0600, Dax Kelson wrote:
> On Wed, 2006-08-23 at 20:42 -0700, Greg KH wrote:
> > On Wed, Aug 23, 2006 at 05:25:51PM -0600, Dax Kelson wrote:
> > > On Wed, 2006-08-02 at 10:53 -0400, James Bottomley wrote:
> > > > On Mon, 2006-07-31 at 20:03 -0700, Andrew Morton wrote:
> > > > > > Ok, so how does this go from here into the mainline kernel?
> > > > > 
> > > > > James has moved the driver into the scsi-misc tree, so I assume he has
> > > > > 2.6.19 plans for it.
> > > > 
> > > > Yes, that's the usual path for scsi-misc.
> > > > 
> > > > James
> > > 
> > > It would be great if the arcmsr driver could be included in 2.6.18 so it
> > > can make into all the new distro releases that will be happening the
> > > last 3-4 months of the year.
> > 
> > What distros would that be?  And how do you know that they are going to
> > freeze their kernels at 2.6.18?
> 
> Well, I don't know for sure, but thankfully most distro development is
> pretty transparent.
> 
> The current Fedora Core 6 development (and consequently RHEL5 and
> CentOS5) is using 2.6.18-rc kernels (actually as of yesterday, your git
> tree).

.. and those distros can't pull in this driver because..... ?

Re: Areca arcmsr kernel integration for 2.6.18?

[Alan Cox]

Ar Iau, 2006-08-24 am 10:34 +0200, ysgrifennodd Arjan van de Ven:
> > The current Fedora Core 6 development (and consequently RHEL5 and
> > CentOS5) is using 2.6.18-rc kernels (actually as of yesterday, your git
> > tree).
> 
> .. and those distros can't pull in this driver because..... ?

FC6 policy is to follow upstream very closely. RHEL may vary.

I don't think its fair to ask Greg to make this decision and James has
already given a clear answer.

Alan

RE: Areca arcmsr kernel integration for 2.6.18?

[Theodore Bullock]

I posted a feature enchancement request to the Novell bugzilla for the
10.2 openSUSE release (currently scheduled for 2.6.18 AFAIK, depends on
upstream timelines), and the response from Greg was this:

"If this driver is in the mainline kernel, it will show up in our
releases, and that's the only way we can accept it at this point in
time.  We will not backport it to an older kernel version."

If the driver goes into 2.6.19 this means 8 to 12 months of time before
our workstations will be supported enough by Novell to start a more
widespread deployment in the company out of the test lab.  We would
prefer this driver to be made available earlier, if only to make
installation more feasible and kernel upgrades possible.

-Ted Bullock
Nortel Networks

-----Original Message-----
From: Alan Cox [mailto:alan@lxorguk.ukuu.org.uk] 
Sent: August 24, 2006 5:41 AM
To: Arjan van de Ven
Cc: Dax Kelson; Greg KH; James Bottomley; Andrew Morton; Bullock,
Theodore (WIC:2J00); robm@fastmail.fm; brong@fastmail.fm;
erich@areca.com.tw; greg@kroah.com; linux-kernel@vger.kernel.org
Subject: Re: Areca arcmsr kernel integration for 2.6.18?

Ar Iau, 2006-08-24 am 10:34 +0200, ysgrifennodd Arjan van de Ven:
> > The current Fedora Core 6 development (and consequently RHEL5 and
> > CentOS5) is using 2.6.18-rc kernels (actually as of yesterday, your 
> > git tree).
> 
> .. and those distros can't pull in this driver because..... ?

FC6 policy is to follow upstream very closely. RHEL may vary.

I don't think its fair to ask Greg to make this decision and James has
already given a clear answer.

Alan

Re: Areca arcmsr kernel integration for 2.6.18?

[Greg KH]

On Thu, Aug 24, 2006 at 12:21:01PM -0400, Theodore Bullock wrote:
> I posted a feature enchancement request to the Novell bugzilla for the
> 10.2 openSUSE release (currently scheduled for 2.6.18 AFAIK, depends on
> upstream timelines), and the response from Greg was this:
> 
> "If this driver is in the mainline kernel, it will show up in our
> releases, and that's the only way we can accept it at this point in
> time.  We will not backport it to an older kernel version."

Yes, that was my suse engineer side talking.  It has _nothing_ to do
with my community work, please don't mix the two.

> If the driver goes into 2.6.19 this means 8 to 12 months of time before
> our workstations will be supported enough by Novell to start a more
> widespread deployment in the company out of the test lab.  We would
> prefer this driver to be made available earlier, if only to make
> installation more feasible and kernel upgrades possible.

No, you can _always_ have Novell create a stand-alone kernel module
package if you want them to (and can convince them to, you need to speak
with some project managers to get that to happen...)  You can also do
this on your own, as it is very simple to do and support over time.

Even if this driver went into 2.6.18, you would _still_ have the same
situation from Novell, so I do not see how you bring this up has any
relevance here.

And you don't even know what the next SuSE release will base their
kernel on yet, as it hasn't been decided :)

thanks,

greg k-h

Re: Areca arcmsr kernel integration for 2.6.18?

[James Bottomley]

On Wed, 2006-08-23 at 17:25 -0600, Dax Kelson wrote:
> It would be great if the arcmsr driver could be included in 2.6.18 so it
> can make into all the new distro releases that will be happening the
> last 3-4 months of the year.
> 
> It is completely self contained and it isn't changing any existing code
> (ergo it can't break anything) so I believe there is quite a bit of
> precedence for "late" inclusion in 2.6.18?
> 
> There are lots of users of this hardware (myself included) that would be
> very appreciative.

Well ... really no.  Kernel development isn't about distro releases.
Most of them pull from the upstream git trees anyway, so the problem
shouldn't arise.

There's precedent for putting relatively uncontroversial drivers into
the -rc tree.  However, arcmsr has had a rather difficult path into the
kernel.  Moving it into scsi-misc is a signal that it goes from being
out of tree to in-tree candidate (i.e. I'm happy with the quality now,
but I'll give other people a chance to voice concerns), but on a slow
path that allows people time to find problems and fix them.  This path
also dictates a time line for any final objections arising (i.e. up
until 2.6.18 is declared).

James

Re: Areca arcmsr kernel integration for 2.6.18?

[Dax Kelson]

On Wed, 2006-08-23 at 23:11 -0500, James Bottomley wrote:
> On Wed, 2006-08-23 at 17:25 -0600, Dax Kelson wrote:
> > It would be great if the arcmsr driver could be included in 2.6.18 so it
> > can make into all the new distro releases that will be happening the
> > last 3-4 months of the year.
> > 
> > It is completely self contained and it isn't changing any existing code
> > (ergo it can't break anything) so I believe there is quite a bit of
> > precedence for "late" inclusion in 2.6.18?

> There's precedent for putting relatively uncontroversial drivers into
> the -rc tree.  

Indeed, this precedent goes back many years.

> However, arcmsr has had a rather difficult path into the
> kernel.  Moving it into scsi-misc is a signal that it goes from being
> out of tree to in-tree candidate (i.e. I'm happy with the quality now,
> but I'll give other people a chance to voice concerns), but on a slow
> path that allows people time to find problems and fix them.  This path
> also dictates a time line for any final objections arising (i.e. up
> until 2.6.18 is declared).

People have been using the driver out of tree (with all the pain that
out of tree drivers entail) for almost 2 years with few if any problems.
The driver worked OK even before it got whipped into submission shape.

Another data point supporting the stability and well tested nature of
the driver is that a widely used distro has been shipping it since June,
Ubuntu 6.06 LTS.

Like Greg's OLS keynote aptly covered, the Linux hardware model is
geared towards and encourages in-tree drivers. 

In fact, from the keynote "... We have a whole sub-architecture that
only has 2 users in the world out there. We have drivers that I know
have only one user, as there was only one piece of hardware ever made
for it. It just isn't true, we will take drivers for anything into our
tree, as we really want it."

Areca hardware owners have been experiencing the out-of-tree pain for a
long time. James is happy with the quality, it is a new driver, well
tested, has been shipped by a major distro. People have had the ample
chance to voice concerns.

Why prolong the pain? Hook a brother up already.

Dax Kelson

Re: Areca arcmsr kernel integration for 2.6.18?

[Bron Gondwana]

On Thu, Aug 24, 2006 at 12:34:34AM -0600, Dax Kelson wrote:
> Areca hardware owners have been experiencing the out-of-tree pain for a
> long time. James is happy with the quality, it is a new driver, well
> tested, has been shipped by a major distro. People have had the ample
> chance to voice concerns.
> 
> Why prolong the pain? Hook a brother up already.

Thanks James and Erich for your great work in getting this driver
ready for inclusion and making sure it will be in 2.6.19.  It will
be nice to not have to apply the patch to every kernel I build!

Bron.

http://kernel.org/pub/linux/kernel/v2.6/snapshots/patch-2.6.18-git10.log

commit d67a70aca200f67be42428e74eb3353f20ad1130 
tree 863562f74854295bfdf60eb26375ff984d8a2d4d 
parent 43d6b68dc38867e489995e21649bb82f6ee7b5d3 
author James Bottomley <James.Bottomley@steeleye.com> Fri, 28 Jul 2006 17:36:46 -0500 
committer James Bottomley <jejb@mulgrave.il.steeleye.com> Wed, 02 Aug 2006 10:53:18 -0400 

    [SCSI] arcmsr: fix up sysfs values
    
    The sysfs files in arcmsr are non-standard in that they aren't simple
    filename value pairs, the values actually contain preceeding text which
    would have to be parsed.  The idea of sysfs files is that the file name
    is the description and the contents is a simple value.
    
    Fix up arcmsr to conform to this standard.
    
    Acked-By: Erich Chen <erich@areca.com.tw>
    Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>

commit 1c57e86d75cf162bdadb3a5fe0cd3f65aa1a9ca3 
tree 166f691c186d6e663218559c4762299063f63657 
parent 0c269e6d3c615403a6e0acbe6e88f1c0da9c2396 
author Erich Chen <erich@areca.com.tw> Wed, 12 Jul 2006 08:59:32 -0700 
committer James Bottomley <jejb@mulgrave.il.steeleye.com> Fri, 28 Jul 2006 14:13:40 -0500 

    [SCSI] arcmsr: initial driver, version 1.20.00.13
    
    arcmsr is a driver for the Areca Raid controller, a host based RAID
    subsystem that speaks SCSI at the firmware level.
    
    This patch is quite a clean up over the initial submission with
    contributions from:
    
    Randy Dunlap <rdunlap@xenotime.net>
    Christoph Hellwig <hch@lst.de>
    Matthew Wilcox <matthew@wil.cx>
    Adrian Bunk <bunk@stusta.de>
    
    Signed-off-by: Erich Chen <erich@areca.com.tw>
    Signed-off-by: Andrew Morton <akpm@osdl.org>
    Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>