Re: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

Re: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[David Mosberger]

Tony and Andrew,

I just checked 2.6.12-rc3 and the fls() fix is indeed missing.  Do you
know what happened?

	--david

>>>>> On Thu, 21 Apr 2005 13:30:50 +0200, Andreas Hirstius <Andreas.Hirstius@cern.ch> said:

  Andreas> Hi, The fls() patch from David solves the problem :-))

  Andreas> Do you have an idea, when it will be in the mainline
  Andreas> kernel??

  Andreas> Andreas



  Andreas> Bartlomiej ZOLNIERKIEWICZ wrote:

  >>  Hi!
  >> 
  >>> A small update.
  >>> 
  >>> Patching mm/filemap.c is not necessary in order to get the
  >>> improved performance!  It's sufficient to remove
  >>> roundup_pow_of_two from |get_init_ra_size ...
  >>> 
  >>> So a simple one-liner changes to picture dramatically.  But why
  >>> ?!?!?
  >> 
  >> 
  >> roundup_pow_of_two() uses fls() and ia64 has buggy fls()
  >> implementation [ seems that David fixed it but patch is not in
  >> the mainline yet]:
  >> 
  >> http://www.mail-archive.com/linux-ia64@vger.kernel.org/msg01196.html
  >> 
  >> That would also explain why you couldn't reproduce the problem on
  >> ia32 Xeon machines.
  >> 
  >> Bartlomiej
  >> 

  Andreas> _______________________________________________
  Andreas> Gelato-technical mailing list
  Andreas> Gelato-technical@gelato.unsw.edu.au
  Andreas> https://www.gelato.unsw.edu.au/mailman/listinfo/gelato-technical

RE: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[Luck, Tony]

>I just checked 2.6.12-rc3 and the fls() fix is indeed missing.  Do you
>know what happened?

If BitKeeper were still in use, I'd have dropped that patch into my
"release" tree and asked Linus to "pull" ... but it's not, and I was
stalled.  I should have a "git" tree up and running in the next couple
of days.  I'll make sure that the fls fix goes in early.

-Tony

RE: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[David Mosberger]

>>>>> On Thu, 21 Apr 2005 10:19:28 -0700, "Luck, Tony" <tony.luck@intel.com> said:

  >> I just checked 2.6.12-rc3 and the fls() fix is indeed missing.
  >> Do you know what happened?

  Tony> If BitKeeper were still in use, I'd have dropped that patch
  Tony> into my "release" tree and asked Linus to "pull" ... but it's
  Tony> not, and I was stalled.  I should have a "git" tree up and
  Tony> running in the next couple of days.  I'll make sure that the
  Tony> fls fix goes in early.

Yeah, I'm facing the same issue.  I started playing with git last
night.  Apart from disk-space usage, it's very nice, though I really
hope someone puts together a web-interface on top of git soon so we
can seek what changed when and by whom.

	--david

Re: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[Randy.Dunlap]

On Thu, 21 Apr 2005 10:33:29 -0700 David Mosberger wrote:

| >>>>> On Thu, 21 Apr 2005 10:19:28 -0700, "Luck, Tony" <tony.luck@intel.com> said:
| 
|   >> I just checked 2.6.12-rc3 and the fls() fix is indeed missing.
|   >> Do you know what happened?
| 
|   Tony> If BitKeeper were still in use, I'd have dropped that patch
|   Tony> into my "release" tree and asked Linus to "pull" ... but it's
|   Tony> not, and I was stalled.  I should have a "git" tree up and
|   Tony> running in the next couple of days.  I'll make sure that the
|   Tony> fls fix goes in early.
| 
| Yeah, I'm facing the same issue.  I started playing with git last
| night.  Apart from disk-space usage, it's very nice, though I really
| hope someone puts together a web-interface on top of git soon so we
| can seek what changed when and by whom.

2 people have already done that.  Examples:
http://ehlo.org/~kay/gitweb.pl
and
http://grmso.net:8090/

and the commits mailing list is now working.
A script to show nightly (or daily:) commits and make
a daily patch tarball is also close to ready.

---
~Randy

RE: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[Luck, Tony]

>Yeah, I'm facing the same issue.  I started playing with git last
>night.  Apart from disk-space usage, it's very nice, though I really
>hope someone puts together a web-interface on top of git soon so we
>can seek what changed when and by whom.

Disk space issues?  A complete git repository of the Linux kernel with
all changesets back to 2.4.0 takes just over 3G ... which is big compared
to BK, but 3G of disk only costs about $1 (for IDE ... if you want 15K rpm
SCSI, then you'll pay a lot more).  Network bandwidth is likely to be a
bigger problem.

There's a prototype web i/f at http://grmso.net:8090/ that's already looking
fairly slick.

-Tony

RE: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[David Mosberger]

>>>>> On Thu, 21 Apr 2005 10:41:52 -0700, "Luck, Tony" <tony.luck@intel.com> said:

  Tony> Disk space issues?  A complete git repository of the Linux
  Tony> kernel with all changesets back to 2.4.0 takes just over 3G
  Tony> ... which is big compared to BK, but 3G of disk only costs
  Tony> about $1 (for IDE ... if you want 15K rpm SCSI, then you'll
  Tony> pay a lot more).  Network bandwidth is likely to be a bigger
  Tony> problem.

Ever heard that data is a gas?  My disks always fill up in no time at
all, no matter how big they are.  I agree that network bandwidth is an
bigger issue, though.

  Tony> There's a prototype web i/f at http://grmso.net:8090/ that's
  Tony> already looking fairly slick.

Indeed.  Plus it has a cool name, too.  Thanks for the pointer.

	--david

Re: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[Stan Bubrouski]

Luck, Tony wrote:
>>Yeah, I'm facing the same issue.  I started playing with git last
>>night.  Apart from disk-space usage, it's very nice, though I really
>>hope someone puts together a web-interface on top of git soon so we
>>can seek what changed when and by whom.
> 
> 
> Disk space issues?  A complete git repository of the Linux kernel with
> all changesets back to 2.4.0 takes just over 3G ... which is big compared
> to BK, but 3G of disk only costs about $1 (for IDE ... if you want 15K rpm
> SCSI, then you'll pay a lot more).  Network bandwidth is likely to be a
> bigger problem.
> 

That said, is there any plan to change how this functions in the future 
to solve these problems?  I.e. have it not use so much diskspace and 
thus use less bandwith.  Am I misunderstanding in assuming that after
say 1000 commits go into the tree it could end up several megs or gigs 
bigger?

If that is the case might it not be more prudent to sort this out now?

> There's a prototype web i/f at http://grmso.net:8090/ that's already looking
> fairly slick.
>

Yes it is very slick.  Kudos to the creator.

-sb


> -Tony

RE: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[Luck, Tony]

>That said, is there any plan to change how this functions in the future 
>to solve these problems?  I.e. have it not use so much diskspace and 
>thus use less bandwith.  Am I misunderstanding in assuming that after
>say 1000 commits go into the tree it could end up several megs or gigs 
>bigger?
>
>If that is the case might it not be more prudent to sort this out now?

Only a new user would have to pull the whole history ... and for most
uses it is sufficient to just pull the current top of the tree. Linus'
own tree only has a history going back to 2.6.12.-rc2 (when he started
using git).

Someday there might be a server daemon that can batch up the changes for
a "pull" to conserve network bandwidth.

There is a mailing list "git@vger.kernel.org" where these issues are
discussed.  Archives are available at marc.theaimsgroup.com and gelato.

-Tony

Re: [Gelato-technical] Re: Serious performance degradation on a RAID with kernel 2.6.10-bk7 and later

[Stan Bubrouski]

Luck, Tony wrote:
<SNIP>
> Only a new user would have to pull the whole history ... and for most
> uses it is sufficient to just pull the current top of the tree. Linus'
> own tree only has a history going back to 2.6.12.-rc2 (when he started
> using git).
> 
> Someday there might be a server daemon that can batch up the changes for
> a "pull" to conserve network bandwidth.
> 
> There is a mailing list "git@vger.kernel.org" where these issues are
> discussed.  Archives are available at marc.theaimsgroup.com and gelato.
> 

Thanks tony I wasn't aware of the list, I'll look there for git info
from now on.

Best Regards,

Stan

> -Tony
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
>