From: Dipankar Sarma <dipankar@in.ibm.com>
To: Andrew Morton <akpm@osdl.org>
Cc: paulmck@us.ibm.com, dada1@cosmosbay.com, torvalds@osdl.org,
linux-kernel@vger.kernel.org,
viro@parcelfarce.linux.theplanet.co.uk, nickpiggin@yahoo.com.au,
hch@infradead.org
Subject: Re: [patch 2/2] fix file counting
Date: Mon, 30 Jan 2006 22:30:28 +0530 [thread overview]
Message-ID: <20060130170028.GA6264@in.ibm.com> (raw)
In-Reply-To: <20060127152857.32066a69.akpm@osdl.org>
On Fri, Jan 27, 2006 at 03:28:57PM -0800, Andrew Morton wrote:
> "Paul E. McKenney" <paulmck@us.ibm.com> wrote:
> >
> > > > I am using a patch that seems sligthly better : It removes the filp_count_lock
> > > > as yours but introduces a percpu variable, and a lazy nr_files . (Its value
> > > > can be off with a delta of +/- 16*num_possible_cpus()
> > >
> > > Yes, I think that is better.
> >
> > I agree that Eric's approach likely improves performance on large systems
> > due to decreased cache thrashing. However, the real problem is getting
> > both good throughput and good latency in RCU callback processing, given
> > Lee Revell's latency testing results. Once we get that in hand, then
> > we should consider Eric's approach.
>
> Dipankar's patch risks worsening large-SMP scalability, doesn't it?
> Putting an atomic op into the file_free path?
Here are some numbers from a 32-way (HT) P4 xeon box with kernbench -
2.6.16-rc1 vanilla
------------------
kernbench 32 5 -m 2
Completed successfully
Elapsed: 109.346s User: 1426.506s System: 178.71s CPU: 1467.6%
1426.63user 177.89system 1:49.07elapsed 1471%CPU (0avgtext+0avgdata
0maxresident)k
1425.48user 179.08system 1:49.09elapsed 1470%CPU (0avgtext+0avgdata
0maxresident)k
1427.02user 179.17system 1:49.82elapsed 1462%CPU (0avgtext+0avgdata
0maxresident)k
1427.13user 179.47system 1:50.00elapsed 1460%CPU (0avgtext+0avgdata
0maxresident)k
1426.27user 177.94system 1:48.75elapsed 1475%CPU (0avgtext+0avgdata
0maxresident)k
2.6.16-rc1+fix-file-counting.patch (with atomic_t nr_files)
-----------------------------------------------------------
kernbench 32 5 -m 2
Completed successfully
Elapsed: 109.338s User: 1427.554s System: 179.764s CPU: 1469.4%
1425.98user 179.32system 1:49.18elapsed 1470%CPU (0avgtext+0avgdata
0maxresident)k
1427.36user 179.60system 1:49.34elapsed 1469%CPU (0avgtext+0avgdata
0maxresident)k
1428.19user 179.92system 1:49.97elapsed 1462%CPU (0avgtext+0avgdata
0maxresident)k
1426.53user 180.45system 1:49.02elapsed 1473%CPU (0avgtext+0avgdata
0maxresident)k
1429.71user 179.53system 1:49.18elapsed 1473%CPU (0avgtext+0avgdata
0maxresident)k
2.6.16-rc1+nr-files.patch (with percpu counters [Eric's patch])
------------------------------------------------
kernbench 32 5 -m 2
Completed successfully
Elapsed: 109.38s User: 1427.684s System: 179.372s CPU: 1468.8%
1429.92user 179.37system 1:48.64elapsed 1481%CPU (0avgtext+0avgdata
0maxresident)k
1427.10user 178.60system 1:48.83elapsed 1475%CPU (0avgtext+0avgdata
0maxresident)k
1425.99user 177.75system 1:49.81elapsed 1460%CPU (0avgtext+0avgdata
0maxresident)k
1426.65user 181.33system 1:49.95elapsed 1462%CPU (0avgtext+0avgdata
0maxresident)k
1428.76user 179.81system 1:49.67elapsed 1466%CPU (0avgtext+0avgdata
0maxresident)k
The difference between atomic_t nr_files and percpu counters are
statistically insignficant. That said, there could be other workloads
where we may get hit badly by the global atomic_t or big SGI boxen
can be a problem.
I will poke around with this a little bit more to see what else
I can analyze.
Thanks
Dipankar
next prev parent reply other threads:[~2006-01-30 17:01 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-01-26 18:40 [patch 0/2] RCU: fix various latency/oom issues Dipankar Sarma
2006-01-26 18:41 ` [patch 1/2] rcu batch tuning Dipankar Sarma
2006-01-26 18:42 ` [patch 2/2] fix file counting Dipankar Sarma
2006-01-26 20:15 ` Eric Dumazet
2006-01-27 22:54 ` Andrew Morton
2006-01-27 23:14 ` Paul E. McKenney
2006-01-27 23:28 ` Andrew Morton
2006-01-28 18:42 ` Dipankar Sarma
2006-01-28 18:51 ` Andrew Morton
2006-01-28 19:10 ` Dipankar Sarma
2006-01-30 17:00 ` Dipankar Sarma [this message]
2006-01-31 10:33 ` Eric Dumazet
2006-01-31 20:19 ` Dipankar Sarma
2006-01-26 19:33 ` [patch 1/2] rcu batch tuning Paul E. McKenney
2006-01-26 19:42 ` Dipankar Sarma
-- strict thread matches above, loose matches on Subject: below --
2006-02-17 15:41 [PATCH 0/2] RCU updates Dipankar Sarma
2006-02-17 15:43 ` [PATCH 1/2] rcu batch tuning Dipankar Sarma
2006-02-17 15:46 ` [PATCH 2/2] fix file counting Dipankar Sarma
2006-02-18 9:04 ` Andrew Morton
2006-02-18 9:25 ` Dipankar Sarma
2006-02-18 9:45 ` Andrew Morton
2006-02-18 10:06 ` Dipankar Sarma
2006-02-18 10:10 ` Andrew Morton
2006-02-18 10:44 ` Dipankar Sarma
2006-02-18 12:14 ` Christoph Hellwig
2006-02-18 12:31 ` Arjan van de Ven
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060130170028.GA6264@in.ibm.com \
--to=dipankar@in.ibm.com \
--cc=akpm@osdl.org \
--cc=dada1@cosmosbay.com \
--cc=hch@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nickpiggin@yahoo.com.au \
--cc=paulmck@us.ibm.com \
--cc=torvalds@osdl.org \
--cc=viro@parcelfarce.linux.theplanet.co.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.