From: Dipankar Sarma <dipankar@in.ibm.com>
To: Andrew Morton <akpm@osdl.org>
Cc: paulmck@us.ibm.com, dada1@cosmosbay.com, torvalds@osdl.org,
linux-kernel@vger.kernel.org,
viro@parcelfarce.linux.theplanet.co.uk, nickpiggin@yahoo.com.au,
hch@infradead.org
Subject: Re: [patch 2/2] fix file counting
Date: Mon, 30 Jan 2006 22:30:28 +0530 [thread overview]
Message-ID: <20060130170028.GA6264@in.ibm.com> (raw)
In-Reply-To: <20060127152857.32066a69.akpm@osdl.org>
On Fri, Jan 27, 2006 at 03:28:57PM -0800, Andrew Morton wrote:
> "Paul E. McKenney" <paulmck@us.ibm.com> wrote:
> >
> > > > I am using a patch that seems sligthly better : It removes the filp_count_lock
> > > > as yours but introduces a percpu variable, and a lazy nr_files . (Its value
> > > > can be off with a delta of +/- 16*num_possible_cpus()
> > >
> > > Yes, I think that is better.
> >
> > I agree that Eric's approach likely improves performance on large systems
> > due to decreased cache thrashing. However, the real problem is getting
> > both good throughput and good latency in RCU callback processing, given
> > Lee Revell's latency testing results. Once we get that in hand, then
> > we should consider Eric's approach.
>
> Dipankar's patch risks worsening large-SMP scalability, doesn't it?
> Putting an atomic op into the file_free path?
Here are some numbers from a 32-way (HT) P4 xeon box with kernbench -
2.6.16-rc1 vanilla
------------------
kernbench 32 5 -m 2
Completed successfully
Elapsed: 109.346s User: 1426.506s System: 178.71s CPU: 1467.6%
1426.63user 177.89system 1:49.07elapsed 1471%CPU (0avgtext+0avgdata
0maxresident)k
1425.48user 179.08system 1:49.09elapsed 1470%CPU (0avgtext+0avgdata
0maxresident)k
1427.02user 179.17system 1:49.82elapsed 1462%CPU (0avgtext+0avgdata
0maxresident)k
1427.13user 179.47system 1:50.00elapsed 1460%CPU (0avgtext+0avgdata
0maxresident)k
1426.27user 177.94system 1:48.75elapsed 1475%CPU (0avgtext+0avgdata
0maxresident)k
2.6.16-rc1+fix-file-counting.patch (with atomic_t nr_files)
-----------------------------------------------------------
kernbench 32 5 -m 2
Completed successfully
Elapsed: 109.338s User: 1427.554s System: 179.764s CPU: 1469.4%
1425.98user 179.32system 1:49.18elapsed 1470%CPU (0avgtext+0avgdata
0maxresident)k
1427.36user 179.60system 1:49.34elapsed 1469%CPU (0avgtext+0avgdata
0maxresident)k
1428.19user 179.92system 1:49.97elapsed 1462%CPU (0avgtext+0avgdata
0maxresident)k
1426.53user 180.45system 1:49.02elapsed 1473%CPU (0avgtext+0avgdata
0maxresident)k
1429.71user 179.53system 1:49.18elapsed 1473%CPU (0avgtext+0avgdata
0maxresident)k
2.6.16-rc1+nr-files.patch (with percpu counters [Eric's patch])
------------------------------------------------
kernbench 32 5 -m 2
Completed successfully
Elapsed: 109.38s User: 1427.684s System: 179.372s CPU: 1468.8%
1429.92user 179.37system 1:48.64elapsed 1481%CPU (0avgtext+0avgdata
0maxresident)k
1427.10user 178.60system 1:48.83elapsed 1475%CPU (0avgtext+0avgdata
0maxresident)k
1425.99user 177.75system 1:49.81elapsed 1460%CPU (0avgtext+0avgdata
0maxresident)k
1426.65user 181.33system 1:49.95elapsed 1462%CPU (0avgtext+0avgdata
0maxresident)k
1428.76user 179.81system 1:49.67elapsed 1466%CPU (0avgtext+0avgdata
0maxresident)k
The difference between atomic_t nr_files and percpu counters are
statistically insignficant. That said, there could be other workloads
where we may get hit badly by the global atomic_t or big SGI boxen
can be a problem.
I will poke around with this a little bit more to see what else
I can analyze.
Thanks
Dipankar
next prev parent reply other threads:[~2006-01-30 17:01 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-01-26 18:40 [patch 0/2] RCU: fix various latency/oom issues Dipankar Sarma
2006-01-26 18:41 ` [patch 1/2] rcu batch tuning Dipankar Sarma
2006-01-26 18:42 ` [patch 2/2] fix file counting Dipankar Sarma
2006-01-26 20:15 ` Eric Dumazet
2006-01-27 22:54 ` Andrew Morton
2006-01-27 23:14 ` Paul E. McKenney
2006-01-27 23:28 ` Andrew Morton
2006-01-28 18:42 ` Dipankar Sarma
2006-01-28 18:51 ` Andrew Morton
2006-01-28 19:10 ` Dipankar Sarma
2006-01-30 17:00 ` Dipankar Sarma [this message]
2006-01-31 10:33 ` Eric Dumazet
2006-01-31 20:19 ` Dipankar Sarma
2006-01-26 19:33 ` [patch 1/2] rcu batch tuning Paul E. McKenney
2006-01-26 19:42 ` Dipankar Sarma
-- strict thread matches above, loose matches on Subject: below --
2006-02-17 15:41 [PATCH 0/2] RCU updates Dipankar Sarma
2006-02-17 15:43 ` [PATCH 1/2] rcu batch tuning Dipankar Sarma
2006-02-17 15:46 ` [PATCH 2/2] fix file counting Dipankar Sarma
2006-02-18 9:04 ` Andrew Morton
2006-02-18 9:25 ` Dipankar Sarma
2006-02-18 9:45 ` Andrew Morton
2006-02-18 10:06 ` Dipankar Sarma
2006-02-18 10:10 ` Andrew Morton
2006-02-18 10:44 ` Dipankar Sarma
2006-02-18 12:14 ` Christoph Hellwig
2006-02-18 12:31 ` Arjan van de Ven
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060130170028.GA6264@in.ibm.com \
--to=dipankar@in.ibm.com \
--cc=akpm@osdl.org \
--cc=dada1@cosmosbay.com \
--cc=hch@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nickpiggin@yahoo.com.au \
--cc=paulmck@us.ibm.com \
--cc=torvalds@osdl.org \
--cc=viro@parcelfarce.linux.theplanet.co.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox