* [Lustre-devel] Thoughts on benchmarking @ 2008-02-25 21:44 Andrew C. Uselton 2008-02-28 16:51 ` Peter Braam 2008-02-29 3:20 ` [Lustre-devel] Andrew's I/O graphs Peter Braam 0 siblings, 2 replies; 7+ messages in thread From: Andrew C. Uselton @ 2008-02-25 21:44 UTC (permalink / raw) To: lustre-devel Howdy all, I'd been in conversation with Cliff White over the last few weeks, and he'd expressed an interest in having me post a draft of a report I've been working on. If you've already heard of it here it is. For those who hadn't I'll try to describe it briefly. In December I assisted with some Lustre benchmark tests on the Franklin Cray XT here at NERSC. Since then I've tried to summarize our analysis and results. The attached pdf is a draft of that summary. The introduction is almost completely useless, so feel free to skip (unless you want to have a laugh at the author's expense). Section 3 has the main details about what we observed and what we thought about it. Section 2 may be amusing for those (like me) who care about methodology. Cheers, Andrew -------------- next part -------------- A non-text attachment was scrubbed... Name: uselton.pdf Type: application/pdf Size: 292842 bytes Desc: not available URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20080225/58c106de/attachment.pdf> ^ permalink raw reply [flat|nested] 7+ messages in thread
* [Lustre-devel] Thoughts on benchmarking 2008-02-25 21:44 [Lustre-devel] Thoughts on benchmarking Andrew C. Uselton @ 2008-02-28 16:51 ` Peter Braam 2008-02-29 3:20 ` [Lustre-devel] Andrew's I/O graphs Peter Braam 1 sibling, 0 replies; 7+ messages in thread From: Peter Braam @ 2008-02-28 16:51 UTC (permalink / raw) To: lustre-devel Oh, and I forgot to say - WOW what a nice paper! - Peter - On 2/25/08 2:44 PM, "Andrew C. Uselton" <acuselton@lbl.gov> wrote: > Howdy all, > I'd been in conversation with Cliff White over the last few weeks, and > he'd expressed an interest in having me post a draft of a report I've > been working on. If you've already heard of it here it is. For those > who hadn't I'll try to describe it briefly. > > In December I assisted with some Lustre benchmark tests on the > Franklin Cray XT here at NERSC. Since then I've tried to summarize our > analysis and results. The attached pdf is a draft of that summary. The > introduction is almost completely useless, so feel free to skip (unless > you want to have a laugh at the author's expense). Section 3 has the > main details about what we observed and what we thought about it. > Section 2 may be amusing for those (like me) who care about methodology. > Cheers, > Andrew > ^ permalink raw reply [flat|nested] 7+ messages in thread
* [Lustre-devel] Andrew's I/O graphs @ 2008-02-29 3:20 ` Peter Braam 2008-02-29 21:10 ` [Lustre-devel] Thoughts on benchmarking Andreas Dilger 0 siblings, 1 reply; 7+ messages in thread From: Peter Braam @ 2008-02-29 3:20 UTC (permalink / raw) To: lustre-devel I see some worrying dips in the graphs - can our I/O specialists comment on which ones are understood and which are not? - peter - On 2/25/08 2:44 PM, "Andrew C. Uselton" <acuselton@lbl.gov> wrote: > Howdy all, > I'd been in conversation with Cliff White over the last few weeks, and > he'd expressed an interest in having me post a draft of a report I've > been working on. If you've already heard of it here it is. For those > who hadn't I'll try to describe it briefly. > > In December I assisted with some Lustre benchmark tests on the > Franklin Cray XT here at NERSC. Since then I've tried to summarize our > analysis and results. The attached pdf is a draft of that summary. The > introduction is almost completely useless, so feel free to skip (unless > you want to have a laugh at the author's expense). Section 3 has the > main details about what we observed and what we thought about it. > Section 2 may be amusing for those (like me) who care about methodology. > Cheers, > Andrew > ------ End of Forwarded Message -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20080228/97465ec9/attachment.htm> ^ permalink raw reply [flat|nested] 7+ messages in thread
* [Lustre-devel] Thoughts on benchmarking 2008-02-29 3:20 ` [Lustre-devel] Andrew's I/O graphs Peter Braam @ 2008-02-29 21:10 ` Andreas Dilger 2008-02-29 21:38 ` Andrew C. Uselton 0 siblings, 1 reply; 7+ messages in thread From: Andreas Dilger @ 2008-02-29 21:10 UTC (permalink / raw) To: lustre-devel On Feb 28, 2008 20:20 -0700, Peter J. Braam wrote: > I see some worrying dips in the graphs - can our I/O specialists comment on > which ones are understood and which are not? I think one of the major problems that Andrew discusses at the end of the test runs is described in bug 7365 "Poor performance when files share an OSC". I didn't see anywhere in the paper which version of Lustre was being tested, but I know we did some work to improve the round-robin allocator to make it more uniform in more recent releases, up to a certain extent. That said, getting completely uniform file distribution will still need some effort, because the MDS doesn't do any correlation between create requests (e.g. from a single job, from a single client, etc). > On 2/25/08 2:44 PM, "Andrew C. Uselton" <acuselton@lbl.gov> wrote: > > I'd been in conversation with Cliff White over the last few weeks, and > > he'd expressed an interest in having me post a draft of a report I've > > been working on. If you've already heard of it here it is. For those > > who hadn't I'll try to describe it briefly. > > > > In December I assisted with some Lustre benchmark tests on the > > Franklin Cray XT here at NERSC. Since then I've tried to summarize our > > analysis and results. The attached pdf is a draft of that summary. The > > introduction is almost completely useless, so feel free to skip (unless > > you want to have a laugh at the author's expense). Section 3 has the > > main details about what we observed and what we thought about it. > > Section 2 may be amusing for those (like me) who care about methodology. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. ^ permalink raw reply [flat|nested] 7+ messages in thread
* [Lustre-devel] Thoughts on benchmarking 2008-02-29 21:10 ` [Lustre-devel] Thoughts on benchmarking Andreas Dilger @ 2008-02-29 21:38 ` Andrew C. Uselton 2008-02-29 22:56 ` Stephen Sugiyama 2008-03-04 14:46 ` Nicholas Henke 0 siblings, 2 replies; 7+ messages in thread From: Andrew C. Uselton @ 2008-02-29 21:38 UTC (permalink / raw) To: lustre-devel Howdy Andreas, Long time no electron :) The work on Franklin (NERSC's shiny new XT4) uses the Lustre delivered and supported by Cray. I believe it's 1.6.x, but I'd have to ask around to get the details. Is there a way to dig the Lustre version out of a client? I'm at a workshop now. I'll try to address this next week. Note that they updated things on Franklin earlier in February. After that we saw a substantial performance increase. The details of what changed have not been communicated to be. Sometime in the near future I'll be interested to follow up on the work I've written about. Feel free to contribute suggestions of tests you'd be interested in. Cheers, Andrew Andreas Dilger wrote: > On Feb 28, 2008 20:20 -0700, Peter J. Braam wrote: >> I see some worrying dips in the graphs - can our I/O specialists comment on >> which ones are understood and which are not? > > I think one of the major problems that Andrew discusses at the end of > the test runs is described in bug 7365 "Poor performance when files share > an OSC". I didn't see anywhere in the paper which version of Lustre was > being tested, but I know we did some work to improve the round-robin > allocator to make it more uniform in more recent releases, up to a > certain extent. > > That said, getting completely uniform file distribution will still need > some effort, because the MDS doesn't do any correlation between create > requests (e.g. from a single job, from a single client, etc). > >> On 2/25/08 2:44 PM, "Andrew C. Uselton" <acuselton@lbl.gov> wrote: >>> I'd been in conversation with Cliff White over the last few weeks, and >>> he'd expressed an interest in having me post a draft of a report I've >>> been working on. If you've already heard of it here it is. For those >>> who hadn't I'll try to describe it briefly. >>> >>> In December I assisted with some Lustre benchmark tests on the >>> Franklin Cray XT here at NERSC. Since then I've tried to summarize our >>> analysis and results. The attached pdf is a draft of that summary. The >>> introduction is almost completely useless, so feel free to skip (unless >>> you want to have a laugh at the author's expense). Section 3 has the >>> main details about what we observed and what we thought about it. >>> Section 2 may be amusing for those (like me) who care about methodology. > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. > ^ permalink raw reply [flat|nested] 7+ messages in thread
* [Lustre-devel] Thoughts on benchmarking 2008-02-29 21:38 ` Andrew C. Uselton @ 2008-02-29 22:56 ` Stephen Sugiyama 2008-03-04 14:46 ` Nicholas Henke 1 sibling, 0 replies; 7+ messages in thread From: Stephen Sugiyama @ 2008-02-29 22:56 UTC (permalink / raw) To: lustre-devel Hi Andrew and Andreas, In December franklin was running Lustre 1.4.9 (plus various patches). Today franklin is running Lustre 1.4.11. Stephen Sugiyama -----Original Message----- From: lustre-devel-bounces@lists.lustre.org [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Andrew C. Uselton Sent: Friday, February 29, 2008 1:38 PM To: lustre-devel at lists.lustre.org Subject: Re: [Lustre-devel] Thoughts on benchmarking Howdy Andreas, Long time no electron :) The work on Franklin (NERSC's shiny new XT4) uses the Lustre delivered and supported by Cray. I believe it's 1.6.x, but I'd have to ask around to get the details. Is there a way to dig the Lustre version out of a client? I'm at a workshop now. I'll try to address this next week. Note that they updated things on Franklin earlier in February. After that we saw a substantial performance increase. The details of what changed have not been communicated to be. Sometime in the near future I'll be interested to follow up on the work I've written about. Feel free to contribute suggestions of tests you'd be interested in. Cheers, Andrew Andreas Dilger wrote: > On Feb 28, 2008 20:20 -0700, Peter J. Braam wrote: >> I see some worrying dips in the graphs - can our I/O specialists comment on >> which ones are understood and which are not? > > I think one of the major problems that Andrew discusses at the end of > the test runs is described in bug 7365 "Poor performance when files share > an OSC". I didn't see anywhere in the paper which version of Lustre was > being tested, but I know we did some work to improve the round-robin > allocator to make it more uniform in more recent releases, up to a > certain extent. > > That said, getting completely uniform file distribution will still need > some effort, because the MDS doesn't do any correlation between create > requests (e.g. from a single job, from a single client, etc). > >> On 2/25/08 2:44 PM, "Andrew C. Uselton" <acuselton@lbl.gov> wrote: >>> I'd been in conversation with Cliff White over the last few weeks, and >>> he'd expressed an interest in having me post a draft of a report I've >>> been working on. If you've already heard of it here it is. For those >>> who hadn't I'll try to describe it briefly. >>> >>> In December I assisted with some Lustre benchmark tests on the >>> Franklin Cray XT here at NERSC. Since then I've tried to summarize our >>> analysis and results. The attached pdf is a draft of that summary. The >>> introduction is almost completely useless, so feel free to skip (unless >>> you want to have a laugh at the author's expense). Section 3 has the >>> main details about what we observed and what we thought about it. >>> Section 2 may be amusing for those (like me) who care about methodology. > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. > _______________________________________________ Lustre-devel mailing list Lustre-devel at lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* [Lustre-devel] Thoughts on benchmarking 2008-02-29 21:38 ` Andrew C. Uselton 2008-02-29 22:56 ` Stephen Sugiyama @ 2008-03-04 14:46 ` Nicholas Henke 1 sibling, 0 replies; 7+ messages in thread From: Nicholas Henke @ 2008-03-04 14:46 UTC (permalink / raw) To: lustre-devel Andrew C. Uselton wrote: > Howdy Andreas, > Long time no electron :) > > The work on Franklin (NERSC's shiny new XT4) uses the Lustre > delivered and supported by Cray. I believe it's 1.6.x, but I'd have to > ask around to get the details. Is there a way to dig the Lustre version > out of a client? 'cat /proc/fs/lustre/version' or strings /path/to/obdclass.ko | grep 'Build Version' Nic ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2008-03-04 14:46 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-02-25 21:44 [Lustre-devel] Thoughts on benchmarking Andrew C. Uselton 2008-02-28 16:51 ` Peter Braam 2008-02-29 3:20 ` [Lustre-devel] Andrew's I/O graphs Peter Braam 2008-02-29 21:10 ` [Lustre-devel] Thoughts on benchmarking Andreas Dilger 2008-02-29 21:38 ` Andrew C. Uselton 2008-02-29 22:56 ` Stephen Sugiyama 2008-03-04 14:46 ` Nicholas Henke
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.