[Lustre-devel] Large readdir RPCs project

[Fan Yong <yong.fan@whamcloud.com>]

  On 10/14/10 12:04 AM, Jeremy Filizetti wrote:
> I've put together a small patch to modify ll_dir_readpage and 
> mdc_readpage to read extra pages (if available) with each RPC.  It is 
> posted under the bug 
> (https://bugzilla.lustre.org/show_bug.cgi?id=17833).  I see you we're 
> the original assignee when you were at Sun/Oracle.
Thanks, I will study such patch. I added Lsy into such bug CC list who 
is interested in it also.

--
Nasf
> Jeremy
>
> 2010/9/29 Fan Yong <yong.fan@whamcloud.com 
> <mailto:yong.fan@whamcloud.com>>
>
>     On 9/30/10 3:01 AM, Jeremy Filizetti wrote:
>>
>>
>>         On the other hand, Large readdir RPCs is basic of another
>>         metadata read performance improvement features - "readdir+",
>>         which is quite useful for "ls -l" operation (for large
>>         directory), and reduce lookup/getattr RPC as much as
>>         possible. In such feature, MDS will pack more dir-item's
>>         attribute (not only name/ino as does currently by readdir,
>>         but also mode/owner, and etc) information back to client-side
>>         in "readdir+" RPC. That means the dir-item count in one
>>         "readdir+" page is less than in the traditional readdir page,
>>         then more pages to be sent back to client. If without "Large
>>         readdir RPCs", the advantage of "readdir+" will be discounted.
>>
>>     I'd be interested in working this as well but probably as a
>>     separate effort since SOM isn't in 1.8 and that's my main focus. 
>>     In our testing, SOM had significant benefits over the WAN and I'd
>>     expect even better from readdir+.  I have tried Oleg's patch for
>>     asynchronous ll_glimpse_size but oddly I've seen someone erradic
>>     performance where at times it was worse then statahead and
>>     synchronous ll_glimpse_size.
>>     Jeremy
>     Yes, SOM is another important feature for metadata reading
>     performance improvement by bypassing the glimpse RPC between
>     client and OSS. Engineers from Lustre Group worked for that for
>     some time, hope can be released soon.
>
>     As for the asynchronous ll_glimpse_size maybe cause bad
>     performance occasionally, one possible reason is that: glimpse RPC
>     maybe not obtain extent lock(s) if some others in using such
>     lock(s), so the file size information obtained by asynchronous
>     glimpse is invalid when it is really used later, means the caller
>     ("stat") has to send synchronous glimpse again. Anyway, I did not
>     study such patch, so it is just a guess.
>
>
>     Cheers,
>     Nasf
>
>     _______________________________________________
>     Lustre-devel mailing list
>     Lustre-devel at lists.lustre.org <mailto:Lustre-devel@lists.lustre.org>
>     http://lists.lustre.org/mailman/listinfo/lustre-devel
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20101014/b5c28ace/attachment.htm>