From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hans Reiser Subject: Re: [PATCH] various allocator optimizations Date: Fri, 14 Mar 2003 13:26:58 +0300 Message-ID: <3E71AE72.3090107@namesys.com> References: <1047400482.8215.312.camel@tiny.suse.com> <20030311194205.A4493@namesys.com> <1047499021.8219.604.camel@tiny.suse.com> <3E6F9DE3.2070902@namesys.com> <1047571151.8219.956.camel@tiny.suse.com> <3E711F34.4060309@namesys.com> <1047605662.8215.1068.camel@tiny.suse.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: list-help: list-unsubscribe: list-post: Errors-To: flx@namesys.com In-Reply-To: <1047605662.8215.1068.camel@tiny.suse.com> List-Id: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Chris Mason Cc: Oleg Drokin , reiserfs-list@namesys.com Chris Mason wrote: >On Thu, 2003-03-13 at 19:15, Hans Reiser wrote: > > > >>>Ok, more testing showed that patch wasn't bad, but wasn't great either. >>>Under a multiprocess test, the buffers in the path would get moved away >>>and there wasn't enough data to make a good decision. So the patch >>>below changes things around a little, and records the span during >>>search_by_key instead of trying to compute it after the search is over. >>> >>> >>> >>Hunh? Why don't you maintain a counter in the directory of the number >>of nodes in it? Or are you afraid of causing extra IO? >> >> >> > >That would mean the parent directory counter would have to be updated >every time we allocated a block in any sub directory. Plus the counter >would have to be inherited down the chain in deep directory structures. >More importantly, I'd rather not waste space in the stat data to store >the information when we can get it during a search ;-) > The space usage is trivial. > >Basing the span on the tree allows us to detect a highly used locality >regardless of what kind of tree object was using it, and we can do it >with almost zero overhead. > > > >>>In other words, I think this is a really good compromise between the >>>current defaults and a more optimal case for fragmentation, and I expect >>>its performance to hold up much better as the filesystem ages. >>> >>> >>> >>Let's get lots of different testers. You may have a nice heuristic here >>though.... >> >> >> > >If everyone agrees the approach is worth trying, I'll make a patch that >enables it via a mount option. > > > >>How big are your packing localities tending to be? >> >> > >Not more than can be pointed to by the leaf level and the level directly >above it. I know that's not very specific, but it varies by the >dataset. packed tails and long directory names lead to more packing >localities per MB. > Which is why it is the wrong measure, yes? > >My 100MB subset of /usr/share/doc has 732 directories in it, and the >copy phase of stress.sh -n 10 -s for that directory tree will produce >1400 packing localities, so it depends on size of the tree in general. > >-chris > > > > > > I am not saying that storing it in the directory is the right thing, because of the risk of extra IO, but I think it should be tried. -- Hans