Re: [PATCH 0/3] xfs: allocation worker causes freelist buffer lock hang

Mark Tinguely
2012-09-26T14:13:04Z

On 09/25/12 17:01, Dave Chinner wrote:
> On Tue, Sep 25, 2012 at 10:14:16AM -0500, Mark Tinguely wrote:

<deletes>

>>>>
>>>> As a bonus, consolidating the loops into one worker actually gives a slight
>>>> performance advantage.
>>>
>>> Can you quantify it?
>>
>> I was comparing the bonnie and iozone benchmarks outputs. I will see
>> if someone can enlighten me on how to quantify those numbers.
>
> Ugh.
>
> Don't bother. Those are two of the worst offenders in the "useless
> benchmarks for regression testing" category. Yeah, they *look* like
> they give decent numbers, but I've wasted so much time looking at
> results from these benhmarks only to find they do basic things wrong
> and give numbers that vary simple because you've made a change that
> increases or decreases the CPU cache footprint of a code path.
>
> e.g. IOZone uses the same memory buffer as the source/destination of
> all it's IO, and does not touch the contents of it at all. Hence for
> small IO, the buffer stays resident in the CPU caches and gives
> unrealsitically high throughput results. Worse is the fact that CPU
> cache residency of the buffer can change according to the kernel
> code path taken, so you can get massive changes in throughput just
> by changing the layout of the code without changing any logic....
>
> IOZone can be useful if you know exactly what you are doing and
> using it to test a specific code path with a specific set of
> configurations. e.g. comparing ext3/4/xfs/btrfs on the same kernel
> and storage is fine. However, the moment you start using it to
> compare different kernels, it's a total crap shoot....

does anyone have a good benchmark XFS should use to share performance
results? A number that we can agree a series does not degrade the
filesystem..

lies, damn lies, statistics and then filesystem benchmarks?! :)

> I guess I don't understand what you mean by "loop on
> xfs_alloc_vextent()" then.
>
> The problem I see above is this:
>
> thread 1 worker 1 worker 2..max
> xfs_bmapi_write(userdata)

loops here calling xfs_bmapi_alloc()

> xfs_bmapi_allocate(user)
> xfs_alloc_vextent(user)
> wait
>
> _xfs_alloc_vextent()
> locks AGF

first loop it takes the lock
one of the next times through the above
loop it cannot get a worker. deadlock here.

I saved the xfs_bmalloca and fs_alloc_arg when
allocating a buffer to verify the paths.

> _xfs_alloc_vextent()
> blocks on AGF lock

> completes allocation
>
> <returns with AGF locked in transaction>
> xfs_bmap_add_extent_hole_real
> xfs_bmap_extents_to_btree
> xfs_alloc_vextent(user)
> wait

this does not need a worker, and since in the same
transaction all locks to the AGF buffer are recursive locks.
no wait here.
>
> <deadlock as no more workers available>

<deletes>

--Mark.