Re: [PATCH] ATA over Ethernet driver for 2.6.9 (with changes)

Ed L Cashin
2004-12-14T13:39:37Z

Jens Axboe <axboe@suse.de> writes:

> On Mon, Dec 13 2004, Ed L Cashin wrote:
>> * use mempool allocation in make_request_fn
>
> It's not good enough, if cannot use a higher allocation priority
> that GFP_NOIO here - basically guarantee that your allocation will
> not block on further io. Currently you have the very same deadlock
> as before, the mempool does not help you since you call into the
> allocator and deadlock before ever blocking on the mempool.

Do you mean that with GFP_KERNEL we may still deadlock on line 199 of
the snippet below (from mm/mempool.c)? That alloc pointer points to
mempool_alloc_slab, which gets called with __GFP_WAIT turned off. The
kmem_cache allocator doesn't get called with the allocation priority
we specify in our make_request_fn, so we won't block there.

190 void * mempool_alloc(mempool_t *pool, int gfp_mask)
191 {
192 void *element;
193 unsigned long flags;
194 DEFINE_WAIT(wait);
195 int gfp_nowait = gfp_mask & ~(__GFP_WAIT | __GFP_IO);
196
197 might_sleep_if(gfp_mask & __GFP_WAIT);
198 repeat_alloc:
199 element = pool->alloc(gfp_nowait|__GFP_NOWARN, pool->pool_data);
200 if (likely(element != NULL))
201 return element;
202

If we block later on the pool, that's because there are 16 objects in
use, which means that mempool_free is going to get called 16 times as
I/O completes, so I/O is throttled and forward progress is guaranteed.
Otherwise, how does the mempool mechanism help in preventing deadlock?

It looks like we can simply change GFP_KERNEL to GFP_IO in our
make_request_fn, but I'd also like to understand why that's necessary
when there's a dedicated pre-allocated pool per aoe device.