Generic I/O

[michi1@michaelblizek.twilightparadox.com (michi1 at michaelblizek.twilightparadox.com)]

Hi!

On 11:40 Tue 15 Nov     , Kai Meyer wrote:
> On 11/15/2011 11:13 AM, michi1 at michaelblizek.twilightparadox.com wrote:
...
> > You might want to take a look at wait queues (the kernel equivalent to pthread
> > "condidions"). Basically you instead of calling msleep(), you call
> > wait_event(). In the function which decrements numbios, you check whether it
> > is 0 and if so call wake_up().
...
> That sounds very promising. When I read up on wait_event here:
> lxr.linux.no/#linux+v2.6.32/include/linux/wait.h#L191
> 
> It sounds like it's basically doing the same thing. I would call it like so:
> 
> wait_event(wq, atomic_read(numbios) == 0);

Yes, you dol something like this.

> To make sure I understand, this seems very much like what I'm doing, 
> except I'm being woken up every time a bio finishes instead of being 
> woken up once every millisecond. That is, I'm assuming I would use the 
> same work queue for all my bios.

You are *not* woken up every time you a bio finishes. You are woken up every
time you call wake_up(). You could do something like:

if (atomic_dec_return(numbios) == 0)
	wake_up(wp);

> During my testing, when I do a lot of disk I/O, I may potentially have 
> hundreds of threads waiting on anywhere between 1 and 32 bios. Help me 
> understand the sort of impact you think I might see between having 
> hundreds waiting for a millisecond, and having hundreds get woken up 
> each time a bio completes. It seems like it would be very helpful in low 
> I/O scenarios, especially when there are fast disks involved. I'm 
> concerned that during heavy I/O loads, I'll be doing a lot of 
> atomic_reads, and I have the impression that atomic_read isn't the 
> cheapest operation.

The wakeups might some some overhead. However, I would worry more about
scheduling overhead on smp systems than atomic_read performance.

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com