On Thu, Sep 09 2010 at  3:43pm -0400,
Mike Snitzer <snitzer@redhat.com> wrote:

> Interestingly, just this loop:
> 
> while true ; do cat /sys/block/vda/serial && date && sleep 1 ; done
> Thu Sep  9 15:29:30 EDT 2010
> ...
> Thu Sep  9 15:31:19 EDT 2010
> 
> caused the following hang:
...
> So it seems like the virtio requests aren't being properly cleaned up?

Yeap, here is the result with the attached debug patch that Vivek wrote
last week to help chase this issue (which adds 'nr_requests_used').  We
thought the mpath device might be leaking requests; concern for other
devices wasn't on our radar:

# cat /sys/block/vda/queue/nr_requests
128

# while true ; do cat /sys/block/vda/queue/nr_requests_used && cat /sys/block/vda/serial && date && sleep 1 ; done
10
Thu Sep  9 16:04:40 EDT 2010
11
Thu Sep  9 16:04:41 EDT 2010
...
Thu Sep  9 16:06:38 EDT 2010
127
Thu Sep  9 16:06:39 EDT 2010
128

I'll have a quick look at the virtio-blk code to see if I can spot where
the request isn't getting cleaned up.  But I welcome others to have a
look too (I've already spent entirely way to much time on this issue).

Mike