Intermittent no space errors

[Dave Cundiff <syshackmin@gmail.com>]

Hello,

On 2.6.35-rc5 I'm seeing some weird behavior under heavy IO loads. I
have a backup process that fires up several rsync processes. These
mirror several dozen servers to individual sub-volumes. Everyday I
snapshot each sub-volume and rsync over it.

The problem I'm seeing is my rsync processes are failing randomly with
"No space left on device". This is a 6 Terabyte volume with plenty of
free space.

Mount options:
/dev/sdb on /backups type btrfs (rw,max_inline=0,compress)

[root@rsync1 ~]# btrfs filesystem df /backups/
Data: total=1.88TB, used=1.88TB
Metadata: total=43.38GB, used=32.06GB
System: total=12.00MB, used=260.00KB

[root@rsync1 ~]# df /dev/sdb
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdb             5781249024 2087273084 3693975940  37% /backups

They don't all fail at once. Normally I have 4-5 running at a time and
1 or 2 will drop out with a no space error. The rest continue on. I've
noticed it will generally occur on ones that are in the middle of
transferring a very large file. If I lighten the load to one rsync at
a time it appears to happen less frequently.

Any known issues I should be aware of?

Thanks,

-- 
Dave Cundiff
System Administrator
A2Hosting, Inc
http://www.a2hosting.com