WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

[David Weber]

Hi,

we are testing Btrfs as a kvm storage system with 
"defaults,space_cache,nodatacow" mount options and regular snapshots. While 
testing we sometimes get these warnings:
WARNING: CPU: 5 PID: 11335 at /home/kernel/COD/linux/fs/btrfs/extent-
tree.c:4029 btrfs_free_reserved_data_space+0x102/0x110 [btrfs]()

Full log:
https://gist.github.com/anonymous/5f672f72899a3d52c20c

Once they start, they flood dmesg with several messages per second. After a 
reboot, it takes a while until they start to appear again. 
We have seen them with Linux 4.0.6 and 4.1

This was already reported a few times:
https://bugzilla.opensuse.org/show_bug.cgi?id=904023
https://bugzilla.redhat.com/show_bug.cgi?id=1173937
http://www.spinics.net/lists/linux-btrfs/msg39069.html
https://lkml.org/lkml/2014/10/22/899

Always with no solution
Is this a real problem or can we safely ignore this?

Cheers,
David

Re: WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

[Lutz Vieweg]

On 06/29/2015 11:35 AM, David Weber wrote:
> we are testing Btrfs as a kvm storage system with
> "defaults,space_cache,nodatacow" mount options and regular snapshots.

That sounds brave - even with "nodatacow" it appeared to me
that using btrfs with often partially overwritten files like
VM images results in excessively fragmented files.
And taking snapshots kind of counteracts "nodatacow".

What does "filefrag" tell about your VM images on btrfs?

(As much as I like btrfs for other purposes, I currently stay
with XFS for VM images, database files and alike.)

Regards,

Lutz Vieweg

Re: WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

[David Weber]

Lutz Vieweg <lvml <at> 5t9.de> writes:

> 
> On 06/29/2015 11:35 AM, David Weber wrote:
> > we are testing Btrfs as a kvm storage system with
> > "defaults,space_cache,nodatacow" mount options and regular snapshots.
> 
> That sounds brave - even with "nodatacow" it appeared to me
> that using btrfs with often partially overwritten files like
> VM images results in excessively fragmented files.
> And taking snapshots kind of counteracts "nodatacow".
> 
> What does "filefrag" tell about your VM images on btrfs?
> 
> (As much as I like btrfs for other purposes, I currently stay
> with XFS for VM images, database files and alike.)

Fragmentation can create a performance hit and you have to ponder if the 
features are worth it.
I use Btrfs with regular snapshots on my workstation with Virtualbox since 
years. Running filefrag on the image doesn't complete after hours but the 
IO of the VM is still "fast enough". But the definition of "fast enough" 
can vary heavily.
If it becomes to slow I will try a btrfs fi deframent or just recreate the 
image with a no reflink copy.

Cheers,
David

Re: WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

[Lutz Vieweg]

On 06/30/2015 11:10 AM, David Weber wrote:
> I use Btrfs with regular snapshots on my workstation with Virtualbox since
> years. Running filefrag on the image doesn't complete after hours but the
> IO of the VM is still "fast enough". But the definition of "fast enough"
> can vary heavily.

How do you backup the VM images? If filefrag takes hours to complete,
I'd assume that attempting a sequential read of the whole file (from a
backup software) takes just as long.

> If it becomes to slow I will try a btrfs fi deframent or just recreate the
> image with a no reflink copy.

Chances are this will also take hours.

Regards,

Lutz Vieweg

Re: WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

[David Weber]

Lutz Vieweg <lvml <at> 5t9.de> writes:

> 
> On 06/30/2015 11:10 AM, David Weber wrote:
> > I use Btrfs with regular snapshots on my workstation with Virtualbox 
since
> > years. Running filefrag on the image doesn't complete after hours but the
> > IO of the VM is still "fast enough". But the definition of "fast enough"
> > can vary heavily.
> 
> How do you backup the VM images? If filefrag takes hours to complete,
> I'd assume that attempting a sequential read of the whole file (from a
> backup software) takes just as long.

I use send/receive. With an SSD it doesn't take that long.

> 
> > If it becomes to slow I will try a btrfs fi deframent or just recreate 
the
> > image with a no reflink copy.
> 
> Chances are this will also take hours.

A sequential read (dd if=Windows7.vdi of=/dev/zero bs=1M) gives nearly raw 
SSD Speed. But CPU consumption is indeed quite high...
> 
> Regards,
> 
> Lutz Vieweg

Re: WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

[Lutz Vieweg]

On 07/07/2015 02:28 PM, David Weber wrote:
>> How do you backup the VM images? If filefrag takes hours to complete,
>> I'd assume that attempting a sequential read of the whole file (from a
>> backup software) takes just as long.
>
> I use send/receive. With an SSD it doesn't take that long.

Hmmm. The kind of backup you can restore only as a whole,
not individual files... plus you have to hope there are never
systematic bugs in btrfs that would make an individual file
as unreadable from the backup (undetected by send/receive)
as it is from the original. :-)

I for one prefer backups to read/write individual files and
to use a different format for storage.

> A sequential read (dd if=Windows7.vdi of=/dev/zero bs=1M) gives nearly raw
> SSD Speed. But CPU consumption is indeed quite high...

Ok, if all the VM image data is on SSDs, the high fragmentation
might be less of an issue to you.

Regards,

Lutz Vieweg

Re: WARNING: at /fs/btrfs/extent-tree.c:4029 btrfs_free_reserved_data_space

[Liu Bo]

On Mon, Jun 29, 2015 at 11:35:12AM +0200, David Weber wrote:
> Hi,
> 
> we are testing Btrfs as a kvm storage system with 
> "defaults,space_cache,nodatacow" mount options and regular snapshots. While 
> testing we sometimes get these warnings:
> WARNING: CPU: 5 PID: 11335 at /home/kernel/COD/linux/fs/btrfs/extent-
> tree.c:4029 btrfs_free_reserved_data_space+0x102/0x110 [btrfs]()
> 
> Full log:
> https://gist.github.com/anonymous/5f672f72899a3d52c20c
> 
> Once they start, they flood dmesg with several messages per second. After a 
> reboot, it takes a while until they start to appear again. 
> We have seen them with Linux 4.0.6 and 4.1
> 
> This was already reported a few times:
> https://bugzilla.opensuse.org/show_bug.cgi?id=904023
> https://bugzilla.redhat.com/show_bug.cgi?id=1173937
> http://www.spinics.net/lists/linux-btrfs/msg39069.html
> https://lkml.org/lkml/2014/10/22/899
> 
> Always with no solution
> Is this a real problem or can we safely ignore this?

Didn't notice this mail but it looks like thia patch is promising to fix
your issue, worth giving a shot.

https://patchwork.kernel.org/patch/6623001/
Btrfs: fix warning of bytes_may_use

BTW, this's been queued for the next merge, so you can try the latest
btrfs from integration-4.2 of the offical btrfs repo[1].

[1]: https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git

Thanks,

-liubo

> 
> Cheers,
> David
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html