WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Rich Rauenzahn]

I've filed a bug on this kernel trace -- I get 100's of these a day.
I'd like to make them go away ....

https://bugzilla.kernel.org/show_bug.cgi?id=196949

[    4.747356] ------------[ cut here ]------------
[    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
btrfs_update_device+0x1c5/0x1d0 [btrfs]
[    4.747377] Modules linked in: nfs_acl lockd grace sunrpc ip_tables
btrfs xor raid6_pq sd_mod crc32c_intel firewire_ohci igb ahci
 firewire_core crc_itu_t dca libahci i915 libata i2c_algo_bit e1000e
drm_kms_helper ptp syscopyarea sysfillrect pps_core sysimgblt f
b_sys_fops drm video
[    4.747385] CPU: 3 PID: 439 Comm: btrfs-cleaner Not tainted
4.13.2-1.el7.elrepo.x86_64 #1
[    4.747385] Hardware name: Supermicro X10SAE/X10SAE, BIOS 2.0a 05/09/2014
[    4.747386] task: ffff88040cdcae80 task.stack: ffffc900021f4000
[    4.747396] RIP: 0010:btrfs_update_device+0x1c5/0x1d0 [btrfs]
[    4.747396] RSP: 0018:ffffc900021f7d00 EFLAGS: 00010206
[    4.747397] RAX: 0000000000000fff RBX: ffff880407b7aa80 RCX: 0000001bc6c71e00
[    4.747397] RDX: ffff880000000000 RSI: ffff880404cd3f3c RDI: ffff880409417b58
[    4.747398] RBP: ffffc900021f7d48 R08: 0000000000003f60 R09: ffffc900021f7cb8
[    4.747398] R10: 0000000000001000 R11: 0000000000000003 R12: ffff88040559f800
[    4.747398] R13: 0000000000000000 R14: ffff880409417b58 R15: 0000000000003f3c
[    4.747399] FS:  0000000000000000(0000) GS:ffff88041fac0000(0000)
knlGS:0000000000000000
[    4.747399] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    4.747400] CR2: 00007f29c3000248 CR3: 00000004056a4000 CR4: 00000000001406e0
[    4.747400] Call Trace:
[    4.747410]  btrfs_remove_chunk+0x2fb/0x8b0 [btrfs]
[    4.747418]  btrfs_delete_unused_bgs+0x363/0x440 [btrfs]
[    4.747426]  cleaner_kthread+0x150/0x180 [btrfs]
[    4.747429]  kthread+0x109/0x140
[    4.747436]  ? btree_invalidatepage+0xa0/0xa0 [btrfs]
[    4.747437]  ? kthread_park+0x60/0x60
[    4.747439]  ret_from_fork+0x25/0x30
[    4.747439] Code: 10 00 00 00 4c 89 fe e8 8a 30 ff ff 4c 89 f7 e8
32 f6 fc ff e9 d3 fe ff ff b8 f4 ff ff ff e9 d4 fe ff ff 0f 1f 00 e8
bb 2e 9e e0 <0f> ff eb af 0f 1f 80 00 00 00 00 0f 1f 44 00 00 55 31 d2
be 02
[    4.747450] ---[ end trace 1ef80a625983d73b ]---

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

On 2017年09月19日 23:56, Rich Rauenzahn wrote:
> I've filed a bug on this kernel trace -- I get 100's of these a day.
> I'd like to make them go away ....
> 
> https://bugzilla.kernel.org/show_bug.cgi?id=196949
> 
> [    4.747356] ------------[ cut here ]------------
> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
> btrfs_update_device+0x1c5/0x1d0 [btrfs]

Is that line the following WARN_ON()?
---
static inline void btrfs_set_device_total_bytes(struct extent_buffer *eb,
						struct btrfs_dev_item *s,
						u64 val)
{
	BUILD_BUG_ON(sizeof(u64) !=
		     sizeof(((struct btrfs_dev_item *)0))->total_bytes);
	WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
	btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), val);
}
---

If so, that means your devices size is not aligned to 4K.

Is your block device still using old 512 block size?
AFAIK nowadays most HDDs are using 4K blocksize and it's recommended to 
use it.

It's not a big problem and one can easily remove the WARN_ON().
But I think we'd better fix the caller to do round_down() before calling 
this function.

Thanks,
Qu

> [    4.747377] Modules linked in: nfs_acl lockd grace sunrpc ip_tables
> btrfs xor raid6_pq sd_mod crc32c_intel firewire_ohci igb ahci
>   firewire_core crc_itu_t dca libahci i915 libata i2c_algo_bit e1000e
> drm_kms_helper ptp syscopyarea sysfillrect pps_core sysimgblt f
> b_sys_fops drm video
> [    4.747385] CPU: 3 PID: 439 Comm: btrfs-cleaner Not tainted
> 4.13.2-1.el7.elrepo.x86_64 #1
> [    4.747385] Hardware name: Supermicro X10SAE/X10SAE, BIOS 2.0a 05/09/2014
> [    4.747386] task: ffff88040cdcae80 task.stack: ffffc900021f4000
> [    4.747396] RIP: 0010:btrfs_update_device+0x1c5/0x1d0 [btrfs]
> [    4.747396] RSP: 0018:ffffc900021f7d00 EFLAGS: 00010206
> [    4.747397] RAX: 0000000000000fff RBX: ffff880407b7aa80 RCX: 0000001bc6c71e00
> [    4.747397] RDX: ffff880000000000 RSI: ffff880404cd3f3c RDI: ffff880409417b58
> [    4.747398] RBP: ffffc900021f7d48 R08: 0000000000003f60 R09: ffffc900021f7cb8
> [    4.747398] R10: 0000000000001000 R11: 0000000000000003 R12: ffff88040559f800
> [    4.747398] R13: 0000000000000000 R14: ffff880409417b58 R15: 0000000000003f3c
> [    4.747399] FS:  0000000000000000(0000) GS:ffff88041fac0000(0000)
> knlGS:0000000000000000
> [    4.747399] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    4.747400] CR2: 00007f29c3000248 CR3: 00000004056a4000 CR4: 00000000001406e0
> [    4.747400] Call Trace:
> [    4.747410]  btrfs_remove_chunk+0x2fb/0x8b0 [btrfs]
> [    4.747418]  btrfs_delete_unused_bgs+0x363/0x440 [btrfs]
> [    4.747426]  cleaner_kthread+0x150/0x180 [btrfs]
> [    4.747429]  kthread+0x109/0x140
> [    4.747436]  ? btree_invalidatepage+0xa0/0xa0 [btrfs]
> [    4.747437]  ? kthread_park+0x60/0x60
> [    4.747439]  ret_from_fork+0x25/0x30
> [    4.747439] Code: 10 00 00 00 4c 89 fe e8 8a 30 ff ff 4c 89 f7 e8
> 32 f6 fc ff e9 d3 fe ff ff b8 f4 ff ff ff e9 d4 fe ff ff 0f 1f 00 e8
> bb 2e 9e e0 <0f> ff eb af 0f 1f 80 00 00 00 00 0f 1f 44 00 00 55 31 d2
> be 02
> [    4.747450] ---[ end trace 1ef80a625983d73b ]---
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Rich Rauenzahn]

On 9/19/2017 5:31 PM, Qu Wenruo wrote:
> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>
> Is that line the following WARN_ON()?
> ---
> static inline void btrfs_set_device_total_bytes(struct extent_buffer *eb,
>                         struct btrfs_dev_item *s,
>                         u64 val)
> {
>     BUILD_BUG_ON(sizeof(u64) !=
>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), 
> val);
> }
> ---
>
> If so, that means your devices size is not aligned to 4K.
>
> Is your block device still using old 512 block size?
> AFAIK nowadays most HDDs are using 4K blocksize and it's recommended 
> to use it.
>
> It's not a big problem and one can easily remove the WARN_ON().
> But I think we'd better fix the caller to do round_down() before 
> calling this function.
>

That's interesting!  I believe I made an effort to align them when I set 
it up years ago, but never knew how to verify.

I have three mirrored filesystems:

$ for i in /dev/sd[abcdef]; do sudo gdisk -l $i; done
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
   MBR: protective
   BSD: not present
   APM: not present
   GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 03FFF12A-2EF5-4916-92D9-59C244EFDF5B
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code Name
    1            2048      3907029134   1.8 TiB     8300 BTRFS BACKUPS
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
   MBR: protective
   BSD: not present
   APM: not present
   GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdb: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): B0CF9AC1-7DD0-46CD-AF62-2E54761686C7
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code Name
    1            2048      3907029134   1.8 TiB     8300 BTRFS BACKUPS
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
   MBR: protective
   BSD: not present
   APM: not present
   GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdc: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 21CA2468-8185-4ECA-B63D-8A9A1557F302
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code Name
    1            2048      3907029134   1.8 TiB     8300 BTRFS MEDIA
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
   MBR: protective
   BSD: not present
   APM: not present
   GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdd: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 5214ED9D-769A-4DF8-886F-8EEC3FDD4D0D
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 8-sector boundaries
Total free space is 6 sectors (3.0 KiB)

Number  Start (sector)    End (sector)  Size       Code Name
    1              40      3907029134   1.8 TiB     8300 BTRFS MEDIA
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
   MBR: protective
   BSD: not present
   APM: not present
   GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sde: 234441648 sectors, 111.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): D0E4B890-0002-4DA1-B011-24CE7FD435FE
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 234441614
Partitions will be aligned on 2048-sector boundaries
Total free space is 2925 sectors (1.4 MiB)

Number  Start (sector)    End (sector)  Size       Code Name
    1            2048          411647   200.0 MiB   EF00 EFI System 
Partition
    2          411648         1435647   500.0 MiB   0700 Primary /boot
    3         1435648       234440703   111.1 GiB   0700 Primary /home
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
   MBR: protective
   BSD: not present
   APM: not present
   GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdf: 234441648 sectors, 111.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): D1523F65-B975-4A94-8519-3D1679A50342
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 234441614
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code Name
    1            2048          411647   200.0 MiB   EF00 EFI System
    2          411648         1435647   500.0 MiB   0700 Secondary /boot
    3         1435648       234441614   111.1 GiB   0700 Secondary /home

.....and one is aligned differently!

Could it be /dev/sdd that's the issue?  But it's aligned at 4096 -- so 
I'm not sure that's the issue after all.

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

On 2017年09月20日 12:49, Rich Rauenzahn wrote:
> 
> 
> On 9/19/2017 5:31 PM, Qu Wenruo wrote:
>> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>>
>> Is that line the following WARN_ON()?
>> ---
>> static inline void btrfs_set_device_total_bytes(struct extent_buffer *eb,
>>                         struct btrfs_dev_item *s,
>>                         u64 val)
>> {
>>     BUILD_BUG_ON(sizeof(u64) !=
>>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), 
>> val);
>> }
>> ---
>>
>> If so, that means your devices size is not aligned to 4K.
>>
>> Is your block device still using old 512 block size?
>> AFAIK nowadays most HDDs are using 4K blocksize and it's recommended 
>> to use it.
>>
>> It's not a big problem and one can easily remove the WARN_ON().
>> But I think we'd better fix the caller to do round_down() before 
>> calling this function.
>>
> 
> That's interesting!  I believe I made an effort to align them when I set 
> it up years ago, but never knew how to verify.

Well, best verifying if that's the line causing the warning, since I 
don't have the source of RedHat kernel.

> 
> I have three mirrored filesystems:
> 
> $ for i in /dev/sd[abcdef]; do sudo gdisk -l $i; done
> GPT fdisk (gdisk) version 0.8.6
> 
> Partition table scan:
>    MBR: protective
>    BSD: not present
>    APM: not present
>    GPT: present
> 
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sda: 3907029168 sectors, 1.8 TiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): 03FFF12A-2EF5-4916-92D9-59C244EFDF5B
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 3907029134
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2014 sectors (1007.0 KiB)
> 
> Number  Start (sector)    End (sector)  Size       Code Name
>     1            2048      3907029134   1.8 TiB     8300 BTRFS BACKUPS
> GPT fdisk (gdisk) version 0.8.6
> 
> Partition table scan:
>    MBR: protective
>    BSD: not present
>    APM: not present
>    GPT: present
> 
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sdb: 3907029168 sectors, 1.8 TiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): B0CF9AC1-7DD0-46CD-AF62-2E54761686C7
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 3907029134
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2014 sectors (1007.0 KiB)
> 
> Number  Start (sector)    End (sector)  Size       Code Name
>     1            2048      3907029134   1.8 TiB     8300 BTRFS BACKUPS
> GPT fdisk (gdisk) version 0.8.6
> 
> Partition table scan:
>    MBR: protective
>    BSD: not present
>    APM: not present
>    GPT: present
> 
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sdc: 3907029168 sectors, 1.8 TiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): 21CA2468-8185-4ECA-B63D-8A9A1557F302
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 3907029134
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2014 sectors (1007.0 KiB)
> 
> Number  Start (sector)    End (sector)  Size       Code Name
>     1            2048      3907029134   1.8 TiB     8300 BTRFS MEDIA
> GPT fdisk (gdisk) version 0.8.6
> 
> Partition table scan:
>    MBR: protective
>    BSD: not present
>    APM: not present
>    GPT: present
> 
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sdd: 3907029168 sectors, 1.8 TiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): 5214ED9D-769A-4DF8-886F-8EEC3FDD4D0D
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 3907029134
> Partitions will be aligned on 8-sector boundaries
> Total free space is 6 sectors (3.0 KiB)
> 
> Number  Start (sector)    End (sector)  Size       Code Name
>     1              40      3907029134   1.8 TiB     8300 BTRFS MEDIA
> GPT fdisk (gdisk) version 0.8.6

At least this size is not aligned to 4K.

> 
> Partition table scan:
>    MBR: protective
>    BSD: not present
>    APM: not present
>    GPT: present
> 
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sde: 234441648 sectors, 111.8 GiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): D0E4B890-0002-4DA1-B011-24CE7FD435FE
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 234441614
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2925 sectors (1.4 MiB)
> 
> Number  Start (sector)    End (sector)  Size       Code Name
>     1            2048          411647   200.0 MiB   EF00 EFI System 
> Partition
>     2          411648         1435647   500.0 MiB   0700 Primary /boot
>     3         1435648       234440703   111.1 GiB   0700 Primary /home
> GPT fdisk (gdisk) version 0.8.6
> 
> Partition table scan:
>    MBR: protective
>    BSD: not present
>    APM: not present
>    GPT: present
> 
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sdf: 234441648 sectors, 111.8 GiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): D1523F65-B975-4A94-8519-3D1679A50342
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 234441614
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2014 sectors (1007.0 KiB)
> 
> Number  Start (sector)    End (sector)  Size       Code Name
>     1            2048          411647   200.0 MiB   EF00 EFI System
>     2          411648         1435647   500.0 MiB   0700 Secondary /boot
>     3         1435648       234441614   111.1 GiB   0700 Secondary /home
> 
> .....and one is aligned differently!
> 
> Could it be /dev/sdd that's the issue?  But it's aligned at 4096 -- so 
> I'm not sure that's the issue after all.

Its start sector is aligned, but end point is not, so the size is not 
aligned either.

BTW, is /dev/sdd added to btrfs using "btrfs device add"?
In my test, if making btrfs on a unaligned file, it will round down to 
its sectorsize boundary.

So I'm wondering if it's caused by added new btrfs device.

Thanks,
Qu

> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

On 2017年09月20日 12:59, Qu Wenruo wrote:
> 
> 
> On 2017年09月20日 12:49, Rich Rauenzahn wrote:
>>
>>
>> On 9/19/2017 5:31 PM, Qu Wenruo wrote:
>>> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>>>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>>>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>>>
>>> Is that line the following WARN_ON()?
>>> ---
>>> static inline void btrfs_set_device_total_bytes(struct extent_buffer 
>>> *eb,
>>>                         struct btrfs_dev_item *s,
>>>                         u64 val)
>>> {
>>>     BUILD_BUG_ON(sizeof(u64) !=
>>>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>>>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>>>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), 
>>> val);
>>> }
>>> ---
>>>
>>> If so, that means your devices size is not aligned to 4K.
>>>
>>> Is your block device still using old 512 block size?
>>> AFAIK nowadays most HDDs are using 4K blocksize and it's recommended 
>>> to use it.
>>>
>>> It's not a big problem and one can easily remove the WARN_ON().
>>> But I think we'd better fix the caller to do round_down() before 
>>> calling this function.
>>>
>>
>> That's interesting!  I believe I made an effort to align them when I 
>> set it up years ago, but never knew how to verify.
> 
> Well, best verifying if that's the line causing the warning, since I 
> don't have the source of RedHat kernel.
> 
>>
>> I have three mirrored filesystems:
>>
[snip]
>>
>> Number  Start (sector)    End (sector)  Size       Code Name
>>     1              40      3907029134   1.8 TiB     8300 BTRFS MEDIA
>> GPT fdisk (gdisk) version 0.8.6
> 
> At least this size is not aligned to 4K.
> 
>>
>> Partition table scan:
[snip]
>>
>> .....and one is aligned differently!
>>
>> Could it be /dev/sdd that's the issue?  But it's aligned at 4096 -- so 
>> I'm not sure that's the issue after all.
> 
> Its start sector is aligned, but end point is not, so the size is not 
> aligned either.
> 
> BTW, is /dev/sdd added to btrfs using "btrfs device add"?
> In my test, if making btrfs on a unaligned file, it will round down to 
> its sectorsize boundary.

Confirmed that "btrfs device add" won't round down the size.
Check the btrfs-debug-tree output:
------
         item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
                 devid 1 total_bytes 10737418240 bytes_used 2172649472
                 io_align 4096 io_width 4096 sector_size 4096 type 0
                 generation 0 start_offset 0 dev_group 0
                 seek_speed 0 bandwidth 0
                 uuid 243a1117-ca31-4d87-8656-81c5630aafb2
                 fsid 6452cde7-14d5-4541-aa07-b265a400bad0
         item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
                 devid 2 total_bytes 1073742336 bytes_used 0
                 io_align 4096 io_width 4096 sector_size 4096 type 0
                 generation 0 start_offset 0 dev_group 0
                 seek_speed 0 bandwidth 0
                 uuid 6bb07260-d230-4e22-88b1-1eabb46622ed
                 fsid 6452cde7-14d5-4541-aa07-b265a400bad0
------

Where first device is completely aligned, the 2nd device which is just 
1G + 512, definitely not aligned.

So if you're using single device purely created by mkfs.btrfs, you're OK.
But if any new device added, you're not OK and causing the false alert.

Any way, it should not be hard to fix.
Just remove the WARN_ON() and add extra round_down when adding device.

Thanks for the report,
Qu

> 
> So I'm wondering if it's caused by added new btrfs device.
> 
> Thanks,
> Qu
> 
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

On 2017年09月20日 13:10, Qu Wenruo wrote:
> 
> 
> On 2017年09月20日 12:59, Qu Wenruo wrote:
>>
>>
>> On 2017年09月20日 12:49, Rich Rauenzahn wrote:
>>>
>>>
>>> On 9/19/2017 5:31 PM, Qu Wenruo wrote:
>>>> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>>>>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>>>>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>>>>
>>>> Is that line the following WARN_ON()?
>>>> ---
>>>> static inline void btrfs_set_device_total_bytes(struct extent_buffer 
>>>> *eb,
>>>>                         struct btrfs_dev_item *s,
>>>>                         u64 val)
>>>> {
>>>>     BUILD_BUG_ON(sizeof(u64) !=
>>>>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>>>>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>>>>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, 
>>>> total_bytes), val);
>>>> }
>>>> ---
>>>>
>>>> If so, that means your devices size is not aligned to 4K.
>>>>
>>>> Is your block device still using old 512 block size?
>>>> AFAIK nowadays most HDDs are using 4K blocksize and it's recommended 
>>>> to use it.
>>>>
>>>> It's not a big problem and one can easily remove the WARN_ON().
>>>> But I think we'd better fix the caller to do round_down() before 
>>>> calling this function.
>>>>
>>>
>>> That's interesting!  I believe I made an effort to align them when I 
>>> set it up years ago, but never knew how to verify.
>>
>> Well, best verifying if that's the line causing the warning, since I 
>> don't have the source of RedHat kernel.
>>
>>>
>>> I have three mirrored filesystems:
>>>
> [snip]
>>>
>>> Number  Start (sector)    End (sector)  Size       Code Name
>>>     1              40      3907029134   1.8 TiB     8300 BTRFS MEDIA
>>> GPT fdisk (gdisk) version 0.8.6
>>
>> At least this size is not aligned to 4K.
>>
>>>
>>> Partition table scan:
> [snip]
>>>
>>> .....and one is aligned differently!
>>>
>>> Could it be /dev/sdd that's the issue?  But it's aligned at 4096 -- 
>>> so I'm not sure that's the issue after all.
>>
>> Its start sector is aligned, but end point is not, so the size is not 
>> aligned either.
>>
>> BTW, is /dev/sdd added to btrfs using "btrfs device add"?
>> In my test, if making btrfs on a unaligned file, it will round down to 
>> its sectorsize boundary.
> 
> Confirmed that "btrfs device add" won't round down the size.
> Check the btrfs-debug-tree output:
> ------
>          item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
>                  devid 1 total_bytes 10737418240 bytes_used 2172649472
>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>                  generation 0 start_offset 0 dev_group 0
>                  seek_speed 0 bandwidth 0
>                  uuid 243a1117-ca31-4d87-8656-81c5630aafb2
>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>          item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
>                  devid 2 total_bytes 1073742336 bytes_used 0
>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>                  generation 0 start_offset 0 dev_group 0
>                  seek_speed 0 bandwidth 0
>                  uuid 6bb07260-d230-4e22-88b1-1eabb46622ed
>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
> ------

Sorry, the output is from v4.12.x, so no kernel warning nor the patch 
rounding down the value.

> 
> Where first device is completely aligned, the 2nd device which is just 
> 1G + 512, definitely not aligned.
> 
> So if you're using single device purely created by mkfs.btrfs, you're OK.
> But if any new device added, you're not OK and causing the false alert.
> 
> Any way, it should not be hard to fix.
> Just remove the WARN_ON() and add extra round_down when adding device.

In v4.13 kernel, the newly added devices are in fact rounded down.
But existing device doesn't get the round down.

So it's recommended to resize (shrink) your fs for very small size to 
fix it if you don't want to wait for the kernel fix.

Thanks,
Qu
> 
> Thanks for the report,
> Qu
> 
>>
>> So I'm wondering if it's caused by added new btrfs device.
>>
>> Thanks,
>> Qu
>>
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe 
>>> linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[nborisov]

On 2017-09-20 07:39, Qu Wenruo wrote:
> On 2017年09月20日 13:10, Qu Wenruo wrote:
>> 
>> 
>> On 2017年09月20日 12:59, Qu Wenruo wrote:
>>> 
>>> 
>>> On 2017年09月20日 12:49, Rich Rauenzahn wrote:
>>>> 
>>>> 
>>>> On 9/19/2017 5:31 PM, Qu Wenruo wrote:
>>>>> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>>>>>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>>>>>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>>>>> 
>>>>> Is that line the following WARN_ON()?
>>>>> ---
>>>>> static inline void btrfs_set_device_total_bytes(struct 
>>>>> extent_buffer *eb,
>>>>>                         struct btrfs_dev_item *s,
>>>>>                         u64 val)
>>>>> {
>>>>>     BUILD_BUG_ON(sizeof(u64) !=
>>>>>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>>>>>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>>>>>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, 
>>>>> total_bytes), val);
>>>>> }
>>>>> ---
>>>>> 
>>>>> If so, that means your devices size is not aligned to 4K.
>>>>> 
>>>>> Is your block device still using old 512 block size?
>>>>> AFAIK nowadays most HDDs are using 4K blocksize and it's 
>>>>> recommended to use it.
>>>>> 
>>>>> It's not a big problem and one can easily remove the WARN_ON().
>>>>> But I think we'd better fix the caller to do round_down() before 
>>>>> calling this function.
>>>>> 
>>>> 
>>>> That's interesting!  I believe I made an effort to align them when I 
>>>> set it up years ago, but never knew how to verify.
>>> 
>>> Well, best verifying if that's the line causing the warning, since I 
>>> don't have the source of RedHat kernel.
>>> 
>>>> 
>>>> I have three mirrored filesystems:
>>>> 
>> [snip]
>>>> 
>>>> Number  Start (sector)    End (sector)  Size       Code Name
>>>>     1              40      3907029134   1.8 TiB     8300 BTRFS MEDIA
>>>> GPT fdisk (gdisk) version 0.8.6
>>> 
>>> At least this size is not aligned to 4K.
>>> 
>>>> 
>>>> Partition table scan:
>> [snip]
>>>> 
>>>> .....and one is aligned differently!
>>>> 
>>>> Could it be /dev/sdd that's the issue?  But it's aligned at 4096 -- 
>>>> so I'm not sure that's the issue after all.
>>> 
>>> Its start sector is aligned, but end point is not, so the size is not 
>>> aligned either.
>>> 
>>> BTW, is /dev/sdd added to btrfs using "btrfs device add"?
>>> In my test, if making btrfs on a unaligned file, it will round down 
>>> to its sectorsize boundary.
>> 
>> Confirmed that "btrfs device add" won't round down the size.
>> Check the btrfs-debug-tree output:
>> ------
>>          item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
>>                  devid 1 total_bytes 10737418240 bytes_used 2172649472
>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>                  generation 0 start_offset 0 dev_group 0
>>                  seek_speed 0 bandwidth 0
>>                  uuid 243a1117-ca31-4d87-8656-81c5630aafb2
>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>>          item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
>>                  devid 2 total_bytes 1073742336 bytes_used 0
>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>                  generation 0 start_offset 0 dev_group 0
>>                  seek_speed 0 bandwidth 0
>>                  uuid 6bb07260-d230-4e22-88b1-1eabb46622ed
>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>> ------
> 
> Sorry, the output is from v4.12.x, so no kernel warning nor the patch
> rounding down the value.
> 
>> 
>> Where first device is completely aligned, the 2nd device which is just 
>> 1G + 512, definitely not aligned.
>> 
>> So if you're using single device purely created by mkfs.btrfs, you're 
>> OK.
>> But if any new device added, you're not OK and causing the false 
>> alert.
>> 
>> Any way, it should not be hard to fix.
>> Just remove the WARN_ON() and add extra round_down when adding device.
> 
> In v4.13 kernel, the newly added devices are in fact rounded down.
> But existing device doesn't get the round down.

We got a report internally at Suse of this problem and it prevented a 
filesystem from being mounted due to the
following check failing: 
http://elixir.free-electrons.com/linux/latest/source/fs/btrfs/volumes.c#L6893.

Hence I added the rounding down fixes. And this warning was put 
specifically to catch future offenders and see
if I had missed a place to patch it. So removing the warning is the 
wrong solution to the problem. Generally if
balancing kicked in had to resize his disk everything would be back to 
normal.

> 
> So it's recommended to resize (shrink) your fs for very small size to
> fix it if you don't want to wait for the kernel fix.
> 
> Thanks,
> Qu
>> 
>> Thanks for the report,
>> Qu
>> 
>>> 
>>> So I'm wondering if it's caused by added new btrfs device.
>>> 
>>> Thanks,
>>> Qu
>>> 
>>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>>> linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> -- To unsubscribe from this list: send the line "unsubscribe 
>> linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" 
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

On 2017年09月20日 14:11, nborisov wrote:
> On 2017-09-20 07:39, Qu Wenruo wrote:
>> On 2017年09月20日 13:10, Qu Wenruo wrote:
>>>
>>>
>>> On 2017年09月20日 12:59, Qu Wenruo wrote:
>>>>
>>>>
>>>> On 2017年09月20日 12:49, Rich Rauenzahn wrote:
>>>>>
>>>>>
>>>>> On 9/19/2017 5:31 PM, Qu Wenruo wrote:
>>>>>> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>>>>>>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>>>>>>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>>>>>>
>>>>>> Is that line the following WARN_ON()?
>>>>>> ---
>>>>>> static inline void btrfs_set_device_total_bytes(struct 
>>>>>> extent_buffer *eb,
>>>>>>                         struct btrfs_dev_item *s,
>>>>>>                         u64 val)
>>>>>> {
>>>>>>     BUILD_BUG_ON(sizeof(u64) !=
>>>>>>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>>>>>>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>>>>>>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, 
>>>>>> total_bytes), val);
>>>>>> }
>>>>>> ---
>>>>>>
>>>>>> If so, that means your devices size is not aligned to 4K.
>>>>>>
>>>>>> Is your block device still using old 512 block size?
>>>>>> AFAIK nowadays most HDDs are using 4K blocksize and it's 
>>>>>> recommended to use it.
>>>>>>
>>>>>> It's not a big problem and one can easily remove the WARN_ON().
>>>>>> But I think we'd better fix the caller to do round_down() before 
>>>>>> calling this function.
>>>>>>
>>>>>
>>>>> That's interesting!  I believe I made an effort to align them when 
>>>>> I set it up years ago, but never knew how to verify.
>>>>
>>>> Well, best verifying if that's the line causing the warning, since I 
>>>> don't have the source of RedHat kernel.
>>>>
>>>>>
>>>>> I have three mirrored filesystems:
>>>>>
>>> [snip]
>>>>>
>>>>> Number  Start (sector)    End (sector)  Size       Code Name
>>>>>     1              40      3907029134   1.8 TiB     8300 BTRFS MEDIA
>>>>> GPT fdisk (gdisk) version 0.8.6
>>>>
>>>> At least this size is not aligned to 4K.
>>>>
>>>>>
>>>>> Partition table scan:
>>> [snip]
>>>>>
>>>>> .....and one is aligned differently!
>>>>>
>>>>> Could it be /dev/sdd that's the issue?  But it's aligned at 4096 -- 
>>>>> so I'm not sure that's the issue after all.
>>>>
>>>> Its start sector is aligned, but end point is not, so the size is 
>>>> not aligned either.
>>>>
>>>> BTW, is /dev/sdd added to btrfs using "btrfs device add"?
>>>> In my test, if making btrfs on a unaligned file, it will round down 
>>>> to its sectorsize boundary.
>>>
>>> Confirmed that "btrfs device add" won't round down the size.
>>> Check the btrfs-debug-tree output:
>>> ------
>>>          item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
>>>                  devid 1 total_bytes 10737418240 bytes_used 2172649472
>>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>>                  generation 0 start_offset 0 dev_group 0
>>>                  seek_speed 0 bandwidth 0
>>>                  uuid 243a1117-ca31-4d87-8656-81c5630aafb2
>>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>>>          item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
>>>                  devid 2 total_bytes 1073742336 bytes_used 0
>>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>>                  generation 0 start_offset 0 dev_group 0
>>>                  seek_speed 0 bandwidth 0
>>>                  uuid 6bb07260-d230-4e22-88b1-1eabb46622ed
>>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>>> ------
>>
>> Sorry, the output is from v4.12.x, so no kernel warning nor the patch
>> rounding down the value.
>>
>>>
>>> Where first device is completely aligned, the 2nd device which is 
>>> just 1G + 512, definitely not aligned.
>>>
>>> So if you're using single device purely created by mkfs.btrfs, you're 
>>> OK.
>>> But if any new device added, you're not OK and causing the false alert.
>>>
>>> Any way, it should not be hard to fix.
>>> Just remove the WARN_ON() and add extra round_down when adding device.
>>
>> In v4.13 kernel, the newly added devices are in fact rounded down.
>> But existing device doesn't get the round down.
> 
> We got a report internally at Suse of this problem and it prevented a 
> filesystem from being mounted due to the
> following check failing: 
> http://elixir.free-electrons.com/linux/latest/source/fs/btrfs/volumes.c#L6893. 
> 
> 
> Hence I added the rounding down fixes. And this warning was put 
> specifically to catch future offenders and see
> if I had missed a place to patch it. So removing the warning is the 
> wrong solution to the problem.

Then at least only enable it for BTRFS_DEBUG.

For end user it's just confusing.

I have submitted a patch to do the check at mounting time, and warning 
user to do shrink to fix it.
(Although still removed the WARN_ON)

I think such warning breaks backward compatibility should be as gentle 
as possible for end users.

Thanks,
Qu

> Generally if
> balancing kicked in had to resize his disk everything would be back to 
> normal. >
>>
>> So it's recommended to resize (shrink) your fs for very small size to
>> fix it if you don't want to wait for the kernel fix.
>>
>> Thanks,
>> Qu
>>>
>>> Thanks for the report,
>>> Qu
>>>
>>>>
>>>> So I'm wondering if it's caused by added new btrfs device.
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>>>> linux-btrfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>> linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[nborisov]

On 2017-09-20 08:23, Qu Wenruo wrote:
> On 2017年09月20日 14:11, nborisov wrote:
>> On 2017-09-20 07:39, Qu Wenruo wrote:
>>> On 2017年09月20日 13:10, Qu Wenruo wrote:
>>>> 
>>>> 
>>>> On 2017年09月20日 12:59, Qu Wenruo wrote:
>>>>> 
>>>>> 
>>>>> On 2017年09月20日 12:49, Rich Rauenzahn wrote:
>>>>>> 
>>>>>> 
>>>>>> On 9/19/2017 5:31 PM, Qu Wenruo wrote:
>>>>>>> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>>>>>>>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>>>>>>>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>>>>>>> 
>>>>>>> Is that line the following WARN_ON()?
>>>>>>> ---
>>>>>>> static inline void btrfs_set_device_total_bytes(struct 
>>>>>>> extent_buffer *eb,
>>>>>>>                         struct btrfs_dev_item *s,
>>>>>>>                         u64 val)
>>>>>>> {
>>>>>>>     BUILD_BUG_ON(sizeof(u64) !=
>>>>>>>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>>>>>>>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>>>>>>>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, 
>>>>>>> total_bytes), val);
>>>>>>> }
>>>>>>> ---
>>>>>>> 
>>>>>>> If so, that means your devices size is not aligned to 4K.
>>>>>>> 
>>>>>>> Is your block device still using old 512 block size?
>>>>>>> AFAIK nowadays most HDDs are using 4K blocksize and it's 
>>>>>>> recommended to use it.
>>>>>>> 
>>>>>>> It's not a big problem and one can easily remove the WARN_ON().
>>>>>>> But I think we'd better fix the caller to do round_down() before 
>>>>>>> calling this function.
>>>>>>> 
>>>>>> 
>>>>>> That's interesting!  I believe I made an effort to align them when 
>>>>>> I set it up years ago, but never knew how to verify.
>>>>> 
>>>>> Well, best verifying if that's the line causing the warning, since 
>>>>> I don't have the source of RedHat kernel.
>>>>> 
>>>>>> 
>>>>>> I have three mirrored filesystems:
>>>>>> 
>>>> [snip]
>>>>>> 
>>>>>> Number  Start (sector)    End (sector)  Size       Code Name
>>>>>>     1              40      3907029134   1.8 TiB     8300 BTRFS 
>>>>>> MEDIA
>>>>>> GPT fdisk (gdisk) version 0.8.6
>>>>> 
>>>>> At least this size is not aligned to 4K.
>>>>> 
>>>>>> 
>>>>>> Partition table scan:
>>>> [snip]
>>>>>> 
>>>>>> .....and one is aligned differently!
>>>>>> 
>>>>>> Could it be /dev/sdd that's the issue?  But it's aligned at 4096 
>>>>>> -- so I'm not sure that's the issue after all.
>>>>> 
>>>>> Its start sector is aligned, but end point is not, so the size is 
>>>>> not aligned either.
>>>>> 
>>>>> BTW, is /dev/sdd added to btrfs using "btrfs device add"?
>>>>> In my test, if making btrfs on a unaligned file, it will round down 
>>>>> to its sectorsize boundary.
>>>> 
>>>> Confirmed that "btrfs device add" won't round down the size.
>>>> Check the btrfs-debug-tree output:
>>>> ------
>>>>          item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
>>>>                  devid 1 total_bytes 10737418240 bytes_used 
>>>> 2172649472
>>>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>>>                  generation 0 start_offset 0 dev_group 0
>>>>                  seek_speed 0 bandwidth 0
>>>>                  uuid 243a1117-ca31-4d87-8656-81c5630aafb2
>>>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>>>>          item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
>>>>                  devid 2 total_bytes 1073742336 bytes_used 0
>>>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>>>                  generation 0 start_offset 0 dev_group 0
>>>>                  seek_speed 0 bandwidth 0
>>>>                  uuid 6bb07260-d230-4e22-88b1-1eabb46622ed
>>>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>>>> ------
>>> 
>>> Sorry, the output is from v4.12.x, so no kernel warning nor the patch
>>> rounding down the value.
>>> 
>>>> 
>>>> Where first device is completely aligned, the 2nd device which is 
>>>> just 1G + 512, definitely not aligned.
>>>> 
>>>> So if you're using single device purely created by mkfs.btrfs, 
>>>> you're OK.
>>>> But if any new device added, you're not OK and causing the false 
>>>> alert.
>>>> 
>>>> Any way, it should not be hard to fix.
>>>> Just remove the WARN_ON() and add extra round_down when adding 
>>>> device.
>>> 
>>> In v4.13 kernel, the newly added devices are in fact rounded down.
>>> But existing device doesn't get the round down.
>> 
>> We got a report internally at Suse of this problem and it prevented a 
>> filesystem from being mounted due to the
>> following check failing: 
>> http://elixir.free-electrons.com/linux/latest/source/fs/btrfs/volumes.c#L6893. 
>> Hence I added the rounding down fixes. And this warning was put 
>> specifically to catch future offenders and see
>> if I had missed a place to patch it. So removing the warning is the 
>> wrong solution to the problem.
> 
> Then at least only enable it for BTRFS_DEBUG.

No, the idea is that if a bug in the code causes such a "corruption" we 
ought to be able to catch it in the first instance and not post factum. 
If a bug in btrfs is introduced in a call path which invokes 
btrfs_update_device with misaligned values we won't see which was the 
culprit. If one then enables BTRFS_DEBUG and starts seeing the warnings 
it will likely yield no useful information since the value will already 
be corrupted.

> 
> For end user it's just confusing.
> 
> I have submitted a patch to do the check at mounting time, and warning
> user to do shrink to fix it.
> (Although still removed the WARN_ON)
> 
> I think such warning breaks backward compatibility should be as gentle
> as possible for end users.

How exactly is it breaking backward compatibility?

> 
> Thanks,
> Qu
> 
>> Generally if
>> balancing kicked in had to resize his disk everything would be back to 
>> normal. >
>>> 
>>> So it's recommended to resize (shrink) your fs for very small size to
>>> fix it if you don't want to wait for the kernel fix.
>>> 
>>> Thanks,
>>> Qu
>>>> 
>>>> Thanks for the report,
>>>> Qu
>>>> 
>>>>> 
>>>>> So I'm wondering if it's caused by added new btrfs device.
>>>>> 
>>>>> Thanks,
>>>>> Qu
>>>>> 
>>>>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>>>>> linux-btrfs" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>>> linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>> linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>> -- To unsubscribe from this list: send the line "unsubscribe 
>> linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

On 2017年09月20日 22:14, nborisov wrote:
> On 2017-09-20 08:23, Qu Wenruo wrote:
>> On 2017年09月20日 14:11, nborisov wrote:
>>> On 2017-09-20 07:39, Qu Wenruo wrote:
>>>> On 2017年09月20日 13:10, Qu Wenruo wrote:
>>>>>
>>>>>
>>>>> On 2017年09月20日 12:59, Qu Wenruo wrote:
>>>>>>
>>>>>>
>>>>>> On 2017年09月20日 12:49, Rich Rauenzahn wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 9/19/2017 5:31 PM, Qu Wenruo wrote:
>>>>>>>> On 2017年09月19日 23:56, Rich Rauenzahn wrote:
>>>>>>>>> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
>>>>>>>>> btrfs_update_device+0x1c5/0x1d0 [btrfs]
>>>>>>>>
>>>>>>>> Is that line the following WARN_ON()?
>>>>>>>> ---
>>>>>>>> static inline void btrfs_set_device_total_bytes(struct 
>>>>>>>> extent_buffer *eb,
>>>>>>>>                         struct btrfs_dev_item *s,
>>>>>>>>                         u64 val)
>>>>>>>> {
>>>>>>>>     BUILD_BUG_ON(sizeof(u64) !=
>>>>>>>>              sizeof(((struct btrfs_dev_item *)0))->total_bytes);
>>>>>>>>     WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize)); <<<
>>>>>>>>     btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, 
>>>>>>>> total_bytes), val);
>>>>>>>> }
>>>>>>>> ---
>>>>>>>>
>>>>>>>> If so, that means your devices size is not aligned to 4K.
>>>>>>>>
>>>>>>>> Is your block device still using old 512 block size?
>>>>>>>> AFAIK nowadays most HDDs are using 4K blocksize and it's 
>>>>>>>> recommended to use it.
>>>>>>>>
>>>>>>>> It's not a big problem and one can easily remove the WARN_ON().
>>>>>>>> But I think we'd better fix the caller to do round_down() before 
>>>>>>>> calling this function.
>>>>>>>>
>>>>>>>
>>>>>>> That's interesting!  I believe I made an effort to align them 
>>>>>>> when I set it up years ago, but never knew how to verify.
>>>>>>
>>>>>> Well, best verifying if that's the line causing the warning, since 
>>>>>> I don't have the source of RedHat kernel.
>>>>>>
>>>>>>>
>>>>>>> I have three mirrored filesystems:
>>>>>>>
>>>>> [snip]
>>>>>>>
>>>>>>> Number  Start (sector)    End (sector)  Size       Code Name
>>>>>>>     1              40      3907029134   1.8 TiB     8300 BTRFS MEDIA
>>>>>>> GPT fdisk (gdisk) version 0.8.6
>>>>>>
>>>>>> At least this size is not aligned to 4K.
>>>>>>
>>>>>>>
>>>>>>> Partition table scan:
>>>>> [snip]
>>>>>>>
>>>>>>> .....and one is aligned differently!
>>>>>>>
>>>>>>> Could it be /dev/sdd that's the issue?  But it's aligned at 4096 
>>>>>>> -- so I'm not sure that's the issue after all.
>>>>>>
>>>>>> Its start sector is aligned, but end point is not, so the size is 
>>>>>> not aligned either.
>>>>>>
>>>>>> BTW, is /dev/sdd added to btrfs using "btrfs device add"?
>>>>>> In my test, if making btrfs on a unaligned file, it will round 
>>>>>> down to its sectorsize boundary.
>>>>>
>>>>> Confirmed that "btrfs device add" won't round down the size.
>>>>> Check the btrfs-debug-tree output:
>>>>> ------
>>>>>          item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
>>>>>                  devid 1 total_bytes 10737418240 bytes_used 2172649472
>>>>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>>>>                  generation 0 start_offset 0 dev_group 0
>>>>>                  seek_speed 0 bandwidth 0
>>>>>                  uuid 243a1117-ca31-4d87-8656-81c5630aafb2
>>>>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>>>>>          item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
>>>>>                  devid 2 total_bytes 1073742336 bytes_used 0
>>>>>                  io_align 4096 io_width 4096 sector_size 4096 type 0
>>>>>                  generation 0 start_offset 0 dev_group 0
>>>>>                  seek_speed 0 bandwidth 0
>>>>>                  uuid 6bb07260-d230-4e22-88b1-1eabb46622ed
>>>>>                  fsid 6452cde7-14d5-4541-aa07-b265a400bad0
>>>>> ------
>>>>
>>>> Sorry, the output is from v4.12.x, so no kernel warning nor the patch
>>>> rounding down the value.
>>>>
>>>>>
>>>>> Where first device is completely aligned, the 2nd device which is 
>>>>> just 1G + 512, definitely not aligned.
>>>>>
>>>>> So if you're using single device purely created by mkfs.btrfs, 
>>>>> you're OK.
>>>>> But if any new device added, you're not OK and causing the false 
>>>>> alert.
>>>>>
>>>>> Any way, it should not be hard to fix.
>>>>> Just remove the WARN_ON() and add extra round_down when adding device.
>>>>
>>>> In v4.13 kernel, the newly added devices are in fact rounded down.
>>>> But existing device doesn't get the round down.
>>>
>>> We got a report internally at Suse of this problem and it prevented a 
>>> filesystem from being mounted due to the
>>> following check failing: 
>>> http://elixir.free-electrons.com/linux/latest/source/fs/btrfs/volumes.c#L6893. 
>>> Hence I added the rounding down fixes. And this warning was put 
>>> specifically to catch future offenders and see
>>> if I had missed a place to patch it. So removing the warning is the 
>>> wrong solution to the problem.
>>
>> Then at least only enable it for BTRFS_DEBUG.
> 
> No, the idea is that if a bug in the code causes such a "corruption" we 
> ought to be able to catch it in the first instance and not post factum. 
> If a bug in btrfs is introduced in a call path which invokes 
> btrfs_update_device with misaligned values we won't see which was the 
> culprit. If one then enables BTRFS_DEBUG and starts seeing the warnings 
> it will likely yield no useful information since the value will already 
> be corrupted.

I totally understand the fix and introduced WARN_ON().

But the truth is, the fix (along with Liu Bo's validation checker) 
doesn't consider existing fs with unaligned bytenr.

No user would be happy to see tons of kernel backtrace just because they 
updated their kernel.

> 
>>
>> For end user it's just confusing.
>>
>> I have submitted a patch to do the check at mounting time, and warning
>> user to do shrink to fix it.
>> (Although still removed the WARN_ON)
>>
>> I think such warning breaks backward compatibility should be as gentle
>> as possible for end users.
> 
> How exactly is it breaking backward compatibility?

The definition of total_bytes of DEV_ITEM.
Old: The size of the device. Which can be unaligned.
New: The size btrfs can use. Which must be aligned.


And old valid (although I know its old total_bytes is not that good) fs 
suddenly causes tons of kernel backtrace in latest kernel, for end user, 
this is already a *REGRESSION*.


I know we should have total_bytes aligned from the very beginning (I 
mean the on-disk format design phase, not after developing it blindly 
for years without any specification), but since we have old FSes using 
the old behavior, we should handle it.

We have a lot of method to fix it:
1) Refuse to mount at all, and tell user how to fix it
    Only if we're at the design phase, we're OK to do this.

2) Introduce new incompat/ro_compat flag, just like
    FREE_SPACE_TREE_VALUD

    May seems overkilled, but for backward compatibility, I think it's
    worthy.
    And free space cache tree just did this, and for a even smaller user
    base (big-endian system only).

3) Silently change the value on RW mount
    Since new behavior is good for old kernel, we can just change it
    silently.
    As long as you can ensure chunk allocator from old kernels won't use
    the tailing unaligned part of device. (And I think that's the case)

4) Allow current behavior to co-exist with aligned one.
    My patch, since it's not that deadly.
    As chunk allocator will do the alignment, it should be OK to keep
    total_bytes unaligned.

    You can make later behavior to align the result, but just denying all
    existing total_bytes without introducing new flag is not a good idea.

Thanks,
Qu



> 
>>
>> Thanks,
>> Qu
>>
>>> Generally if
>>> balancing kicked in had to resize his disk everything would be back 
>>> to normal. >
>>>>
>>>> So it's recommended to resize (shrink) your fs for very small size to
>>>> fix it if you don't want to wait for the kernel fix.
>>>>
>>>> Thanks,
>>>> Qu
>>>>>
>>>>> Thanks for the report,
>>>>> Qu
>>>>>
>>>>>>
>>>>>> So I'm wondering if it's caused by added new btrfs device.
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>>
>>>>>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>>>>>> linux-btrfs" in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>>>> linux-btrfs" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>>> linux-btrfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> -- To unsubscribe from this list: send the line "unsubscribe 
>>> linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Rich Rauenzahn]

For the warning -- is there anyway to add in the filesystem/disk causing 
the issue?  I didn't see any identifier in the message that told me 
which it was.

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Rich Rauenzahn]

On 9/19/2017 10:39 PM, Qu Wenruo wrote:
>
> In v4.13 kernel, the newly added devices are in fact rounded down.
> But existing device doesn't get the round down.
>
> So it's recommended to resize (shrink) your fs for very small size to 
> fix it if you don't want to wait for the kernel fix.
>

What's the most direct way to do that?  (Was about to risk breaking the 
mirror and repartitioning!  I'd rather not!)

Hmm -- maybe this worked:

$ sudo btrfs filesystem resize -1m /.MEDIA/
Resize '/.MEDIA/' of '-1m'

> Thanks,
> Qu
>>>
>>> So I'm wondering if it's caused by added new btrfs device.
>>>

Possibly... I've added and removed disks to some of the btrfs fs's.

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Rich Rauenzahn]

On 9/20/2017 9:58 AM, Rich Rauenzahn wrote:
>
> What's the most direct way to do that?  (Was about to risk breaking 
> the mirror and repartitioning!  I'd rather not!)
>
> Hmm -- maybe this worked:
>
> $ sudo btrfs filesystem resize -1m /.MEDIA/
> Resize '/.MEDIA/' of '-1m'
No, doesn't seem to have worked.  I used btrfs fi show --raw /.MEDIA to 
see the size in bytes, and then ...

sudo btrfs filesystem resize N:SIZE /.MEDIA/

Where N was the disk number in the show output, and SIZE was the number 
of bytes to set it to that was a multiple of 4k.

Rich

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

On 2017年09月21日 02:10, Rich Rauenzahn wrote:
> 
> On 9/20/2017 9:58 AM, Rich Rauenzahn wrote:
>>
>> What's the most direct way to do that?  (Was about to risk breaking 
>> the mirror and repartitioning!  I'd rather not!)
>>
>> Hmm -- maybe this worked:
>>
>> $ sudo btrfs filesystem resize -1m /.MEDIA/
>> Resize '/.MEDIA/' of '-1m'
> No, doesn't seem to have worked.  I used btrfs fi show --raw /.MEDIA to 
> see the size in bytes, and then ...
> 
> sudo btrfs filesystem resize N:SIZE /.MEDIA/
> 

Yes, this is what I did to my fs.
While in fact you can do it better by using:

btrfs fi resize N:max <path>

Or

btrfs fi resize N:-1 <path>

The first one will resize the devid N to its max size.
The 2nd one will resize the devid N to current size - 1 (byte).

Both will go through shrink routine, which will do the round_down.

Thanks,
Qu

> Where N was the disk number in the show output, and SIZE was the number 
> of bytes to set it to that was a multiple of 4k.
> 
> Rich

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3218 bytes --]



On 2017/9/19 下午11:56, Rich Rauenzahn wrote:
> I've filed a bug on this kernel trace -- I get 100's of these a day.
> I'd like to make them go away ....

If you want to solve the problem sooner, then mail list is faster than
kernel bugzilla.

For your case, it's just a warning for unaligned device size, normally
caused by older mkfs or older kernel.

You can fix it by using "btrfs rescue fix-device-size <device>" on a
unmounted fs, or use "btrfs resize <devid>:-4K <mnt>" to resize *each*
device of your mounted fs.

I'd prefer "btrfs rescue fix-device-size <device>" if you have the
latest version of btrfs-progs, as it does not only fix the unaligned
device size, but also fix your super block total bytes.

Thanks,
Qu
> 
> https://bugzilla.kernel.org/show_bug.cgi?id=196949
> 
> [    4.747356] ------------[ cut here ]------------
> [    4.747377] WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559
> btrfs_update_device+0x1c5/0x1d0 [btrfs]
> [    4.747377] Modules linked in: nfs_acl lockd grace sunrpc ip_tables
> btrfs xor raid6_pq sd_mod crc32c_intel firewire_ohci igb ahci
>  firewire_core crc_itu_t dca libahci i915 libata i2c_algo_bit e1000e
> drm_kms_helper ptp syscopyarea sysfillrect pps_core sysimgblt f
> b_sys_fops drm video
> [    4.747385] CPU: 3 PID: 439 Comm: btrfs-cleaner Not tainted
> 4.13.2-1.el7.elrepo.x86_64 #1
> [    4.747385] Hardware name: Supermicro X10SAE/X10SAE, BIOS 2.0a 05/09/2014
> [    4.747386] task: ffff88040cdcae80 task.stack: ffffc900021f4000
> [    4.747396] RIP: 0010:btrfs_update_device+0x1c5/0x1d0 [btrfs]
> [    4.747396] RSP: 0018:ffffc900021f7d00 EFLAGS: 00010206
> [    4.747397] RAX: 0000000000000fff RBX: ffff880407b7aa80 RCX: 0000001bc6c71e00
> [    4.747397] RDX: ffff880000000000 RSI: ffff880404cd3f3c RDI: ffff880409417b58
> [    4.747398] RBP: ffffc900021f7d48 R08: 0000000000003f60 R09: ffffc900021f7cb8
> [    4.747398] R10: 0000000000001000 R11: 0000000000000003 R12: ffff88040559f800
> [    4.747398] R13: 0000000000000000 R14: ffff880409417b58 R15: 0000000000003f3c
> [    4.747399] FS:  0000000000000000(0000) GS:ffff88041fac0000(0000)
> knlGS:0000000000000000
> [    4.747399] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    4.747400] CR2: 00007f29c3000248 CR3: 00000004056a4000 CR4: 00000000001406e0
> [    4.747400] Call Trace:
> [    4.747410]  btrfs_remove_chunk+0x2fb/0x8b0 [btrfs]
> [    4.747418]  btrfs_delete_unused_bgs+0x363/0x440 [btrfs]
> [    4.747426]  cleaner_kthread+0x150/0x180 [btrfs]
> [    4.747429]  kthread+0x109/0x140
> [    4.747436]  ? btree_invalidatepage+0xa0/0xa0 [btrfs]
> [    4.747437]  ? kthread_park+0x60/0x60
> [    4.747439]  ret_from_fork+0x25/0x30
> [    4.747439] Code: 10 00 00 00 4c 89 fe e8 8a 30 ff ff 4c 89 f7 e8
> 32 f6 fc ff e9 d3 fe ff ff b8 f4 ff ff ff e9 d4 fe ff ff 0f 1f 00 e8
> bb 2e 9e e0 <0f> ff eb af 0f 1f 80 00 00 00 00 0f 1f 44 00 00 55 31 d2
> be 02
> [    4.747450] ---[ end trace 1ef80a625983d73b ]---
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]