Compatibility matrix kernel/tools

Compatibility matrix kernel/tools

[Cyril Scetbon]

Hi,

Where can I find the compatibility matrix to know which btrfs-tools version should work with a chosen linux kernel ?
-- 
Cyril SCETBON

Re: Compatibility matrix kernel/tools

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

On Wed, Nov 05, 2014 at 09:57:31PM +0100, Cyril Scetbon wrote:
> Hi,
> 
> Where can I find the compatibility matrix to know which btrfs-tools version should work with a chosen linux kernel ?

   Any of them should work with any kernel.

   For normal operation, if the tools are too old, they may not
support newer kernel features -- but that will simply mean you can't
access the feature, not that anything will be broken.

   If you're doing recovery work (btrfs check and friends) then using
the latest released version of the tools is strongly recommended.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
            --- Putting U back in Honor,  Valor, and Trth ---            

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Compatibility matrix kernel/tools

[Qu Wenruo]

-------- Original Message --------
Subject: Re: Compatibility matrix kernel/tools
From: Hugo Mills <hugo@carfax.org.uk>
To: Cyril Scetbon <cyril.scetbon@free.fr>
Date: 2014年11月06日 05:45
> On Wed, Nov 05, 2014 at 09:57:31PM +0100, Cyril Scetbon wrote:
>> Hi,
>>
>> Where can I find the compatibility matrix to know which btrfs-tools version should work with a chosen linux kernel ?
>     Any of them should work with any kernel.
>
>     For normal operation, if the tools are too old, they may not
> support newer kernel features -- but that will simply mean you can't
> access the feature, not that anything will be broken.
>
>     If you're doing recovery work (btrfs check and friends) then using
> the latest released version of the tools is strongly recommended.
>
>     Hugo.
>
As Hugo mentioned, overall any kernel/user tool combination should be OK 
and won't cause disaster.

[Online operations]
More specifically, online(btrfs mounted) operations mostly depend on the 
kernel.
Like subvolume/snapshot device add/remove/replace balance/scrub and 
send/receive should mainly depend on the
kernel version.

[If version not match]
1. Kernel ver > progs ver.
It may happens that progs don't know the new added ioctls, so some 
bleeding edge functions may not be available,
but everything should work fine without problem.(except some known/fixed 
progs bugs)

2. Progs ver > kernel ver.
It may happen some ioctl not supported by kernel, if there is a fallback 
method everything should work without problem,
or progs should prompt the fact that some bleeding edge function is not 
supported by kernel.
Everything should also works fine except some known/fixed kernel bugs, 
which maybe worse than progs bugs.

[Offline operations]
Offline operations includes mkfs, btrfsck and all its 
friends(btrfs-find-roots btrfs-show-super btrfs-debug-tree ...)
Since offline operations needs to do some dirty job like reading/writing 
the superblock, it may cause big compatibility problem
if using some new incompact features.

[If version not match]
1. kernel ver > progs ver.
There maybe some btrfs filesystems that kernel can mount but offline 
tools can't open.
For example, before v0.20-rc1 progs doesn't support skinny metadata, and 
3.17 kernel supports it,
a btrfs created with skinny metadata can't be fscked using v0.20-rc1, 
but kernel can still mount it,
and online operations should be fine.

If not using the new features, it would be OK.

2. kernel ver < progs ver
This maybe even worse.
If you create a fs with latest progs, which may enable some new feature 
like big metadata on *DEFAULT*,
kernels before v3.3 will be unable to even mount it since it can't 
understand the new incompact features.


[Conclusion]
If not using offline operations often, compatibility won't be a big problem.
If using offline operations often, better keep kernel/progs version from 
differing too much.

Also if you want to keep a matrix for it, it may take some time to git 
blame/log/describe...
Tips will be focus on INCOMPACT flags in fs/btrfs/ctree.h and ioctls 
change in fs/btrfs/ioctl.c.

Thanks,
Qu

Re: Compatibility matrix kernel/tools

[Anand Jain]

On 11/06/2014 09:51 AM, Qu Wenruo wrote:
>
> -------- Original Message --------
> Subject: Re: Compatibility matrix kernel/tools
> From: Hugo Mills <hugo@carfax.org.uk>
> To: Cyril Scetbon <cyril.scetbon@free.fr>
> Date: 2014年11月06日 05:45
>> On Wed, Nov 05, 2014 at 09:57:31PM +0100, Cyril Scetbon wrote:
>>> Hi,
>>>
>>> Where can I find the compatibility matrix to know which btrfs-tools
>>> version should work with a chosen linux kernel ?
>>     Any of them should work with any kernel.
>>
>>     For normal operation, if the tools are too old, they may not
>> support newer kernel features -- but that will simply mean you can't
>> access the feature, not that anything will be broken.
>>
>>     If you're doing recovery work (btrfs check and friends) then using
>> the latest released version of the tools is strongly recommended.
>>
>>     Hugo.
>>
> As Hugo mentioned, overall any kernel/user tool combination should be OK
> and won't cause disaster.
>
> [Online operations]
> More specifically, online(btrfs mounted) operations mostly depend on the
> kernel.
> Like subvolume/snapshot device add/remove/replace balance/scrub and
> send/receive should mainly depend on the
> kernel version.
>
> [If version not match]
> 1. Kernel ver > progs ver.
> It may happens that progs don't know the new added ioctls, so some
> bleeding edge functions may not be available,
> but everything should work fine without problem.(except some known/fixed
> progs bugs)
>
> 2. Progs ver > kernel ver.
> It may happen some ioctl not supported by kernel, if there is a fallback
> method everything should work without problem,
> or progs should prompt the fact that some bleeding edge function is not
> supported by kernel.
> Everything should also works fine except some known/fixed kernel bugs,
> which maybe worse than progs bugs.
>
> [Offline operations]
> Offline operations includes mkfs, btrfsck and all its
> friends(btrfs-find-roots btrfs-show-super btrfs-debug-tree ...)
> Since offline operations needs to do some dirty job like reading/writing
> the superblock, it may cause big compatibility problem
> if using some new incompact features.
>
> [If version not match]
> 1. kernel ver > progs ver.
> There maybe some btrfs filesystems that kernel can mount but offline
> tools can't open.
> For example, before v0.20-rc1 progs doesn't support skinny metadata, and
> 3.17 kernel supports it,
> a btrfs created with skinny metadata can't be fscked using v0.20-rc1,
> but kernel can still mount it,
> and online operations should be fine.
>
> If not using the new features, it would be OK.
>
> 2. kernel ver < progs ver
> This maybe even worse.
> If you create a fs with latest progs, which may enable some new feature
> like big metadata on *DEFAULT*,
> kernels before v3.3 will be unable to even mount it since it can't
> understand the new incompact features.
>
>
> [Conclusion]
> If not using offline operations often, compatibility won't be a big
> problem.
> If using offline operations often, better keep kernel/progs version from
> differing too much.
>
> Also if you want to keep a matrix for it, it may take some time to git
> blame/log/describe...
> Tips will be focus on INCOMPACT flags in fs/btrfs/ctree.h and ioctls
> change in fs/btrfs/ioctl.c.

  for the long term its better if (some critical) offline tools are
  moved into the kernel and operate on the disks through the
  /dev/btrfs-control ioctl which in case the disks doesn't have to
  be mounted. btrfs is also a volume manger there should be a way
  to manage the disks in the kernel not necessary only when the device
  is mounted.

Thanks, Anand

> Thanks,
> Qu
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Compatibility matrix kernel/tools

[Cyril Scetbon]

Thank you guys for this status ! I think you should add it somewhere in the documentation cause I'm pretty sure this is a repeated question from users.
-- 
Cyril SCETBON

> On 06 Nov 2014, at 02:51, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
> 
> 
> -------- Original Message --------
> Subject: Re: Compatibility matrix kernel/tools
> From: Hugo Mills <hugo@carfax.org.uk>
> To: Cyril Scetbon <cyril.scetbon@free.fr>
> Date: 2014年11月06日 05:45
>> On Wed, Nov 05, 2014 at 09:57:31PM +0100, Cyril Scetbon wrote:
>>> Hi,
>>> 
>>> Where can I find the compatibility matrix to know which btrfs-tools version should work with a chosen linux kernel ?
>>    Any of them should work with any kernel.
>> 
>>    For normal operation, if the tools are too old, they may not
>> support newer kernel features -- but that will simply mean you can't
>> access the feature, not that anything will be broken.
>> 
>>    If you're doing recovery work (btrfs check and friends) then using
>> the latest released version of the tools is strongly recommended.
>> 
>>    Hugo.
>> 
> As Hugo mentioned, overall any kernel/user tool combination should be OK and won't cause disaster.
> 
> [Online operations]
> More specifically, online(btrfs mounted) operations mostly depend on the kernel.
> Like subvolume/snapshot device add/remove/replace balance/scrub and send/receive should mainly depend on the
> kernel version.
> 
> [If version not match]
> 1. Kernel ver > progs ver.
> It may happens that progs don't know the new added ioctls, so some bleeding edge functions may not be available,
> but everything should work fine without problem.(except some known/fixed progs bugs)
> 
> 2. Progs ver > kernel ver.
> It may happen some ioctl not supported by kernel, if there is a fallback method everything should work without problem,
> or progs should prompt the fact that some bleeding edge function is not supported by kernel.
> Everything should also works fine except some known/fixed kernel bugs, which maybe worse than progs bugs.
> 
> [Offline operations]
> Offline operations includes mkfs, btrfsck and all its friends(btrfs-find-roots btrfs-show-super btrfs-debug-tree ...)
> Since offline operations needs to do some dirty job like reading/writing the superblock, it may cause big compatibility problem
> if using some new incompact features.
> 
> [If version not match]
> 1. kernel ver > progs ver.
> There maybe some btrfs filesystems that kernel can mount but offline tools can't open.
> For example, before v0.20-rc1 progs doesn't support skinny metadata, and 3.17 kernel supports it,
> a btrfs created with skinny metadata can't be fscked using v0.20-rc1, but kernel can still mount it,
> and online operations should be fine.
> 
> If not using the new features, it would be OK.
> 
> 2. kernel ver < progs ver
> This maybe even worse.
> If you create a fs with latest progs, which may enable some new feature like big metadata on *DEFAULT*,
> kernels before v3.3 will be unable to even mount it since it can't understand the new incompact features.
> 
> 
> [Conclusion]
> If not using offline operations often, compatibility won't be a big problem.
> If using offline operations often, better keep kernel/progs version from differing too much.
> 
> Also if you want to keep a matrix for it, it may take some time to git blame/log/describe...
> Tips will be focus on INCOMPACT flags in fs/btrfs/ctree.h and ioctls change in fs/btrfs/ioctl.c.
> 
> Thanks,
> Qu

Re: Compatibility matrix kernel/tools

[Duncan]

Cyril Scetbon posted on Thu, 06 Nov 2014 10:21:47 +0100 as excerpted:

>>> On Wed, Nov 05, 2014 at 09:57:31PM +0100, Cyril Scetbon wrote:
>>>> 
>>>> Where can I find the compatibility matrix to know which btrfs-tools
>>>> version should work with a chosen linux kernel ?

> Thank you guys for this status ! I think you should add it somewhere in
> the documentation cause I'm pretty sure this is a repeated question from
> users.

Looking backward it hasn't been that much of a FAQ, because (despite what 
various commercial distros were willing to support for money) as far as 
the list was concerned btrfs was experimental (and is currently still not 
fully stable), so the very strong recommendation has always been, and 
remains now altho the strength of the recommendation is gradually fading 
as btrfs stabilizes, that users should always run the latest kernel as 
otherwise they are missing fixes for known problems that may well bite 
them if they don't.

Similarly but not as critically and with some differences for userspace.  
One difference was that until fairly recently userspace didn't have 
regular releases, so users of an after all experimental filesystem were 
expected to run git snapshots either built themselves and updated 
regularly, or from the distro, provided the distro updated their 
snapshots at least a couple times a year.  Older versions were mainly 
missing features for the online stuff (as Chris mentioned), and users 
were told they'd obtain best results for the offline stuff with current 
live-git (where the master branch was and is release-maturity-only code) 
or with specific testing patches.

Looking forward, however, as btrfs matures and stabilizes and as part of 
that process the btrfs community and documentation begins to accommodate 
users who routinely run years outdated code for the stability, and who 
expect to do the same thing with btrfs, this question is as you 
suggested, certain to BECOME a FAQ.

So adding it "somewhere", with the most appropriate initial "somewhere" 
likely being the user documentation section FAQ on the wiki, is indeed a 
good idea.

But it's a wiki and as such the expectation is that users themselves do 
the editing.  Go right ahead.  Chris Mason's authoritative explanation's 
a great start. =:^)

OTOH if you're like me you're more comfortable on the list, and editing 
the wiki is a big hassle.  The information will probably get there 
eventually either way, but if you do it you make sure it's done /now/, 
not whenever someone else gets to it.  So if you are comfortable updating 
the wiki, by all means do so, but if not, well, I've never done so 
either, so...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman