Btrfs progs build fails for 4.8

Btrfs progs build fails for 4.8

[Abhay Sachan]

Hi,
I tried building latest btrfs progs on CentOS 6, "./configure" failed
with the error:

checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
configure: error: no definition of FIEMAP_EXTENT_SHARED found

Any ideas what am I lacking in the system?

Regards,
Abhay

Re: Btrfs progs build fails for 4.8

[Qu Wenruo]

On 10/12/2016 09:58 PM, Abhay Sachan wrote:
> Hi,
> I tried building latest btrfs progs on CentOS 6, "./configure" failed
> with the error:
>
> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
> configure: error: no definition of FIEMAP_EXTENT_SHARED found
>
> Any ideas what am I lacking in the system?

Your kernel is too old and its header doesn't support 
FIEMAP_EXTENT_SHARED flag for fiemap ioctl.

And since that flag is very important for tools like "btrfs filesystem 
du", for old kernel doesn't support EXTENT_SHARED flag, we have no 
choice but abort configuration.

Thanks,
Qu

>
> Regards,
> Abhay
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Btrfs progs build fails for 4.8

[Abhay Sachan]

Hi Qu,
I am running latest 4.8.1, which I compiled on the machine itself.

Linux vm88 4.8.1 #1 SMP Thu Oct 13 10:33:08 IST 2016 x86_64 x86_64
x86_64 GNU/Linux

Regards,
Abhay

On Wed, Oct 12, 2016 at 7:31 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
> On 10/12/2016 09:58 PM, Abhay Sachan wrote:
>>
>> Hi,
>> I tried building latest btrfs progs on CentOS 6, "./configure" failed
>> with the error:
>>
>> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
>> configure: error: no definition of FIEMAP_EXTENT_SHARED found
>>
>> Any ideas what am I lacking in the system?
>
>
> Your kernel is too old and its header doesn't support FIEMAP_EXTENT_SHARED
> flag for fiemap ioctl.
>
> And since that flag is very important for tools like "btrfs filesystem du",
> for old kernel doesn't support EXTENT_SHARED flag, we have no choice but
> abort configuration.
>
> Thanks,
> Qu
>
>>
>> Regards,
>> Abhay
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>



-- 
Abhay

Re: Btrfs progs build fails for 4.8

[Eric Sandeen]

On 10/12/16 9:45 AM, Abhay Sachan wrote:
> Hi Qu,
> I am running latest 4.8.1, which I compiled on the machine itself.

You likely still have the fiemap.h from Centos' kernel-headers
rpm, which is from an older kernel and does not define it.
 
-Eric

> Linux vm88 4.8.1 #1 SMP Thu Oct 13 10:33:08 IST 2016 x86_64 x86_64
> x86_64 GNU/Linux
> 
> Regards,
> Abhay
> 
> On Wed, Oct 12, 2016 at 7:31 PM, Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>> On 10/12/2016 09:58 PM, Abhay Sachan wrote:
>>>
>>> Hi,
>>> I tried building latest btrfs progs on CentOS 6, "./configure" failed
>>> with the error:
>>>
>>> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
>>> configure: error: no definition of FIEMAP_EXTENT_SHARED found
>>>
>>> Any ideas what am I lacking in the system?
>>
>>
>> Your kernel is too old and its header doesn't support FIEMAP_EXTENT_SHARED
>> flag for fiemap ioctl.
>>
>> And since that flag is very important for tools like "btrfs filesystem du",
>> for old kernel doesn't support EXTENT_SHARED flag, we have no choice but
>> abort configuration.
>>
>> Thanks,
>> Qu
>>
>>>
>>> Regards,
>>> Abhay
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
> 
> 
> 

Re: Btrfs progs build fails for 4.8

[David Sterba]

On Wed, Oct 12, 2016 at 10:01:27PM +0800, Qu Wenruo wrote:
> 
> 
> On 10/12/2016 09:58 PM, Abhay Sachan wrote:
> > Hi,
> > I tried building latest btrfs progs on CentOS 6, "./configure" failed
> > with the error:
> >
> > checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
> > configure: error: no definition of FIEMAP_EXTENT_SHARED found
> >
> > Any ideas what am I lacking in the system?
> 
> Your kernel is too old and its header doesn't support 
> FIEMAP_EXTENT_SHARED flag for fiemap ioctl.

As this is not the first time, I wonder if we could provide a defintion
of the macro in progs, regardless of the installed headers. Note that
this does not mean the running kernel is old. Previously the user said
he was running a 4.4 kernel [1] (or it could be any other kernel
version). For that combination of headers and running kernel, I think
it's ok to add a fallback definition.

> And since that flag is very important for tools like "btrfs filesystem 
> du", for old kernel doesn't support EXTENT_SHARED flag, we have no 
> choice but abort configuration.

The check was added to configure so it's caught earlier than during
build. The 'fi du' command is useful, but not IMO critical to stop the
build.

[1] http://lkml.kernel.org/r/2f451ba15f1d3dc508ab5439cf4dc8b4@crc.id.au

Re: Btrfs progs build fails for 4.8

[Qu Wenruo]

At 10/13/2016 01:26 AM, David Sterba wrote:
> On Wed, Oct 12, 2016 at 10:01:27PM +0800, Qu Wenruo wrote:
>>
>>
>> On 10/12/2016 09:58 PM, Abhay Sachan wrote:
>>> Hi,
>>> I tried building latest btrfs progs on CentOS 6, "./configure" failed
>>> with the error:
>>>
>>> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
>>> configure: error: no definition of FIEMAP_EXTENT_SHARED found
>>>
>>> Any ideas what am I lacking in the system?
>>
>> Your kernel is too old and its header doesn't support
>> FIEMAP_EXTENT_SHARED flag for fiemap ioctl.
>
> As this is not the first time, I wonder if we could provide a defintion
> of the macro in progs, regardless of the installed headers. Note that
> this does not mean the running kernel is old. Previously the user said
> he was running a 4.4 kernel [1] (or it could be any other kernel
> version). For that combination of headers and running kernel, I think
> it's ok to add a fallback definition.

Yes, fallback definition is good for case like running kernel supports 
SHARED_EXTENT flag, but not kernel header.

But I'm a little concerned about such fallback definition will lead to 
corrupted result for "fi du".

>
>> And since that flag is very important for tools like "btrfs filesystem
>> du", for old kernel doesn't support EXTENT_SHARED flag, we have no
>> choice but abort configuration.
>
> The check was added to configure so it's caught earlier than during
> build. The 'fi du' command is useful, but not IMO critical to stop the
> build.

What about just disabling subcommands using SHARED_EXTENT flag?
Like "fi du".

Thanks,
Qu

>
> [1] http://lkml.kernel.org/r/2f451ba15f1d3dc508ab5439cf4dc8b4@crc.id.au
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

Re: Btrfs progs build fails for 4.8

[David Sterba]

On Thu, Oct 13, 2016 at 03:25:57PM +0800, Qu Wenruo wrote:
> At 10/13/2016 01:26 AM, David Sterba wrote:
> > On Wed, Oct 12, 2016 at 10:01:27PM +0800, Qu Wenruo wrote:
> >>
> >>
> >> On 10/12/2016 09:58 PM, Abhay Sachan wrote:
> >>> Hi,
> >>> I tried building latest btrfs progs on CentOS 6, "./configure" failed
> >>> with the error:
> >>>
> >>> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
> >>> configure: error: no definition of FIEMAP_EXTENT_SHARED found
> >>>
> >>> Any ideas what am I lacking in the system?
> >>
> >> Your kernel is too old and its header doesn't support
> >> FIEMAP_EXTENT_SHARED flag for fiemap ioctl.
> >
> > As this is not the first time, I wonder if we could provide a defintion
> > of the macro in progs, regardless of the installed headers. Note that
> > this does not mean the running kernel is old. Previously the user said
> > he was running a 4.4 kernel [1] (or it could be any other kernel
> > version). For that combination of headers and running kernel, I think
> > it's ok to add a fallback definition.
> 
> Yes, fallback definition is good for case like running kernel supports 
> SHARED_EXTENT flag, but not kernel header.
> 
> But I'm a little concerned about such fallback definition will lead to 
> corrupted result for "fi du".

Well, not corrupted bug wrong. The shared data will be reported as
exclusive.

> >> And since that flag is very important for tools like "btrfs filesystem
> >> du", for old kernel doesn't support EXTENT_SHARED flag, we have no
> >> choice but abort configuration.
> >
> > The check was added to configure so it's caught earlier than during
> > build. The 'fi du' command is useful, but not IMO critical to stop the
> > build.
> 
> What about just disabling subcommands using SHARED_EXTENT flag?
> Like "fi du".

This would need to be checked at runtime, based only on kernel version.
I think we can do that, print a warning in 'fi du' but otherwise let it
continue.

Re: Btrfs progs build fails for 4.8

[Eric Sandeen]

On 10/14/16 10:49 AM, David Sterba wrote:
> On Thu, Oct 13, 2016 at 03:25:57PM +0800, Qu Wenruo wrote:
>> At 10/13/2016 01:26 AM, David Sterba wrote:
>>> On Wed, Oct 12, 2016 at 10:01:27PM +0800, Qu Wenruo wrote:
>>>>
>>>>
>>>> On 10/12/2016 09:58 PM, Abhay Sachan wrote:
>>>>> Hi,
>>>>> I tried building latest btrfs progs on CentOS 6, "./configure" failed
>>>>> with the error:
>>>>>
>>>>> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
>>>>> configure: error: no definition of FIEMAP_EXTENT_SHARED found
>>>>>
>>>>> Any ideas what am I lacking in the system?
>>>>
>>>> Your kernel is too old and its header doesn't support
>>>> FIEMAP_EXTENT_SHARED flag for fiemap ioctl.
>>>
>>> As this is not the first time, I wonder if we could provide a defintion
>>> of the macro in progs, regardless of the installed headers. Note that
>>> this does not mean the running kernel is old. Previously the user said
>>> he was running a 4.4 kernel [1] (or it could be any other kernel
>>> version). For that combination of headers and running kernel, I think
>>> it's ok to add a fallback definition.
>>
>> Yes, fallback definition is good for case like running kernel supports 
>> SHARED_EXTENT flag, but not kernel header.
>>
>> But I'm a little concerned about such fallback definition will lead to 
>> corrupted result for "fi du".
> 
> Well, not corrupted bug wrong. The shared data will be reported as
> exclusive.
> 
>>>> And since that flag is very important for tools like "btrfs filesystem
>>>> du", for old kernel doesn't support EXTENT_SHARED flag, we have no
>>>> choice but abort configuration.
>>>
>>> The check was added to configure so it's caught earlier than during
>>> build. The 'fi du' command is useful, but not IMO critical to stop the
>>> build.
>>
>> What about just disabling subcommands using SHARED_EXTENT flag?
>> Like "fi du".
> 
> This would need to be checked at runtime, based only on kernel version.
> I think we can do that, print a warning in 'fi du' but otherwise let it
> continue.

Checking kernel version is pretty unreliable - distro kernels that backport
will quite possibly fail the version test.  Sure, it's possible to patch
the check back out of userspace in the distro, but one must first know that
such things exist, and look out for them...

Maybe warn on kernel version < 2.6.33 /and/ no shared extents found.

Or, why not just:

1) #ifndef FIEMAP_EXTENT_SHARED / #define FIEMAP_EXTENT_SHARED 0x00002000 / #endif
2) carry on as usual.

Worst case is that you don't get shared accounting on very old kernels.

And FIEMAP_EXTENT_SHARED has been around since 2.6.33 so it's not going
to bite too many people, right?

(If you really wanted to, you could export has_fiemap_shared in the btrfs
features sysfs file, and runtime could see if shared extents are
detectable and if not, print a warning or something ... but given that
this would be a new feature much newer than the fiemap flag I'm not
sure that would do much good.)

-Eric
 

[PATCH] btrfs-progs: define FIEMAP_EXTENT_SHARED locally if needed

[Eric Sandeen]

As it stands today, btrfs-progs won't build if the installed
kernel headers don't define FIEMAP_EXTENT_SHARED.

But that doesn't tell us anything about the capabilities of
the /running/ kernel - so just define it locally if not found
in the fiemap header, and allow the build to proceed.

If run against an old kernel, worst case scenario is that
no shared extents will be reported via the du command.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
---

you can take it or leave it, but I had this locally 
anyway, so if it's helpful here you go :)

diff --git a/cmds-fi-du.c b/cmds-fi-du.c
index a5b66e6..fa8f421 100644
--- a/cmds-fi-du.c
+++ b/cmds-fi-du.c
@@ -29,6 +29,11 @@
 #include <linux/fs.h>
 #include <linux/fiemap.h>
 
+/* Appeared upstream in 2.6.33 */
+#ifndef FIEMAP_EXTENT_SHARED
+#define FIEMAP_EXTENT_SHARED 0x00002000
+#endif
+
 #include "utils.h"
 #include "commands.h"
 #include "kerncompat.h"
diff --git a/configure.ac b/configure.ac
index 8fd8f42..9b87b24 100644
--- a/configure.ac
+++ b/configure.ac
@@ -144,8 +144,6 @@ if test "$DISABLE_BTRFSCONVERT" = 0 && test "x$convertfs" = "x"; then
 	AC_MSG_ERROR([no filesystems for convert, use --disable-convert instead])
 fi
 
-AX_CHECK_DEFINE([linux/fiemap.h], [FIEMAP_EXTENT_SHARED], [],
-		[AC_MSG_ERROR([no definition of FIEMAP_EXTENT_SHARED found])])
 
 dnl Define <NAME>_LIBS= and <NAME>_CFLAGS= by pkg-config
 dnl

Re: Btrfs progs build fails for 4.8

[David Sterba]

On Fri, Oct 14, 2016 at 12:00:56PM -0500, Eric Sandeen wrote:
> On 10/14/16 10:49 AM, David Sterba wrote:
> > On Thu, Oct 13, 2016 at 03:25:57PM +0800, Qu Wenruo wrote:
> >> At 10/13/2016 01:26 AM, David Sterba wrote:
> >>> On Wed, Oct 12, 2016 at 10:01:27PM +0800, Qu Wenruo wrote:
> >>>>
> >>>>
> >>>> On 10/12/2016 09:58 PM, Abhay Sachan wrote:
> >>>>> Hi,
> >>>>> I tried building latest btrfs progs on CentOS 6, "./configure" failed
> >>>>> with the error:
> >>>>>
> >>>>> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
> >>>>> configure: error: no definition of FIEMAP_EXTENT_SHARED found
> >>>>>
> >>>>> Any ideas what am I lacking in the system?
> >>>>
> >>>> Your kernel is too old and its header doesn't support
> >>>> FIEMAP_EXTENT_SHARED flag for fiemap ioctl.
> >>>
> >>> As this is not the first time, I wonder if we could provide a defintion
> >>> of the macro in progs, regardless of the installed headers. Note that
> >>> this does not mean the running kernel is old. Previously the user said
> >>> he was running a 4.4 kernel [1] (or it could be any other kernel
> >>> version). For that combination of headers and running kernel, I think
> >>> it's ok to add a fallback definition.
> >>
> >> Yes, fallback definition is good for case like running kernel supports 
> >> SHARED_EXTENT flag, but not kernel header.
> >>
> >> But I'm a little concerned about such fallback definition will lead to 
> >> corrupted result for "fi du".
> > 
> > Well, not corrupted bug wrong. The shared data will be reported as
> > exclusive.
> > 
> >>>> And since that flag is very important for tools like "btrfs filesystem
> >>>> du", for old kernel doesn't support EXTENT_SHARED flag, we have no
> >>>> choice but abort configuration.
> >>>
> >>> The check was added to configure so it's caught earlier than during
> >>> build. The 'fi du' command is useful, but not IMO critical to stop the
> >>> build.
> >>
> >> What about just disabling subcommands using SHARED_EXTENT flag?
> >> Like "fi du".
> > 
> > This would need to be checked at runtime, based only on kernel version.
> > I think we can do that, print a warning in 'fi du' but otherwise let it
> > continue.
> 
> Checking kernel version is pretty unreliable - distro kernels that backport
> will quite possibly fail the version test.  Sure, it's possible to patch
> the check back out of userspace in the distro, but one must first know that
> such things exist, and look out for them...
> 
> Maybe warn on kernel version < 2.6.33 /and/ no shared extents found.

Yeah version is not reliable, 'fi du' will just warn. As the warning is
printed at the beginning, we don't know if there will be any shared
extent.

> Or, why not just:
> 
> 1) #ifndef FIEMAP_EXTENT_SHARED / #define FIEMAP_EXTENT_SHARED 0x00002000 / #endif
> 2) carry on as usual.
> 
> Worst case is that you don't get shared accounting on very old kernels.

Yes, implemented it that way.

> And FIEMAP_EXTENT_SHARED has been around since 2.6.33 so it's not going
> to bite too many people, right?

Right, I want to avoid another report that the build failed on centos 6.

Re: Btrfs progs build fails for 4.8

[David Sterba]

On Wed, Oct 12, 2016 at 07:28:20PM +0530, Abhay Sachan wrote:
> Hi,
> I tried building latest btrfs progs on CentOS 6, "./configure" failed
> with the error:
> 
> checking for FIEMAP_EXTENT_SHARED defined in linux/fiemap.h... no
> configure: error: no definition of FIEMAP_EXTENT_SHARED found
> 
> Any ideas what am I lacking in the system?

Can you please test the build from current devel branch? [1] Configure
will just print a warning if the flag is not defined, 'btrfs fi du' will
print another warning if it's running on kernel < 2.6.33 .

[1] git://github.com/kdave/btrfs-progs (branch devel)