[PATCH] fanotify: fix support of large files

[PATCH] fanotify: fix support of large files

[Justin Maggard]

Opening a file of >2GB in an area of the filesystem that has been marked for
fanotify events currently results in an EOVERFLOW error.  This is particularly
problematic if you are using fanotify permissions checking, because it prevents
large files from being opened at all.  Fix this by setting the O_LARGEFILE flag
on the new file handle.

Signed-off-by: Justin Maggard <jmaggard10@gmail.com> 
---
 fs/notify/fanotify/fanotify_user.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index 9ff4a5e..ef02b3d 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -64,6 +64,7 @@ static int create_fd(struct fsnotify_group *group,
 			struct file **file)
 {
 	int client_fd;
+	int f_flags;
 	struct file *new_file;
 
 	pr_debug("%s: group=%p event=%p\n", __func__, group, event);
@@ -82,12 +83,11 @@ static int create_fd(struct fsnotify_group *group,
 	 * we need a new file handle for the userspace program so it can read even if it was
 	 * originally opened O_WRONLY.
 	 */
+	f_flags = group->fanotify_data.f_flags | FMODE_NONOTIFY | O_LARGEFILE;
 	/* it's possible this event was an overflow event.  in that case dentry and mnt
 	 * are NULL;  That's fine, just don't call dentry open */
 	if (event->path.dentry && event->path.mnt)
-		new_file = dentry_open(&event->path,
-				       group->fanotify_data.f_flags | FMODE_NONOTIFY,
-				       current_cred());
+		new_file = dentry_open(&event->path, f_flags, current_cred());
 	else
 		new_file = ERR_PTR(-EOVERFLOW);
 	if (IS_ERR(new_file)) {
-- 
1.7.9.5

Re: fanotify: fix support of large files

[Heinrich Schuchardt]

Dear Justin,

looking at the example at
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
the large file support is enabled by passing O_LARGEFILE to fanotify_init:

   if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
                                     O_RDONLY | O_CLOEXEC | 
O_LARGEFILE)) < 0)

Could you, please, check if this solves your issue.

(I am resending this message because HTML was rejected by
linux-kernel@vger.kernel.org).

Best regards

Heinrich Schuchardt

On 14.02.2013 18:29, Justin Maggard wrote:
> Opening a file of>2GB in an area of the filesystem that has been marked for
> fanotify events currently results in an EOVERFLOW error.  This is particularly
> problematic if you are using fanotify permissions checking, because it prevents
> large files from being opened at all.  Fix this by setting the O_LARGEFILE flag
> on the new file handle.
>
> Signed-off-by: Justin Maggard<jmaggard10@gmail.com>
>
> ---
> fs/notify/fanotify/fanotify_user.c |    6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
> index 9ff4a5e..ef02b3d 100644
> --- a/fs/notify/fanotify/fanotify_user.c
> +++ b/fs/notify/fanotify/fanotify_user.c
> @@ -64,6 +64,7 @@ static int create_fd(struct fsnotify_group *group,
>   			struct file **file)
>   {
>   	int client_fd;
> +	int f_flags;
>   	struct file *new_file;
>
>   	pr_debug("%s: group=%p event=%p\n", __func__, group, event);
> @@ -82,12 +83,11 @@ static int create_fd(struct fsnotify_group *group,
>   	 * we need a new file handle for the userspace program so it can read even if it was
>   	 * originally opened O_WRONLY.
>   	 */
> +	f_flags = group->fanotify_data.f_flags | FMODE_NONOTIFY | O_LARGEFILE;
>   	/* it's possible this event was an overflow event.  in that case dentry and mnt
>   	 * are NULL;  That's fine, just don't call dentry open */
>   	if (event->path.dentry&&  event->path.mnt)
> -		new_file = dentry_open(&event->path,
> -				       group->fanotify_data.f_flags | FMODE_NONOTIFY,
> -				       current_cred());
> +		new_file = dentry_open(&event->path, f_flags, current_cred());
>   	else
>   		new_file = ERR_PTR(-EOVERFLOW);
>   	if (IS_ERR(new_file)) {
>

Re: fanotify: fix support of large files

[Justin Maggard]

On Sat, Apr 13, 2013 at 4:54 AM, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> Dear Justin,
>
> looking at the example at
> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
> the large file support is enabled by passing O_LARGEFILE to fanotify_init:
>
>   if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
>                                     O_RDONLY | O_CLOEXEC | O_LARGEFILE)) <
> 0)
>
> Could you, please, check if this solves your issue.
>
> (I am resending this message because HTML was rejected by
> linux-kernel@vger.kernel.org).
>
> Best regards
>
> Heinrich Schuchardt
>

No, unfortunately that doesn't help.  I slightly modifed
fanotify-example.c to call perror() when read() fails, and here's the
output:

jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example . &
[1] 7248
jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
truncate -s 2047m 2047m
Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
FAN_OPEN
FAN_CLOSE_WRITE
jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
read: Value too large for defined data type

-Justin

Re: fanotify: fix support of large files

[Heinrich Schuchardt]

Hello Justin,

I downloaded the example,
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
compiled it without modification
$ gcc fanotify-example.c -o fanotify-example
and started the executable. In a separate windows I executed
truncate -s 2048m 2048m

This is the output of the first window:
$ sudo ./fanotify-example /home/user/temp/
Started monitoring directory '/home/user/temp/'...
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
         FAN_OPEN
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
         FAN_CLOSE_WRITE

$ uname -a
Linux family2 3.8.0 #1 SMP Fri Feb 22 22:07:58 CET 2013 i686 GNU/Linux

To reproduce your problem, could you, please, provide the Linux 
configuration file (look in your /boot directory) and a link to the 
source of the Linux kernel version you use. Then I can use your 
configuration file to compile that kernel version.

Best regards

Heinrich Schuchardt

On 19.04.2013 21:23, Justin Maggard wrote:
> On Sat, Apr 13, 2013 at 4:54 AM, Heinrich Schuchardt<xypron.glpk@gmx.de>  wrote:
>> Dear Justin,
>>
>> looking at the example at
>> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
>> the large file support is enabled by passing O_LARGEFILE to fanotify_init:
>>
>>    if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
>>                                      O_RDONLY | O_CLOEXEC | O_LARGEFILE))<
>> 0)
>>
>> Could you, please, check if this solves your issue.
>>
>> (I am resending this message because HTML was rejected by
>> linux-kernel@vger.kernel.org).
>>
>> Best regards
>>
>> Heinrich Schuchardt
>>
>
> No, unfortunately that doesn't help.  I slightly modifed
> fanotify-example.c to call perror() when read() fails, and here's the
> output:
>
> jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example .&
> [1] 7248
> jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
> truncate -s 2047m 2047m
> Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
> FAN_OPEN
> FAN_CLOSE_WRITE
> jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
> read: Value too large for defined data type
>
> -Justin
>

Re: fanotify: fix support of large files

[Justin Maggard]

On Mon, Apr 22, 2013 at 11:49 AM, Heinrich Schuchardt
<xypron.glpk@gmx.de> wrote:
>
> Hello Justin,
>
> I downloaded the example,
> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
> compiled it without modification
> $ gcc fanotify-example.c -o fanotify-example
> and started the executable. In a separate windows I executed
> truncate -s 2048m 2048m
>
> This is the output of the first window:
> $ sudo ./fanotify-example /home/user/temp/
> Started monitoring directory '/home/user/temp/'...
> Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
>         FAN_OPEN
> Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
>         FAN_CLOSE_WRITE
>
> $ uname -a
> Linux family2 3.8.0 #1 SMP Fri Feb 22 22:07:58 CET 2013 i686 GNU/Linux
>

Ah yes, that's the reason.  I'm running on x86_64.  32-bit platforms
actually work if you set O_LARGEFILE.  From /usr/incude/bits/fcntl.h:

#ifdef __USE_LARGEFILE64
# if __WORDSIZE == 64
#  define O_LARGEFILE   0
# else
#  define O_LARGEFILE   0100000
# endif
#endif

If I include <asm/fcntl.h> instead of <fcntl.h>, it gets set to
0x8000, and my large file fanotify test works.  Or, of course, I can
define that flag locally and use it.  But it feels like I shouldn't
have to do that.

-Justin

>
> To reproduce your problem, could you, please, provide the Linux configuration file (look in your /boot directory) and a link to the source of the Linux kernel version you use. Then I can use your configuration file to compile that kernel version.
>
> Best regards
>
> Heinrich Schuchardt
>
>
> On 19.04.2013 21:23, Justin Maggard wrote:
>>
>> On Sat, Apr 13, 2013 at 4:54 AM, Heinrich Schuchardt<xypron.glpk@gmx.de>  wrote:
>>>
>>> Dear Justin,
>>>
>>> looking at the example at
>>> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
>>> the large file support is enabled by passing O_LARGEFILE to fanotify_init:
>>>
>>>    if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
>>>                                      O_RDONLY | O_CLOEXEC | O_LARGEFILE))<
>>> 0)
>>>
>>> Could you, please, check if this solves your issue.
>>>
>>> (I am resending this message because HTML was rejected by
>>> linux-kernel@vger.kernel.org).
>>>
>>> Best regards
>>>
>>> Heinrich Schuchardt
>>>
>>
>> No, unfortunately that doesn't help.  I slightly modifed
>> fanotify-example.c to call perror() when read() fails, and here's the
>> output:
>>
>> jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example .&
>> [1] 7248
>> jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
>> truncate -s 2047m 2047m
>> Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
>> FAN_OPEN
>> FAN_CLOSE_WRITE
>> jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
>> read: Value too large for defined data type
>>
>> -Justin
>>
>