E2BIG from listxattr

E2BIG from listxattr

[Sage Weil]

Hi,

If you create a bunch of xattrs on an inode, you can get into a sitation 
where you can no longer list them.  The reproducer below just does 
setxattr until it gets an error or listxattr starts returning E2BIG.  Once 
you're in this state you can getxattr attrs you know the name of, but you 
can't list them, which is a bit inconvenient.

Presumably the fix is for setxattr to verify that the list of all xattr 
names will fit into a 64K buffer or else return ENOSPC...

sage


#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/xattr.h>
#include <string.h>
#include <errno.h>

int main(int argc, char **argv)
{
	char *fn = argv[1];
	char buf[655360];
	int buflen = sizeof(buf);
	int i;

	for (i = 0; i < 10000; ++i) {
		int r = listxattr(fn, buf, buflen);
		if (r < 0) {
			perror("listxattr");
			return 1;
		}
		char n[100];
		sprintf(n, "user.foooooooooooooooooooooooooooooooooooooo%d", i);
		r = setxattr(fn, n, "", 0, 0);
		if (r < 0) {
			perror("setxattr");
			return 1;
		}
		printf("set %d\n", i);
	}
	return 0;
}

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: E2BIG from listxattr

[Brian Foster]

On Tue, Mar 25, 2014 at 10:48:08PM -0700, Sage Weil wrote:
> Hi,
> 
> If you create a bunch of xattrs on an inode, you can get into a sitation 
> where you can no longer list them.  The reproducer below just does 
> setxattr until it gets an error or listxattr starts returning E2BIG.  Once 
> you're in this state you can getxattr attrs you know the name of, but you 
> can't list them, which is a bit inconvenient.
> 
> Presumably the fix is for setxattr to verify that the list of all xattr 
> names will fit into a 64K buffer or else return ENOSPC...
> 

Hi Sage,

I think doing something like this would be an artificial restriction on
XFS. The listxattr() function in the vfs actually caps the provided
buffer to 64k. This in turn causes an error in XFS, which is capable of
filling a larger buffer, and the VFS converts that error into E2BIG.

I ran a quick experiment to remove that bit of code from listxattr() and
modified your program to use a 128k buffer. Everything still seems to
work Ok until I exhaust that buffer. Given that and the fact that
listxattr() does an allocation of the user-provided size, I suspect the
64k restriction here is some kind of policy based on memory allocation,
but that's a guess. The right fix might be something more involved, such
as fixing the interface to handle larger sets of data (e.g., copy direct
to a user buffer, or introduce an iterative interface, etc.).

Brian

> sage
> 
> 
> #include <stdio.h>
> #include <unistd.h>
> #include <sys/types.h>
> #include <sys/xattr.h>
> #include <string.h>
> #include <errno.h>
> 
> int main(int argc, char **argv)
> {
> 	char *fn = argv[1];
> 	char buf[655360];
> 	int buflen = sizeof(buf);
> 	int i;
> 
> 	for (i = 0; i < 10000; ++i) {
> 		int r = listxattr(fn, buf, buflen);
> 		if (r < 0) {
> 			perror("listxattr");
> 			return 1;
> 		}
> 		char n[100];
> 		sprintf(n, "user.foooooooooooooooooooooooooooooooooooooo%d", i);
> 		r = setxattr(fn, n, "", 0, 0);
> 		if (r < 0) {
> 			perror("setxattr");
> 			return 1;
> 		}
> 		printf("set %d\n", i);
> 	}
> 	return 0;
> }
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: E2BIG from listxattr

[Sage Weil]

On Wed, 26 Mar 2014, Brian Foster wrote:
> On Tue, Mar 25, 2014 at 10:48:08PM -0700, Sage Weil wrote:
> > Hi,
> > 
> > If you create a bunch of xattrs on an inode, you can get into a sitation 
> > where you can no longer list them.  The reproducer below just does 
> > setxattr until it gets an error or listxattr starts returning E2BIG.  Once 
> > you're in this state you can getxattr attrs you know the name of, but you 
> > can't list them, which is a bit inconvenient.
> > 
> > Presumably the fix is for setxattr to verify that the list of all xattr 
> > names will fit into a 64K buffer or else return ENOSPC...
> > 
> 
> Hi Sage,
> 
> I think doing something like this would be an artificial restriction on
> XFS. The listxattr() function in the vfs actually caps the provided
> buffer to 64k. This in turn causes an error in XFS, which is capable of
> filling a larger buffer, and the VFS converts that error into E2BIG.
> 
> I ran a quick experiment to remove that bit of code from listxattr() and
> modified your program to use a 128k buffer. Everything still seems to
> work Ok until I exhaust that buffer. Given that and the fact that
> listxattr() does an allocation of the user-provided size, I suspect the
> 64k restriction here is some kind of policy based on memory allocation,
> but that's a guess. The right fix might be something more involved, such
> as fixing the interface to handle larger sets of data (e.g., copy direct
> to a user buffer, or introduce an iterative interface, etc.).

Hi Brian,

I don't have a real preference whether the solution is to allow larger 
buffers or to error out of setxattr even though XFS could handle it, as 
long as it is no longer possible for an XFS user to get into this state 
where there is no (easy) way to recover.  I think we've hard-coded Ceph to 
only try up to 64 KB here, but I don't see that limitation documented 
anywhere and a savvy user can pass in a NULL buffer to get an 
appropriate length back.

In our case, we only hit this limit when something else has already gone 
a bit off the rails, so getting ENOSPC is just fine.

sage



> 
> Brian
> 
> > sage
> > 
> > 
> > #include <stdio.h>
> > #include <unistd.h>
> > #include <sys/types.h>
> > #include <sys/xattr.h>
> > #include <string.h>
> > #include <errno.h>
> > 
> > int main(int argc, char **argv)
> > {
> > 	char *fn = argv[1];
> > 	char buf[655360];
> > 	int buflen = sizeof(buf);
> > 	int i;
> > 
> > 	for (i = 0; i < 10000; ++i) {
> > 		int r = listxattr(fn, buf, buflen);
> > 		if (r < 0) {
> > 			perror("listxattr");
> > 			return 1;
> > 		}
> > 		char n[100];
> > 		sprintf(n, "user.foooooooooooooooooooooooooooooooooooooo%d", i);
> > 		r = setxattr(fn, n, "", 0, 0);
> > 		if (r < 0) {
> > 			perror("setxattr");
> > 			return 1;
> > 		}
> > 		printf("set %d\n", i);
> > 	}
> > 	return 0;
> > }
> > 
> > _______________________________________________
> > xfs mailing list
> > xfs@oss.sgi.com
> > http://oss.sgi.com/mailman/listinfo/xfs
> 
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs