[meta-selinux][PATCHv2 0/8] Label file system in build.

[meta-selinux][PATCHv2 0/8] Label file system in build.

[Philip Tricca]

This is the second version of a patch series that allows the file system
of SELinux images to be labeled as part of the build process. This will
allow SELinux images to boot read only file systems and remove the need to
label the file system on first boot.

To do this we must label the file system in the build as well as add
support for extended attributes to the mke2fs utility in the e2fsprogs
package. The first version of this patch series is here:
https://lists.yoctoproject.org/pipermail/yocto/2015-June/025141.html
The approach described in this previous RFC remains the same.

Changes in v2:
This second version has two significant changes: First I've done a bunch
of cleanup. This includes work to make the descriptions in the patch
headers / commit messages more exact as well as combining some commits
with related functionality. Secondly I've reimplemented the xattr cache
so that it actually works.

I've made the patch headers as descriptive as possible and kept the git
commit messages minimal. If the preference is for more verbose commit
messages I'm happy to oblige if advised.

The cache is just a single linked list that's searched for duplicates after
the creation of each new xattr block. The previous implementation was similar
but, aside from not working properly, it was overly complex in its attempt to
keep the list sorted.

Tests:
To test this new implementation I used the core-image-selinux-minimal image
from the unmodified master branch as a control. This image has 2536 unique
file system objects including the root fs directory. The ext4 file system
produced by the build has 71492 blocks with 13621 free.

As an additional test I added the patches from this set WITHOUT the cache
patches. This causes each file system object with an associated extended
attribute to use up an additional block for the xattr. This should cause
(hypothesis) the output file system to have 13621 - 2536 = 11085 free
blocks. The build producing an ext4 file system with 71492 blocks and 11088
free. That's an additional 2533 blocks used instead of the 2536 expected.
These 3 missing xattr blocks can be accounted for in that there are 3
unlabeled files in the file system.

Introducing the cache allows files with identical xattr blocks to share
them to reduce the number of used blocks. Since we're only storing SELinux
labels in the xattrs we can say that every file with the same SELinux label
should share an xattr block. Counting the unique SELinux labels on file
objects we know that there are 83 in total. The second hypothesis we have
to test then is that using the cache will reduce the number of used blocks
from 2533 down to 83.

Applying the patch that enables the cache produces a third and final ext4
file system. This one again report 71492 total blocks but this time 13538
free. This is 83 blocks fewer than the unlabled file system from the
initial test as we expected. The code added by this patch set is also
instrumented to count the objects in the cache when they're freed. With
this debug output enabled it reports the same number of objects in the
cache.

From the test results I'm pretty confident that the cache functions as
expected. It's still a very basic implementation but given the small
number of unique SELinux labels in the reference file systems it's
likely sufficient for a first version. Feedback / comments on both the
implementation and testing approach would be appreciated.

Regards,
Philip
----

Philip Tricca (8):
  policycoreutils: Patch setfiles to add FTS_NOCHDIR to fts_flags.
  selinux-image: Add new image class to label the rootfs, use it for
    selinux images.
  e2fsprogs: Add bbappend and stub for xattr module.
  e2fsprogs: Insert calls to xattr module into mke2fs and build xattr
    code.
  e2fsprogs: Add xattr security prefix data to
    lib/ext2fs/ext2_ext_attr.h
  e2fsprogs: Copy xattr block from source file.
  e2fsprogs: Add stub functions for an xattr cache and struct to hold
    the header and block data.
  e2fsprogs: Implement xattr block cache with simple linked list.

 classes/selinux-image.bbclass                      |   8 +
 ...ib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch |  20 ++
 .../misc-xattr-add-xattr-module-stub.patch         |  57 ++++
 .../misc-xattr-create-xattr-block-node.patch       | 175 +++++++++++
 .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
 .../e2fsprogs/misc-xattr-create-xattr-cache.patch  | 181 +++++++++++
 .../mke2fs.c-create_inode.c-copy-xattrs.patch      | 164 ++++++++++
 .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |  10 +
 .../images/core-image-selinux-minimal.bb           |   2 +-
 recipes-security/images/core-image-selinux.bb      |   2 +-
 .../policycoreutils-fts_flags-FTS_NOCHDIR.patch    |  25 ++
 recipes-security/selinux/policycoreutils_2.3.bb    |   1 +
 12 files changed, 984 insertions(+), 2 deletions(-)
 create mode 100644 classes/selinux-image.bbclass
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
 create mode 100644 recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch

-- 
2.1.4

[meta-selinux][PATCHv2 1/8] policycoreutils: Patch setfiles to add FTS_NOCHDIR to fts_flags.

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 .../policycoreutils-fts_flags-FTS_NOCHDIR.patch    | 25 ++++++++++++++++++++++
 recipes-security/selinux/policycoreutils_2.3.bb    |  1 +
 2 files changed, 26 insertions(+)
 create mode 100644 recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch

diff --git a/recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch b/recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch
new file mode 100644
index 0000000..fd04b51
--- /dev/null
+++ b/recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch
@@ -0,0 +1,25 @@
+This works around a bug in the default libc used by OE. FTS functions are
+supposed to change the current working directory as they walk the file
+hierarchy. The fts_accpath member of the FTSENT structure relies on this
+behavior and without it the path is not relative to cwd as expected.
+Supplying the FTS_NOCHDIR flag disables this optimization and causes
+fts_accpath to be relative to the directory where the traversal started.
+Use of this flag doesn't effect compatibility with glibc.
+
+An alternative fix could replace the use of fts_accpath with fts_path
+which is absolute.
+
+Signed-off-by: Philip Tricca <flihp@twobit.us>
+Index: policycoreutils/setfiles/setfiles.c
+===================================================================
+--- policycoreutils.orig/setfiles/setfiles.c
++++ policycoreutils/setfiles/setfiles.c
+@@ -194,7 +194,7 @@ int main(int argc, char **argv)
+ 		r_opts.expand_realpath = 0;
+ 		r_opts.abort_on_error = 1;
+ 		r_opts.add_assoc = 1;
+-		r_opts.fts_flags = FTS_PHYSICAL | FTS_XDEV;
++		r_opts.fts_flags = FTS_PHYSICAL | FTS_XDEV | FTS_NOCHDIR;
+ 		ctx_validate = 1;
+ 	} else {
+ 		/*
diff --git a/recipes-security/selinux/policycoreutils_2.3.bb b/recipes-security/selinux/policycoreutils_2.3.bb
index 447e6c9..d3c36dc 100644
--- a/recipes-security/selinux/policycoreutils_2.3.bb
+++ b/recipes-security/selinux/policycoreutils_2.3.bb
@@ -13,4 +13,5 @@ SRC_URI += "\
 	file://policycoreutils-semanage-edit-user.patch \
 	file://policycoreutils-process-ValueError-for-sepolicy-seobject.patch \
 	file://policycoreutils-fix-TypeError-for-seobject.py.patch \
+	file://policycoreutils-fts_flags-FTS_NOCHDIR.patch \
 	"
-- 
2.1.4

[meta-selinux][PATCHv2 2/8] selinux-image: Add new image class to label the rootfs, use it for selinux images.

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 classes/selinux-image.bbclass                         | 8 ++++++++
 recipes-security/images/core-image-selinux-minimal.bb | 2 +-
 recipes-security/images/core-image-selinux.bb         | 2 +-
 3 files changed, 10 insertions(+), 2 deletions(-)
 create mode 100644 classes/selinux-image.bbclass

diff --git a/classes/selinux-image.bbclass b/classes/selinux-image.bbclass
new file mode 100644
index 0000000..362ca5c
--- /dev/null
+++ b/classes/selinux-image.bbclass
@@ -0,0 +1,8 @@
+selinux_set_labels () {
+    POL_TYPE=$(sed -n -e "s&SELINUXTYPE[:space:]*=[:space:]*\([0-9A-Za-z_]\+\)&\1&p" ${IMAGE_ROOTFS}/${sysconfdir}/selinux/config)
+    setfiles -r ${IMAGE_ROOTFS} ${IMAGE_ROOTFS}/${sysconfdir}/selinux/${POL_TYPE}/contexts/files/file_contexts ${IMAGE_ROOTFS} || exit 1;
+}
+
+IMAGE_PREPROCESS_COMMAND += "selinux_set_labels ;"
+
+inherit core-image
diff --git a/recipes-security/images/core-image-selinux-minimal.bb b/recipes-security/images/core-image-selinux-minimal.bb
index 45cd847..45bb099 100644
--- a/recipes-security/images/core-image-selinux-minimal.bb
+++ b/recipes-security/images/core-image-selinux-minimal.bb
@@ -12,4 +12,4 @@ IMAGE_INSTALL = "\
 	packagegroup-selinux-minimal \
 "
 
-inherit core-image
+inherit selinux-image
diff --git a/recipes-security/images/core-image-selinux.bb b/recipes-security/images/core-image-selinux.bb
index e186bdc..70b525e 100644
--- a/recipes-security/images/core-image-selinux.bb
+++ b/recipes-security/images/core-image-selinux.bb
@@ -11,4 +11,4 @@ IMAGE_INSTALL = "\
 	packagegroup-core-selinux \
 "   
 
-inherit core-image
+inherit selinux-image
-- 
2.1.4

[meta-selinux][PATCHv2 3/8] e2fsprogs: Add bbappend and stub for xattr module.

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 .../misc-xattr-add-xattr-module-stub.patch         | 57 ++++++++++++++++++++++
 .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |  5 ++
 2 files changed, 62 insertions(+)
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend

diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch
new file mode 100644
index 0000000..5a39abe
--- /dev/null
+++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch
@@ -0,0 +1,57 @@
+This patch adds the structure for a module to duplicate xattr blocks as part
+of copying files with the '-d' option to mke2fs. The function stubs here are
+intended to be the public interface to the module. We also define a macro
+for dumping debug data specific to this module.
+
+Signed-off-by: Philip Tricca <flihp@twobit.us>
+
+Index: e2fsprogs-1.42.9/misc/xattr.c
+===================================================================
+--- /dev/null
++++ e2fsprogs-1.42.9/misc/xattr.c
+@@ -0,0 +1,34 @@
++#include "xattr.h"
++
++#include <stdio.h>
++
++#ifdef XATTR_DEBUG
++#define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args)
++#else
++#define XATTR_STDERR(fmt, args...) do {} while (0)
++#endif
++
++
++/* Free remaining resources after all files have been processed. */
++void
++xattr_cleanup ()
++{
++	XATTR_STDERR ("Cleaning up resources from xattrs.\n");
++}
++
++/* This is the entry point to the xattr module. This function copies the xattrs
++ * from the file at 'path' to the file system object at 'ino'.
++ *
++ * Parameters:
++ *   fs:  the file system object for the fs we're operating on
++ *   ino: inode for the object we're labeling
++ *   path: path to the object we're copying xattrs from
++ */
++errcode_t
++set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)
++{
++	errcode_t ret = 0;
++
++	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path, ino);
++	return ret;
++}
+Index: e2fsprogs-1.42.9/misc/xattr.h
+===================================================================
+--- /dev/null
++++ e2fsprogs-1.42.9/misc/xattr.h
+@@ -0,0 +1,6 @@
++#include "et/com_err.h"
++#include "ext2fs/ext2fs.h"
++
++/* Copy xattrs from source file to destination inode */
++errcode_t set_inode_xattrs(ext2_filsys fs, ext2_ino_t ino, const char *name);
++void xattr_cleanup ();
diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
new file mode 100644
index 0000000..02318ea
--- /dev/null
+++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
@@ -0,0 +1,5 @@
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
+
+SRC_URI += " \
+    file://misc-xattr-add-xattr-module-stub.patch \
+"
-- 
2.1.4

[meta-selinux][PATCHv2 4/8] e2fsprogs: Insert calls to xattr module into mke2fs and build xattr code.

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 .../mke2fs.c-create_inode.c-copy-xattrs.patch      | 164 +++++++++++++++++++++
 .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
 2 files changed, 165 insertions(+)
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch

diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch b/recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch
new file mode 100644
index 0000000..8b0109f
--- /dev/null
+++ b/recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch
@@ -0,0 +1,164 @@
+Insert calls into xattr module to copy xattrs for each file system object
+copied by mke2fs when '-d' is specified. This requires a call to
+set_inode_xattr in two places:
+misc/create_inode.c: to copy xattrs as part of the recursive traversal
+of the source directory
+misc/mke2fs.c: to copy the xattr block associated with the root of the
+file system
+
+Insert a call to xattr_cleanup to free any resources that need special
+handling in the xattr module.
+
+We also add the necessary rules to Makefile.in to build the xattr module
+and link it into executables as required.
+
+Signed-off-by: Philip Tricca <flihp@twobit.us>
+
+Index: e2fsprogs-1.42.9/misc/mke2fs.c
+===================================================================
+--- e2fsprogs-1.42.9.orig/misc/mke2fs.c
++++ e2fsprogs-1.42.9/misc/mke2fs.c
+@@ -56,6 +56,7 @@ extern int optind;
+ #include "../version.h"
+ #include "quota/mkquota.h"
+ #include "create_inode.h"
++#include "xattr.h"
+ 
+ #define STRIDE_LENGTH 8
+ 
+@@ -2744,6 +2745,11 @@ no_journal:
+ 		hdlinks.count = 0;
+ 		current_fs = fs;
+ 		root = EXT2_ROOT_INO;
++		retval = set_inode_xattr (fs, EXT2_ROOT_INO, root_dir);
++		if (retval) {
++			fprintf(stderr, "%s", _("Error setting xattr for inode: 0x%x\n"), EXT2_ROOT_INO);
++			return retval;
++		}
+ 		retval = populate_fs(root, root_dir);
+ 		if (retval) {
+ 			fprintf(stderr, "%s",
+@@ -2751,6 +2757,7 @@ no_journal:
+ 			return retval;
+ 		} else if (!quiet)
+ 			printf("%s", _("done\n"));
++		xattr_cleanup();
+ 	}
+ 
+ 	if (!quiet)
+Index: e2fsprogs-1.42.9/misc/create_inode.c
+===================================================================
+--- e2fsprogs-1.42.9.orig/misc/create_inode.c
++++ e2fsprogs-1.42.9/misc/create_inode.c
+@@ -1,5 +1,6 @@
+ #include "create_inode.h"
+ #include <limits.h>
++#include "xattr.h"
+ 
+ #if __STDC_VERSION__ < 199901L
+ # if __GNUC__ >= 2
+@@ -549,6 +550,12 @@ errcode_t populate_fs(ext2_ino_t parent_
+ 			return retval;
+ 		}
+ 
++		if ((retval = set_inode_xattr (current_fs, ino, name))) {
++			com_err(__func__, retval,
++				_("while setting xattrs for \"%s\""), name);
++			return retval;
++		}
++
+ 		/* Save the hardlink ino */
+ 		if (save_inode) {
+ 			/*
+Index: e2fsprogs-1.42.9/debugfs/Makefile.in
+===================================================================
+--- e2fsprogs-1.42.9.orig/debugfs/Makefile.in
++++ e2fsprogs-1.42.9/debugfs/Makefile.in
+@@ -18,7 +18,7 @@ MK_CMDS=	_SS_DIR_OVERRIDE=../lib/ss ../l
+ 
+ DEBUG_OBJS= debug_cmds.o debugfs.o util.o ncheck.o icheck.o ls.o \
+ 	lsdel.o dump.o set_fields.o logdump.o htree.o unused.o e2freefrag.o \
+-	filefrag.o extent_cmds.o extent_inode.o zap.o create_inode.o
++	filefrag.o extent_cmds.o extent_inode.o zap.o create_inode.o xattr.o
+ 
+ RO_DEBUG_OBJS= ro_debug_cmds.o ro_debugfs.o util.o ncheck.o icheck.o ls.o \
+ 	lsdel.o logdump.o htree.o e2freefrag.o filefrag.o extent_cmds.o \
+@@ -29,12 +29,16 @@ SRCS= debug_cmds.c $(srcdir)/debugfs.c $
+ 	$(srcdir)/dump.c $(srcdir)/set_fields.c ${srcdir}/logdump.c \
+ 	$(srcdir)/htree.c $(srcdir)/unused.c ${srcdir}/../misc/e2freefrag.c \
+ 	$(srcdir)/filefrag.c $(srcdir)/extent_inode.c $(srcdir)/zap.c \
+-	$(srcdir)/../misc/create_inode.c
++	$(srcdir)/../misc/create_inode.c $(srcdir)/../misc/xattr.c
+ 
+ CREATE_INODE_DEPS= $(srcdir)/../misc/create_inode.h \
+ 	$(srcdir)/../misc/create_inode.c $(top_builddir)/lib/config.h \
+ 	$(srcdir)/../lib/ext2fs/ext2fs.h $(srcdir)/../lib/et/com_err.h \
+-	$(srcdir)/../lib/e2p/e2p.h $(srcdir)/../misc/nls-enable.h
++	$(srcdir)/../lib/e2p/e2p.h $(srcdir)/../misc/nls-enable.h \
++	$(srcdir)/../misc/xattr.h
++
++XATTR_DEPS= $(srcdir)/../misc/xattr.h $(srcdir)/../misc/xattr.h \
++    $(srcdir)/../lib/et/com_err.h $(srcdir)/../lib/ext2fs/ext2fs.h
+ 
+ LIBS= $(LIBEXT2FS) $(LIBE2P) $(LIBSS) $(LIBCOM_ERR) $(LIBBLKID) \
+ 	$(LIBUUID)
+@@ -92,6 +96,11 @@ create_inode.o: $(CREATE_INODE_DEPS)
+ 	$(Q) $(CC) -c $(ALL_CFLAGS) -I$(srcdir) \
+ 		 $(srcdir)/../misc/create_inode.c -DDEBUGFS -o $@
+ 
++xattr.o: $(XATTR_DEPS)
++	$(E) "  CC $@"
++	$(Q) $(CC) -c $(ALL_CFLAGS) -I$(srcdir) \
++		$(srcdir)/../misc/xattr.c -o $@
++
+ debugfs.8: $(DEP_SUBSTITUTE) $(srcdir)/debugfs.8.in
+ 	$(E) "	SUBST $@"
+ 	$(Q) $(SUBSTITUTE_UPTIME) $(srcdir)/debugfs.8.in debugfs.8
+Index: e2fsprogs-1.42.9/misc/Makefile.in
+===================================================================
+--- e2fsprogs-1.42.9.orig/misc/Makefile.in
++++ e2fsprogs-1.42.9/misc/Makefile.in
+@@ -43,7 +43,7 @@ LPROGS=		@E2INITRD_PROG@
+ TUNE2FS_OBJS=	tune2fs.o util.o
+ MKLPF_OBJS=	mklost+found.o
+ MKE2FS_OBJS=	mke2fs.o util.o profile.o prof_err.o default_profile.o \
+-			create_inode.o
++			create_inode.o xattr.o
+ CHATTR_OBJS=	chattr.o
+ LSATTR_OBJS=	lsattr.o
+ UUIDGEN_OBJS=	uuidgen.o
+@@ -62,7 +62,7 @@ PROFILED_TUNE2FS_OBJS=	profiled/tune2fs.
+ PROFILED_MKLPF_OBJS=	profiled/mklost+found.o
+ PROFILED_MKE2FS_OBJS=	profiled/mke2fs.o profiled/util.o profiled/profile.o \
+ 			profiled/prof_err.o profiled/default_profile.o \
+-			profiled/create_inode.o
++			profiled/create_inode.o profiled/xattr.o
+ PROFILED_CHATTR_OBJS=	profiled/chattr.o
+ PROFILED_LSATTR_OBJS=	profiled/lsattr.o
+ PROFILED_UUIDGEN_OBJS=	profiled/uuidgen.o
+@@ -84,7 +84,8 @@ SRCS=	$(srcdir)/tune2fs.c $(srcdir)/mklo
+ 		$(srcdir)/uuidgen.c $(srcdir)/blkid.c $(srcdir)/logsave.c \
+ 		$(srcdir)/filefrag.c $(srcdir)/base_device.c \
+ 		$(srcdir)/ismounted.c $(srcdir)/../e2fsck/profile.c \
+-		$(srcdir)/e2undo.c $(srcdir)/e2freefrag.c $(srcdir)/create_inode.c
++		$(srcdir)/e2undo.c $(srcdir)/e2freefrag.c $(srcdir)/create_inode.c \
++		$(srcdir)/xattr.c
+ 
+ LIBS= $(LIBEXT2FS) $(LIBCOM_ERR) 
+ DEPLIBS= $(LIBEXT2FS) $(DEPLIBCOM_ERR)
+@@ -634,7 +635,8 @@ mke2fs.o: $(srcdir)/mke2fs.c $(top_build
+  $(srcdir)/util.h profile.h prof_err.h $(top_srcdir)/version.h \
+  $(srcdir)/nls-enable.h $(top_srcdir)/lib/quota/mkquota.h $(srcdir)/create_inode.h\
+  $(top_srcdir)/lib/quota/quotaio.h $(top_srcdir)/lib/quota/dqblk_v2.h \
+- $(top_srcdir)/lib/quota/quotaio_tree.h $(top_srcdir)/lib/../e2fsck/dict.h
++ $(top_srcdir)/lib/quota/quotaio_tree.h $(top_srcdir)/lib/../e2fsck/dict.h \
++ $(srcdir)/xattr.h
+ chattr.o: $(srcdir)/chattr.c $(top_builddir)/lib/config.h \
+  $(top_builddir)/lib/dirpaths.h $(top_srcdir)/lib/ext2fs/ext2_fs.h \
+  $(top_builddir)/lib/ext2fs/ext2_types.h $(top_srcdir)/lib/et/com_err.h \
+@@ -716,3 +718,5 @@ create_inode.o: $(srcdir)/create_inode.h
+  $(top_builddir)/lib/config.h $(top_srcdir)/lib/ext2fs/ext2fs.h \
+  $(top_srcdir)/lib/et/com_err.h $(top_srcdir)/lib/e2p/e2p.h \
+  $(srcdir)/nls-enable.h
++xattr.o: $(srcdir)/xattr.h $(srcdir)/xattr.c \
++ $(top_builddir)/lib/et/com_err.h $(top_srcdir)/lib/ext2fs/ext2fs.h
diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
index 02318ea..3950bf7 100644
--- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
+++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
@@ -2,4 +2,5 @@ FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
 
 SRC_URI += " \
     file://misc-xattr-add-xattr-module-stub.patch \
+    file://mke2fs.c-create_inode.c-copy-xattrs.patch \
 "
-- 
2.1.4

[meta-selinux][PATCHv2 5/8] e2fsprogs: Add xattr security prefix data to lib/ext2fs/ext2_ext_attr.h

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 .../lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch | 20 ++++++++++++++++++++
 recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend |  1 +
 2 files changed, 21 insertions(+)
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch

diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch b/recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch
new file mode 100644
index 0000000..b87c414
--- /dev/null
+++ b/recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch
@@ -0,0 +1,20 @@
+Add xattr name index for xattrs with the 'security' prefix. These are defined
+in the ext(2|3|4)/xattr.h in the kernel. We use the EXT2 prefix for consistency
+with e2fslibs naming.
+
+Signed-off-by: Philip Tricca <flihp@twobit.us>
+
+Index: e2fsprogs-1.42.9/lib/ext2fs/ext2_ext_attr.h
+===================================================================
+--- e2fsprogs-1.42.9.orig/lib/ext2fs/ext2_ext_attr.h
++++ e2fsprogs-1.42.9/lib/ext2fs/ext2_ext_attr.h
+@@ -15,6 +15,9 @@
+ /* Maximum number of references to one attribute block */
+ #define EXT2_EXT_ATTR_REFCOUNT_MAX	1024
+ 
++/* Name indexes */
++#define EXT2_XATTR_INDEX_SECURITY               6
++
+ struct ext2_ext_attr_header {
+ 	__u32	h_magic;	/* magic number for identification */
+ 	__u32	h_refcount;	/* reference count */
diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
index 3950bf7..a4576b1 100644
--- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
+++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
@@ -3,4 +3,5 @@ FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
 SRC_URI += " \
     file://misc-xattr-add-xattr-module-stub.patch \
     file://mke2fs.c-create_inode.c-copy-xattrs.patch \
+    file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
 "
-- 
2.1.4

[meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
 .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
 2 files changed, 342 insertions(+)
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch

diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
new file mode 100644
index 0000000..5955b44
--- /dev/null
+++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
@@ -0,0 +1,341 @@
+To build the xattr disk block we process the output from listxattr and
+lgetxattr from the source file system object. This data is formated in a disk
+block according to the format specified in the kernel ext2 file system driver.
+See the comment block at the beginning of fs/ext2/xattr.c for details.
+
+Currently we only process attributes with the 'security.' prefix as our use
+case is SELinux labels and IMA. Additional prefixes can likely be supported with
+minimal effort but none have been tested.
+
+Once the xattr block has been created it is written to disk. The xattr block is
+associated with the appropriate file system object through the i_file_acl inode
+member and the inode is updated on disk.
+
+Signed-off-by: Philip Tricca <flihp@twobit.us>
+
+Index: e2fsprogs-1.42.9/misc/xattr.c
+===================================================================
+--- e2fsprogs-1.42.9.orig/misc/xattr.c
++++ e2fsprogs-1.42.9/misc/xattr.c
+@@ -1,6 +1,23 @@
+ #include "xattr.h"
+ 
++#include <attr/xattr.h>
++#include <ctype.h>
++#include <errno.h>
++#include <ext2fs/ext2_ext_attr.h>
++#include <linux/xattr.h>
++#include <stdint.h>
+ #include <stdio.h>
++#include <stdlib.h>
++#include <string.h>
++#include <sys/stat.h>
++#include <sys/types.h>
++#include <unistd.h>
++
++#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y))
++#define HEADER(ptr) ((struct ext2_ext_attr_header *)(ptr))
++#define ENTRY(ptr)  ((struct ext2_ext_attr_entry *)(ptr))
++#define FIRST_ENTRY(ptr) ENTRY(HEADER(ptr) + 1)
++#define VALUE(hdr, ent) (((char*)hdr) + (ent->e_value_offs))
+ 
+ #ifdef XATTR_DEBUG
+ #define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args)
+@@ -8,6 +25,28 @@
+ #define XATTR_STDERR(fmt, args...) do {} while (0)
+ #endif
+ 
++/* structure for mapping xattr name prefix data */
++typedef struct xattr_prefix {
++	int index;
++	char *name;
++	size_t length;
++} xattr_prefix_t;
++
++xattr_prefix_t xattr_prefixes [] = {
++/* Only interested in security prefix. Can support others though.
++	{
++		.index = EXT2_XATTR_INDEX_USER,
++		.name = XATTR_USER_PREFIX,
++		.length = XATTR_USER_PREFIX_LEN,
++	},
++*/
++	{
++		.index = EXT2_XATTR_INDEX_SECURITY,
++		.name = XATTR_SECURITY_PREFIX,
++		.length = XATTR_SECURITY_PREFIX_LEN,
++	},
++	{ 0 },
++};
+ 
+ /* Free remaining resources after all files have been processed. */
+ void
+@@ -16,6 +55,211 @@ xattr_cleanup ()
+ 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");
+ }
+ 
++/* Get value for named xattr from file at path.
++ * Returns pointer to allocated block for value and length in length param.
++ * If no value, return NULL pointer and length of 0.
++ * On error return NULL pointer and length set to -1.
++ */
++static char*
++xattr_get_value (const char *path, const char *name, ssize_t *length)
++{
++	char *value = NULL;
++
++	*length = lgetxattr (path, name, NULL, 0);
++	if (*length == -1) {
++		com_err (__func__, errno, "lgetattr");
++		goto out;
++	}
++	if (*length == 0) {
++		fprintf (stderr, "xattr %s has value length 0\n", name);
++		goto out;
++	}
++	value = calloc (1, *length);
++	if (value == NULL) {
++		com_err (__func__, errno, "calloc");
++		goto out;
++	}
++	*length = lgetxattr (path, name, value, *length);
++	if (*length == -1) {
++		com_err (__func__, errno, "lgetattr");
++		goto value_err;
++	}
++out:
++	return value;
++
++value_err:
++	if (value)
++		free (value);
++	return NULL;
++}
++
++/* Get all attribute names for file at path. Return pointer to allocated memory
++ * block holding all names and the length through parameter size.
++ * If no xattrs: return NULL and set size to 0
++ * If error: return NULL and set size to -1
++ */
++static char*
++xattr_get_names (const char *path, ssize_t *size)
++{
++	char *names = NULL;
++
++	*size = llistxattr (path, NULL, 0);
++	if (*size == -1) {
++		com_err (__func__, errno, "llistxattr");
++		goto out;
++	}
++	if (*size == 0) {
++		/* no xattrs */
++		goto out;
++	}
++	names = calloc (1, *size);
++	if (names == NULL) {
++		com_err (__func__, errno, "calloc");
++		goto out;
++	}
++	*size = llistxattr (path, names, *size);
++	if (*size == -1) {
++		com_err (__func__, errno, "llistxattr");
++		goto cleanup;
++	}
++	if (*size == 0) {
++		fprintf (stdout, "Conflicting data, no xattrs for file: %s\n", path);
++		goto cleanup;
++	}
++out:
++	return names;
++
++cleanup:
++	if (names)
++		free (names);
++	return NULL;
++}
++
++/* return pointer to next string in xattr name block, don't go beyond length
++ */
++static inline char*
++next_name (char *name, size_t length)
++{
++	int i = 0;
++
++	for (i = 0; i < length; ++i)
++		if (name [i] == '\0') {
++			++i;
++			break;
++		}
++
++	return name + i;
++}
++
++/* Find entry in xattr_table with matching prefix. */
++static const xattr_prefix_t*
++xattr_find_prefix (char *name, size_t length)
++{
++	int i = 0;
++
++	XATTR_STDERR ("find_attr_data: searching for prefix from xattr name: %s\n", name);
++	for (i = 0; xattr_prefixes[i].index != 0; ++i) {
++		if (!strncmp (name, xattr_prefixes[i].name, MIN (length, xattr_prefixes[i].length))) {
++			XATTR_STDERR ("found match in table with index: %d\n", xattr_prefixes[i].index);
++			return &xattr_prefixes[i];
++		}
++	}
++	return NULL;
++}
++
++/* Query file at path for attributes. build up structure the file system
++ * expects of an extended attribute disk block (header parameter).
++ *
++ * The main loop walks through the xattr names one at a time. It gets the value
++ * for each named xattr and copies the data into the xattr block pointed to by
++ * the header parameter. To do this the loop also tracks the location of the
++ * associated entry and value. Values start at the end of the buffer and grow
++ * back towards the header while the entries start immediately after the header
++ * and grow towards the end of the block.
++ *
++ * See the comment block at the beginning of the xattr.c file in the ext2 file
++ * system code in the kernel: fs/ext2/xattr.c
++ * Assume the xattr block pointed to by header parameter is initialized to 0s.
++ */
++static int
++xattr_build_block (const char *path,
++		struct ext2_ext_attr_header **header,
++		size_t header_length)
++{
++	struct ext2_ext_attr_entry *entry = NULL;
++	char *names = NULL, *value = NULL, *name_curr = NULL;
++	ssize_t names_length = 0, value_length = 0;
++	size_t name_length = 0, value_index = 0, len_rem = 0;
++	const xattr_prefix_t *prefix = NULL;
++	int ret = 0;
++
++	XATTR_STDERR ("xattr_build_block for file: %s\n", path);
++	*header = NULL;
++	names = xattr_get_names (path, &names_length);
++	if (names == NULL) {
++		// no xattrs for file @ path or error
++		ret = names_length;
++		goto out;
++	}
++	*header = calloc (1, header_length);
++	if (*header == NULL) {
++		com_err (__func__, errno, "calloc");
++		goto out;
++	}
++	(*header)->h_magic = EXT2_EXT_ATTR_MAGIC;
++	(*header)->h_blocks = 1;
++	/* Values start at end of buffer. NOTE: It must be moved before a value can
++	 * be stored.
++	 */
++	value_index = header_length;
++	for (name_curr = names, entry = FIRST_ENTRY(*header), len_rem = names_length;
++		name_curr < names + names_length;
++		len_rem = names_length - (name_curr - names),
++			name_curr = next_name (name_curr, len_rem),
++			entry = EXT2_EXT_ATTR_NEXT(entry))
++	{
++		XATTR_STDERR ("xattr_build_block: processing xattr with name %s\n", name_curr);
++		name_length = strnlen (name_curr, len_rem);
++		prefix = xattr_find_prefix (name_curr, name_length);
++		if (!prefix) {
++			fprintf (stderr, "Don't currently handle xattr: %s\n", name_curr);
++			continue;
++		}
++		value = xattr_get_value (path, name_curr, &value_length);
++		if (value == NULL) {
++			// no xattr value or error
++			fprintf (stderr, "failed to get value, skipping\n");
++			goto next;
++		}
++		/* setup offsets and lengths for name and value */
++		entry->e_name_len = name_length - prefix->length;
++		entry->e_name_index = prefix->index;
++		/* Can't know these till we know the length of the value. */
++		entry->e_value_offs = value_index -= EXT2_EXT_ATTR_SIZE(value_length);
++		entry->e_value_size = value_length;
++		/* Check to be sure entry name and value don't overlap before copy. */
++		if (EXT2_EXT_ATTR_NAME(entry) + entry->e_name_len > VALUE(*header, entry)) {
++			fprintf (stderr, "xattr entry name and value overlap! Too much xattr data.");
++			ret = -1;
++			goto out;
++		}
++		/* copy name and value data then calculate the hash */
++		memcpy (EXT2_EXT_ATTR_NAME(entry),
++			name_curr + prefix->length,
++			entry->e_name_len);
++		memcpy (VALUE(*header, entry), value, entry->e_value_size);
++		entry->e_hash = ext2fs_ext_attr_hash_entry (entry, VALUE(*header, entry));
++next:
++		if (value)
++			free (value);
++	}
++	XATTR_STDERR ("xattr_build_block: done building xattr buffer\n");
++out:
++	if (names)
++		free (names);
++	return ret;
++}
++
+ /* This is the entry point to the xattr module. This function copies the xattrs
+  * from the file at 'path' to the file system object at 'ino'.
+  *
+@@ -28,7 +272,56 @@ errcode_t
+ set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)
+ {
+ 	errcode_t ret = 0;
++	char *buf = NULL;
++	struct ext2_inode inode = { 0 };
++	blk_t block = 0;
++	struct ext2_ext_attr_header *header = NULL;
++	uint32_t newcount = 0;
+ 
+ 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path, ino);
++	/* Populate the xattr block for the file at path */
++	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
++		goto out;
++	}
++	if (header == NULL) {
++		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
++		goto out;
++	}
++	if (ret = ext2fs_read_inode (fs, ino, &inode)) {
++		com_err(__func__, ret, "ext2fs_read_inode");
++		goto out;
++	}
++	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
++		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
++		goto out;
++	}
++	ext2fs_mark_block_bitmap2 (fs->block_map, block);
++	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
++	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
++		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
++		goto out;
++	}
++	/* point inode for current file to xattr block, update block count and
++	 * write inode to disk
++	 */
++	inode.i_file_acl = block;
++	if (ret = ext2fs_adjust_ea_refcount2(fs,
++					block,
++					(char*)header,
++					1,
++					&newcount))
++	{
++		com_err(__func__, ret, "ext2fs_adjust_ea_refcount");
++		goto out;
++	}
++	if (ret = ext2fs_iblk_add_blocks (fs, &inode, 1)) {
++		com_err(__func__, ret, "ext2fs_iblk_add_blocks failed");
++		goto out;
++	}
++	if (ret = ext2fs_write_inode (fs, ino, &inode))
++		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
++out:
++	if (header)
++		free (header);
+ 	return ret;
+ }
diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
index a4576b1..edc94d8 100644
--- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
+++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
@@ -4,4 +4,5 @@ SRC_URI += " \
     file://misc-xattr-add-xattr-module-stub.patch \
     file://mke2fs.c-create_inode.c-copy-xattrs.patch \
     file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
+    file://misc-xattr-create-xattr-block.patch \
 "
-- 
2.1.4

Re: [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.

[Huang, Jie (Jackie)]

> -----Original Message-----
> From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Philip
> Tricca
> Sent: Thursday, June 18, 2015 6:31 AM
> To: yocto@yoctoproject.org
> Subject: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
> 
> Signed-off-by: Philip Tricca <flihp@twobit.us>
> ---
>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
>  2 files changed, 342 insertions(+)
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
> 
> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch b/recipes-
> devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
> new file mode 100644
> index 0000000..5955b44
> --- /dev/null
> +++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block
> +++ .patch
> @@ -0,0 +1,341 @@
> +To build the xattr disk block we process the output from listxattr and
> +lgetxattr from the source file system object. This data is formated in
> +a disk block according to the format specified in the kernel ext2 file system driver.
> +See the comment block at the beginning of fs/ext2/xattr.c for details.
> +
> +Currently we only process attributes with the 'security.' prefix as our
> +use case is SELinux labels and IMA. Additional prefixes can likely be
> +supported with minimal effort but none have been tested.
> +
> +Once the xattr block has been created it is written to disk. The xattr
> +block is associated with the appropriate file system object through the
> +i_file_acl inode member and the inode is updated on disk.
> +
> +Signed-off-by: Philip Tricca <flihp@twobit.us>
> +
> +Index: e2fsprogs-1.42.9/misc/xattr.c
> +===================================================================
> +--- e2fsprogs-1.42.9.orig/misc/xattr.c
> ++++ e2fsprogs-1.42.9/misc/xattr.c
> +@@ -1,6 +1,23 @@
> + #include "xattr.h"
> +
> ++#include <attr/xattr.h>
> ++#include <ctype.h>
> ++#include <errno.h>
> ++#include <ext2fs/ext2_ext_attr.h>
> ++#include <linux/xattr.h>
> ++#include <stdint.h>
> + #include <stdio.h>
> ++#include <stdlib.h>
> ++#include <string.h>
> ++#include <sys/stat.h>
> ++#include <sys/types.h>
> ++#include <unistd.h>
> ++
> ++#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y)) #define HEADER(ptr)
> ++((struct ext2_ext_attr_header *)(ptr)) #define ENTRY(ptr)  ((struct
> ++ext2_ext_attr_entry *)(ptr)) #define FIRST_ENTRY(ptr)
> ++ENTRY(HEADER(ptr) + 1) #define VALUE(hdr, ent) (((char*)hdr) +
> ++(ent->e_value_offs))
> +
> + #ifdef XATTR_DEBUG
> + #define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args) @@
> +-8,6 +25,28 @@  #define XATTR_STDERR(fmt, args...) do {} while (0)
> +#endif
> +
> ++/* structure for mapping xattr name prefix data */ typedef struct
> ++xattr_prefix {
> ++	int index;
> ++	char *name;
> ++	size_t length;
> ++} xattr_prefix_t;
> ++
> ++xattr_prefix_t xattr_prefixes [] = {
> ++/* Only interested in security prefix. Can support others though.
> ++	{
> ++		.index = EXT2_XATTR_INDEX_USER,
> ++		.name = XATTR_USER_PREFIX,
> ++		.length = XATTR_USER_PREFIX_LEN,
> ++	},
> ++*/
> ++	{
> ++		.index = EXT2_XATTR_INDEX_SECURITY,
> ++		.name = XATTR_SECURITY_PREFIX,
> ++		.length = XATTR_SECURITY_PREFIX_LEN,

Hi Philip,

This cause build errors on some host OS when building e2fsprogs-native:

| /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-1.42.9/debugfs/../misc/xattr.c:62:11: error: 'XATTR_SECURITY_PREFIX' undeclared here (not in a function)
|    .name = XATTR_SECURITY_PREFIX,
|            ^
| /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-1.42.9/debugfs/../misc/xattr.c:63:13: error: 'XATTR_SECURITY_PREFIX_LEN' undeclared here (not in a function)
|    .length = XATTR_SECURITY_PREFIX_LEN,
|              ^

I did some investigate and found that your patch needs the header linux/xattr.h,
which is provided by linux-libc-headers, but for -native package, there is no linux-libc-headers-native,
so it search the one from host OS, but the problem is, there is no XATTR_SECURITY_PREFIX definition
in the linux/xattr.h on some host OS like: SUSE 11.x, centos 6, etc.

I'm not sure if your patch is really needed by the e2fsprogs-native, if not, I think we can make this
patch only apply for target package. If yes, you may need to make it avoid the dependency on host's
header or you may have a better idea about this.

Thanks,
Jackie

> ++	},
> ++	{ 0 },
> ++};
> +
> + /* Free remaining resources after all files have been processed. */
> +void @@ -16,6 +55,211 @@ xattr_cleanup ()
> + 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");  }
> +
> ++/* Get value for named xattr from file at path.
> ++ * Returns pointer to allocated block for value and length in length param.
> ++ * If no value, return NULL pointer and length of 0.
> ++ * On error return NULL pointer and length set to -1.
> ++ */
> ++static char*
> ++xattr_get_value (const char *path, const char *name, ssize_t *length)
> ++{
> ++	char *value = NULL;
> ++
> ++	*length = lgetxattr (path, name, NULL, 0);
> ++	if (*length == -1) {
> ++		com_err (__func__, errno, "lgetattr");
> ++		goto out;
> ++	}
> ++	if (*length == 0) {
> ++		fprintf (stderr, "xattr %s has value length 0\n", name);
> ++		goto out;
> ++	}
> ++	value = calloc (1, *length);
> ++	if (value == NULL) {
> ++		com_err (__func__, errno, "calloc");
> ++		goto out;
> ++	}
> ++	*length = lgetxattr (path, name, value, *length);
> ++	if (*length == -1) {
> ++		com_err (__func__, errno, "lgetattr");
> ++		goto value_err;
> ++	}
> ++out:
> ++	return value;
> ++
> ++value_err:
> ++	if (value)
> ++		free (value);
> ++	return NULL;
> ++}
> ++
> ++/* Get all attribute names for file at path. Return pointer to
> ++allocated memory
> ++ * block holding all names and the length through parameter size.
> ++ * If no xattrs: return NULL and set size to 0
> ++ * If error: return NULL and set size to -1  */ static char*
> ++xattr_get_names (const char *path, ssize_t *size) {
> ++	char *names = NULL;
> ++
> ++	*size = llistxattr (path, NULL, 0);
> ++	if (*size == -1) {
> ++		com_err (__func__, errno, "llistxattr");
> ++		goto out;
> ++	}
> ++	if (*size == 0) {
> ++		/* no xattrs */
> ++		goto out;
> ++	}
> ++	names = calloc (1, *size);
> ++	if (names == NULL) {
> ++		com_err (__func__, errno, "calloc");
> ++		goto out;
> ++	}
> ++	*size = llistxattr (path, names, *size);
> ++	if (*size == -1) {
> ++		com_err (__func__, errno, "llistxattr");
> ++		goto cleanup;
> ++	}
> ++	if (*size == 0) {
> ++		fprintf (stdout, "Conflicting data, no xattrs for file: %s\n", path);
> ++		goto cleanup;
> ++	}
> ++out:
> ++	return names;
> ++
> ++cleanup:
> ++	if (names)
> ++		free (names);
> ++	return NULL;
> ++}
> ++
> ++/* return pointer to next string in xattr name block, don't go beyond
> ++length  */ static inline char* next_name (char *name, size_t length) {
> ++	int i = 0;
> ++
> ++	for (i = 0; i < length; ++i)
> ++		if (name [i] == '\0') {
> ++			++i;
> ++			break;
> ++		}
> ++
> ++	return name + i;
> ++}
> ++
> ++/* Find entry in xattr_table with matching prefix. */ static const
> ++xattr_prefix_t* xattr_find_prefix (char *name, size_t length) {
> ++	int i = 0;
> ++
> ++	XATTR_STDERR ("find_attr_data: searching for prefix from xattr name: %s\n", name);
> ++	for (i = 0; xattr_prefixes[i].index != 0; ++i) {
> ++		if (!strncmp (name, xattr_prefixes[i].name, MIN (length, xattr_prefixes[i].length))) {
> ++			XATTR_STDERR ("found match in table with index: %d\n",
> xattr_prefixes[i].index);
> ++			return &xattr_prefixes[i];
> ++		}
> ++	}
> ++	return NULL;
> ++}
> ++
> ++/* Query file at path for attributes. build up structure the file
> ++system
> ++ * expects of an extended attribute disk block (header parameter).
> ++ *
> ++ * The main loop walks through the xattr names one at a time. It gets
> ++the value
> ++ * for each named xattr and copies the data into the xattr block
> ++pointed to by
> ++ * the header parameter. To do this the loop also tracks the location
> ++of the
> ++ * associated entry and value. Values start at the end of the buffer
> ++and grow
> ++ * back towards the header while the entries start immediately after
> ++the header
> ++ * and grow towards the end of the block.
> ++ *
> ++ * See the comment block at the beginning of the xattr.c file in the
> ++ext2 file
> ++ * system code in the kernel: fs/ext2/xattr.c
> ++ * Assume the xattr block pointed to by header parameter is initialized to 0s.
> ++ */
> ++static int
> ++xattr_build_block (const char *path,
> ++		struct ext2_ext_attr_header **header,
> ++		size_t header_length)
> ++{
> ++	struct ext2_ext_attr_entry *entry = NULL;
> ++	char *names = NULL, *value = NULL, *name_curr = NULL;
> ++	ssize_t names_length = 0, value_length = 0;
> ++	size_t name_length = 0, value_index = 0, len_rem = 0;
> ++	const xattr_prefix_t *prefix = NULL;
> ++	int ret = 0;
> ++
> ++	XATTR_STDERR ("xattr_build_block for file: %s\n", path);
> ++	*header = NULL;
> ++	names = xattr_get_names (path, &names_length);
> ++	if (names == NULL) {
> ++		// no xattrs for file @ path or error
> ++		ret = names_length;
> ++		goto out;
> ++	}
> ++	*header = calloc (1, header_length);
> ++	if (*header == NULL) {
> ++		com_err (__func__, errno, "calloc");
> ++		goto out;
> ++	}
> ++	(*header)->h_magic = EXT2_EXT_ATTR_MAGIC;
> ++	(*header)->h_blocks = 1;
> ++	/* Values start at end of buffer. NOTE: It must be moved before a value can
> ++	 * be stored.
> ++	 */
> ++	value_index = header_length;
> ++	for (name_curr = names, entry = FIRST_ENTRY(*header), len_rem = names_length;
> ++		name_curr < names + names_length;
> ++		len_rem = names_length - (name_curr - names),
> ++			name_curr = next_name (name_curr, len_rem),
> ++			entry = EXT2_EXT_ATTR_NEXT(entry))
> ++	{
> ++		XATTR_STDERR ("xattr_build_block: processing xattr with name %s\n", name_curr);
> ++		name_length = strnlen (name_curr, len_rem);
> ++		prefix = xattr_find_prefix (name_curr, name_length);
> ++		if (!prefix) {
> ++			fprintf (stderr, "Don't currently handle xattr: %s\n", name_curr);
> ++			continue;
> ++		}
> ++		value = xattr_get_value (path, name_curr, &value_length);
> ++		if (value == NULL) {
> ++			// no xattr value or error
> ++			fprintf (stderr, "failed to get value, skipping\n");
> ++			goto next;
> ++		}
> ++		/* setup offsets and lengths for name and value */
> ++		entry->e_name_len = name_length - prefix->length;
> ++		entry->e_name_index = prefix->index;
> ++		/* Can't know these till we know the length of the value. */
> ++		entry->e_value_offs = value_index -= EXT2_EXT_ATTR_SIZE(value_length);
> ++		entry->e_value_size = value_length;
> ++		/* Check to be sure entry name and value don't overlap before copy. */
> ++		if (EXT2_EXT_ATTR_NAME(entry) + entry->e_name_len > VALUE(*header, entry)) {
> ++			fprintf (stderr, "xattr entry name and value overlap! Too much xattr data.");
> ++			ret = -1;
> ++			goto out;
> ++		}
> ++		/* copy name and value data then calculate the hash */
> ++		memcpy (EXT2_EXT_ATTR_NAME(entry),
> ++			name_curr + prefix->length,
> ++			entry->e_name_len);
> ++		memcpy (VALUE(*header, entry), value, entry->e_value_size);
> ++		entry->e_hash = ext2fs_ext_attr_hash_entry (entry, VALUE(*header,
> ++entry));
> ++next:
> ++		if (value)
> ++			free (value);
> ++	}
> ++	XATTR_STDERR ("xattr_build_block: done building xattr buffer\n");
> ++out:
> ++	if (names)
> ++		free (names);
> ++	return ret;
> ++}
> ++
> + /* This is the entry point to the xattr module. This function copies
> +the xattrs
> +  * from the file at 'path' to the file system object at 'ino'.
> +  *
> +@@ -28,7 +272,56 @@ errcode_t
> + set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)  {
> + 	errcode_t ret = 0;
> ++	char *buf = NULL;
> ++	struct ext2_inode inode = { 0 };
> ++	blk_t block = 0;
> ++	struct ext2_ext_attr_header *header = NULL;
> ++	uint32_t newcount = 0;
> +
> + 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path, ino);
> ++	/* Populate the xattr block for the file at path */
> ++	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
> ++		goto out;
> ++	}
> ++	if (header == NULL) {
> ++		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
> ++		goto out;
> ++	}
> ++	if (ret = ext2fs_read_inode (fs, ino, &inode)) {
> ++		com_err(__func__, ret, "ext2fs_read_inode");
> ++		goto out;
> ++	}
> ++	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
> ++		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
> ++		goto out;
> ++	}
> ++	ext2fs_mark_block_bitmap2 (fs->block_map, block);
> ++	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
> ++	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
> ++		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
> ++		goto out;
> ++	}
> ++	/* point inode for current file to xattr block, update block count and
> ++	 * write inode to disk
> ++	 */
> ++	inode.i_file_acl = block;
> ++	if (ret = ext2fs_adjust_ea_refcount2(fs,
> ++					block,
> ++					(char*)header,
> ++					1,
> ++					&newcount))
> ++	{
> ++		com_err(__func__, ret, "ext2fs_adjust_ea_refcount");
> ++		goto out;
> ++	}
> ++	if (ret = ext2fs_iblk_add_blocks (fs, &inode, 1)) {
> ++		com_err(__func__, ret, "ext2fs_iblk_add_blocks failed");
> ++		goto out;
> ++	}
> ++	if (ret = ext2fs_write_inode (fs, ino, &inode))
> ++		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
> ++out:
> ++	if (header)
> ++		free (header);
> + 	return ret;
> + }
> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-
> devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> index a4576b1..edc94d8 100644
> --- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> +++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> @@ -4,4 +4,5 @@ SRC_URI += " \
>      file://misc-xattr-add-xattr-module-stub.patch \
>      file://mke2fs.c-create_inode.c-copy-xattrs.patch \
>      file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
> +    file://misc-xattr-create-xattr-block.patch \
>  "
> --
> 2.1.4
> 
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.

[Philip Tricca]

Hey Jackie,

On 08/20/2015 11:25 PM, Huang, Jie (Jackie) wrote:
>> -----Original Message-----
>> From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Philip
>> Tricca
>> Sent: Thursday, June 18, 2015 6:31 AM
>> To: yocto@yoctoproject.org
>> Subject: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
>>
>> Signed-off-by: Philip Tricca <flihp@twobit.us>
>> ---
>>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
>>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
>>  2 files changed, 342 insertions(+)
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>>
>> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch b/recipes-
>> devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>> new file mode 100644
>> index 0000000..5955b44
>> --- /dev/null
>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block
>> +++ .patch
>> @@ -0,0 +1,341 @@
>> +To build the xattr disk block we process the output from listxattr and
>> +lgetxattr from the source file system object. This data is formated in
>> +a disk block according to the format specified in the kernel ext2 file system driver.
>> +See the comment block at the beginning of fs/ext2/xattr.c for details.
>> +
>> +Currently we only process attributes with the 'security.' prefix as our
>> +use case is SELinux labels and IMA. Additional prefixes can likely be
>> +supported with minimal effort but none have been tested.
>> +
>> +Once the xattr block has been created it is written to disk. The xattr
>> +block is associated with the appropriate file system object through the
>> +i_file_acl inode member and the inode is updated on disk.
>> +
>> +Signed-off-by: Philip Tricca <flihp@twobit.us>
>> +
>> +Index: e2fsprogs-1.42.9/misc/xattr.c
>> +===================================================================
>> +--- e2fsprogs-1.42.9.orig/misc/xattr.c
>> ++++ e2fsprogs-1.42.9/misc/xattr.c
>> +@@ -1,6 +1,23 @@
>> + #include "xattr.h"
>> +
>> ++#include <attr/xattr.h>
>> ++#include <ctype.h>
>> ++#include <errno.h>
>> ++#include <ext2fs/ext2_ext_attr.h>
>> ++#include <linux/xattr.h>
>> ++#include <stdint.h>
>> + #include <stdio.h>
>> ++#include <stdlib.h>
>> ++#include <string.h>
>> ++#include <sys/stat.h>
>> ++#include <sys/types.h>
>> ++#include <unistd.h>
>> ++
>> ++#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y)) #define HEADER(ptr)
>> ++((struct ext2_ext_attr_header *)(ptr)) #define ENTRY(ptr)  ((struct
>> ++ext2_ext_attr_entry *)(ptr)) #define FIRST_ENTRY(ptr)
>> ++ENTRY(HEADER(ptr) + 1) #define VALUE(hdr, ent) (((char*)hdr) +
>> ++(ent->e_value_offs))
>> +
>> + #ifdef XATTR_DEBUG
>> + #define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args) @@
>> +-8,6 +25,28 @@  #define XATTR_STDERR(fmt, args...) do {} while (0)
>> +#endif
>> +
>> ++/* structure for mapping xattr name prefix data */ typedef struct
>> ++xattr_prefix {
>> ++	int index;
>> ++	char *name;
>> ++	size_t length;
>> ++} xattr_prefix_t;
>> ++
>> ++xattr_prefix_t xattr_prefixes [] = {
>> ++/* Only interested in security prefix. Can support others though.
>> ++	{
>> ++		.index = EXT2_XATTR_INDEX_USER,
>> ++		.name = XATTR_USER_PREFIX,
>> ++		.length = XATTR_USER_PREFIX_LEN,
>> ++	},
>> ++*/
>> ++	{
>> ++		.index = EXT2_XATTR_INDEX_SECURITY,
>> ++		.name = XATTR_SECURITY_PREFIX,
>> ++		.length = XATTR_SECURITY_PREFIX_LEN,
> 
> Hi Philip,
> 
> This cause build errors on some host OS when building e2fsprogs-native:
> 
> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-1.42.9/debugfs/../misc/xattr.c:62:11: error: 'XATTR_SECURITY_PREFIX' undeclared here (not in a function)
> |    .name = XATTR_SECURITY_PREFIX,
> |            ^
> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-1.42.9/debugfs/../misc/xattr.c:63:13: error: 'XATTR_SECURITY_PREFIX_LEN' undeclared here (not in a function)
> |    .length = XATTR_SECURITY_PREFIX_LEN,
> |              ^
> 
> I did some investigate and found that your patch needs the header linux/xattr.h,
> which is provided by linux-libc-headers, but for -native package, there is no linux-libc-headers-native,
> so it search the one from host OS, but the problem is, there is no XATTR_SECURITY_PREFIX definition
> in the linux/xattr.h on some host OS like: SUSE 11.x, centos 6, etc.
> 
> I'm not sure if your patch is really needed by the e2fsprogs-native, if not, I think we can make this
> patch only apply for target package. If yes, you may need to make it avoid the dependency on host's
> header or you may have a better idea about this.

Apologies for the breakage and thanks for testing. This patch is only
intended for e2fsprogs-native since the use-case is to preserve xattrs
when OE creates root filesystems. So it looks to me like you found a
real problem. Likely OE was falling back to using the headers on my
build host.

I don't have an immediate fix but I'll take a look over the weekend.
Thanks again for catching this.

Best,
hilip

>> ++	},
>> ++	{ 0 },
>> ++};
>> +
>> + /* Free remaining resources after all files have been processed. */
>> +void @@ -16,6 +55,211 @@ xattr_cleanup ()
>> + 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");  }
>> +
>> ++/* Get value for named xattr from file at path.
>> ++ * Returns pointer to allocated block for value and length in length param.
>> ++ * If no value, return NULL pointer and length of 0.
>> ++ * On error return NULL pointer and length set to -1.
>> ++ */
>> ++static char*
>> ++xattr_get_value (const char *path, const char *name, ssize_t *length)
>> ++{
>> ++	char *value = NULL;
>> ++
>> ++	*length = lgetxattr (path, name, NULL, 0);
>> ++	if (*length == -1) {
>> ++		com_err (__func__, errno, "lgetattr");
>> ++		goto out;
>> ++	}
>> ++	if (*length == 0) {
>> ++		fprintf (stderr, "xattr %s has value length 0\n", name);
>> ++		goto out;
>> ++	}
>> ++	value = calloc (1, *length);
>> ++	if (value == NULL) {
>> ++		com_err (__func__, errno, "calloc");
>> ++		goto out;
>> ++	}
>> ++	*length = lgetxattr (path, name, value, *length);
>> ++	if (*length == -1) {
>> ++		com_err (__func__, errno, "lgetattr");
>> ++		goto value_err;
>> ++	}
>> ++out:
>> ++	return value;
>> ++
>> ++value_err:
>> ++	if (value)
>> ++		free (value);
>> ++	return NULL;
>> ++}
>> ++
>> ++/* Get all attribute names for file at path. Return pointer to
>> ++allocated memory
>> ++ * block holding all names and the length through parameter size.
>> ++ * If no xattrs: return NULL and set size to 0
>> ++ * If error: return NULL and set size to -1  */ static char*
>> ++xattr_get_names (const char *path, ssize_t *size) {
>> ++	char *names = NULL;
>> ++
>> ++	*size = llistxattr (path, NULL, 0);
>> ++	if (*size == -1) {
>> ++		com_err (__func__, errno, "llistxattr");
>> ++		goto out;
>> ++	}
>> ++	if (*size == 0) {
>> ++		/* no xattrs */
>> ++		goto out;
>> ++	}
>> ++	names = calloc (1, *size);
>> ++	if (names == NULL) {
>> ++		com_err (__func__, errno, "calloc");
>> ++		goto out;
>> ++	}
>> ++	*size = llistxattr (path, names, *size);
>> ++	if (*size == -1) {
>> ++		com_err (__func__, errno, "llistxattr");
>> ++		goto cleanup;
>> ++	}
>> ++	if (*size == 0) {
>> ++		fprintf (stdout, "Conflicting data, no xattrs for file: %s\n", path);
>> ++		goto cleanup;
>> ++	}
>> ++out:
>> ++	return names;
>> ++
>> ++cleanup:
>> ++	if (names)
>> ++		free (names);
>> ++	return NULL;
>> ++}
>> ++
>> ++/* return pointer to next string in xattr name block, don't go beyond
>> ++length  */ static inline char* next_name (char *name, size_t length) {
>> ++	int i = 0;
>> ++
>> ++	for (i = 0; i < length; ++i)
>> ++		if (name [i] == '\0') {
>> ++			++i;
>> ++			break;
>> ++		}
>> ++
>> ++	return name + i;
>> ++}
>> ++
>> ++/* Find entry in xattr_table with matching prefix. */ static const
>> ++xattr_prefix_t* xattr_find_prefix (char *name, size_t length) {
>> ++	int i = 0;
>> ++
>> ++	XATTR_STDERR ("find_attr_data: searching for prefix from xattr name: %s\n", name);
>> ++	for (i = 0; xattr_prefixes[i].index != 0; ++i) {
>> ++		if (!strncmp (name, xattr_prefixes[i].name, MIN (length, xattr_prefixes[i].length))) {
>> ++			XATTR_STDERR ("found match in table with index: %d\n",
>> xattr_prefixes[i].index);
>> ++			return &xattr_prefixes[i];
>> ++		}
>> ++	}
>> ++	return NULL;
>> ++}
>> ++
>> ++/* Query file at path for attributes. build up structure the file
>> ++system
>> ++ * expects of an extended attribute disk block (header parameter).
>> ++ *
>> ++ * The main loop walks through the xattr names one at a time. It gets
>> ++the value
>> ++ * for each named xattr and copies the data into the xattr block
>> ++pointed to by
>> ++ * the header parameter. To do this the loop also tracks the location
>> ++of the
>> ++ * associated entry and value. Values start at the end of the buffer
>> ++and grow
>> ++ * back towards the header while the entries start immediately after
>> ++the header
>> ++ * and grow towards the end of the block.
>> ++ *
>> ++ * See the comment block at the beginning of the xattr.c file in the
>> ++ext2 file
>> ++ * system code in the kernel: fs/ext2/xattr.c
>> ++ * Assume the xattr block pointed to by header parameter is initialized to 0s.
>> ++ */
>> ++static int
>> ++xattr_build_block (const char *path,
>> ++		struct ext2_ext_attr_header **header,
>> ++		size_t header_length)
>> ++{
>> ++	struct ext2_ext_attr_entry *entry = NULL;
>> ++	char *names = NULL, *value = NULL, *name_curr = NULL;
>> ++	ssize_t names_length = 0, value_length = 0;
>> ++	size_t name_length = 0, value_index = 0, len_rem = 0;
>> ++	const xattr_prefix_t *prefix = NULL;
>> ++	int ret = 0;
>> ++
>> ++	XATTR_STDERR ("xattr_build_block for file: %s\n", path);
>> ++	*header = NULL;
>> ++	names = xattr_get_names (path, &names_length);
>> ++	if (names == NULL) {
>> ++		// no xattrs for file @ path or error
>> ++		ret = names_length;
>> ++		goto out;
>> ++	}
>> ++	*header = calloc (1, header_length);
>> ++	if (*header == NULL) {
>> ++		com_err (__func__, errno, "calloc");
>> ++		goto out;
>> ++	}
>> ++	(*header)->h_magic = EXT2_EXT_ATTR_MAGIC;
>> ++	(*header)->h_blocks = 1;
>> ++	/* Values start at end of buffer. NOTE: It must be moved before a value can
>> ++	 * be stored.
>> ++	 */
>> ++	value_index = header_length;
>> ++	for (name_curr = names, entry = FIRST_ENTRY(*header), len_rem = names_length;
>> ++		name_curr < names + names_length;
>> ++		len_rem = names_length - (name_curr - names),
>> ++			name_curr = next_name (name_curr, len_rem),
>> ++			entry = EXT2_EXT_ATTR_NEXT(entry))
>> ++	{
>> ++		XATTR_STDERR ("xattr_build_block: processing xattr with name %s\n", name_curr);
>> ++		name_length = strnlen (name_curr, len_rem);
>> ++		prefix = xattr_find_prefix (name_curr, name_length);
>> ++		if (!prefix) {
>> ++			fprintf (stderr, "Don't currently handle xattr: %s\n", name_curr);
>> ++			continue;
>> ++		}
>> ++		value = xattr_get_value (path, name_curr, &value_length);
>> ++		if (value == NULL) {
>> ++			// no xattr value or error
>> ++			fprintf (stderr, "failed to get value, skipping\n");
>> ++			goto next;
>> ++		}
>> ++		/* setup offsets and lengths for name and value */
>> ++		entry->e_name_len = name_length - prefix->length;
>> ++		entry->e_name_index = prefix->index;
>> ++		/* Can't know these till we know the length of the value. */
>> ++		entry->e_value_offs = value_index -= EXT2_EXT_ATTR_SIZE(value_length);
>> ++		entry->e_value_size = value_length;
>> ++		/* Check to be sure entry name and value don't overlap before copy. */
>> ++		if (EXT2_EXT_ATTR_NAME(entry) + entry->e_name_len > VALUE(*header, entry)) {
>> ++			fprintf (stderr, "xattr entry name and value overlap! Too much xattr data.");
>> ++			ret = -1;
>> ++			goto out;
>> ++		}
>> ++		/* copy name and value data then calculate the hash */
>> ++		memcpy (EXT2_EXT_ATTR_NAME(entry),
>> ++			name_curr + prefix->length,
>> ++			entry->e_name_len);
>> ++		memcpy (VALUE(*header, entry), value, entry->e_value_size);
>> ++		entry->e_hash = ext2fs_ext_attr_hash_entry (entry, VALUE(*header,
>> ++entry));
>> ++next:
>> ++		if (value)
>> ++			free (value);
>> ++	}
>> ++	XATTR_STDERR ("xattr_build_block: done building xattr buffer\n");
>> ++out:
>> ++	if (names)
>> ++		free (names);
>> ++	return ret;
>> ++}
>> ++
>> + /* This is the entry point to the xattr module. This function copies
>> +the xattrs
>> +  * from the file at 'path' to the file system object at 'ino'.
>> +  *
>> +@@ -28,7 +272,56 @@ errcode_t
>> + set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)  {
>> + 	errcode_t ret = 0;
>> ++	char *buf = NULL;
>> ++	struct ext2_inode inode = { 0 };
>> ++	blk_t block = 0;
>> ++	struct ext2_ext_attr_header *header = NULL;
>> ++	uint32_t newcount = 0;
>> +
>> + 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path, ino);
>> ++	/* Populate the xattr block for the file at path */
>> ++	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
>> ++		goto out;
>> ++	}
>> ++	if (header == NULL) {
>> ++		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_read_inode (fs, ino, &inode)) {
>> ++		com_err(__func__, ret, "ext2fs_read_inode");
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
>> ++		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
>> ++		goto out;
>> ++	}
>> ++	ext2fs_mark_block_bitmap2 (fs->block_map, block);
>> ++	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
>> ++	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
>> ++		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
>> ++		goto out;
>> ++	}
>> ++	/* point inode for current file to xattr block, update block count and
>> ++	 * write inode to disk
>> ++	 */
>> ++	inode.i_file_acl = block;
>> ++	if (ret = ext2fs_adjust_ea_refcount2(fs,
>> ++					block,
>> ++					(char*)header,
>> ++					1,
>> ++					&newcount))
>> ++	{
>> ++		com_err(__func__, ret, "ext2fs_adjust_ea_refcount");
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_iblk_add_blocks (fs, &inode, 1)) {
>> ++		com_err(__func__, ret, "ext2fs_iblk_add_blocks failed");
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_write_inode (fs, ino, &inode))
>> ++		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
>> ++out:
>> ++	if (header)
>> ++		free (header);
>> + 	return ret;
>> + }
>> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-
>> devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>> index a4576b1..edc94d8 100644
>> --- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>> @@ -4,4 +4,5 @@ SRC_URI += " \
>>      file://misc-xattr-add-xattr-module-stub.patch \
>>      file://mke2fs.c-create_inode.c-copy-xattrs.patch \
>>      file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
>> +    file://misc-xattr-create-xattr-block.patch \
>>  "
>> --
>> 2.1.4
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto

Re: [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.

[Philip Tricca]

Hello Jackie,

On 08/20/2015 11:25 PM, Huang, Jie (Jackie) wrote:
>> -----Original Message-----
>> From: yocto-bounces@yoctoproject.org [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Philip
>> Tricca
>> Sent: Thursday, June 18, 2015 6:31 AM
>> To: yocto@yoctoproject.org
>> Subject: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
>>
>> Signed-off-by: Philip Tricca <flihp@twobit.us>
>> ---
>>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
>>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
>>  2 files changed, 342 insertions(+)
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>>
>> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch b/recipes-
>> devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>> new file mode 100644
>> index 0000000..5955b44
>> --- /dev/null
>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block
>> +++ .patch
>> @@ -0,0 +1,341 @@
>> +To build the xattr disk block we process the output from listxattr and
>> +lgetxattr from the source file system object. This data is formated in
>> +a disk block according to the format specified in the kernel ext2 file system driver.
>> +See the comment block at the beginning of fs/ext2/xattr.c for details.
>> +
>> +Currently we only process attributes with the 'security.' prefix as our
>> +use case is SELinux labels and IMA. Additional prefixes can likely be
>> +supported with minimal effort but none have been tested.
>> +
>> +Once the xattr block has been created it is written to disk. The xattr
>> +block is associated with the appropriate file system object through the
>> +i_file_acl inode member and the inode is updated on disk.
>> +
>> +Signed-off-by: Philip Tricca <flihp@twobit.us>
>> +
>> +Index: e2fsprogs-1.42.9/misc/xattr.c
>> +===================================================================
>> +--- e2fsprogs-1.42.9.orig/misc/xattr.c
>> ++++ e2fsprogs-1.42.9/misc/xattr.c
>> +@@ -1,6 +1,23 @@
>> + #include "xattr.h"
>> +
>> ++#include <attr/xattr.h>
>> ++#include <ctype.h>
>> ++#include <errno.h>
>> ++#include <ext2fs/ext2_ext_attr.h>
>> ++#include <linux/xattr.h>
>> ++#include <stdint.h>
>> + #include <stdio.h>
>> ++#include <stdlib.h>
>> ++#include <string.h>
>> ++#include <sys/stat.h>
>> ++#include <sys/types.h>
>> ++#include <unistd.h>
>> ++
>> ++#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y)) #define HEADER(ptr)
>> ++((struct ext2_ext_attr_header *)(ptr)) #define ENTRY(ptr)  ((struct
>> ++ext2_ext_attr_entry *)(ptr)) #define FIRST_ENTRY(ptr)
>> ++ENTRY(HEADER(ptr) + 1) #define VALUE(hdr, ent) (((char*)hdr) +
>> ++(ent->e_value_offs))
>> +
>> + #ifdef XATTR_DEBUG
>> + #define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args) @@
>> +-8,6 +25,28 @@  #define XATTR_STDERR(fmt, args...) do {} while (0)
>> +#endif
>> +
>> ++/* structure for mapping xattr name prefix data */ typedef struct
>> ++xattr_prefix {
>> ++	int index;
>> ++	char *name;
>> ++	size_t length;
>> ++} xattr_prefix_t;
>> ++
>> ++xattr_prefix_t xattr_prefixes [] = {
>> ++/* Only interested in security prefix. Can support others though.
>> ++	{
>> ++		.index = EXT2_XATTR_INDEX_USER,
>> ++		.name = XATTR_USER_PREFIX,
>> ++		.length = XATTR_USER_PREFIX_LEN,
>> ++	},
>> ++*/
>> ++	{
>> ++		.index = EXT2_XATTR_INDEX_SECURITY,
>> ++		.name = XATTR_SECURITY_PREFIX,
>> ++		.length = XATTR_SECURITY_PREFIX_LEN,
> 
> Hi Philip,
> 
> This cause build errors on some host OS when building e2fsprogs-native:
> 
> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-1.42.9/debugfs/../misc/xattr.c:62:11: error: 'XATTR_SECURITY_PREFIX' undeclared here (not in a function)
> |    .name = XATTR_SECURITY_PREFIX,
> |            ^
> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-1.42.9/debugfs/../misc/xattr.c:63:13: error: 'XATTR_SECURITY_PREFIX_LEN' undeclared here (not in a function)
> |    .length = XATTR_SECURITY_PREFIX_LEN,
> |              ^
> 
> I did some investigate and found that your patch needs the header linux/xattr.h,
> which is provided by linux-libc-headers, but for -native package, there is no linux-libc-headers-native,
> so it search the one from host OS, but the problem is, there is no XATTR_SECURITY_PREFIX definition
> in the linux/xattr.h on some host OS like: SUSE 11.x, centos 6, etc.
> 
> I'm not sure if your patch is really needed by the e2fsprogs-native, if not, I think we can make this
> patch only apply for target package. If yes, you may need to make it avoid the dependency on host's
> header or you may have a better idea about this.

I took a look at this yesterday. Creating a bbappend for
linux-libc-headers and adding 'native' to BBCLASSEXTEND seems like the
most direct fix. Add a new dependency to the e2fsprogs package and I
think the issue is resolved. I'll send patches to the list and if you're
willing to test them I'd appreciate it.

Thanks,
Philip

>> ++	},
>> ++	{ 0 },
>> ++};
>> +
>> + /* Free remaining resources after all files have been processed. */
>> +void @@ -16,6 +55,211 @@ xattr_cleanup ()
>> + 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");  }
>> +
>> ++/* Get value for named xattr from file at path.
>> ++ * Returns pointer to allocated block for value and length in length param.
>> ++ * If no value, return NULL pointer and length of 0.
>> ++ * On error return NULL pointer and length set to -1.
>> ++ */
>> ++static char*
>> ++xattr_get_value (const char *path, const char *name, ssize_t *length)
>> ++{
>> ++	char *value = NULL;
>> ++
>> ++	*length = lgetxattr (path, name, NULL, 0);
>> ++	if (*length == -1) {
>> ++		com_err (__func__, errno, "lgetattr");
>> ++		goto out;
>> ++	}
>> ++	if (*length == 0) {
>> ++		fprintf (stderr, "xattr %s has value length 0\n", name);
>> ++		goto out;
>> ++	}
>> ++	value = calloc (1, *length);
>> ++	if (value == NULL) {
>> ++		com_err (__func__, errno, "calloc");
>> ++		goto out;
>> ++	}
>> ++	*length = lgetxattr (path, name, value, *length);
>> ++	if (*length == -1) {
>> ++		com_err (__func__, errno, "lgetattr");
>> ++		goto value_err;
>> ++	}
>> ++out:
>> ++	return value;
>> ++
>> ++value_err:
>> ++	if (value)
>> ++		free (value);
>> ++	return NULL;
>> ++}
>> ++
>> ++/* Get all attribute names for file at path. Return pointer to
>> ++allocated memory
>> ++ * block holding all names and the length through parameter size.
>> ++ * If no xattrs: return NULL and set size to 0
>> ++ * If error: return NULL and set size to -1  */ static char*
>> ++xattr_get_names (const char *path, ssize_t *size) {
>> ++	char *names = NULL;
>> ++
>> ++	*size = llistxattr (path, NULL, 0);
>> ++	if (*size == -1) {
>> ++		com_err (__func__, errno, "llistxattr");
>> ++		goto out;
>> ++	}
>> ++	if (*size == 0) {
>> ++		/* no xattrs */
>> ++		goto out;
>> ++	}
>> ++	names = calloc (1, *size);
>> ++	if (names == NULL) {
>> ++		com_err (__func__, errno, "calloc");
>> ++		goto out;
>> ++	}
>> ++	*size = llistxattr (path, names, *size);
>> ++	if (*size == -1) {
>> ++		com_err (__func__, errno, "llistxattr");
>> ++		goto cleanup;
>> ++	}
>> ++	if (*size == 0) {
>> ++		fprintf (stdout, "Conflicting data, no xattrs for file: %s\n", path);
>> ++		goto cleanup;
>> ++	}
>> ++out:
>> ++	return names;
>> ++
>> ++cleanup:
>> ++	if (names)
>> ++		free (names);
>> ++	return NULL;
>> ++}
>> ++
>> ++/* return pointer to next string in xattr name block, don't go beyond
>> ++length  */ static inline char* next_name (char *name, size_t length) {
>> ++	int i = 0;
>> ++
>> ++	for (i = 0; i < length; ++i)
>> ++		if (name [i] == '\0') {
>> ++			++i;
>> ++			break;
>> ++		}
>> ++
>> ++	return name + i;
>> ++}
>> ++
>> ++/* Find entry in xattr_table with matching prefix. */ static const
>> ++xattr_prefix_t* xattr_find_prefix (char *name, size_t length) {
>> ++	int i = 0;
>> ++
>> ++	XATTR_STDERR ("find_attr_data: searching for prefix from xattr name: %s\n", name);
>> ++	for (i = 0; xattr_prefixes[i].index != 0; ++i) {
>> ++		if (!strncmp (name, xattr_prefixes[i].name, MIN (length, xattr_prefixes[i].length))) {
>> ++			XATTR_STDERR ("found match in table with index: %d\n",
>> xattr_prefixes[i].index);
>> ++			return &xattr_prefixes[i];
>> ++		}
>> ++	}
>> ++	return NULL;
>> ++}
>> ++
>> ++/* Query file at path for attributes. build up structure the file
>> ++system
>> ++ * expects of an extended attribute disk block (header parameter).
>> ++ *
>> ++ * The main loop walks through the xattr names one at a time. It gets
>> ++the value
>> ++ * for each named xattr and copies the data into the xattr block
>> ++pointed to by
>> ++ * the header parameter. To do this the loop also tracks the location
>> ++of the
>> ++ * associated entry and value. Values start at the end of the buffer
>> ++and grow
>> ++ * back towards the header while the entries start immediately after
>> ++the header
>> ++ * and grow towards the end of the block.
>> ++ *
>> ++ * See the comment block at the beginning of the xattr.c file in the
>> ++ext2 file
>> ++ * system code in the kernel: fs/ext2/xattr.c
>> ++ * Assume the xattr block pointed to by header parameter is initialized to 0s.
>> ++ */
>> ++static int
>> ++xattr_build_block (const char *path,
>> ++		struct ext2_ext_attr_header **header,
>> ++		size_t header_length)
>> ++{
>> ++	struct ext2_ext_attr_entry *entry = NULL;
>> ++	char *names = NULL, *value = NULL, *name_curr = NULL;
>> ++	ssize_t names_length = 0, value_length = 0;
>> ++	size_t name_length = 0, value_index = 0, len_rem = 0;
>> ++	const xattr_prefix_t *prefix = NULL;
>> ++	int ret = 0;
>> ++
>> ++	XATTR_STDERR ("xattr_build_block for file: %s\n", path);
>> ++	*header = NULL;
>> ++	names = xattr_get_names (path, &names_length);
>> ++	if (names == NULL) {
>> ++		// no xattrs for file @ path or error
>> ++		ret = names_length;
>> ++		goto out;
>> ++	}
>> ++	*header = calloc (1, header_length);
>> ++	if (*header == NULL) {
>> ++		com_err (__func__, errno, "calloc");
>> ++		goto out;
>> ++	}
>> ++	(*header)->h_magic = EXT2_EXT_ATTR_MAGIC;
>> ++	(*header)->h_blocks = 1;
>> ++	/* Values start at end of buffer. NOTE: It must be moved before a value can
>> ++	 * be stored.
>> ++	 */
>> ++	value_index = header_length;
>> ++	for (name_curr = names, entry = FIRST_ENTRY(*header), len_rem = names_length;
>> ++		name_curr < names + names_length;
>> ++		len_rem = names_length - (name_curr - names),
>> ++			name_curr = next_name (name_curr, len_rem),
>> ++			entry = EXT2_EXT_ATTR_NEXT(entry))
>> ++	{
>> ++		XATTR_STDERR ("xattr_build_block: processing xattr with name %s\n", name_curr);
>> ++		name_length = strnlen (name_curr, len_rem);
>> ++		prefix = xattr_find_prefix (name_curr, name_length);
>> ++		if (!prefix) {
>> ++			fprintf (stderr, "Don't currently handle xattr: %s\n", name_curr);
>> ++			continue;
>> ++		}
>> ++		value = xattr_get_value (path, name_curr, &value_length);
>> ++		if (value == NULL) {
>> ++			// no xattr value or error
>> ++			fprintf (stderr, "failed to get value, skipping\n");
>> ++			goto next;
>> ++		}
>> ++		/* setup offsets and lengths for name and value */
>> ++		entry->e_name_len = name_length - prefix->length;
>> ++		entry->e_name_index = prefix->index;
>> ++		/* Can't know these till we know the length of the value. */
>> ++		entry->e_value_offs = value_index -= EXT2_EXT_ATTR_SIZE(value_length);
>> ++		entry->e_value_size = value_length;
>> ++		/* Check to be sure entry name and value don't overlap before copy. */
>> ++		if (EXT2_EXT_ATTR_NAME(entry) + entry->e_name_len > VALUE(*header, entry)) {
>> ++			fprintf (stderr, "xattr entry name and value overlap! Too much xattr data.");
>> ++			ret = -1;
>> ++			goto out;
>> ++		}
>> ++		/* copy name and value data then calculate the hash */
>> ++		memcpy (EXT2_EXT_ATTR_NAME(entry),
>> ++			name_curr + prefix->length,
>> ++			entry->e_name_len);
>> ++		memcpy (VALUE(*header, entry), value, entry->e_value_size);
>> ++		entry->e_hash = ext2fs_ext_attr_hash_entry (entry, VALUE(*header,
>> ++entry));
>> ++next:
>> ++		if (value)
>> ++			free (value);
>> ++	}
>> ++	XATTR_STDERR ("xattr_build_block: done building xattr buffer\n");
>> ++out:
>> ++	if (names)
>> ++		free (names);
>> ++	return ret;
>> ++}
>> ++
>> + /* This is the entry point to the xattr module. This function copies
>> +the xattrs
>> +  * from the file at 'path' to the file system object at 'ino'.
>> +  *
>> +@@ -28,7 +272,56 @@ errcode_t
>> + set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)  {
>> + 	errcode_t ret = 0;
>> ++	char *buf = NULL;
>> ++	struct ext2_inode inode = { 0 };
>> ++	blk_t block = 0;
>> ++	struct ext2_ext_attr_header *header = NULL;
>> ++	uint32_t newcount = 0;
>> +
>> + 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path, ino);
>> ++	/* Populate the xattr block for the file at path */
>> ++	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
>> ++		goto out;
>> ++	}
>> ++	if (header == NULL) {
>> ++		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_read_inode (fs, ino, &inode)) {
>> ++		com_err(__func__, ret, "ext2fs_read_inode");
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
>> ++		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
>> ++		goto out;
>> ++	}
>> ++	ext2fs_mark_block_bitmap2 (fs->block_map, block);
>> ++	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
>> ++	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
>> ++		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
>> ++		goto out;
>> ++	}
>> ++	/* point inode for current file to xattr block, update block count and
>> ++	 * write inode to disk
>> ++	 */
>> ++	inode.i_file_acl = block;
>> ++	if (ret = ext2fs_adjust_ea_refcount2(fs,
>> ++					block,
>> ++					(char*)header,
>> ++					1,
>> ++					&newcount))
>> ++	{
>> ++		com_err(__func__, ret, "ext2fs_adjust_ea_refcount");
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_iblk_add_blocks (fs, &inode, 1)) {
>> ++		com_err(__func__, ret, "ext2fs_iblk_add_blocks failed");
>> ++		goto out;
>> ++	}
>> ++	if (ret = ext2fs_write_inode (fs, ino, &inode))
>> ++		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
>> ++out:
>> ++	if (header)
>> ++		free (header);
>> + 	return ret;
>> + }
>> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-
>> devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>> index a4576b1..edc94d8 100644
>> --- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>> @@ -4,4 +4,5 @@ SRC_URI += " \
>>      file://misc-xattr-add-xattr-module-stub.patch \
>>      file://mke2fs.c-create_inode.c-copy-xattrs.patch \
>>      file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
>> +    file://misc-xattr-create-xattr-block.patch \
>>  "
>> --
>> 2.1.4
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto

Re: [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.

[Huang, Jie (Jackie)]

> -----Original Message-----
> From: Philip Tricca [mailto:flihp@twobit.us]
> Sent: Monday, August 24, 2015 6:25 AM
> To: Huang, Jie (Jackie)
> Cc: yocto@yoctoproject.org
> Subject: Re: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
> 
> Hello Jackie,
> 
> On 08/20/2015 11:25 PM, Huang, Jie (Jackie) wrote:
> >> -----Original Message-----
> >> From: yocto-bounces@yoctoproject.org
> >> [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Philip Tricca
> >> Sent: Thursday, June 18, 2015 6:31 AM
> >> To: yocto@yoctoproject.org
> >> Subject: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
> >>
> >> Signed-off-by: Philip Tricca <flihp@twobit.us>
> >> ---
> >>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
> >>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
> >>  2 files changed, 342 insertions(+)
> >>  create mode 100644
> >> recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.pa
> >> tch
> >>
> >> diff --git
> >> a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.
> >> patch b/recipes-
> >> devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
> >> new file mode 100644
> >> index 0000000..5955b44
> >> --- /dev/null
> >> +++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-bl
> >> +++ ock
> >> +++ .patch
> >> @@ -0,0 +1,341 @@
> >> +To build the xattr disk block we process the output from listxattr
> >> +and lgetxattr from the source file system object. This data is
> >> +formated in a disk block according to the format specified in the kernel ext2 file system driver.
> >> +See the comment block at the beginning of fs/ext2/xattr.c for details.
> >> +
> >> +Currently we only process attributes with the 'security.' prefix as
> >> +our use case is SELinux labels and IMA. Additional prefixes can
> >> +likely be supported with minimal effort but none have been tested.
> >> +
> >> +Once the xattr block has been created it is written to disk. The
> >> +xattr block is associated with the appropriate file system object
> >> +through the i_file_acl inode member and the inode is updated on disk.
> >> +
> >> +Signed-off-by: Philip Tricca <flihp@twobit.us>
> >> +
> >> +Index: e2fsprogs-1.42.9/misc/xattr.c
> >> +===================================================================
> >> +--- e2fsprogs-1.42.9.orig/misc/xattr.c
> >> ++++ e2fsprogs-1.42.9/misc/xattr.c
> >> +@@ -1,6 +1,23 @@
> >> + #include "xattr.h"
> >> +
> >> ++#include <attr/xattr.h>
> >> ++#include <ctype.h>
> >> ++#include <errno.h>
> >> ++#include <ext2fs/ext2_ext_attr.h>
> >> ++#include <linux/xattr.h>
> >> ++#include <stdint.h>
> >> + #include <stdio.h>
> >> ++#include <stdlib.h>
> >> ++#include <string.h>
> >> ++#include <sys/stat.h>
> >> ++#include <sys/types.h>
> >> ++#include <unistd.h>
> >> ++
> >> ++#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y)) #define HEADER(ptr)
> >> ++((struct ext2_ext_attr_header *)(ptr)) #define ENTRY(ptr)  ((struct
> >> ++ext2_ext_attr_entry *)(ptr)) #define FIRST_ENTRY(ptr)
> >> ++ENTRY(HEADER(ptr) + 1) #define VALUE(hdr, ent) (((char*)hdr) +
> >> ++(ent->e_value_offs))
> >> +
> >> + #ifdef XATTR_DEBUG
> >> + #define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args) @@
> >> +-8,6 +25,28 @@  #define XATTR_STDERR(fmt, args...) do {} while (0)
> >> +#endif
> >> +
> >> ++/* structure for mapping xattr name prefix data */ typedef struct
> >> ++xattr_prefix {
> >> ++	int index;
> >> ++	char *name;
> >> ++	size_t length;
> >> ++} xattr_prefix_t;
> >> ++
> >> ++xattr_prefix_t xattr_prefixes [] = {
> >> ++/* Only interested in security prefix. Can support others though.
> >> ++	{
> >> ++		.index = EXT2_XATTR_INDEX_USER,
> >> ++		.name = XATTR_USER_PREFIX,
> >> ++		.length = XATTR_USER_PREFIX_LEN,
> >> ++	},
> >> ++*/
> >> ++	{
> >> ++		.index = EXT2_XATTR_INDEX_SECURITY,
> >> ++		.name = XATTR_SECURITY_PREFIX,
> >> ++		.length = XATTR_SECURITY_PREFIX_LEN,
> >
> > Hi Philip,
> >
> > This cause build errors on some host OS when building e2fsprogs-native:
> >
> > | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-
> 1.42.9/debugfs/../misc/xattr.c:62:11: error: 'XATTR_SECURITY_PREFIX' undeclared here (not in a
> function)
> > |    .name = XATTR_SECURITY_PREFIX,
> > |            ^
> > | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-
> 1.42.9/debugfs/../misc/xattr.c:63:13: error: 'XATTR_SECURITY_PREFIX_LEN' undeclared here (not in a
> function)
> > |    .length = XATTR_SECURITY_PREFIX_LEN,
> > |              ^
> >
> > I did some investigate and found that your patch needs the header
> > linux/xattr.h, which is provided by linux-libc-headers, but for
> > -native package, there is no linux-libc-headers-native, so it search
> > the one from host OS, but the problem is, there is no XATTR_SECURITY_PREFIX definition in the
> linux/xattr.h on some host OS like: SUSE 11.x, centos 6, etc.
> >
> > I'm not sure if your patch is really needed by the e2fsprogs-native,
> > if not, I think we can make this patch only apply for target package.
> > If yes, you may need to make it avoid the dependency on host's header or you may have a better
> idea about this.
> 
> I took a look at this yesterday. Creating a bbappend for linux-libc-headers and adding 'native' to
> BBCLASSEXTEND seems like the most direct fix. Add a new dependency to the e2fsprogs package and I
> think the issue is resolved. I'll send patches to the list and if you're willing to test them I'd appreciate it.

I tested with your patch and it works fine, thanks!

Thanks,
Jackie

> 
> Thanks,
> Philip
> 
> >> ++	},
> >> ++	{ 0 },
> >> ++};
> >> +
> >> + /* Free remaining resources after all files have been processed. */
> >> +void @@ -16,6 +55,211 @@ xattr_cleanup ()
> >> + 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");  }
> >> +
> >> ++/* Get value for named xattr from file at path.
> >> ++ * Returns pointer to allocated block for value and length in length param.
> >> ++ * If no value, return NULL pointer and length of 0.
> >> ++ * On error return NULL pointer and length set to -1.
> >> ++ */
> >> ++static char*
> >> ++xattr_get_value (const char *path, const char *name, ssize_t
> >> ++*length) {
> >> ++	char *value = NULL;
> >> ++
> >> ++	*length = lgetxattr (path, name, NULL, 0);
> >> ++	if (*length == -1) {
> >> ++		com_err (__func__, errno, "lgetattr");
> >> ++		goto out;
> >> ++	}
> >> ++	if (*length == 0) {
> >> ++		fprintf (stderr, "xattr %s has value length 0\n", name);
> >> ++		goto out;
> >> ++	}
> >> ++	value = calloc (1, *length);
> >> ++	if (value == NULL) {
> >> ++		com_err (__func__, errno, "calloc");
> >> ++		goto out;
> >> ++	}
> >> ++	*length = lgetxattr (path, name, value, *length);
> >> ++	if (*length == -1) {
> >> ++		com_err (__func__, errno, "lgetattr");
> >> ++		goto value_err;
> >> ++	}
> >> ++out:
> >> ++	return value;
> >> ++
> >> ++value_err:
> >> ++	if (value)
> >> ++		free (value);
> >> ++	return NULL;
> >> ++}
> >> ++
> >> ++/* Get all attribute names for file at path. Return pointer to
> >> ++allocated memory
> >> ++ * block holding all names and the length through parameter size.
> >> ++ * If no xattrs: return NULL and set size to 0
> >> ++ * If error: return NULL and set size to -1  */ static char*
> >> ++xattr_get_names (const char *path, ssize_t *size) {
> >> ++	char *names = NULL;
> >> ++
> >> ++	*size = llistxattr (path, NULL, 0);
> >> ++	if (*size == -1) {
> >> ++		com_err (__func__, errno, "llistxattr");
> >> ++		goto out;
> >> ++	}
> >> ++	if (*size == 0) {
> >> ++		/* no xattrs */
> >> ++		goto out;
> >> ++	}
> >> ++	names = calloc (1, *size);
> >> ++	if (names == NULL) {
> >> ++		com_err (__func__, errno, "calloc");
> >> ++		goto out;
> >> ++	}
> >> ++	*size = llistxattr (path, names, *size);
> >> ++	if (*size == -1) {
> >> ++		com_err (__func__, errno, "llistxattr");
> >> ++		goto cleanup;
> >> ++	}
> >> ++	if (*size == 0) {
> >> ++		fprintf (stdout, "Conflicting data, no xattrs for file: %s\n", path);
> >> ++		goto cleanup;
> >> ++	}
> >> ++out:
> >> ++	return names;
> >> ++
> >> ++cleanup:
> >> ++	if (names)
> >> ++		free (names);
> >> ++	return NULL;
> >> ++}
> >> ++
> >> ++/* return pointer to next string in xattr name block, don't go
> >> ++beyond length  */ static inline char* next_name (char *name, size_t length) {
> >> ++	int i = 0;
> >> ++
> >> ++	for (i = 0; i < length; ++i)
> >> ++		if (name [i] == '\0') {
> >> ++			++i;
> >> ++			break;
> >> ++		}
> >> ++
> >> ++	return name + i;
> >> ++}
> >> ++
> >> ++/* Find entry in xattr_table with matching prefix. */ static const
> >> ++xattr_prefix_t* xattr_find_prefix (char *name, size_t length) {
> >> ++	int i = 0;
> >> ++
> >> ++	XATTR_STDERR ("find_attr_data: searching for prefix from xattr name: %s\n", name);
> >> ++	for (i = 0; xattr_prefixes[i].index != 0; ++i) {
> >> ++		if (!strncmp (name, xattr_prefixes[i].name, MIN (length, xattr_prefixes[i].length))) {
> >> ++			XATTR_STDERR ("found match in table with index: %d\n",
> >> xattr_prefixes[i].index);
> >> ++			return &xattr_prefixes[i];
> >> ++		}
> >> ++	}
> >> ++	return NULL;
> >> ++}
> >> ++
> >> ++/* Query file at path for attributes. build up structure the file
> >> ++system
> >> ++ * expects of an extended attribute disk block (header parameter).
> >> ++ *
> >> ++ * The main loop walks through the xattr names one at a time. It
> >> ++gets the value
> >> ++ * for each named xattr and copies the data into the xattr block
> >> ++pointed to by
> >> ++ * the header parameter. To do this the loop also tracks the
> >> ++location of the
> >> ++ * associated entry and value. Values start at the end of the
> >> ++buffer and grow
> >> ++ * back towards the header while the entries start immediately
> >> ++after the header
> >> ++ * and grow towards the end of the block.
> >> ++ *
> >> ++ * See the comment block at the beginning of the xattr.c file in
> >> ++the
> >> ++ext2 file
> >> ++ * system code in the kernel: fs/ext2/xattr.c
> >> ++ * Assume the xattr block pointed to by header parameter is initialized to 0s.
> >> ++ */
> >> ++static int
> >> ++xattr_build_block (const char *path,
> >> ++		struct ext2_ext_attr_header **header,
> >> ++		size_t header_length)
> >> ++{
> >> ++	struct ext2_ext_attr_entry *entry = NULL;
> >> ++	char *names = NULL, *value = NULL, *name_curr = NULL;
> >> ++	ssize_t names_length = 0, value_length = 0;
> >> ++	size_t name_length = 0, value_index = 0, len_rem = 0;
> >> ++	const xattr_prefix_t *prefix = NULL;
> >> ++	int ret = 0;
> >> ++
> >> ++	XATTR_STDERR ("xattr_build_block for file: %s\n", path);
> >> ++	*header = NULL;
> >> ++	names = xattr_get_names (path, &names_length);
> >> ++	if (names == NULL) {
> >> ++		// no xattrs for file @ path or error
> >> ++		ret = names_length;
> >> ++		goto out;
> >> ++	}
> >> ++	*header = calloc (1, header_length);
> >> ++	if (*header == NULL) {
> >> ++		com_err (__func__, errno, "calloc");
> >> ++		goto out;
> >> ++	}
> >> ++	(*header)->h_magic = EXT2_EXT_ATTR_MAGIC;
> >> ++	(*header)->h_blocks = 1;
> >> ++	/* Values start at end of buffer. NOTE: It must be moved before a value can
> >> ++	 * be stored.
> >> ++	 */
> >> ++	value_index = header_length;
> >> ++	for (name_curr = names, entry = FIRST_ENTRY(*header), len_rem = names_length;
> >> ++		name_curr < names + names_length;
> >> ++		len_rem = names_length - (name_curr - names),
> >> ++			name_curr = next_name (name_curr, len_rem),
> >> ++			entry = EXT2_EXT_ATTR_NEXT(entry))
> >> ++	{
> >> ++		XATTR_STDERR ("xattr_build_block: processing xattr with name %s\n", name_curr);
> >> ++		name_length = strnlen (name_curr, len_rem);
> >> ++		prefix = xattr_find_prefix (name_curr, name_length);
> >> ++		if (!prefix) {
> >> ++			fprintf (stderr, "Don't currently handle xattr: %s\n", name_curr);
> >> ++			continue;
> >> ++		}
> >> ++		value = xattr_get_value (path, name_curr, &value_length);
> >> ++		if (value == NULL) {
> >> ++			// no xattr value or error
> >> ++			fprintf (stderr, "failed to get value, skipping\n");
> >> ++			goto next;
> >> ++		}
> >> ++		/* setup offsets and lengths for name and value */
> >> ++		entry->e_name_len = name_length - prefix->length;
> >> ++		entry->e_name_index = prefix->index;
> >> ++		/* Can't know these till we know the length of the value. */
> >> ++		entry->e_value_offs = value_index -= EXT2_EXT_ATTR_SIZE(value_length);
> >> ++		entry->e_value_size = value_length;
> >> ++		/* Check to be sure entry name and value don't overlap before copy. */
> >> ++		if (EXT2_EXT_ATTR_NAME(entry) + entry->e_name_len > VALUE(*header, entry)) {
> >> ++			fprintf (stderr, "xattr entry name and value overlap! Too much xattr data.");
> >> ++			ret = -1;
> >> ++			goto out;
> >> ++		}
> >> ++		/* copy name and value data then calculate the hash */
> >> ++		memcpy (EXT2_EXT_ATTR_NAME(entry),
> >> ++			name_curr + prefix->length,
> >> ++			entry->e_name_len);
> >> ++		memcpy (VALUE(*header, entry), value, entry->e_value_size);
> >> ++		entry->e_hash = ext2fs_ext_attr_hash_entry (entry, VALUE(*header,
> >> ++entry));
> >> ++next:
> >> ++		if (value)
> >> ++			free (value);
> >> ++	}
> >> ++	XATTR_STDERR ("xattr_build_block: done building xattr buffer\n");
> >> ++out:
> >> ++	if (names)
> >> ++		free (names);
> >> ++	return ret;
> >> ++}
> >> ++
> >> + /* This is the entry point to the xattr module. This function
> >> +copies the xattrs
> >> +  * from the file at 'path' to the file system object at 'ino'.
> >> +  *
> >> +@@ -28,7 +272,56 @@ errcode_t
> >> + set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)  {
> >> + 	errcode_t ret = 0;
> >> ++	char *buf = NULL;
> >> ++	struct ext2_inode inode = { 0 };
> >> ++	blk_t block = 0;
> >> ++	struct ext2_ext_attr_header *header = NULL;
> >> ++	uint32_t newcount = 0;
> >> +
> >> + 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path,
> >> + ino);
> >> ++	/* Populate the xattr block for the file at path */
> >> ++	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
> >> ++		goto out;
> >> ++	}
> >> ++	if (header == NULL) {
> >> ++		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
> >> ++		goto out;
> >> ++	}
> >> ++	if (ret = ext2fs_read_inode (fs, ino, &inode)) {
> >> ++		com_err(__func__, ret, "ext2fs_read_inode");
> >> ++		goto out;
> >> ++	}
> >> ++	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
> >> ++		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
> >> ++		goto out;
> >> ++	}
> >> ++	ext2fs_mark_block_bitmap2 (fs->block_map, block);
> >> ++	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
> >> ++	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
> >> ++		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
> >> ++		goto out;
> >> ++	}
> >> ++	/* point inode for current file to xattr block, update block count and
> >> ++	 * write inode to disk
> >> ++	 */
> >> ++	inode.i_file_acl = block;
> >> ++	if (ret = ext2fs_adjust_ea_refcount2(fs,
> >> ++					block,
> >> ++					(char*)header,
> >> ++					1,
> >> ++					&newcount))
> >> ++	{
> >> ++		com_err(__func__, ret, "ext2fs_adjust_ea_refcount");
> >> ++		goto out;
> >> ++	}
> >> ++	if (ret = ext2fs_iblk_add_blocks (fs, &inode, 1)) {
> >> ++		com_err(__func__, ret, "ext2fs_iblk_add_blocks failed");
> >> ++		goto out;
> >> ++	}
> >> ++	if (ret = ext2fs_write_inode (fs, ino, &inode))
> >> ++		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
> >> ++out:
> >> ++	if (header)
> >> ++		free (header);
> >> + 	return ret;
> >> + }
> >> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >> b/recipes- devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >> index a4576b1..edc94d8 100644
> >> --- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >> +++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >> @@ -4,4 +4,5 @@ SRC_URI += " \
> >>      file://misc-xattr-add-xattr-module-stub.patch \
> >>      file://mke2fs.c-create_inode.c-copy-xattrs.patch \
> >>      file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
> >> +    file://misc-xattr-create-xattr-block.patch \
> >>  "
> >> --
> >> 2.1.4
> >>
> >> --
> >> _______________________________________________
> >> yocto mailing list
> >> yocto@yoctoproject.org
> >> https://lists.yoctoproject.org/listinfo/yocto

Re: [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.

[Philip Tricca]

Hey Joe,

Any thoughts on this? It looks like since my patch set went into master
the meta-selinux build is broken for a number of build hosts (anything
w/o linux-libc-headers). I think Jackie and I have worked out a solution
but it's waiting for a maintainer to merge (or reject).

I hate to think I'm breaking builds for some folks but we've made as
much progress as we can w/o a maintainer getting involved. Any help you
can provide would be appreciated.

Thanks,
Philip

On 08/23/2015 10:27 PM, Huang, Jie (Jackie) wrote:
> 
> 
>> -----Original Message-----
>> From: Philip Tricca [mailto:flihp@twobit.us]
>> Sent: Monday, August 24, 2015 6:25 AM
>> To: Huang, Jie (Jackie)
>> Cc: yocto@yoctoproject.org
>> Subject: Re: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
>>
>> Hello Jackie,
>>
>> On 08/20/2015 11:25 PM, Huang, Jie (Jackie) wrote:
>>>> -----Original Message-----
>>>> From: yocto-bounces@yoctoproject.org
>>>> [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Philip Tricca
>>>> Sent: Thursday, June 18, 2015 6:31 AM
>>>> To: yocto@yoctoproject.org
>>>> Subject: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
>>>>
>>>> Signed-off-by: Philip Tricca <flihp@twobit.us>
>>>> ---
>>>>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
>>>>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
>>>>  2 files changed, 342 insertions(+)
>>>>  create mode 100644
>>>> recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.pa
>>>> tch
>>>>
>>>> diff --git
>>>> a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.
>>>> patch b/recipes-
>>>> devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>>>> new file mode 100644
>>>> index 0000000..5955b44
>>>> --- /dev/null
>>>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-bl
>>>> +++ ock
>>>> +++ .patch
>>>> @@ -0,0 +1,341 @@
>>>> +To build the xattr disk block we process the output from listxattr
>>>> +and lgetxattr from the source file system object. This data is
>>>> +formated in a disk block according to the format specified in the kernel ext2 file system driver.
>>>> +See the comment block at the beginning of fs/ext2/xattr.c for details.
>>>> +
>>>> +Currently we only process attributes with the 'security.' prefix as
>>>> +our use case is SELinux labels and IMA. Additional prefixes can
>>>> +likely be supported with minimal effort but none have been tested.
>>>> +
>>>> +Once the xattr block has been created it is written to disk. The
>>>> +xattr block is associated with the appropriate file system object
>>>> +through the i_file_acl inode member and the inode is updated on disk.
>>>> +
>>>> +Signed-off-by: Philip Tricca <flihp@twobit.us>
>>>> +
>>>> +Index: e2fsprogs-1.42.9/misc/xattr.c
>>>> +===================================================================
>>>> +--- e2fsprogs-1.42.9.orig/misc/xattr.c
>>>> ++++ e2fsprogs-1.42.9/misc/xattr.c
>>>> +@@ -1,6 +1,23 @@
>>>> + #include "xattr.h"
>>>> +
>>>> ++#include <attr/xattr.h>
>>>> ++#include <ctype.h>
>>>> ++#include <errno.h>
>>>> ++#include <ext2fs/ext2_ext_attr.h>
>>>> ++#include <linux/xattr.h>
>>>> ++#include <stdint.h>
>>>> + #include <stdio.h>
>>>> ++#include <stdlib.h>
>>>> ++#include <string.h>
>>>> ++#include <sys/stat.h>
>>>> ++#include <sys/types.h>
>>>> ++#include <unistd.h>
>>>> ++
>>>> ++#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y)) #define HEADER(ptr)
>>>> ++((struct ext2_ext_attr_header *)(ptr)) #define ENTRY(ptr)  ((struct
>>>> ++ext2_ext_attr_entry *)(ptr)) #define FIRST_ENTRY(ptr)
>>>> ++ENTRY(HEADER(ptr) + 1) #define VALUE(hdr, ent) (((char*)hdr) +
>>>> ++(ent->e_value_offs))
>>>> +
>>>> + #ifdef XATTR_DEBUG
>>>> + #define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args) @@
>>>> +-8,6 +25,28 @@  #define XATTR_STDERR(fmt, args...) do {} while (0)
>>>> +#endif
>>>> +
>>>> ++/* structure for mapping xattr name prefix data */ typedef struct
>>>> ++xattr_prefix {
>>>> ++	int index;
>>>> ++	char *name;
>>>> ++	size_t length;
>>>> ++} xattr_prefix_t;
>>>> ++
>>>> ++xattr_prefix_t xattr_prefixes [] = {
>>>> ++/* Only interested in security prefix. Can support others though.
>>>> ++	{
>>>> ++		.index = EXT2_XATTR_INDEX_USER,
>>>> ++		.name = XATTR_USER_PREFIX,
>>>> ++		.length = XATTR_USER_PREFIX_LEN,
>>>> ++	},
>>>> ++*/
>>>> ++	{
>>>> ++		.index = EXT2_XATTR_INDEX_SECURITY,
>>>> ++		.name = XATTR_SECURITY_PREFIX,
>>>> ++		.length = XATTR_SECURITY_PREFIX_LEN,
>>>
>>> Hi Philip,
>>>
>>> This cause build errors on some host OS when building e2fsprogs-native:
>>>
>>> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-
>> 1.42.9/debugfs/../misc/xattr.c:62:11: error: 'XATTR_SECURITY_PREFIX' undeclared here (not in a
>> function)
>>> |    .name = XATTR_SECURITY_PREFIX,
>>> |            ^
>>> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-
>> 1.42.9/debugfs/../misc/xattr.c:63:13: error: 'XATTR_SECURITY_PREFIX_LEN' undeclared here (not in a
>> function)
>>> |    .length = XATTR_SECURITY_PREFIX_LEN,
>>> |              ^
>>>
>>> I did some investigate and found that your patch needs the header
>>> linux/xattr.h, which is provided by linux-libc-headers, but for
>>> -native package, there is no linux-libc-headers-native, so it search
>>> the one from host OS, but the problem is, there is no XATTR_SECURITY_PREFIX definition in the
>> linux/xattr.h on some host OS like: SUSE 11.x, centos 6, etc.
>>>
>>> I'm not sure if your patch is really needed by the e2fsprogs-native,
>>> if not, I think we can make this patch only apply for target package.
>>> If yes, you may need to make it avoid the dependency on host's header or you may have a better
>> idea about this.
>>
>> I took a look at this yesterday. Creating a bbappend for linux-libc-headers and adding 'native' to
>> BBCLASSEXTEND seems like the most direct fix. Add a new dependency to the e2fsprogs package and I
>> think the issue is resolved. I'll send patches to the list and if you're willing to test them I'd appreciate it.
> 
> I tested with your patch and it works fine, thanks!
> 
> Thanks,
> Jackie
> 
>>
>> Thanks,
>> Philip
>>
>>>> ++	},
>>>> ++	{ 0 },
>>>> ++};
>>>> +
>>>> + /* Free remaining resources after all files have been processed. */
>>>> +void @@ -16,6 +55,211 @@ xattr_cleanup ()
>>>> + 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");  }
>>>> +
>>>> ++/* Get value for named xattr from file at path.
>>>> ++ * Returns pointer to allocated block for value and length in length param.
>>>> ++ * If no value, return NULL pointer and length of 0.
>>>> ++ * On error return NULL pointer and length set to -1.
>>>> ++ */
>>>> ++static char*
>>>> ++xattr_get_value (const char *path, const char *name, ssize_t
>>>> ++*length) {
>>>> ++	char *value = NULL;
>>>> ++
>>>> ++	*length = lgetxattr (path, name, NULL, 0);
>>>> ++	if (*length == -1) {
>>>> ++		com_err (__func__, errno, "lgetattr");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	if (*length == 0) {
>>>> ++		fprintf (stderr, "xattr %s has value length 0\n", name);
>>>> ++		goto out;
>>>> ++	}
>>>> ++	value = calloc (1, *length);
>>>> ++	if (value == NULL) {
>>>> ++		com_err (__func__, errno, "calloc");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	*length = lgetxattr (path, name, value, *length);
>>>> ++	if (*length == -1) {
>>>> ++		com_err (__func__, errno, "lgetattr");
>>>> ++		goto value_err;
>>>> ++	}
>>>> ++out:
>>>> ++	return value;
>>>> ++
>>>> ++value_err:
>>>> ++	if (value)
>>>> ++		free (value);
>>>> ++	return NULL;
>>>> ++}
>>>> ++
>>>> ++/* Get all attribute names for file at path. Return pointer to
>>>> ++allocated memory
>>>> ++ * block holding all names and the length through parameter size.
>>>> ++ * If no xattrs: return NULL and set size to 0
>>>> ++ * If error: return NULL and set size to -1  */ static char*
>>>> ++xattr_get_names (const char *path, ssize_t *size) {
>>>> ++	char *names = NULL;
>>>> ++
>>>> ++	*size = llistxattr (path, NULL, 0);
>>>> ++	if (*size == -1) {
>>>> ++		com_err (__func__, errno, "llistxattr");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	if (*size == 0) {
>>>> ++		/* no xattrs */
>>>> ++		goto out;
>>>> ++	}
>>>> ++	names = calloc (1, *size);
>>>> ++	if (names == NULL) {
>>>> ++		com_err (__func__, errno, "calloc");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	*size = llistxattr (path, names, *size);
>>>> ++	if (*size == -1) {
>>>> ++		com_err (__func__, errno, "llistxattr");
>>>> ++		goto cleanup;
>>>> ++	}
>>>> ++	if (*size == 0) {
>>>> ++		fprintf (stdout, "Conflicting data, no xattrs for file: %s\n", path);
>>>> ++		goto cleanup;
>>>> ++	}
>>>> ++out:
>>>> ++	return names;
>>>> ++
>>>> ++cleanup:
>>>> ++	if (names)
>>>> ++		free (names);
>>>> ++	return NULL;
>>>> ++}
>>>> ++
>>>> ++/* return pointer to next string in xattr name block, don't go
>>>> ++beyond length  */ static inline char* next_name (char *name, size_t length) {
>>>> ++	int i = 0;
>>>> ++
>>>> ++	for (i = 0; i < length; ++i)
>>>> ++		if (name [i] == '\0') {
>>>> ++			++i;
>>>> ++			break;
>>>> ++		}
>>>> ++
>>>> ++	return name + i;
>>>> ++}
>>>> ++
>>>> ++/* Find entry in xattr_table with matching prefix. */ static const
>>>> ++xattr_prefix_t* xattr_find_prefix (char *name, size_t length) {
>>>> ++	int i = 0;
>>>> ++
>>>> ++	XATTR_STDERR ("find_attr_data: searching for prefix from xattr name: %s\n", name);
>>>> ++	for (i = 0; xattr_prefixes[i].index != 0; ++i) {
>>>> ++		if (!strncmp (name, xattr_prefixes[i].name, MIN (length, xattr_prefixes[i].length))) {
>>>> ++			XATTR_STDERR ("found match in table with index: %d\n",
>>>> xattr_prefixes[i].index);
>>>> ++			return &xattr_prefixes[i];
>>>> ++		}
>>>> ++	}
>>>> ++	return NULL;
>>>> ++}
>>>> ++
>>>> ++/* Query file at path for attributes. build up structure the file
>>>> ++system
>>>> ++ * expects of an extended attribute disk block (header parameter).
>>>> ++ *
>>>> ++ * The main loop walks through the xattr names one at a time. It
>>>> ++gets the value
>>>> ++ * for each named xattr and copies the data into the xattr block
>>>> ++pointed to by
>>>> ++ * the header parameter. To do this the loop also tracks the
>>>> ++location of the
>>>> ++ * associated entry and value. Values start at the end of the
>>>> ++buffer and grow
>>>> ++ * back towards the header while the entries start immediately
>>>> ++after the header
>>>> ++ * and grow towards the end of the block.
>>>> ++ *
>>>> ++ * See the comment block at the beginning of the xattr.c file in
>>>> ++the
>>>> ++ext2 file
>>>> ++ * system code in the kernel: fs/ext2/xattr.c
>>>> ++ * Assume the xattr block pointed to by header parameter is initialized to 0s.
>>>> ++ */
>>>> ++static int
>>>> ++xattr_build_block (const char *path,
>>>> ++		struct ext2_ext_attr_header **header,
>>>> ++		size_t header_length)
>>>> ++{
>>>> ++	struct ext2_ext_attr_entry *entry = NULL;
>>>> ++	char *names = NULL, *value = NULL, *name_curr = NULL;
>>>> ++	ssize_t names_length = 0, value_length = 0;
>>>> ++	size_t name_length = 0, value_index = 0, len_rem = 0;
>>>> ++	const xattr_prefix_t *prefix = NULL;
>>>> ++	int ret = 0;
>>>> ++
>>>> ++	XATTR_STDERR ("xattr_build_block for file: %s\n", path);
>>>> ++	*header = NULL;
>>>> ++	names = xattr_get_names (path, &names_length);
>>>> ++	if (names == NULL) {
>>>> ++		// no xattrs for file @ path or error
>>>> ++		ret = names_length;
>>>> ++		goto out;
>>>> ++	}
>>>> ++	*header = calloc (1, header_length);
>>>> ++	if (*header == NULL) {
>>>> ++		com_err (__func__, errno, "calloc");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	(*header)->h_magic = EXT2_EXT_ATTR_MAGIC;
>>>> ++	(*header)->h_blocks = 1;
>>>> ++	/* Values start at end of buffer. NOTE: It must be moved before a value can
>>>> ++	 * be stored.
>>>> ++	 */
>>>> ++	value_index = header_length;
>>>> ++	for (name_curr = names, entry = FIRST_ENTRY(*header), len_rem = names_length;
>>>> ++		name_curr < names + names_length;
>>>> ++		len_rem = names_length - (name_curr - names),
>>>> ++			name_curr = next_name (name_curr, len_rem),
>>>> ++			entry = EXT2_EXT_ATTR_NEXT(entry))
>>>> ++	{
>>>> ++		XATTR_STDERR ("xattr_build_block: processing xattr with name %s\n", name_curr);
>>>> ++		name_length = strnlen (name_curr, len_rem);
>>>> ++		prefix = xattr_find_prefix (name_curr, name_length);
>>>> ++		if (!prefix) {
>>>> ++			fprintf (stderr, "Don't currently handle xattr: %s\n", name_curr);
>>>> ++			continue;
>>>> ++		}
>>>> ++		value = xattr_get_value (path, name_curr, &value_length);
>>>> ++		if (value == NULL) {
>>>> ++			// no xattr value or error
>>>> ++			fprintf (stderr, "failed to get value, skipping\n");
>>>> ++			goto next;
>>>> ++		}
>>>> ++		/* setup offsets and lengths for name and value */
>>>> ++		entry->e_name_len = name_length - prefix->length;
>>>> ++		entry->e_name_index = prefix->index;
>>>> ++		/* Can't know these till we know the length of the value. */
>>>> ++		entry->e_value_offs = value_index -= EXT2_EXT_ATTR_SIZE(value_length);
>>>> ++		entry->e_value_size = value_length;
>>>> ++		/* Check to be sure entry name and value don't overlap before copy. */
>>>> ++		if (EXT2_EXT_ATTR_NAME(entry) + entry->e_name_len > VALUE(*header, entry)) {
>>>> ++			fprintf (stderr, "xattr entry name and value overlap! Too much xattr data.");
>>>> ++			ret = -1;
>>>> ++			goto out;
>>>> ++		}
>>>> ++		/* copy name and value data then calculate the hash */
>>>> ++		memcpy (EXT2_EXT_ATTR_NAME(entry),
>>>> ++			name_curr + prefix->length,
>>>> ++			entry->e_name_len);
>>>> ++		memcpy (VALUE(*header, entry), value, entry->e_value_size);
>>>> ++		entry->e_hash = ext2fs_ext_attr_hash_entry (entry, VALUE(*header,
>>>> ++entry));
>>>> ++next:
>>>> ++		if (value)
>>>> ++			free (value);
>>>> ++	}
>>>> ++	XATTR_STDERR ("xattr_build_block: done building xattr buffer\n");
>>>> ++out:
>>>> ++	if (names)
>>>> ++		free (names);
>>>> ++	return ret;
>>>> ++}
>>>> ++
>>>> + /* This is the entry point to the xattr module. This function
>>>> +copies the xattrs
>>>> +  * from the file at 'path' to the file system object at 'ino'.
>>>> +  *
>>>> +@@ -28,7 +272,56 @@ errcode_t
>>>> + set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)  {
>>>> + 	errcode_t ret = 0;
>>>> ++	char *buf = NULL;
>>>> ++	struct ext2_inode inode = { 0 };
>>>> ++	blk_t block = 0;
>>>> ++	struct ext2_ext_attr_header *header = NULL;
>>>> ++	uint32_t newcount = 0;
>>>> +
>>>> + 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path,
>>>> + ino);
>>>> ++	/* Populate the xattr block for the file at path */
>>>> ++	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
>>>> ++		goto out;
>>>> ++	}
>>>> ++	if (header == NULL) {
>>>> ++		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
>>>> ++		goto out;
>>>> ++	}
>>>> ++	if (ret = ext2fs_read_inode (fs, ino, &inode)) {
>>>> ++		com_err(__func__, ret, "ext2fs_read_inode");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
>>>> ++		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
>>>> ++		goto out;
>>>> ++	}
>>>> ++	ext2fs_mark_block_bitmap2 (fs->block_map, block);
>>>> ++	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
>>>> ++	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
>>>> ++		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
>>>> ++		goto out;
>>>> ++	}
>>>> ++	/* point inode for current file to xattr block, update block count and
>>>> ++	 * write inode to disk
>>>> ++	 */
>>>> ++	inode.i_file_acl = block;
>>>> ++	if (ret = ext2fs_adjust_ea_refcount2(fs,
>>>> ++					block,
>>>> ++					(char*)header,
>>>> ++					1,
>>>> ++					&newcount))
>>>> ++	{
>>>> ++		com_err(__func__, ret, "ext2fs_adjust_ea_refcount");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	if (ret = ext2fs_iblk_add_blocks (fs, &inode, 1)) {
>>>> ++		com_err(__func__, ret, "ext2fs_iblk_add_blocks failed");
>>>> ++		goto out;
>>>> ++	}
>>>> ++	if (ret = ext2fs_write_inode (fs, ino, &inode))
>>>> ++		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
>>>> ++out:
>>>> ++	if (header)
>>>> ++		free (header);
>>>> + 	return ret;
>>>> + }
>>>> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>>>> b/recipes- devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>>>> index a4576b1..edc94d8 100644
>>>> --- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>>>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>>>> @@ -4,4 +4,5 @@ SRC_URI += " \
>>>>      file://misc-xattr-add-xattr-module-stub.patch \
>>>>      file://mke2fs.c-create_inode.c-copy-xattrs.patch \
>>>>      file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
>>>> +    file://misc-xattr-create-xattr-block.patch \
>>>>  "
>>>> --
>>>> 2.1.4
>>>>
>>>> --
>>>> _______________________________________________
>>>> yocto mailing list
>>>> yocto@yoctoproject.org
>>>> https://lists.yoctoproject.org/listinfo/yocto
> 

Re: [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.

[Joe MacDonald]

[-- Attachment #1: Type: text/plain, Size: 19125 bytes --]

Hi Philip,

[Re: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.] On 15.09.05 (Sat 10:59) Philip Tricca wrote:

> Hey Joe,
> 
> Any thoughts on this? It looks like since my patch set went into master
> the meta-selinux build is broken for a number of build hosts (anything
> w/o linux-libc-headers). I think Jackie and I have worked out a solution
> but it's waiting for a maintainer to merge (or reject).
> 
> I hate to think I'm breaking builds for some folks but we've made as
> much progress as we can w/o a maintainer getting involved. Any help you
> can provide would be appreciated.

Sure thing, I had been watching for this but it got lost in my inbox
somehow, so I appreciate the follow-up on it.  I'm hoping to have the
meta-selinux patchwork in place very soon, so we're less dependent on
proper care and feeding of our respective mailboxes.  In the meantime,
I'm working through the meta-selinux queue right now (at least
everything I can put my hands on).  I expect I'll be through it by the
end of the day, so if something's not go some kind of response by
tomorrow, it's fair to assume it's been lost and a re-send would be
helpful.  Of course, if the plan changes between now and sundown, I'll
follow up again with a new ETA.  :-)

-J.

> 
> Thanks,
> Philip
> 
> On 08/23/2015 10:27 PM, Huang, Jie (Jackie) wrote:
> > 
> > 
> >> -----Original Message-----
> >> From: Philip Tricca [mailto:flihp@twobit.us]
> >> Sent: Monday, August 24, 2015 6:25 AM
> >> To: Huang, Jie (Jackie)
> >> Cc: yocto@yoctoproject.org
> >> Subject: Re: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
> >>
> >> Hello Jackie,
> >>
> >> On 08/20/2015 11:25 PM, Huang, Jie (Jackie) wrote:
> >>>> -----Original Message-----
> >>>> From: yocto-bounces@yoctoproject.org
> >>>> [mailto:yocto-bounces@yoctoproject.org] On Behalf Of Philip Tricca
> >>>> Sent: Thursday, June 18, 2015 6:31 AM
> >>>> To: yocto@yoctoproject.org
> >>>> Subject: [yocto] [meta-selinux][PATCHv2 6/8] e2fsprogs: Copy xattr block from source file.
> >>>>
> >>>> Signed-off-by: Philip Tricca <flihp@twobit.us>
> >>>> ---
> >>>>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
> >>>>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
> >>>>  2 files changed, 342 insertions(+)
> >>>>  create mode 100644
> >>>> recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.pa
> >>>> tch
> >>>>
> >>>> diff --git
> >>>> a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.
> >>>> patch b/recipes-
> >>>> devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
> >>>> new file mode 100644
> >>>> index 0000000..5955b44
> >>>> --- /dev/null
> >>>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-bl
> >>>> +++ ock
> >>>> +++ .patch
> >>>> @@ -0,0 +1,341 @@
> >>>> +To build the xattr disk block we process the output from listxattr
> >>>> +and lgetxattr from the source file system object. This data is
> >>>> +formated in a disk block according to the format specified in the kernel ext2 file system driver.
> >>>> +See the comment block at the beginning of fs/ext2/xattr.c for details.
> >>>> +
> >>>> +Currently we only process attributes with the 'security.' prefix as
> >>>> +our use case is SELinux labels and IMA. Additional prefixes can
> >>>> +likely be supported with minimal effort but none have been tested.
> >>>> +
> >>>> +Once the xattr block has been created it is written to disk. The
> >>>> +xattr block is associated with the appropriate file system object
> >>>> +through the i_file_acl inode member and the inode is updated on disk.
> >>>> +
> >>>> +Signed-off-by: Philip Tricca <flihp@twobit.us>
> >>>> +
> >>>> +Index: e2fsprogs-1.42.9/misc/xattr.c
> >>>> +===================================================================
> >>>> +--- e2fsprogs-1.42.9.orig/misc/xattr.c
> >>>> ++++ e2fsprogs-1.42.9/misc/xattr.c
> >>>> +@@ -1,6 +1,23 @@
> >>>> + #include "xattr.h"
> >>>> +
> >>>> ++#include <attr/xattr.h>
> >>>> ++#include <ctype.h>
> >>>> ++#include <errno.h>
> >>>> ++#include <ext2fs/ext2_ext_attr.h>
> >>>> ++#include <linux/xattr.h>
> >>>> ++#include <stdint.h>
> >>>> + #include <stdio.h>
> >>>> ++#include <stdlib.h>
> >>>> ++#include <string.h>
> >>>> ++#include <sys/stat.h>
> >>>> ++#include <sys/types.h>
> >>>> ++#include <unistd.h>
> >>>> ++
> >>>> ++#define MIN(X, Y) (((X) < (Y)) ? (X) : (Y)) #define HEADER(ptr)
> >>>> ++((struct ext2_ext_attr_header *)(ptr)) #define ENTRY(ptr)  ((struct
> >>>> ++ext2_ext_attr_entry *)(ptr)) #define FIRST_ENTRY(ptr)
> >>>> ++ENTRY(HEADER(ptr) + 1) #define VALUE(hdr, ent) (((char*)hdr) +
> >>>> ++(ent->e_value_offs))
> >>>> +
> >>>> + #ifdef XATTR_DEBUG
> >>>> + #define XATTR_STDERR(fmt, args...) fprintf (stderr, fmt, ##args) @@
> >>>> +-8,6 +25,28 @@  #define XATTR_STDERR(fmt, args...) do {} while (0)
> >>>> +#endif
> >>>> +
> >>>> ++/* structure for mapping xattr name prefix data */ typedef struct
> >>>> ++xattr_prefix {
> >>>> ++	int index;
> >>>> ++	char *name;
> >>>> ++	size_t length;
> >>>> ++} xattr_prefix_t;
> >>>> ++
> >>>> ++xattr_prefix_t xattr_prefixes [] = {
> >>>> ++/* Only interested in security prefix. Can support others though.
> >>>> ++	{
> >>>> ++		.index = EXT2_XATTR_INDEX_USER,
> >>>> ++		.name = XATTR_USER_PREFIX,
> >>>> ++		.length = XATTR_USER_PREFIX_LEN,
> >>>> ++	},
> >>>> ++*/
> >>>> ++	{
> >>>> ++		.index = EXT2_XATTR_INDEX_SECURITY,
> >>>> ++		.name = XATTR_SECURITY_PREFIX,
> >>>> ++		.length = XATTR_SECURITY_PREFIX_LEN,
> >>>
> >>> Hi Philip,
> >>>
> >>> This cause build errors on some host OS when building e2fsprogs-native:
> >>>
> >>> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-
> >> 1.42.9/debugfs/../misc/xattr.c:62:11: error: 'XATTR_SECURITY_PREFIX' undeclared here (not in a
> >> function)
> >>> |    .name = XATTR_SECURITY_PREFIX,
> >>> |            ^
> >>> | /build/yp/y_x64_150821/tmp/work/x86_64-linux/e2fsprogs-native/1.42.9-r0/e2fsprogs-
> >> 1.42.9/debugfs/../misc/xattr.c:63:13: error: 'XATTR_SECURITY_PREFIX_LEN' undeclared here (not in a
> >> function)
> >>> |    .length = XATTR_SECURITY_PREFIX_LEN,
> >>> |              ^
> >>>
> >>> I did some investigate and found that your patch needs the header
> >>> linux/xattr.h, which is provided by linux-libc-headers, but for
> >>> -native package, there is no linux-libc-headers-native, so it search
> >>> the one from host OS, but the problem is, there is no XATTR_SECURITY_PREFIX definition in the
> >> linux/xattr.h on some host OS like: SUSE 11.x, centos 6, etc.
> >>>
> >>> I'm not sure if your patch is really needed by the e2fsprogs-native,
> >>> if not, I think we can make this patch only apply for target package.
> >>> If yes, you may need to make it avoid the dependency on host's header or you may have a better
> >> idea about this.
> >>
> >> I took a look at this yesterday. Creating a bbappend for linux-libc-headers and adding 'native' to
> >> BBCLASSEXTEND seems like the most direct fix. Add a new dependency to the e2fsprogs package and I
> >> think the issue is resolved. I'll send patches to the list and if you're willing to test them I'd appreciate it.
> > 
> > I tested with your patch and it works fine, thanks!
> > 
> > Thanks,
> > Jackie
> > 
> >>
> >> Thanks,
> >> Philip
> >>
> >>>> ++	},
> >>>> ++	{ 0 },
> >>>> ++};
> >>>> +
> >>>> + /* Free remaining resources after all files have been processed. */
> >>>> +void @@ -16,6 +55,211 @@ xattr_cleanup ()
> >>>> + 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");  }
> >>>> +
> >>>> ++/* Get value for named xattr from file at path.
> >>>> ++ * Returns pointer to allocated block for value and length in length param.
> >>>> ++ * If no value, return NULL pointer and length of 0.
> >>>> ++ * On error return NULL pointer and length set to -1.
> >>>> ++ */
> >>>> ++static char*
> >>>> ++xattr_get_value (const char *path, const char *name, ssize_t
> >>>> ++*length) {
> >>>> ++	char *value = NULL;
> >>>> ++
> >>>> ++	*length = lgetxattr (path, name, NULL, 0);
> >>>> ++	if (*length == -1) {
> >>>> ++		com_err (__func__, errno, "lgetattr");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	if (*length == 0) {
> >>>> ++		fprintf (stderr, "xattr %s has value length 0\n", name);
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	value = calloc (1, *length);
> >>>> ++	if (value == NULL) {
> >>>> ++		com_err (__func__, errno, "calloc");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	*length = lgetxattr (path, name, value, *length);
> >>>> ++	if (*length == -1) {
> >>>> ++		com_err (__func__, errno, "lgetattr");
> >>>> ++		goto value_err;
> >>>> ++	}
> >>>> ++out:
> >>>> ++	return value;
> >>>> ++
> >>>> ++value_err:
> >>>> ++	if (value)
> >>>> ++		free (value);
> >>>> ++	return NULL;
> >>>> ++}
> >>>> ++
> >>>> ++/* Get all attribute names for file at path. Return pointer to
> >>>> ++allocated memory
> >>>> ++ * block holding all names and the length through parameter size.
> >>>> ++ * If no xattrs: return NULL and set size to 0
> >>>> ++ * If error: return NULL and set size to -1  */ static char*
> >>>> ++xattr_get_names (const char *path, ssize_t *size) {
> >>>> ++	char *names = NULL;
> >>>> ++
> >>>> ++	*size = llistxattr (path, NULL, 0);
> >>>> ++	if (*size == -1) {
> >>>> ++		com_err (__func__, errno, "llistxattr");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	if (*size == 0) {
> >>>> ++		/* no xattrs */
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	names = calloc (1, *size);
> >>>> ++	if (names == NULL) {
> >>>> ++		com_err (__func__, errno, "calloc");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	*size = llistxattr (path, names, *size);
> >>>> ++	if (*size == -1) {
> >>>> ++		com_err (__func__, errno, "llistxattr");
> >>>> ++		goto cleanup;
> >>>> ++	}
> >>>> ++	if (*size == 0) {
> >>>> ++		fprintf (stdout, "Conflicting data, no xattrs for file: %s\n", path);
> >>>> ++		goto cleanup;
> >>>> ++	}
> >>>> ++out:
> >>>> ++	return names;
> >>>> ++
> >>>> ++cleanup:
> >>>> ++	if (names)
> >>>> ++		free (names);
> >>>> ++	return NULL;
> >>>> ++}
> >>>> ++
> >>>> ++/* return pointer to next string in xattr name block, don't go
> >>>> ++beyond length  */ static inline char* next_name (char *name, size_t length) {
> >>>> ++	int i = 0;
> >>>> ++
> >>>> ++	for (i = 0; i < length; ++i)
> >>>> ++		if (name [i] == '\0') {
> >>>> ++			++i;
> >>>> ++			break;
> >>>> ++		}
> >>>> ++
> >>>> ++	return name + i;
> >>>> ++}
> >>>> ++
> >>>> ++/* Find entry in xattr_table with matching prefix. */ static const
> >>>> ++xattr_prefix_t* xattr_find_prefix (char *name, size_t length) {
> >>>> ++	int i = 0;
> >>>> ++
> >>>> ++	XATTR_STDERR ("find_attr_data: searching for prefix from xattr name: %s\n", name);
> >>>> ++	for (i = 0; xattr_prefixes[i].index != 0; ++i) {
> >>>> ++		if (!strncmp (name, xattr_prefixes[i].name, MIN (length, xattr_prefixes[i].length))) {
> >>>> ++			XATTR_STDERR ("found match in table with index: %d\n",
> >>>> xattr_prefixes[i].index);
> >>>> ++			return &xattr_prefixes[i];
> >>>> ++		}
> >>>> ++	}
> >>>> ++	return NULL;
> >>>> ++}
> >>>> ++
> >>>> ++/* Query file at path for attributes. build up structure the file
> >>>> ++system
> >>>> ++ * expects of an extended attribute disk block (header parameter).
> >>>> ++ *
> >>>> ++ * The main loop walks through the xattr names one at a time. It
> >>>> ++gets the value
> >>>> ++ * for each named xattr and copies the data into the xattr block
> >>>> ++pointed to by
> >>>> ++ * the header parameter. To do this the loop also tracks the
> >>>> ++location of the
> >>>> ++ * associated entry and value. Values start at the end of the
> >>>> ++buffer and grow
> >>>> ++ * back towards the header while the entries start immediately
> >>>> ++after the header
> >>>> ++ * and grow towards the end of the block.
> >>>> ++ *
> >>>> ++ * See the comment block at the beginning of the xattr.c file in
> >>>> ++the
> >>>> ++ext2 file
> >>>> ++ * system code in the kernel: fs/ext2/xattr.c
> >>>> ++ * Assume the xattr block pointed to by header parameter is initialized to 0s.
> >>>> ++ */
> >>>> ++static int
> >>>> ++xattr_build_block (const char *path,
> >>>> ++		struct ext2_ext_attr_header **header,
> >>>> ++		size_t header_length)
> >>>> ++{
> >>>> ++	struct ext2_ext_attr_entry *entry = NULL;
> >>>> ++	char *names = NULL, *value = NULL, *name_curr = NULL;
> >>>> ++	ssize_t names_length = 0, value_length = 0;
> >>>> ++	size_t name_length = 0, value_index = 0, len_rem = 0;
> >>>> ++	const xattr_prefix_t *prefix = NULL;
> >>>> ++	int ret = 0;
> >>>> ++
> >>>> ++	XATTR_STDERR ("xattr_build_block for file: %s\n", path);
> >>>> ++	*header = NULL;
> >>>> ++	names = xattr_get_names (path, &names_length);
> >>>> ++	if (names == NULL) {
> >>>> ++		// no xattrs for file @ path or error
> >>>> ++		ret = names_length;
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	*header = calloc (1, header_length);
> >>>> ++	if (*header == NULL) {
> >>>> ++		com_err (__func__, errno, "calloc");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	(*header)->h_magic = EXT2_EXT_ATTR_MAGIC;
> >>>> ++	(*header)->h_blocks = 1;
> >>>> ++	/* Values start at end of buffer. NOTE: It must be moved before a value can
> >>>> ++	 * be stored.
> >>>> ++	 */
> >>>> ++	value_index = header_length;
> >>>> ++	for (name_curr = names, entry = FIRST_ENTRY(*header), len_rem = names_length;
> >>>> ++		name_curr < names + names_length;
> >>>> ++		len_rem = names_length - (name_curr - names),
> >>>> ++			name_curr = next_name (name_curr, len_rem),
> >>>> ++			entry = EXT2_EXT_ATTR_NEXT(entry))
> >>>> ++	{
> >>>> ++		XATTR_STDERR ("xattr_build_block: processing xattr with name %s\n", name_curr);
> >>>> ++		name_length = strnlen (name_curr, len_rem);
> >>>> ++		prefix = xattr_find_prefix (name_curr, name_length);
> >>>> ++		if (!prefix) {
> >>>> ++			fprintf (stderr, "Don't currently handle xattr: %s\n", name_curr);
> >>>> ++			continue;
> >>>> ++		}
> >>>> ++		value = xattr_get_value (path, name_curr, &value_length);
> >>>> ++		if (value == NULL) {
> >>>> ++			// no xattr value or error
> >>>> ++			fprintf (stderr, "failed to get value, skipping\n");
> >>>> ++			goto next;
> >>>> ++		}
> >>>> ++		/* setup offsets and lengths for name and value */
> >>>> ++		entry->e_name_len = name_length - prefix->length;
> >>>> ++		entry->e_name_index = prefix->index;
> >>>> ++		/* Can't know these till we know the length of the value. */
> >>>> ++		entry->e_value_offs = value_index -= EXT2_EXT_ATTR_SIZE(value_length);
> >>>> ++		entry->e_value_size = value_length;
> >>>> ++		/* Check to be sure entry name and value don't overlap before copy. */
> >>>> ++		if (EXT2_EXT_ATTR_NAME(entry) + entry->e_name_len > VALUE(*header, entry)) {
> >>>> ++			fprintf (stderr, "xattr entry name and value overlap! Too much xattr data.");
> >>>> ++			ret = -1;
> >>>> ++			goto out;
> >>>> ++		}
> >>>> ++		/* copy name and value data then calculate the hash */
> >>>> ++		memcpy (EXT2_EXT_ATTR_NAME(entry),
> >>>> ++			name_curr + prefix->length,
> >>>> ++			entry->e_name_len);
> >>>> ++		memcpy (VALUE(*header, entry), value, entry->e_value_size);
> >>>> ++		entry->e_hash = ext2fs_ext_attr_hash_entry (entry, VALUE(*header,
> >>>> ++entry));
> >>>> ++next:
> >>>> ++		if (value)
> >>>> ++			free (value);
> >>>> ++	}
> >>>> ++	XATTR_STDERR ("xattr_build_block: done building xattr buffer\n");
> >>>> ++out:
> >>>> ++	if (names)
> >>>> ++		free (names);
> >>>> ++	return ret;
> >>>> ++}
> >>>> ++
> >>>> + /* This is the entry point to the xattr module. This function
> >>>> +copies the xattrs
> >>>> +  * from the file at 'path' to the file system object at 'ino'.
> >>>> +  *
> >>>> +@@ -28,7 +272,56 @@ errcode_t
> >>>> + set_inode_xattr (ext2_filsys fs, ext2_ino_t ino, const char *path)  {
> >>>> + 	errcode_t ret = 0;
> >>>> ++	char *buf = NULL;
> >>>> ++	struct ext2_inode inode = { 0 };
> >>>> ++	blk_t block = 0;
> >>>> ++	struct ext2_ext_attr_header *header = NULL;
> >>>> ++	uint32_t newcount = 0;
> >>>> +
> >>>> + 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path,
> >>>> + ino);
> >>>> ++	/* Populate the xattr block for the file at path */
> >>>> ++	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	if (header == NULL) {
> >>>> ++		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	if (ret = ext2fs_read_inode (fs, ino, &inode)) {
> >>>> ++		com_err(__func__, ret, "ext2fs_read_inode");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
> >>>> ++		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	ext2fs_mark_block_bitmap2 (fs->block_map, block);
> >>>> ++	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
> >>>> ++	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
> >>>> ++		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	/* point inode for current file to xattr block, update block count and
> >>>> ++	 * write inode to disk
> >>>> ++	 */
> >>>> ++	inode.i_file_acl = block;
> >>>> ++	if (ret = ext2fs_adjust_ea_refcount2(fs,
> >>>> ++					block,
> >>>> ++					(char*)header,
> >>>> ++					1,
> >>>> ++					&newcount))
> >>>> ++	{
> >>>> ++		com_err(__func__, ret, "ext2fs_adjust_ea_refcount");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	if (ret = ext2fs_iblk_add_blocks (fs, &inode, 1)) {
> >>>> ++		com_err(__func__, ret, "ext2fs_iblk_add_blocks failed");
> >>>> ++		goto out;
> >>>> ++	}
> >>>> ++	if (ret = ext2fs_write_inode (fs, ino, &inode))
> >>>> ++		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
> >>>> ++out:
> >>>> ++	if (header)
> >>>> ++		free (header);
> >>>> + 	return ret;
> >>>> + }
> >>>> diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >>>> b/recipes- devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >>>> index a4576b1..edc94d8 100644
> >>>> --- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >>>> +++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
> >>>> @@ -4,4 +4,5 @@ SRC_URI += " \
> >>>>      file://misc-xattr-add-xattr-module-stub.patch \
> >>>>      file://mke2fs.c-create_inode.c-copy-xattrs.patch \
> >>>>      file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
> >>>> +    file://misc-xattr-create-xattr-block.patch \
> >>>>  "
> >>>> --
> >>>> 2.1.4
> >>>>
> >>>> --
> >>>> _______________________________________________
> >>>> yocto mailing list
> >>>> yocto@yoctoproject.org
> >>>> https://lists.yoctoproject.org/listinfo/yocto
> > 
> 

-- 
-Joe MacDonald.
:wq

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

[meta-selinux][PATCHv2 7/8] e2fsprogs: Add stub functions for an xattr cache and struct to hold the header and block data.

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 .../misc-xattr-create-xattr-block-node.patch       | 175 +++++++++++++++++++++
 .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
 2 files changed, 176 insertions(+)
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch

diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch
new file mode 100644
index 0000000..f2e4582
--- /dev/null
+++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch
@@ -0,0 +1,175 @@
+Add two new functions to manage a cache of xattr blocks as well as a new struct
+to build a linked list of xattr blocks.
+
+xattr_add_block: Adds a block to the cache. This function is supplied with a
+pointer to the head of a linked list of xattr blocks and an xattr block to add.
+The purpose of this cache is to allow sharing of what would otherwise be
+duplicate xattr blocks and so duplicates are not allowed in this cache. If an
+identical block is already in the cache a pointer to this block will be
+returned.
+
+xattr_rm_block: Removes a block from the cache. This function is supplied with
+a pointer to the cache and a node that shall be removed from the cache. The
+cache is searched for the node and the node is removed if found.
+
+Both functions are integrated into the 'set_inode_xattr'. Here the logic is
+rearranged to cope with associating preexisting xattr blocks with inodes as
+well as creating new blocks when necessary.
+
+Signed-off-by: Philip Tricca <flihp@twobit.us>
+
+Index: e2fsprogs-1.42.9/misc/xattr.c
+===================================================================
+--- e2fsprogs-1.42.9.orig/misc/xattr.c
++++ e2fsprogs-1.42.9/misc/xattr.c
+@@ -25,6 +25,19 @@
+ #define XATTR_STDERR(fmt, args...) do {} while (0)
+ #endif
+ 
++/* nodes for simple linked list to track xattr blocks, calling it a cache
++ * would be a stretch ...
++ */
++typedef struct xattr_node xattr_node_t;
++
++struct xattr_node {
++	struct ext2_ext_attr_header *header;
++	blk_t block;
++	struct xattr_node *next;
++};
++
++xattr_node_t *xattr_list_head = NULL;
++
+ /* structure for mapping xattr name prefix data */
+ typedef struct xattr_prefix {
+ 	int index;
+@@ -48,6 +61,17 @@ xattr_prefix_t xattr_prefixes [] = {
+ 	{ 0 },
+ };
+ 
++/* free xattr node and the buffer holding the xattr block */
++static void
++xattr_free_node (xattr_node_t *node)
++{
++	if (node) {
++		if (node->header)
++			free (node->header);
++		free (node);
++	}
++}
++
+ /* Free remaining resources after all files have been processed. */
+ void
+ xattr_cleanup ()
+@@ -260,6 +284,28 @@ out:
+ 	return ret;
+ }
+ 
++/* Add an xattr node to the list specified by head. This function will update
++ * head as necessary. It will return a pointer to the xattr_node_t added to the
++ * list. In the event that an identical xattr block is already on the list this
++ * function will return a pointer to the pre-existing node.
++ */
++static xattr_node_t*
++xattr_add_block (xattr_node_t **head, xattr_node_t *node)
++{
++	XATTR_STDERR ("Adding xattr to the the node list.\n");
++	return node;
++}
++
++/* Remove xattr node from list. Returns pointer to the node being removed.
++ * NOTE: We're not comparing the xattr blocks, just the nodes.
++ */
++static xattr_node_t*
++xattr_rm_block (xattr_node_t **head, xattr_node_t *node)
++{
++	XATTR_STDERR ("Removing xattr from the node list.\n");
++	return node;
++}
++
+ /* This is the entry point to the xattr module. This function copies the xattrs
+  * from the file at 'path' to the file system object at 'ino'.
+  *
+@@ -274,16 +320,21 @@ set_inode_xattr (ext2_filsys fs, ext2_in
+ 	errcode_t ret = 0;
+ 	char *buf = NULL;
+ 	struct ext2_inode inode = { 0 };
+-	blk_t block = 0;
+-	struct ext2_ext_attr_header *header = NULL;
++	xattr_node_t *node = NULL, *node_tmp = NULL;
+ 	uint32_t newcount = 0;
+ 
+ 	XATTR_STDERR ("Copying xattrs from %s to inode 0x%x.\n", path, ino);
++	/* Create an xattr_node_t for it and add it to the cache if appropriate */
++	if (!(node = calloc (1, sizeof (xattr_node_t)))) {
++		com_err (__func__, errno, "calloc");
++		ret = errno;
++		goto out;
++	}
+ 	/* Populate the xattr block for the file at path */
+-	if (ret = xattr_build_block (path, &header, fs->blocksize)) {
++	if (ret = xattr_build_block (path, &(node->header), fs->blocksize)) {
+ 		goto out;
+ 	}
+-	if (header == NULL) {
++	if (node->header == NULL) {
+ 		XATTR_STDERR ("set_inode_xattrs: no xattrs for %s\n", path);
+ 		goto out;
+ 	}
+@@ -291,23 +342,38 @@ set_inode_xattr (ext2_filsys fs, ext2_in
+ 		com_err(__func__, ret, "ext2fs_read_inode");
+ 		goto out;
+ 	}
+-	if (ret = ext2fs_alloc_block (fs, 0, NULL, &block)) {
+-		com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
++	if (!(node_tmp = xattr_add_block (&xattr_list_head, node))) {
++		fprintf (stderr, "Cannot add NULL node to xattr_block list.\n");
+ 		goto out;
+ 	}
+-	ext2fs_mark_block_bitmap2 (fs->block_map, block);
+-	XATTR_STDERR ("writing xattr block 0x%x to disk:\n", block);
+-	if (ret = ext2fs_write_ext_attr (fs, block, header)) {
+-		com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
+-		goto out;
++	if (node == node_tmp) {
++		/* xattr block is new: add to list & write to disk */
++		XATTR_STDERR ("no pre-existing xattr block, creating / adding\n");
++		if (ret = ext2fs_alloc_block (fs, 0, NULL, &node->block)) {
++			com_err(__func__, ret, "ext2fs_alloc_block: returned %d", ret);
++			xattr_rm_block (&xattr_list_head, node);
++			goto out;
++		}
++		ext2fs_mark_block_bitmap2 (fs->block_map, node->block);
++		XATTR_STDERR ("writing xattr block 0x%x to disk:\n", node->block);
++		if (ret = ext2fs_write_ext_attr (fs, node->block, node->header)) {
++			com_err(__func__, ret, "ext2fs_write_ext_attr: returned %d", ret);
++			xattr_rm_block (&xattr_list_head, node);
++			goto out;
++		}
++	} else {
++		/* already have an identical xattr block, free the one we just made */
++		XATTR_STDERR ("using xattr from existing block: 0x%x\n", node_tmp->block);
++		xattr_free_node (node);
++		node = node_tmp;
+ 	}
+ 	/* point inode for current file to xattr block, update block count and
+ 	 * write inode to disk
+ 	 */
+-	inode.i_file_acl = block;
++	inode.i_file_acl = node->block;
+ 	if (ret = ext2fs_adjust_ea_refcount2(fs,
+-					block,
+-					(char*)header,
++					node->block,
++					(char*)(node->header),
+ 					1,
+ 					&newcount))
+ 	{
+@@ -321,7 +387,6 @@ set_inode_xattr (ext2_filsys fs, ext2_in
+ 	if (ret = ext2fs_write_inode (fs, ino, &inode))
+ 		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
+ out:
+-	if (header)
+-		free (header);
++	xattr_free_node (node);
+ 	return ret;
+ }
diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
index edc94d8..0705ec4 100644
--- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
+++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
@@ -5,4 +5,5 @@ SRC_URI += " \
     file://mke2fs.c-create_inode.c-copy-xattrs.patch \
     file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
     file://misc-xattr-create-xattr-block.patch \
+    file://misc-xattr-create-xattr-block-node.patch \
 "
-- 
2.1.4

[meta-selinux][PATCHv2 8/8] e2fsprogs: Implement xattr block cache with simple linked list.

[Philip Tricca]

Signed-off-by: Philip Tricca <flihp@twobit.us>
---
 .../e2fsprogs/misc-xattr-create-xattr-cache.patch  | 181 +++++++++++++++++++++
 .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |   1 +
 2 files changed, 182 insertions(+)
 create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch

diff --git a/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch
new file mode 100644
index 0000000..38e6454
--- /dev/null
+++ b/recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch
@@ -0,0 +1,181 @@
+Implement the xattr block cache as an unsorted linked list. This requires the
+add and rm functions be able to test for equality between xattr blocks. This
+is implemented as two functions. The first compares individual entries and the
+second compares the whole xattr block by iterating over the individual entries.
+
+The xattr block cache keeps memory allocated on the heap around across
+invocations of the set_inode_xattr function. To free this memory we implement
+an xattr_cleanup function that iterates over the cache freeing resources
+associated with each node.
+
+Signed-off-by: Philip Tricca <flihp@twobit.us>
+
+Index: e2fsprogs-1.42.9/misc/xattr.c
+===================================================================
+--- e2fsprogs-1.42.9.orig/misc/xattr.c
++++ e2fsprogs-1.42.9/misc/xattr.c
+@@ -5,6 +5,7 @@
+ #include <errno.h>
+ #include <ext2fs/ext2_ext_attr.h>
+ #include <linux/xattr.h>
++#include <stdbool.h>
+ #include <stdint.h>
+ #include <stdio.h>
+ #include <stdlib.h>
+@@ -25,6 +26,9 @@
+ #define XATTR_STDERR(fmt, args...) do {} while (0)
+ #endif
+ 
++static size_t cache_hit = 0;
++static size_t cache_miss = 0;
++
+ /* nodes for simple linked list to track xattr blocks, calling it a cache
+  * would be a stretch ...
+  */
+@@ -76,7 +80,18 @@ xattr_free_node (xattr_node_t *node)
+ void
+ xattr_cleanup ()
+ {
++	xattr_node_t *curr = NULL, *tmp = NULL;
++	size_t count = 0;
++
+ 	XATTR_STDERR ("Cleaning up resources from xattrs.\n");
++	for (curr = xattr_list_head; curr != NULL; ++count) {
++		tmp = curr;
++		curr = curr->next;
++		xattr_free_node (tmp);
++	}
++	XATTR_STDERR ("Freed %d xattr_node_ts.\n", count);
++	XATTR_STDERR ("Cache hits: %u\n", cache_hit);
++	XATTR_STDERR ("Cache miss: %u\n", cache_miss);
+ }
+ 
+ /* Get value for named xattr from file at path.
+@@ -284,6 +299,58 @@ out:
+ 	return ret;
+ }
+ 
++static bool
++xattr_entry_isequal (struct ext2_ext_attr_header *header_a,
++			struct ext2_ext_attr_entry *entry_a,
++			struct ext2_ext_attr_header *header_b,
++			struct ext2_ext_attr_entry *entry_b)
++{
++	if (entry_a->e_hash == entry_b->e_hash ||
++		entry_a->e_name_index == entry_b->e_name_index ||
++		entry_a->e_name_len == entry_b->e_name_len ||
++		entry_a->e_value_size == entry_b->e_value_size)
++	{
++		/* If entry header data all matches we check name and value */
++		if (memcmp (EXT2_EXT_ATTR_NAME(entry_a),
++				EXT2_EXT_ATTR_NAME(entry_b),
++				MIN (entry_a->e_name_len, entry_b->e_name_len)) != 0)
++			return false;
++		if (memcmp (VALUE(header_a, entry_a),
++				VALUE(header_b, entry_b),
++				MIN(entry_a->e_value_size, entry_b->e_value_size)) != 0)
++			return false;
++		return true;
++	} else {
++		return false;
++	}
++}
++
++static bool
++xattr_block_isequal (struct ext2_ext_attr_header *header_a,
++			struct ext2_ext_attr_header *header_b)
++{
++	struct ext2_ext_attr_entry *entry_a = NULL, *entry_b = NULL;
++
++	XATTR_STDERR ("equality test: xattr blocks at 0x%x and 0x%x\n", header_a, header_b);
++	for (entry_a = FIRST_ENTRY(header_a), entry_b = FIRST_ENTRY(header_b);
++		!EXT2_EXT_IS_LAST_ENTRY(entry_a) && !EXT2_EXT_IS_LAST_ENTRY(entry_b);
++		entry_a = EXT2_EXT_ATTR_NEXT(entry_a), entry_b = EXT2_EXT_ATTR_NEXT(entry_b))
++	{
++		if (!xattr_entry_isequal (header_a, entry_a, header_b, entry_b)) {
++			/* bail as soon as we find entries that don't match */
++			XATTR_STDERR ("entries do not match\n");
++			return false;
++		}
++	}
++	/* Be sure we're on the last element from each block. */
++	if (EXT2_EXT_IS_LAST_ENTRY(entry_a) && EXT2_EXT_IS_LAST_ENTRY(entry_b)) {
++		XATTR_STDERR ("entries match\n");
++		return true;
++	} else {
++		return false;
++	}
++}
++
+ /* Add an xattr node to the list specified by head. This function will update
+  * head as necessary. It will return a pointer to the xattr_node_t added to the
+  * list. In the event that an identical xattr block is already on the list this
+@@ -292,7 +359,31 @@ out:
+ static xattr_node_t*
+ xattr_add_block (xattr_node_t **head, xattr_node_t *node)
+ {
++	xattr_node_t *curr_node = NULL, *prev_node = NULL;
++
+ 	XATTR_STDERR ("Adding xattr to the the node list.\n");
++	if (node == NULL)
++		return NULL;
++	/* list is empty, node becomes first node */
++	if (!(*head)) {
++		*head = node;
++		return node;
++	}
++	for (curr_node = *head; curr_node != NULL; curr_node = curr_node->next)
++	{
++		/* cache hit */
++		if (xattr_block_isequal (node->header, curr_node->header)) {
++			++cache_hit;
++			return curr_node;
++		}
++		/* end of list */
++		if (curr_node->next == NULL) {
++			++cache_miss;
++			curr_node->next = node;
++			return node;
++		}
++	}
++	/* should never reach: assert? */
+ 	return node;
+ }
+ 
+@@ -302,8 +393,27 @@ xattr_add_block (xattr_node_t **head, xa
+ static xattr_node_t*
+ xattr_rm_block (xattr_node_t **head, xattr_node_t *node)
+ {
++	xattr_node_t *curr_node = NULL, *prev_node = NULL;
++
+ 	XATTR_STDERR ("Removing xattr from the node list.\n");
+-	return node;
++	/* no list, or empty list: nothing to search though */
++	if (!head || !(*head))
++		return NULL;
++
++	for (prev_node = NULL, curr_node = *head;
++		curr_node != NULL;
++		prev_node = curr_node, curr_node = curr_node->next)
++	{
++		if (node == curr_node) {
++			if (prev_node)
++				prev_node->next = curr_node->next;
++			else
++				*head = curr_node->next;
++			return curr_node;
++		}
++	}
++	/* reached end of list, no match */
++	return NULL;
+ }
+ 
+ /* This is the entry point to the xattr module. This function copies the xattrs
+@@ -386,6 +496,7 @@ set_inode_xattr (ext2_filsys fs, ext2_in
+ 	}
+ 	if (ret = ext2fs_write_inode (fs, ino, &inode))
+ 		com_err(__func__, ret, "ext2fs_write_inode: returned %d", ret);
++	return ret;
+ out:
+ 	xattr_free_node (node);
+ 	return ret;
diff --git a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
index 0705ec4..d8de9da 100644
--- a/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
+++ b/recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
@@ -6,4 +6,5 @@ SRC_URI += " \
     file://lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch \
     file://misc-xattr-create-xattr-block.patch \
     file://misc-xattr-create-xattr-block-node.patch \
+    file://misc-xattr-create-xattr-cache.patch \
 "
-- 
2.1.4

Re: [meta-selinux][PATCHv2 0/8] Label file system in build.

[Joe MacDonald]

[-- Attachment #1: Type: text/plain, Size: 6814 bytes --]

Hi Phil,

I'm sorry this has been in the merge queue for so long.  I've merged it
after taking the policy updates from Shrikant and a few other small
patches that had been hanging around too.  I didn't drop it on master
yet, though, since I wanted to give everyone else a little bit of time
to try it out (myself included, I'm finally able to come up for air on
some of the day job things :-)).  Instead it is currently living on the
fs_label branch, but I rebased the patches on the current master HEAD
commit.  That means, though, that if you get a chance I'd like to take a
look at the branch to ensure I didn't mangle your patch set too much.

-J.

[[meta-selinux][PATCHv2 0/8] Label file system in build.] On 15.06.17 (Wed 15:30) Philip Tricca wrote:

> This is the second version of a patch series that allows the file system
> of SELinux images to be labeled as part of the build process. This will
> allow SELinux images to boot read only file systems and remove the need to
> label the file system on first boot.
> 
> To do this we must label the file system in the build as well as add
> support for extended attributes to the mke2fs utility in the e2fsprogs
> package. The first version of this patch series is here:
> https://lists.yoctoproject.org/pipermail/yocto/2015-June/025141.html
> The approach described in this previous RFC remains the same.
> 
> Changes in v2:
> This second version has two significant changes: First I've done a bunch
> of cleanup. This includes work to make the descriptions in the patch
> headers / commit messages more exact as well as combining some commits
> with related functionality. Secondly I've reimplemented the xattr cache
> so that it actually works.
> 
> I've made the patch headers as descriptive as possible and kept the git
> commit messages minimal. If the preference is for more verbose commit
> messages I'm happy to oblige if advised.
> 
> The cache is just a single linked list that's searched for duplicates after
> the creation of each new xattr block. The previous implementation was similar
> but, aside from not working properly, it was overly complex in its attempt to
> keep the list sorted.
> 
> Tests:
> To test this new implementation I used the core-image-selinux-minimal image
> from the unmodified master branch as a control. This image has 2536 unique
> file system objects including the root fs directory. The ext4 file system
> produced by the build has 71492 blocks with 13621 free.
> 
> As an additional test I added the patches from this set WITHOUT the cache
> patches. This causes each file system object with an associated extended
> attribute to use up an additional block for the xattr. This should cause
> (hypothesis) the output file system to have 13621 - 2536 = 11085 free
> blocks. The build producing an ext4 file system with 71492 blocks and 11088
> free. That's an additional 2533 blocks used instead of the 2536 expected.
> These 3 missing xattr blocks can be accounted for in that there are 3
> unlabeled files in the file system.
> 
> Introducing the cache allows files with identical xattr blocks to share
> them to reduce the number of used blocks. Since we're only storing SELinux
> labels in the xattrs we can say that every file with the same SELinux label
> should share an xattr block. Counting the unique SELinux labels on file
> objects we know that there are 83 in total. The second hypothesis we have
> to test then is that using the cache will reduce the number of used blocks
> from 2533 down to 83.
> 
> Applying the patch that enables the cache produces a third and final ext4
> file system. This one again report 71492 total blocks but this time 13538
> free. This is 83 blocks fewer than the unlabled file system from the
> initial test as we expected. The code added by this patch set is also
> instrumented to count the objects in the cache when they're freed. With
> this debug output enabled it reports the same number of objects in the
> cache.
> 
> From the test results I'm pretty confident that the cache functions as
> expected. It's still a very basic implementation but given the small
> number of unique SELinux labels in the reference file systems it's
> likely sufficient for a first version. Feedback / comments on both the
> implementation and testing approach would be appreciated.
> 
> Regards,
> Philip
> ----
> 
> Philip Tricca (8):
>   policycoreutils: Patch setfiles to add FTS_NOCHDIR to fts_flags.
>   selinux-image: Add new image class to label the rootfs, use it for
>     selinux images.
>   e2fsprogs: Add bbappend and stub for xattr module.
>   e2fsprogs: Insert calls to xattr module into mke2fs and build xattr
>     code.
>   e2fsprogs: Add xattr security prefix data to
>     lib/ext2fs/ext2_ext_attr.h
>   e2fsprogs: Copy xattr block from source file.
>   e2fsprogs: Add stub functions for an xattr cache and struct to hold
>     the header and block data.
>   e2fsprogs: Implement xattr block cache with simple linked list.
> 
>  classes/selinux-image.bbclass                      |   8 +
>  ...ib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch |  20 ++
>  .../misc-xattr-add-xattr-module-stub.patch         |  57 ++++
>  .../misc-xattr-create-xattr-block-node.patch       | 175 +++++++++++
>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
>  .../e2fsprogs/misc-xattr-create-xattr-cache.patch  | 181 +++++++++++
>  .../mke2fs.c-create_inode.c-copy-xattrs.patch      | 164 ++++++++++
>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |  10 +
>  .../images/core-image-selinux-minimal.bb           |   2 +-
>  recipes-security/images/core-image-selinux.bb      |   2 +-
>  .../policycoreutils-fts_flags-FTS_NOCHDIR.patch    |  25 ++
>  recipes-security/selinux/policycoreutils_2.3.bb    |   1 +
>  12 files changed, 984 insertions(+), 2 deletions(-)
>  create mode 100644 classes/selinux-image.bbclass
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch
>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>  create mode 100644 recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch
> 
-- 
-Joe MacDonald.
:wq

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

Re: [meta-selinux][PATCHv2 0/8] Label file system in build.

[Philip Tricca]

Hey Joe,

On 08/08/2015 02:00 PM, Joe MacDonald wrote:
> I'm sorry this has been in the merge queue for so long.

Better late than never :)

> I've merged it
> after taking the policy updates from Shrikant and a few other small
> patches that had been hanging around too.  I didn't drop it on master
> yet, though, since I wanted to give everyone else a little bit of time
> to try it out (myself included, I'm finally able to come up for air on
> some of the day job things :-)).  Instead it is currently living on the
> fs_label branch, but I rebased the patches on the current master HEAD
> commit.  That means, though, that if you get a chance I'd like to take a
> look at the branch to ensure I didn't mangle your patch set too much.

Will do.

Philip

> [[meta-selinux][PATCHv2 0/8] Label file system in build.] On 15.06.17 (Wed 15:30) Philip Tricca wrote:
> 
>> This is the second version of a patch series that allows the file system
>> of SELinux images to be labeled as part of the build process. This will
>> allow SELinux images to boot read only file systems and remove the need to
>> label the file system on first boot.
>>
>> To do this we must label the file system in the build as well as add
>> support for extended attributes to the mke2fs utility in the e2fsprogs
>> package. The first version of this patch series is here:
>> https://lists.yoctoproject.org/pipermail/yocto/2015-June/025141.html
>> The approach described in this previous RFC remains the same.
>>
>> Changes in v2:
>> This second version has two significant changes: First I've done a bunch
>> of cleanup. This includes work to make the descriptions in the patch
>> headers / commit messages more exact as well as combining some commits
>> with related functionality. Secondly I've reimplemented the xattr cache
>> so that it actually works.
>>
>> I've made the patch headers as descriptive as possible and kept the git
>> commit messages minimal. If the preference is for more verbose commit
>> messages I'm happy to oblige if advised.
>>
>> The cache is just a single linked list that's searched for duplicates after
>> the creation of each new xattr block. The previous implementation was similar
>> but, aside from not working properly, it was overly complex in its attempt to
>> keep the list sorted.
>>
>> Tests:
>> To test this new implementation I used the core-image-selinux-minimal image
>> from the unmodified master branch as a control. This image has 2536 unique
>> file system objects including the root fs directory. The ext4 file system
>> produced by the build has 71492 blocks with 13621 free.
>>
>> As an additional test I added the patches from this set WITHOUT the cache
>> patches. This causes each file system object with an associated extended
>> attribute to use up an additional block for the xattr. This should cause
>> (hypothesis) the output file system to have 13621 - 2536 = 11085 free
>> blocks. The build producing an ext4 file system with 71492 blocks and 11088
>> free. That's an additional 2533 blocks used instead of the 2536 expected.
>> These 3 missing xattr blocks can be accounted for in that there are 3
>> unlabeled files in the file system.
>>
>> Introducing the cache allows files with identical xattr blocks to share
>> them to reduce the number of used blocks. Since we're only storing SELinux
>> labels in the xattrs we can say that every file with the same SELinux label
>> should share an xattr block. Counting the unique SELinux labels on file
>> objects we know that there are 83 in total. The second hypothesis we have
>> to test then is that using the cache will reduce the number of used blocks
>> from 2533 down to 83.
>>
>> Applying the patch that enables the cache produces a third and final ext4
>> file system. This one again report 71492 total blocks but this time 13538
>> free. This is 83 blocks fewer than the unlabled file system from the
>> initial test as we expected. The code added by this patch set is also
>> instrumented to count the objects in the cache when they're freed. With
>> this debug output enabled it reports the same number of objects in the
>> cache.
>>
>> From the test results I'm pretty confident that the cache functions as
>> expected. It's still a very basic implementation but given the small
>> number of unique SELinux labels in the reference file systems it's
>> likely sufficient for a first version. Feedback / comments on both the
>> implementation and testing approach would be appreciated.
>>
>> Regards,
>> Philip
>> ----
>>
>> Philip Tricca (8):
>>   policycoreutils: Patch setfiles to add FTS_NOCHDIR to fts_flags.
>>   selinux-image: Add new image class to label the rootfs, use it for
>>     selinux images.
>>   e2fsprogs: Add bbappend and stub for xattr module.
>>   e2fsprogs: Insert calls to xattr module into mke2fs and build xattr
>>     code.
>>   e2fsprogs: Add xattr security prefix data to
>>     lib/ext2fs/ext2_ext_attr.h
>>   e2fsprogs: Copy xattr block from source file.
>>   e2fsprogs: Add stub functions for an xattr cache and struct to hold
>>     the header and block data.
>>   e2fsprogs: Implement xattr block cache with simple linked list.
>>
>>  classes/selinux-image.bbclass                      |   8 +
>>  ...ib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch |  20 ++
>>  .../misc-xattr-add-xattr-module-stub.patch         |  57 ++++
>>  .../misc-xattr-create-xattr-block-node.patch       | 175 +++++++++++
>>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
>>  .../e2fsprogs/misc-xattr-create-xattr-cache.patch  | 181 +++++++++++
>>  .../mke2fs.c-create_inode.c-copy-xattrs.patch      | 164 ++++++++++
>>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |  10 +
>>  .../images/core-image-selinux-minimal.bb           |   2 +-
>>  recipes-security/images/core-image-selinux.bb      |   2 +-
>>  .../policycoreutils-fts_flags-FTS_NOCHDIR.patch    |  25 ++
>>  recipes-security/selinux/policycoreutils_2.3.bb    |   1 +
>>  12 files changed, 984 insertions(+), 2 deletions(-)
>>  create mode 100644 classes/selinux-image.bbclass
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch
>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>>  create mode 100644 recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch
>>

Re: [meta-selinux][PATCHv2 0/8] Label file system in build.

[Philip Tricca]

On 08/10/2015 09:10 PM, Philip Tricca wrote:
> On 08/08/2015 02:00 PM, Joe MacDonald wrote:
>> I'm sorry this has been in the merge queue for so long.
> 
> Better late than never :)
> 
>> I've merged it
>> after taking the policy updates from Shrikant and a few other small
>> patches that had been hanging around too.  I didn't drop it on master
>> yet, though, since I wanted to give everyone else a little bit of time
>> to try it out (myself included, I'm finally able to come up for air on
>> some of the day job things :-)).  Instead it is currently living on the
>> fs_label branch, but I rebased the patches on the current master HEAD
>> commit.  That means, though, that if you get a chance I'd like to take a
>> look at the branch to ensure I didn't mangle your patch set too much.
> 
> Will do.

Built and tested. Works as expected. There are a couple AVCs from
rpcbind related to stuff in /tmp but those are present in the master
branch so they're not related.

Philip

>> [[meta-selinux][PATCHv2 0/8] Label file system in build.] On 15.06.17 (Wed 15:30) Philip Tricca wrote:
>>
>>> This is the second version of a patch series that allows the file system
>>> of SELinux images to be labeled as part of the build process. This will
>>> allow SELinux images to boot read only file systems and remove the need to
>>> label the file system on first boot.
>>>
>>> To do this we must label the file system in the build as well as add
>>> support for extended attributes to the mke2fs utility in the e2fsprogs
>>> package. The first version of this patch series is here:
>>> https://lists.yoctoproject.org/pipermail/yocto/2015-June/025141.html
>>> The approach described in this previous RFC remains the same.
>>>
>>> Changes in v2:
>>> This second version has two significant changes: First I've done a bunch
>>> of cleanup. This includes work to make the descriptions in the patch
>>> headers / commit messages more exact as well as combining some commits
>>> with related functionality. Secondly I've reimplemented the xattr cache
>>> so that it actually works.
>>>
>>> I've made the patch headers as descriptive as possible and kept the git
>>> commit messages minimal. If the preference is for more verbose commit
>>> messages I'm happy to oblige if advised.
>>>
>>> The cache is just a single linked list that's searched for duplicates after
>>> the creation of each new xattr block. The previous implementation was similar
>>> but, aside from not working properly, it was overly complex in its attempt to
>>> keep the list sorted.
>>>
>>> Tests:
>>> To test this new implementation I used the core-image-selinux-minimal image
>>> from the unmodified master branch as a control. This image has 2536 unique
>>> file system objects including the root fs directory. The ext4 file system
>>> produced by the build has 71492 blocks with 13621 free.
>>>
>>> As an additional test I added the patches from this set WITHOUT the cache
>>> patches. This causes each file system object with an associated extended
>>> attribute to use up an additional block for the xattr. This should cause
>>> (hypothesis) the output file system to have 13621 - 2536 = 11085 free
>>> blocks. The build producing an ext4 file system with 71492 blocks and 11088
>>> free. That's an additional 2533 blocks used instead of the 2536 expected.
>>> These 3 missing xattr blocks can be accounted for in that there are 3
>>> unlabeled files in the file system.
>>>
>>> Introducing the cache allows files with identical xattr blocks to share
>>> them to reduce the number of used blocks. Since we're only storing SELinux
>>> labels in the xattrs we can say that every file with the same SELinux label
>>> should share an xattr block. Counting the unique SELinux labels on file
>>> objects we know that there are 83 in total. The second hypothesis we have
>>> to test then is that using the cache will reduce the number of used blocks
>>> from 2533 down to 83.
>>>
>>> Applying the patch that enables the cache produces a third and final ext4
>>> file system. This one again report 71492 total blocks but this time 13538
>>> free. This is 83 blocks fewer than the unlabled file system from the
>>> initial test as we expected. The code added by this patch set is also
>>> instrumented to count the objects in the cache when they're freed. With
>>> this debug output enabled it reports the same number of objects in the
>>> cache.
>>>
>>> From the test results I'm pretty confident that the cache functions as
>>> expected. It's still a very basic implementation but given the small
>>> number of unique SELinux labels in the reference file systems it's
>>> likely sufficient for a first version. Feedback / comments on both the
>>> implementation and testing approach would be appreciated.
>>>
>>> Regards,
>>> Philip
>>> ----
>>>
>>> Philip Tricca (8):
>>>   policycoreutils: Patch setfiles to add FTS_NOCHDIR to fts_flags.
>>>   selinux-image: Add new image class to label the rootfs, use it for
>>>     selinux images.
>>>   e2fsprogs: Add bbappend and stub for xattr module.
>>>   e2fsprogs: Insert calls to xattr module into mke2fs and build xattr
>>>     code.
>>>   e2fsprogs: Add xattr security prefix data to
>>>     lib/ext2fs/ext2_ext_attr.h
>>>   e2fsprogs: Copy xattr block from source file.
>>>   e2fsprogs: Add stub functions for an xattr cache and struct to hold
>>>     the header and block data.
>>>   e2fsprogs: Implement xattr block cache with simple linked list.
>>>
>>>  classes/selinux-image.bbclass                      |   8 +
>>>  ...ib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch |  20 ++
>>>  .../misc-xattr-add-xattr-module-stub.patch         |  57 ++++
>>>  .../misc-xattr-create-xattr-block-node.patch       | 175 +++++++++++
>>>  .../e2fsprogs/misc-xattr-create-xattr-block.patch  | 341 +++++++++++++++++++++
>>>  .../e2fsprogs/misc-xattr-create-xattr-cache.patch  | 181 +++++++++++
>>>  .../mke2fs.c-create_inode.c-copy-xattrs.patch      | 164 ++++++++++
>>>  .../e2fsprogs/e2fsprogs_1.42.9.bbappend            |  10 +
>>>  .../images/core-image-selinux-minimal.bb           |   2 +-
>>>  recipes-security/images/core-image-selinux.bb      |   2 +-
>>>  .../policycoreutils-fts_flags-FTS_NOCHDIR.patch    |  25 ++
>>>  recipes-security/selinux/policycoreutils_2.3.bb    |   1 +
>>>  12 files changed, 984 insertions(+), 2 deletions(-)
>>>  create mode 100644 classes/selinux-image.bbclass
>>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/lib-ext2fs-ext2_ext_attr.h-add-xattr-index.patch
>>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-add-xattr-module-stub.patch
>>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block-node.patch
>>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-block.patch
>>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/misc-xattr-create-xattr-cache.patch
>>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs/mke2fs.c-create_inode.c-copy-xattrs.patch
>>>  create mode 100644 recipes-devtools/e2fsprogs/e2fsprogs_1.42.9.bbappend
>>>  create mode 100644 recipes-security/selinux/policycoreutils/policycoreutils-fts_flags-FTS_NOCHDIR.patch
>>>
>