[PATCH 0/1] v2 pseudo 1.7 (now 1.7.1)

[PATCH 0/1] v2 pseudo 1.7 (now 1.7.1)

[Peter Seebach]

So, this is just like the previous pseudo 1.7.0 patch, except
that the very strange and sporadic failures that were showing up
in the RPM backend are now fixed.

The actual issue: realpath(foo, NULL) is supposed to allocate storage.
When I stopped having path canonicalization allocate new storage, I
fixed the points where the returned values were being freed, but I
did not spot this particular instance of a place where the value was
being returned with the expectation that the caller would free it.

Also changed the lstat() calls in path canonicalization so they will
use lstat64 where appropriate (such as all the time on Linux), because
otherwise lstat() can fail if a file has an *inode value* out of range,
which is distinct from the "size out of range" issues previously corrected.
This may have broken path lookups in some cases on XFS.

The xattrdb stuff has not been otherwise changed, so it's still
considered experimental.

The following changes since commit bdeb32b4cdbe316f17c2fd854d59e05e8f2e8ffc:

  rt-tests: drop unnecessary added-missing-dependencies.patch (2015-09-01 11:43:38 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib seebs/pseudo171
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=seebs/pseudo171

Peter Seebach (1):
  pseudo_1.7.1.bb: New version of pseudo

 meta/recipes-devtools/pseudo/pseudo_1.7.1.bb | 19 +++++++++++++++++++
 meta/recipes-devtools/pseudo/pseudo_git.bb   |  4 ++--
 2 files changed, 21 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/pseudo/pseudo_1.7.1.bb

-- 
2.3.1

[PATCH 1/1] pseudo_1.7.1.bb: New version of pseudo

[Peter Seebach]

Pseudo 1.7 adds an experimental feature (which I think needs more testing
before it becomes the default) allowing the pseudo client to store modes
and uid/gid values in extended attributes rather than using the sqlite
database. On most Linux-like systems, this works only if the underlying
file is a plain file or a directory.

Also added is a profiling feature to allow some amount of reporting on
the wall-clock time the client spends in wrappers, processing operations,
or in IPC. This feature is not intendeded to be precisely accurate, but
gives a good overview of where time is going.

Based on the results from the profiling feature, the client now suppresses
OP_OPEN and OP_EXEC messages if the server is not logging messages, and
no longer uses constant dynamic allocation and free cycles for canonicalized
paths.

There's a few other likely-looking optimizations being considered, but
this seemed like a good cutoff for now.

(1.7.1 fixes two bugs, one affecting mostly XFS systems with 64-bit
inode values, and one affecting code that called realpath(x, NULL), such
as the RPM backend.)

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
---
 meta/recipes-devtools/pseudo/pseudo_1.7.1.bb | 19 +++++++++++++++++++
 meta/recipes-devtools/pseudo/pseudo_git.bb   |  4 ++--
 2 files changed, 21 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/pseudo/pseudo_1.7.1.bb

diff --git a/meta/recipes-devtools/pseudo/pseudo_1.7.1.bb b/meta/recipes-devtools/pseudo/pseudo_1.7.1.bb
new file mode 100644
index 0000000..447092b
--- /dev/null
+++ b/meta/recipes-devtools/pseudo/pseudo_1.7.1.bb
@@ -0,0 +1,19 @@
+require pseudo.inc
+
+SRC_URI = " \
+    http://downloads.yoctoproject.org/releases/pseudo/${BPN}-${PV}.tar.bz2 \
+    file://fallback-passwd \
+    file://fallback-group \
+"
+
+SRC_URI[md5sum] = "f7ee16ebf6ab0f2bf80d4ca64862de09"
+SRC_URI[sha256sum] = "aac9659cb15b7a40ff27d5de29ef37f033890ec1671d3fa41abbf2d57fdc0358"
+
+PSEUDO_EXTRA_OPTS ?= "--enable-force-async --without-passwd-fallback"
+
+do_install_append_class-native () {
+	install -d ${D}${sysconfdir}
+	# The fallback files should never be modified
+	install -m 444 ${WORKDIR}/fallback-passwd ${D}${sysconfdir}/passwd
+	install -m 444 ${WORKDIR}/fallback-group ${D}${sysconfdir}/group
+}
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb b/meta/recipes-devtools/pseudo/pseudo_git.bb
index aa315d3..c336151 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -1,7 +1,7 @@
 require pseudo.inc
 
-SRCREV = "db758fb11167c79d9682a17d359568e2a3c4acd5"
-PV = "1.6.5+git${SRCPV}"
+SRCREV = "a49c56fbf1fdff5a3050d30b47851c52b4e35fc2"
+PV = "1.7.1+git${SRCPV}"
 
 DEFAULT_PREFERENCE = "-1"
 
-- 
2.3.1

Re: [PATCH 1/1] pseudo_1.7.1.bb: New version of pseudo

[Richard Purdie]

On Wed, 2015-09-02 at 13:47 -0500, Peter Seebach wrote:
> Pseudo 1.7 adds an experimental feature (which I think needs more testing
> before it becomes the default) allowing the pseudo client to store modes
> and uid/gid values in extended attributes rather than using the sqlite
> database. On most Linux-like systems, this works only if the underlying
> file is a plain file or a directory.
> 
> Also added is a profiling feature to allow some amount of reporting on
> the wall-clock time the client spends in wrappers, processing operations,
> or in IPC. This feature is not intendeded to be precisely accurate, but
> gives a good overview of where time is going.
> 
> Based on the results from the profiling feature, the client now suppresses
> OP_OPEN and OP_EXEC messages if the server is not logging messages, and
> no longer uses constant dynamic allocation and free cycles for canonicalized
> paths.
> 
> There's a few other likely-looking optimizations being considered, but
> this seemed like a good cutoff for now.
> 
> (1.7.1 fixes two bugs, one affecting mostly XFS systems with 64-bit
> inode values, and one affecting code that called realpath(x, NULL), such
> as the RPM backend.)

This is better however opkg is still unhappy. This is build with this
pseudo patch applied:

https://autobuilder.yoctoproject.org/main/builders/nightly-ipk/builds/469

and this is a build with the same patches expect for the pseudo one:

https://autobuilder.yoctoproject.org/main/builders/nightly-ipk/builds/470

so something is still unhappy :(. The rpm builds were a lot better
though.

Cheers,

Richard

Re: [PATCH 1/1] pseudo_1.7.1.bb: New version of pseudo

[Peter Seebach]

On Thu, 3 Sep 2015 09:43:50 +0100
Richard Purdie <richard.purdie@linuxfoundation.org> wrote:

> This is better however opkg is still unhappy. This is build with this
> pseudo patch applied:
> 
> https://autobuilder.yoctoproject.org/main/builders/nightly-ipk/builds/469

I've found the cause of that particular bit. During my attempts to reproduce
this, I ran into segfaults in cross-localedef that I cannot now reproduce.

The cause of this was fixing up filesystem modes for the case where umask
contains 0200, which wasn't originally expected to work, but which was
breaking the xattrdb code. (There are a few things in the system which create
files with umask 0777, it turns out; this is arguably bad, but now there's a
workaround.)

But when I put in the workaround, I used 0600 as the "definitely at least
these bits" mode, instead of the requested mode, and that breaks scripts
which were expecting, say, 0700. So the things which have a specified mode
now use that | 0600 instead of just 0600.

I want to run a couple more things to see whether I can get any of the other
failures to come back again.

I also partially know why I wasn't seeing this on one of my systems, but only
partially. When I do my builds on CentOS 6.5, the .postinst files in question
either don't come into existence, or get run successfully anyway. So far as
I can tell, opkg is doing "sh -c /path/to/script" on those, which requires
0700 to run. I'm not sure what's causing the difference between the hosts.

-s
-- 
Listen, get this.  Nobody with a good compiler needs to be justified.