* [PATCH 0/1] Uprev to pseudo 1.2
@ 2011-11-02 21:11 Mark Hatle
2011-11-02 21:11 ` [PATCH 1/1] pseudo: Uprev pseudo to version 1.2 Mark Hatle
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Mark Hatle @ 2011-11-02 21:11 UTC (permalink / raw)
To: openembedded-core
Uprev to pseudo 1.2.
This was heavily tested by building oe-core with numerous image
configurations. Each configuration generated the same image before and
after the change from the current to new version of pseudo.
The primary purpose of this change is to enable the PSEUDO_UNLOAD
functionality that may be used for performance enhancements in future
versions of oe-core.
The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/pseudo
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
Mark Hatle (1):
pseudo: Uprev pseudo to version 1.2
.../conf/distro/include/distro_tracking_fields.inc | 6 +-
.../pseudo/pseudo/realpath_fix.patch | 165 --------------------
.../pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} | 7 +-
meta/recipes-devtools/pseudo/pseudo_git.bb | 6 +-
4 files changed, 9 insertions(+), 175 deletions(-)
delete mode 100644 meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
rename meta/recipes-devtools/pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} (48%)
--
1.7.3.4
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/1] pseudo: Uprev pseudo to version 1.2
2011-11-02 21:11 [PATCH 0/1] Uprev to pseudo 1.2 Mark Hatle
@ 2011-11-02 21:11 ` Mark Hatle
2011-11-02 21:22 ` [PATCH 0/1] Uprev to pseudo 1.2 Martin Jansa
2011-11-08 22:58 ` Saul Wold
2 siblings, 0 replies; 10+ messages in thread
From: Mark Hatle @ 2011-11-02 21:11 UTC (permalink / raw)
To: openembedded-core
This adds a new feature, PSEUDO_UNLOAD, which can be used to eliminate
overhead of LD_PRELOAD when no longer necessary.
Also the, clone(2), support on Linux has been updated to resolve some
potential defects in the previous implementation.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
.../conf/distro/include/distro_tracking_fields.inc | 6 +-
.../pseudo/pseudo/realpath_fix.patch | 165 --------------------
.../pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} | 7 +-
meta/recipes-devtools/pseudo/pseudo_git.bb | 6 +-
4 files changed, 9 insertions(+), 175 deletions(-)
delete mode 100644 meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
rename meta/recipes-devtools/pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} (48%)
diff --git a/meta/conf/distro/include/distro_tracking_fields.inc b/meta/conf/distro/include/distro_tracking_fields.inc
index 998eabf..6995abb 100644
--- a/meta/conf/distro/include/distro_tracking_fields.inc
+++ b/meta/conf/distro/include/distro_tracking_fields.inc
@@ -5944,11 +5944,11 @@ RECIPE_LAST_UPDATE_pn-gettext = "May 24, 2011"
RECIPE_MANUAL_CHECK_DATE_pn-gettext = "Sep 29, 2011"
RECIPE_STATUS_pn-pseudo = "green"
-RECIPE_LATEST_VERSION_pn-pseudo = "1.1.1"
-RECIPE_LAST_UPDATE_pn-pseudo = "Jun 06, 2011"
+RECIPE_LATEST_VERSION_pn-pseudo = "1.2"
+RECIPE_LAST_UPDATE_pn-pseudo = "Nov 02, 2011"
RECIPE_MAINTAINER_pn-pseudo = "Mark Hatle <mark.hatle@windriver.com>"
RECIPE_COMMENTS_pn-pseudo = "Yocto Project maintained"
-RECIPE_MANUAL_CHECK_DATE_pn-pseudo = "Jun 06, 2011"
+RECIPE_MANUAL_CHECK_DATE_pn-pseudo = "Nov 02, 2011"
DISTRO_PN_ALIAS_pn-pseudo = "Windriver"
DISTRO_PN_ALIAS_pn-rt-tests = "Debian=rt-tests Ubuntu=rt-tests"
diff --git a/meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch b/meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
deleted file mode 100644
index 4a107e0..0000000
--- a/meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
+++ /dev/null
@@ -1,165 +0,0 @@
-commit c2f7c5ad8ef0f9c94a2a8382c109c8c6e16c8b18
-Author: Peter Seebach <peter.seebach@windriver.com>
-Date: Thu Jun 9 10:53:32 2011 -0500
-
- Fix realpath(name, NULL) when PSEUDO_DISABLED=1
-
- On some Linux systems, dlsym("realpath", RTLD_NEXT) prefers
- for reasons of its own to give a symbol that is also known
- as old_realpath, which fails and yields EINVAL when called
- with a null pointer as the second argument. This can be
- avoided, on some systems, by using dlvsym() to request
- the GLIBC_2.3 version of the symbol.
-
- The wrapper logic is enhanced to allow for specifying
- versions, although this currently only works for Linux
- (Darwin has no dlvsym, apparently?). The test case is
- a trivial program which calls realpath(name, NULL) run
- with PSEUDO_DISABLED=1.
-
-diff --git a/ChangeLog.txt b/ChangeLog.txt
-index 7ffb74a..a2bbb61 100644
---- a/ChangeLog.txt
-+++ b/ChangeLog.txt
-@@ -1,3 +1,8 @@
-+2011-06-08:
-+ * (seebs) Get the modern realpath from glibc instead of the old
-+ one inexplicably proferred by RTLD_NEXT. Fixes realpath(path, NULL)
-+ when PSEUDO_DISABLED=1.
-+
- 2011-06-06:
- * (seebs) revise system() handler substantially. It now
- pollutes the environment but works.
-diff --git a/makewrappers b/makewrappers
-index 6dcf889..20bbf2b 100755
---- a/makewrappers
-+++ b/makewrappers
-@@ -211,6 +211,7 @@ class Function:
- self.flags = '0'
- self.port = port
- self.directory = ''
-+ self.version = 'NULL'
- # On Darwin, some functions are SECRETLY converted to foo$INODE64
- # when called. So we have to look those up for real_*
- self.inode64 = None
-diff --git a/ports/unix/wrapfuncs.in b/ports/unix/wrapfuncs.in
-index 74bad89..e06e404 100644
---- a/ports/unix/wrapfuncs.in
-+++ b/ports/unix/wrapfuncs.in
-@@ -18,7 +18,7 @@ int lutimes(const char *path, const struct timeval *tv);
- char *mkdtemp(char *template);
- char *mktemp(char *template);
- long pathconf(const char *path, int name);
--char *realpath(const char *name, char *resolved_name);
-+char *realpath(const char *name, char *resolved_name); /* version="GLIBC_2.3" */
- int remove(const char *path); /* flags=AT_SYMLINK_NOFOLLOW */
- DIR *opendir(const char *path);
- char *tempnam(const char *template, const char *pfx);
-diff --git a/pseudo_wrappers.c b/pseudo_wrappers.c
-index 600a918..07a4429 100644
---- a/pseudo_wrappers.c
-+++ b/pseudo_wrappers.c
-@@ -90,6 +90,42 @@ pseudo_reinit_libpseudo(void) {
- _libpseudo_init();
- }
-
-+static void
-+pseudo_init_one_wrapper(pseudo_function *func) {
-+ int (*f)(void) = (int (*)(void)) NULL;
-+ char *e;
-+ if (*func->real != NULL) {
-+ /* already initialized */
-+ return;
-+ }
-+ dlerror();
-+
-+#if PSEUDO_PORT_LINUX
-+ if (func->version)
-+ f = dlvsym(RTLD_NEXT, func->name, func->version);
-+ /* fall through to the general case, if that failed */
-+ if (!f)
-+#endif
-+ f = dlsym(RTLD_NEXT, func->name);
-+ if (f) {
-+ *func->real = f;
-+ } else {
-+ e = dlerror();
-+#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS
-+ char *s = func->name;
-+ s += strlen(s) - 2;
-+ /* *at() don't have to exist */
-+ if (!strcmp(s, "at")) {
-+ continue;
-+ }
-+#else
-+ if (e != NULL) {
-+ pseudo_diag("No real function for %s: %s\n", func->name, e);
-+ }
-+#endif
-+ }
-+}
-+
- void
- pseudo_init_wrappers(void) {
- int i;
-@@ -103,29 +139,7 @@ pseudo_init_wrappers(void) {
- */
- if (!done) {
- for (i = 0; pseudo_functions[i].name; ++i) {
-- if (*pseudo_functions[i].real == NULL) {
-- int (*f)(void);
-- char *e;
-- dlerror();
-- f = dlsym(RTLD_NEXT, pseudo_functions[i].name);
-- if (f) {
-- *pseudo_functions[i].real = f;
-- } else {
-- e = dlerror();
--#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS
-- char *s = pseudo_functions[i].name;
-- s += strlen(s) - 2;
-- /* *at() don't have to exist */
-- if (!strcmp(s, "at")) {
-- continue;
-- }
--#else
-- if (e != NULL) {
-- pseudo_diag("No real function for %s: %s\n", pseudo_functions[i].name, e);
-- }
--#endif
-- }
-- }
-+ pseudo_init_one_wrapper(&pseudo_functions[i]);
- }
- done = 1;
- }
-diff --git a/templates/wrapper_table b/templates/wrapper_table
-index 2e79fcd..bb30530 100644
---- a/templates/wrapper_table
-+++ b/templates/wrapper_table
-@@ -4,17 +4,21 @@
-
- /* This file is generated and should not be modified. See the makewrappers
- * script if you want to modify this. */
--static struct {
-+typedef struct {
- char *name; /* the name */
- int (**real)(void); /* the underlying syscall */
- int (*wrapper)(void); /* the wrapper from guts/name.c */
--} pseudo_functions[] = {
-+ char *version; /* the version, if we know and care */
-+} pseudo_function;
-+
-+static pseudo_function pseudo_functions[] = {
- @body
- { /* ${comment}; */
- "${name}${maybe_inode64}",
- (int (**)(void)) &real_${name},
-- (int (*)(void)) wrap_${name}
-+ (int (*)(void)) wrap_${name},
-+ ${version}
- },
- @footer
-- { NULL, NULL, NULL },
-+ { NULL, NULL, NULL, NULL },
- };
diff --git a/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb b/meta/recipes-devtools/pseudo/pseudo_1.2.bb
similarity index 48%
rename from meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
rename to meta/recipes-devtools/pseudo/pseudo_1.2.bb
index 0a61aec..1d5fe1a 100644
--- a/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_1.2.bb
@@ -1,11 +1,10 @@
require pseudo.inc
-PR = "r2"
+PR = "r3"
SRC_URI = "http://www.yoctoproject.org/downloads/${BPN}/${BPN}-${PV}.tar.bz2 \
file://oe-config.patch \
- file://realpath_fix.patch \
file://static_sqlite.patch"
-SRC_URI[md5sum] = "dd59766c17e199fe6144fce8a2c67802"
-SRC_URI[sha256sum] = "c697f643577d661c3ce826504b9dcd11fa98e78a5d10e3c83931da8942f6bfad"
+SRC_URI[md5sum] = "a2819084bab7e991f06626d02cf55048"
+SRC_URI[sha256sum] = "4749a22df687f44d24c26e97170d4781a1bd52d5ee092364a40877e4d96ff058"
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb b/meta/recipes-devtools/pseudo/pseudo_git.bb
index c1f0432..5ed8cf7 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -1,8 +1,8 @@
require pseudo.inc
-SRCREV = "c2f7c5ad8ef0f9c94a2a8382c109c8c6e16c8b18"
-PV = "1.1.1+git${SRCPV}"
-PR = "r19"
+SRCREV = "17c2233f93692f79684792750001ee6d13e03925"
+PV = "1.2+git${SRCPV}"
+PR = "r20"
DEFAULT_PREFERENCE = "-1"
--
1.7.3.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-02 21:11 [PATCH 0/1] Uprev to pseudo 1.2 Mark Hatle
2011-11-02 21:11 ` [PATCH 1/1] pseudo: Uprev pseudo to version 1.2 Mark Hatle
@ 2011-11-02 21:22 ` Martin Jansa
2011-11-02 21:39 ` Mark Hatle
2011-11-08 22:58 ` Saul Wold
2 siblings, 1 reply; 10+ messages in thread
From: Martin Jansa @ 2011-11-02 21:22 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 2539 bytes --]
On Wed, Nov 02, 2011 at 04:11:57PM -0500, Mark Hatle wrote:
> Uprev to pseudo 1.2.
>
> This was heavily tested by building oe-core with numerous image
> configurations. Each configuration generated the same image before and
> after the change from the current to new version of pseudo.
>
> The primary purpose of this change is to enable the PSEUDO_UNLOAD
> functionality that may be used for performance enhancements in future
> versions of oe-core.
>
> The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
>
> meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
>
> are available in the git repository at:
> git://git.pokylinux.org/poky-contrib mhatle/pseudo
> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
Is there branch with pseudo-1.2 for oe-core?
I would like to try it because we have strange errors when libstc++
isn't found when building with pseudo, but same binaries are working
fine without pseudo (seems related to /etc/ld.so.cache being ignored
sometimes from pseudo env).
And while I was trying to debug it I've noticed that files.db and
pseudo.log is quite big.
-rw-r--r-- 1 bitbake bitbake 240M Aug 15 13:49 files.db
-rw-r--r-- 1 bitbake bitbake 91K Aug 15 13:50 logs.db
-rw-r--r-- 1 bitbake bitbake 305M Aug 15 13:49 pseudo.log
with pseudo.log full of errors like this
pseudo: path mismatch [12 links]: ino 39721500 db
'/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor'
req
'/OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/task-shr-feed-1.0-r96'.
Cheers,
>
> Mark Hatle (1):
> pseudo: Uprev pseudo to version 1.2
>
> .../conf/distro/include/distro_tracking_fields.inc | 6 +-
> .../pseudo/pseudo/realpath_fix.patch | 165 --------------------
> .../pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} | 7 +-
> meta/recipes-devtools/pseudo/pseudo_git.bb | 6 +-
> 4 files changed, 9 insertions(+), 175 deletions(-)
> delete mode 100644 meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
> rename meta/recipes-devtools/pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} (48%)
>
> --
> 1.7.3.4
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-02 21:22 ` [PATCH 0/1] Uprev to pseudo 1.2 Martin Jansa
@ 2011-11-02 21:39 ` Mark Hatle
2011-11-02 22:11 ` Martin Jansa
0 siblings, 1 reply; 10+ messages in thread
From: Mark Hatle @ 2011-11-02 21:39 UTC (permalink / raw)
To: openembedded-core
On 11/2/11 4:22 PM, Martin Jansa wrote:
> On Wed, Nov 02, 2011 at 04:11:57PM -0500, Mark Hatle wrote:
>> Uprev to pseudo 1.2.
>>
>> This was heavily tested by building oe-core with numerous image
>> configurations. Each configuration generated the same image before and
>> after the change from the current to new version of pseudo.
>>
>> The primary purpose of this change is to enable the PSEUDO_UNLOAD
>> functionality that may be used for performance enhancements in future
>> versions of oe-core.
>>
>> The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
>>
>> meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
>>
>> are available in the git repository at:
>> git://git.pokylinux.org/poky-contrib mhatle/pseudo
>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
>
> Is there branch with pseudo-1.2 for oe-core?
^^^^^^^^^^^^^^^^^^^^^^^^
git://git.pokylinux.org/poky-contrib mhatle/pseudo
Grab that -- or simply grab the top commit and add it to your oe-core checkout.
> I would like to try it because we have strange errors when libstc++
> isn't found when building with pseudo, but same binaries are working
> fine without pseudo (seems related to /etc/ld.so.cache being ignored
> sometimes from pseudo env).
>
> And while I was trying to debug it I've noticed that files.db and
> pseudo.log is quite big.
> -rw-r--r-- 1 bitbake bitbake 240M Aug 15 13:49 files.db
> -rw-r--r-- 1 bitbake bitbake 91K Aug 15 13:50 logs.db
> -rw-r--r-- 1 bitbake bitbake 305M Aug 15 13:49 pseudo.log
The files.db is based on the number of files accessed during the time pseudo is
running.
logs.db and pseudo.log are based on the debug settings of pseudo. If you don't
have debugging enabled (you would have to do this manually) and the logs are
that big, something is going seriously wrong with the path/inode translation...
It's fairly normal to have a set of logs that are in the 10-20k range for
complex packages.. (These are documenting cases where pseudo has to guess about
inode to path mapping..)
> with pseudo.log full of errors like this
> pseudo: path mismatch [12 links]: ino 39721500 db
> '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor'
> req
> '/OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/task-shr-feed-1.0-r96'.
Are you building on a NFS filesystem? NFS (and a few other filesystems) have a
habit of remapping inodes out from under the filesystem. pseudo uses the inodes
as a validation that the file hasn't changed. If these inodes are changing,
pseudo will do it's best to keep things in sync, but there is a chance it won't
always succeed.
(This is done because someone could access a file in pseudo control that was
created outside.. so pseudo could only capture the inode.. then if in a future
action the file is accessed the inode/path can be checked. If the inode matches
it assumes it's the same file and now a path name exists.. if someone makes
changes outside of pseudo control a similar check happens.. this time it'll see
the path is the same but inode has changed.. so it guesses the file was modified
and I believe resets everything based on the new inode. Each of these will
generate a warning in the logs.)
--Mark
> Cheers,
>
>
>>
>> Mark Hatle (1):
>> pseudo: Uprev pseudo to version 1.2
>>
>> .../conf/distro/include/distro_tracking_fields.inc | 6 +-
>> .../pseudo/pseudo/realpath_fix.patch | 165 --------------------
>> .../pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} | 7 +-
>> meta/recipes-devtools/pseudo/pseudo_git.bb | 6 +-
>> 4 files changed, 9 insertions(+), 175 deletions(-)
>> delete mode 100644 meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
>> rename meta/recipes-devtools/pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} (48%)
>>
>> --
>> 1.7.3.4
>>
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-02 21:39 ` Mark Hatle
@ 2011-11-02 22:11 ` Martin Jansa
2011-11-02 22:31 ` Mark Hatle
0 siblings, 1 reply; 10+ messages in thread
From: Martin Jansa @ 2011-11-02 22:11 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 4878 bytes --]
On Wed, Nov 02, 2011 at 04:39:42PM -0500, Mark Hatle wrote:
> On 11/2/11 4:22 PM, Martin Jansa wrote:
> > On Wed, Nov 02, 2011 at 04:11:57PM -0500, Mark Hatle wrote:
> >> Uprev to pseudo 1.2.
> >>
> >> This was heavily tested by building oe-core with numerous image
> >> configurations. Each configuration generated the same image before and
> >> after the change from the current to new version of pseudo.
> >>
> >> The primary purpose of this change is to enable the PSEUDO_UNLOAD
> >> functionality that may be used for performance enhancements in future
> >> versions of oe-core.
> >>
> >> The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
> >>
> >> meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
> >>
> >> are available in the git repository at:
> >> git://git.pokylinux.org/poky-contrib mhatle/pseudo
> >> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
> >
> > Is there branch with pseudo-1.2 for oe-core?
>
> ^^^^^^^^^^^^^^^^^^^^^^^^
>
> git://git.pokylinux.org/poky-contrib mhatle/pseudo
>
> Grab that -- or simply grab the top commit and add it to your oe-core checkout.
Yes, but git cherry-pick would be faster and adding poky-contrib as
another git remote is not very efficient as it doesn't share any git
objects with oe-core repo so it checkouts "whole poky" again :/.
So I'll use 2nd easiest option "pw-am.sh 14227".
> > I would like to try it because we have strange errors when libstc++
> > isn't found when building with pseudo, but same binaries are working
> > fine without pseudo (seems related to /etc/ld.so.cache being ignored
> > sometimes from pseudo env).
> >
> > And while I was trying to debug it I've noticed that files.db and
> > pseudo.log is quite big.
> > -rw-r--r-- 1 bitbake bitbake 240M Aug 15 13:49 files.db
> > -rw-r--r-- 1 bitbake bitbake 91K Aug 15 13:50 logs.db
> > -rw-r--r-- 1 bitbake bitbake 305M Aug 15 13:49 pseudo.log
>
> The files.db is based on the number of files accessed during the time pseudo is
> running.
Ah ok, this was in sysroot which was created "Aug 9" so it gets pretty
big fast, lets hope that sqlite3 can handle it efficiently, or is there
some way to "cleanup" stale entries in files table?
ie I'm using eglibc-2.14, but files table is full of eglibc-2.13 entries
and _none_ from eglibc-2.14, is it supposed to be like this or is it
sign of some pseudo problems?
sqlite> select count(*) from files where path like '%eglibc-2.13%';
20221
sqlite> select count(*) from files where path like '%eglibc-2.14%';
0
sqlite> select count(*) from files;
1085380
> logs.db and pseudo.log are based on the debug settings of pseudo. If you don't
> have debugging enabled (you would have to do this manually) and the logs are
> that big, something is going seriously wrong with the path/inode translation...
> It's fairly normal to have a set of logs that are in the 10-20k range for
> complex packages.. (These are documenting cases where pseudo has to guess about
> inode to path mapping..)
This was generated without debug enabled.. but maybe because of rm_work,
see below...
> > with pseudo.log full of errors like this
> > pseudo: path mismatch [12 links]: ino 39721500 db
> > '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor'
> > req
> > '/OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/task-shr-feed-1.0-r96'.
>
> Are you building on a NFS filesystem? NFS (and a few other filesystems) have a
> habit of remapping inodes out from under the filesystem. pseudo uses the inodes
> as a validation that the file hasn't changed. If these inodes are changing,
> pseudo will do it's best to keep things in sync, but there is a chance it won't
> always succeed.
No I'm building on local ext3 filesystem, but I'm also using rm_work, so
/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor
is probably long gone and inode 39721500 wasn't marked as unused in
pseudo world or do I read it wrong?
> (This is done because someone could access a file in pseudo control that was
> created outside.. so pseudo could only capture the inode.. then if in a future
> action the file is accessed the inode/path can be checked. If the inode matches
> it assumes it's the same file and now a path name exists.. if someone makes
> changes outside of pseudo control a similar check happens.. this time it'll see
> the path is the same but inode has changed.. so it guesses the file was modified
> and I believe resets everything based on the new inode. Each of these will
> generate a warning in the logs.)
Thanks for clarifying that.
Cheers,
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-02 22:11 ` Martin Jansa
@ 2011-11-02 22:31 ` Mark Hatle
2011-11-02 22:47 ` Martin Jansa
0 siblings, 1 reply; 10+ messages in thread
From: Mark Hatle @ 2011-11-02 22:31 UTC (permalink / raw)
To: openembedded-core
On 11/2/11 5:11 PM, Martin Jansa wrote:
> On Wed, Nov 02, 2011 at 04:39:42PM -0500, Mark Hatle wrote:
>> On 11/2/11 4:22 PM, Martin Jansa wrote:
>>> On Wed, Nov 02, 2011 at 04:11:57PM -0500, Mark Hatle wrote:
>>>> Uprev to pseudo 1.2.
>>>>
>>>> This was heavily tested by building oe-core with numerous image
>>>> configurations. Each configuration generated the same image before and
>>>> after the change from the current to new version of pseudo.
>>>>
>>>> The primary purpose of this change is to enable the PSEUDO_UNLOAD
>>>> functionality that may be used for performance enhancements in future
>>>> versions of oe-core.
>>>>
>>>> The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
>>>>
>>>> meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
>>>>
>>>> are available in the git repository at:
>>>> git://git.pokylinux.org/poky-contrib mhatle/pseudo
>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
>>>
>>> Is there branch with pseudo-1.2 for oe-core?
>>
>> ^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> git://git.pokylinux.org/poky-contrib mhatle/pseudo
>>
>> Grab that -- or simply grab the top commit and add it to your oe-core checkout.
>
> Yes, but git cherry-pick would be faster and adding poky-contrib as
> another git remote is not very efficient as it doesn't share any git
> objects with oe-core repo so it checkouts "whole poky" again :/.
>
> So I'll use 2nd easiest option "pw-am.sh 14227".
>
>>> I would like to try it because we have strange errors when libstc++
>>> isn't found when building with pseudo, but same binaries are working
>>> fine without pseudo (seems related to /etc/ld.so.cache being ignored
>>> sometimes from pseudo env).
>>>
>>> And while I was trying to debug it I've noticed that files.db and
>>> pseudo.log is quite big.
>>> -rw-r--r-- 1 bitbake bitbake 240M Aug 15 13:49 files.db
>>> -rw-r--r-- 1 bitbake bitbake 91K Aug 15 13:50 logs.db
>>> -rw-r--r-- 1 bitbake bitbake 305M Aug 15 13:49 pseudo.log
>>
>> The files.db is based on the number of files accessed during the time pseudo is
>> running.
>
> Ah ok, this was in sysroot which was created "Aug 9" so it gets pretty
> big fast, lets hope that sqlite3 can handle it efficiently, or is there
> some way to "cleanup" stale entries in files table?
When you clean the build it will restart the pseudo DB.
> ie I'm using eglibc-2.14, but files table is full of eglibc-2.13 entries
> and _none_ from eglibc-2.14, is it supposed to be like this or is it
> sign of some pseudo problems?
>
> sqlite> select count(*) from files where path like '%eglibc-2.13%';
> 20221
> sqlite> select count(*) from files where path like '%eglibc-2.14%';
> 0
> sqlite> select count(*) from files;
> 1085380
The pseudo DB is specific to the tmp work directory. If the work directory
hasn't changed (i.e. you aren't updating versions and pr) then the pseudo DB
will continue to grow to accommodate the new files.
There are flags within the DB if an erase operation occurs, but entries are only
tagged as erased -- but not removed. This is due to speed issues. pseudo is
significantly slower on remove and replace operations if the DB entries are removed.
>> logs.db and pseudo.log are based on the debug settings of pseudo. If you don't
>> have debugging enabled (you would have to do this manually) and the logs are
>> that big, something is going seriously wrong with the path/inode translation...
>> It's fairly normal to have a set of logs that are in the 10-20k range for
>> complex packages.. (These are documenting cases where pseudo has to guess about
>> inode to path mapping..)
>
> This was generated without debug enabled.. but maybe because of rm_work,
> see below...
>
>>> with pseudo.log full of errors like this
>>> pseudo: path mismatch [12 links]: ino 39721500 db
>>> '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor'
>>> req
>>> '/OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/task-shr-feed-1.0-r96'.
>>
>> Are you building on a NFS filesystem? NFS (and a few other filesystems) have a
>> habit of remapping inodes out from under the filesystem. pseudo uses the inodes
>> as a validation that the file hasn't changed. If these inodes are changing,
>> pseudo will do it's best to keep things in sync, but there is a chance it won't
>> always succeed.
>
> No I'm building on local ext3 filesystem, but I'm also using rm_work, so
> /OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor
> is probably long gone and inode 39721500 wasn't marked as unused in
> pseudo world or do I read it wrong?
If you use rm_work, then the pseudo DB located within the work directory will
also be removed. So I'm not sure why you would be having such a large DB.
>> (This is done because someone could access a file in pseudo control that was
>> created outside.. so pseudo could only capture the inode.. then if in a future
>> action the file is accessed the inode/path can be checked. If the inode matches
>> it assumes it's the same file and now a path name exists.. if someone makes
>> changes outside of pseudo control a similar check happens.. this time it'll see
>> the path is the same but inode has changed.. so it guesses the file was modified
>> and I believe resets everything based on the new inode. Each of these will
>> generate a warning in the logs.)
>
> Thanks for clarifying that.
>
> Cheers,
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-02 22:31 ` Mark Hatle
@ 2011-11-02 22:47 ` Martin Jansa
2011-11-03 1:52 ` Mark Hatle
0 siblings, 1 reply; 10+ messages in thread
From: Martin Jansa @ 2011-11-02 22:47 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
[-- Attachment #1: Type: text/plain, Size: 6427 bytes --]
On Wed, Nov 02, 2011 at 05:31:59PM -0500, Mark Hatle wrote:
> On 11/2/11 5:11 PM, Martin Jansa wrote:
> > On Wed, Nov 02, 2011 at 04:39:42PM -0500, Mark Hatle wrote:
> >> On 11/2/11 4:22 PM, Martin Jansa wrote:
> >>> On Wed, Nov 02, 2011 at 04:11:57PM -0500, Mark Hatle wrote:
> >>>> Uprev to pseudo 1.2.
> >>>>
> >>>> This was heavily tested by building oe-core with numerous image
> >>>> configurations. Each configuration generated the same image before and
> >>>> after the change from the current to new version of pseudo.
> >>>>
> >>>> The primary purpose of this change is to enable the PSEUDO_UNLOAD
> >>>> functionality that may be used for performance enhancements in future
> >>>> versions of oe-core.
> >>>>
> >>>> The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
> >>>>
> >>>> meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
> >>>>
> >>>> are available in the git repository at:
> >>>> git://git.pokylinux.org/poky-contrib mhatle/pseudo
> >>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
> >>>
> >>> Is there branch with pseudo-1.2 for oe-core?
> >>
> >> ^^^^^^^^^^^^^^^^^^^^^^^^
> >>
> >> git://git.pokylinux.org/poky-contrib mhatle/pseudo
> >>
> >> Grab that -- or simply grab the top commit and add it to your oe-core checkout.
> >
> > Yes, but git cherry-pick would be faster and adding poky-contrib as
> > another git remote is not very efficient as it doesn't share any git
> > objects with oe-core repo so it checkouts "whole poky" again :/.
> >
> > So I'll use 2nd easiest option "pw-am.sh 14227".
> >
> >>> I would like to try it because we have strange errors when libstc++
> >>> isn't found when building with pseudo, but same binaries are working
> >>> fine without pseudo (seems related to /etc/ld.so.cache being ignored
> >>> sometimes from pseudo env).
> >>>
> >>> And while I was trying to debug it I've noticed that files.db and
> >>> pseudo.log is quite big.
> >>> -rw-r--r-- 1 bitbake bitbake 240M Aug 15 13:49 files.db
> >>> -rw-r--r-- 1 bitbake bitbake 91K Aug 15 13:50 logs.db
> >>> -rw-r--r-- 1 bitbake bitbake 305M Aug 15 13:49 pseudo.log
> >>
> >> The files.db is based on the number of files accessed during the time pseudo is
> >> running.
> >
> > Ah ok, this was in sysroot which was created "Aug 9" so it gets pretty
> > big fast, lets hope that sqlite3 can handle it efficiently, or is there
> > some way to "cleanup" stale entries in files table?
>
> When you clean the build it will restart the pseudo DB.
>
> > ie I'm using eglibc-2.14, but files table is full of eglibc-2.13 entries
> > and _none_ from eglibc-2.14, is it supposed to be like this or is it
> > sign of some pseudo problems?
> >
> > sqlite> select count(*) from files where path like '%eglibc-2.13%';
> > 20221
> > sqlite> select count(*) from files where path like '%eglibc-2.14%';
> > 0
> > sqlite> select count(*) from files;
> > 1085380
>
> The pseudo DB is specific to the tmp work directory. If the work directory
> hasn't changed (i.e. you aren't updating versions and pr) then the pseudo DB
> will continue to grow to accommodate the new files.
>
> There are flags within the DB if an erase operation occurs, but entries are only
> tagged as erased -- but not removed. This is due to speed issues. pseudo is
> significantly slower on remove and replace operations if the DB entries are removed.
>
> >> logs.db and pseudo.log are based on the debug settings of pseudo. If you don't
> >> have debugging enabled (you would have to do this manually) and the logs are
> >> that big, something is going seriously wrong with the path/inode translation...
> >> It's fairly normal to have a set of logs that are in the 10-20k range for
> >> complex packages.. (These are documenting cases where pseudo has to guess about
> >> inode to path mapping..)
> >
> > This was generated without debug enabled.. but maybe because of rm_work,
> > see below...
> >
> >>> with pseudo.log full of errors like this
> >>> pseudo: path mismatch [12 links]: ino 39721500 db
> >>> '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor'
> >>> req
> >>> '/OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/task-shr-feed-1.0-r96'.
> >>
> >> Are you building on a NFS filesystem? NFS (and a few other filesystems) have a
> >> habit of remapping inodes out from under the filesystem. pseudo uses the inodes
> >> as a validation that the file hasn't changed. If these inodes are changing,
> >> pseudo will do it's best to keep things in sync, but there is a chance it won't
> >> always succeed.
> >
> > No I'm building on local ext3 filesystem, but I'm also using rm_work, so
> > /OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor
> > is probably long gone and inode 39721500 wasn't marked as unused in
> > pseudo world or do I read it wrong?
>
> If you use rm_work, then the pseudo DB located within the work directory will
> also be removed. So I'm not sure why you would be having such a large DB.
But I was talking about pseudo db here:
tmp/sysroots/x86_64-linux/var/pseudo/
now with pseudo-1.2 I still have quite a few warnings about inodes in
pseudo.log now in work directory, ie python build:
OE @ ~/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1 $
-rw-r--r-- 1 bitbake bitbake 5.2M Nov 2 23:44 files.db
-rw-r--r-- 1 bitbake bitbake 4.0K Nov 2 23:44 logs.db
-rw-r--r-- 1 bitbake bitbake 46K Nov 2 23:44 pseudo.log
pseudo: dir err : 1240102 ['/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/deploy-ipks/armv4t/IPKG_BUILD.25064'] (db 'no path') db mode 0100644, header mode 040755 (unlinking db)
pseudo: creat for '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/deploy-ipks/armv4t/stvITxha' replaces existing 1240107 ['no path'].
wxr-xr-x 1 bitbake bitbake 0 Nov 2 23:44 pseudo.socket
pseudo: path mismatch [2 links]: ino 1240673 db '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/sstate-build-package/package/usr/bin/python2.7' req '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/sstate-build-package/package/usr/bin/python'.
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-02 22:47 ` Martin Jansa
@ 2011-11-03 1:52 ` Mark Hatle
2011-11-03 12:47 ` Richard Purdie
0 siblings, 1 reply; 10+ messages in thread
From: Mark Hatle @ 2011-11-03 1:52 UTC (permalink / raw)
To: openembedded-core
On 11/2/11 5:47 PM, Martin Jansa wrote:
> On Wed, Nov 02, 2011 at 05:31:59PM -0500, Mark Hatle wrote:
>> On 11/2/11 5:11 PM, Martin Jansa wrote:
>>> On Wed, Nov 02, 2011 at 04:39:42PM -0500, Mark Hatle wrote:
>>>> On 11/2/11 4:22 PM, Martin Jansa wrote:
>>>>> On Wed, Nov 02, 2011 at 04:11:57PM -0500, Mark Hatle wrote:
>>>>>> Uprev to pseudo 1.2.
>>>>>>
>>>>>> This was heavily tested by building oe-core with numerous image
>>>>>> configurations. Each configuration generated the same image before and
>>>>>> after the change from the current to new version of pseudo.
>>>>>>
>>>>>> The primary purpose of this change is to enable the PSEUDO_UNLOAD
>>>>>> functionality that may be used for performance enhancements in future
>>>>>> versions of oe-core.
>>>>>>
>>>>>> The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
>>>>>>
>>>>>> meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
>>>>>>
>>>>>> are available in the git repository at:
>>>>>> git://git.pokylinux.org/poky-contrib mhatle/pseudo
>>>>>> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
>>>>>
>>>>> Is there branch with pseudo-1.2 for oe-core?
>>>>
>>>> ^^^^^^^^^^^^^^^^^^^^^^^^
>>>>
>>>> git://git.pokylinux.org/poky-contrib mhatle/pseudo
>>>>
>>>> Grab that -- or simply grab the top commit and add it to your oe-core checkout.
>>>
>>> Yes, but git cherry-pick would be faster and adding poky-contrib as
>>> another git remote is not very efficient as it doesn't share any git
>>> objects with oe-core repo so it checkouts "whole poky" again :/.
>>>
>>> So I'll use 2nd easiest option "pw-am.sh 14227".
>>>
>>>>> I would like to try it because we have strange errors when libstc++
>>>>> isn't found when building with pseudo, but same binaries are working
>>>>> fine without pseudo (seems related to /etc/ld.so.cache being ignored
>>>>> sometimes from pseudo env).
>>>>>
>>>>> And while I was trying to debug it I've noticed that files.db and
>>>>> pseudo.log is quite big.
>>>>> -rw-r--r-- 1 bitbake bitbake 240M Aug 15 13:49 files.db
>>>>> -rw-r--r-- 1 bitbake bitbake 91K Aug 15 13:50 logs.db
>>>>> -rw-r--r-- 1 bitbake bitbake 305M Aug 15 13:49 pseudo.log
>>>>
>>>> The files.db is based on the number of files accessed during the time pseudo is
>>>> running.
>>>
>>> Ah ok, this was in sysroot which was created "Aug 9" so it gets pretty
>>> big fast, lets hope that sqlite3 can handle it efficiently, or is there
>>> some way to "cleanup" stale entries in files table?
>>
>> When you clean the build it will restart the pseudo DB.
>>
>>> ie I'm using eglibc-2.14, but files table is full of eglibc-2.13 entries
>>> and _none_ from eglibc-2.14, is it supposed to be like this or is it
>>> sign of some pseudo problems?
>>>
>>> sqlite> select count(*) from files where path like '%eglibc-2.13%';
>>> 20221
>>> sqlite> select count(*) from files where path like '%eglibc-2.14%';
>>> 0
>>> sqlite> select count(*) from files;
>>> 1085380
>>
>> The pseudo DB is specific to the tmp work directory. If the work directory
>> hasn't changed (i.e. you aren't updating versions and pr) then the pseudo DB
>> will continue to grow to accommodate the new files.
>>
>> There are flags within the DB if an erase operation occurs, but entries are only
>> tagged as erased -- but not removed. This is due to speed issues. pseudo is
>> significantly slower on remove and replace operations if the DB entries are removed.
>>
>>>> logs.db and pseudo.log are based on the debug settings of pseudo. If you don't
>>>> have debugging enabled (you would have to do this manually) and the logs are
>>>> that big, something is going seriously wrong with the path/inode translation...
>>>> It's fairly normal to have a set of logs that are in the 10-20k range for
>>>> complex packages.. (These are documenting cases where pseudo has to guess about
>>>> inode to path mapping..)
>>>
>>> This was generated without debug enabled.. but maybe because of rm_work,
>>> see below...
>>>
>>>>> with pseudo.log full of errors like this
>>>>> pseudo: path mismatch [12 links]: ino 39721500 db
>>>>> '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor'
>>>>> req
>>>>> '/OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/task-shr-feed-1.0-r96'.
>>>>
>>>> Are you building on a NFS filesystem? NFS (and a few other filesystems) have a
>>>> habit of remapping inodes out from under the filesystem. pseudo uses the inodes
>>>> as a validation that the file hasn't changed. If these inodes are changing,
>>>> pseudo will do it's best to keep things in sync, but there is a chance it won't
>>>> always succeed.
>>>
>>> No I'm building on local ext3 filesystem, but I'm also using rm_work, so
>>> /OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor
>>> is probably long gone and inode 39721500 wasn't marked as unused in
>>> pseudo world or do I read it wrong?
>>
>> If you use rm_work, then the pseudo DB located within the work directory will
>> also be removed. So I'm not sure why you would be having such a large DB.
>
> But I was talking about pseudo db here:
> tmp/sysroots/x86_64-linux/var/pseudo/
I just checked my system and the logs in the sysroot isn't very large. I'm
curious as to why things are as big on your system.
> now with pseudo-1.2 I still have quite a few warnings about inodes in
> pseudo.log now in work directory, ie python build:
> OE @ ~/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1 $
> -rw-r--r-- 1 bitbake bitbake 5.2M Nov 2 23:44 files.db
> -rw-r--r-- 1 bitbake bitbake 4.0K Nov 2 23:44 logs.db
> -rw-r--r-- 1 bitbake bitbake 46K Nov 2 23:44 pseudo.log
>
> pseudo: dir err : 1240102 ['/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/deploy-ipks/armv4t/IPKG_BUILD.25064'] (db 'no path') db mode 0100644, header mode 040755 (unlinking db)
> pseudo: creat for '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/deploy-ipks/armv4t/stvITxha' replaces existing 1240107 ['no path'].
> wxr-xr-x 1 bitbake bitbake 0 Nov 2 23:44 pseudo.socket
> pseudo: path mismatch [2 links]: ino 1240673 db '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/sstate-build-package/package/usr/bin/python2.7' req '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/python-2.7.2-r0.1/sstate-build-package/package/usr/bin/python'.
None of the inode processing was changed in 1.2, only code that was specific to
pseudo preloading and the clone(2) function wrapper.
The 'no path' messages.. (see the first and second ones above) are the ones
where a file and/or directory was created outside of pseudo, and then referenced
by the inode directly. Once referenced by name and not inode it was able to
correlate them. This is somewhat expected and a model that will be looked at in
the near? future by the pseudo maintainer.. (he and I talked about it.. and the
fact we're getting so many of those messages is surprising.)
The later one indicates that the inode 1240673 was defined as belonging to one
item, but a second item showed up instead. This usually means that something
was renamed outside of the control of pseudo. This is more likely to be a
problem then the first two.
--Mark
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-03 1:52 ` Mark Hatle
@ 2011-11-03 12:47 ` Richard Purdie
0 siblings, 0 replies; 10+ messages in thread
From: Richard Purdie @ 2011-11-03 12:47 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
On Wed, 2011-11-02 at 20:52 -0500, Mark Hatle wrote:
> On 11/2/11 5:47 PM, Martin Jansa wrote:
> > On Wed, Nov 02, 2011 at 05:31:59PM -0500, Mark Hatle wrote:
> >> On 11/2/11 5:11 PM, Martin Jansa wrote:
> >>> On Wed, Nov 02, 2011 at 04:39:42PM -0500, Mark Hatle wrote:
> >>>>> with pseudo.log full of errors like this
> >>>>> pseudo: path mismatch [12 links]: ino 39721500 db
> >>>>> '/OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor'
> >>>>> req
> >>>>> '/OE/shr-core/tmp/work/om_gta02-oe-linux-gnueabi/task-shr-feed-1.0-r96'.
> >>>>
> >>>> Are you building on a NFS filesystem? NFS (and a few other filesystems) have a
> >>>> habit of remapping inodes out from under the filesystem. pseudo uses the inodes
> >>>> as a validation that the file hasn't changed. If these inodes are changing,
> >>>> pseudo will do it's best to keep things in sync, but there is a chance it won't
> >>>> always succeed.
> >>>
> >>> No I'm building on local ext3 filesystem, but I'm also using rm_work, so
> >>> /OE/shr-core/tmp/work/armv4t-oe-linux-gnueabi/vim-7.2.446-r10.3/image/usr/share/vim/vim72/tutor
> >>> is probably long gone and inode 39721500 wasn't marked as unused in
> >>> pseudo world or do I read it wrong?
> >>
> >> If you use rm_work, then the pseudo DB located within the work directory will
> >> also be removed. So I'm not sure why you would be having such a large DB.
> >
> > But I was talking about pseudo db here:
> > tmp/sysroots/x86_64-linux/var/pseudo/
>
> I just checked my system and the logs in the sysroot isn't very large. I'm
> curious as to why things are as big on your system.
We did have a problem a while back where pseudo was active when it
shouldn't have been and I noticed huge db/logs on my system too. The
environment changes that were pushed shortly before 1.1 fixed that
problem as far as I know but if this is an old build directory, it could
have those left behind.
This actually accounted for the significant load time bitbake had on one
of my systems too - it was spending the time loading the pseudo db.
Cheers,
Richard
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 0/1] Uprev to pseudo 1.2
2011-11-02 21:11 [PATCH 0/1] Uprev to pseudo 1.2 Mark Hatle
2011-11-02 21:11 ` [PATCH 1/1] pseudo: Uprev pseudo to version 1.2 Mark Hatle
2011-11-02 21:22 ` [PATCH 0/1] Uprev to pseudo 1.2 Martin Jansa
@ 2011-11-08 22:58 ` Saul Wold
2 siblings, 0 replies; 10+ messages in thread
From: Saul Wold @ 2011-11-08 22:58 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
On 11/02/2011 02:11 PM, Mark Hatle wrote:
> Uprev to pseudo 1.2.
>
> This was heavily tested by building oe-core with numerous image
> configurations. Each configuration generated the same image before and
> after the change from the current to new version of pseudo.
>
> The primary purpose of this change is to enable the PSEUDO_UNLOAD
> functionality that may be used for performance enhancements in future
> versions of oe-core.
>
> The following changes since commit 37579d7d74d127c90c1e078d05c5bf4ba0b3f755:
>
> meta: glib-2.0: don't apply qsort_r test removable patch for native version (2011-11-02 09:08:21 +0000)
>
> are available in the git repository at:
> git://git.pokylinux.org/poky-contrib mhatle/pseudo
> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/pseudo
>
> Mark Hatle (1):
> pseudo: Uprev pseudo to version 1.2
>
> .../conf/distro/include/distro_tracking_fields.inc | 6 +-
> .../pseudo/pseudo/realpath_fix.patch | 165 --------------------
> .../pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} | 7 +-
> meta/recipes-devtools/pseudo/pseudo_git.bb | 6 +-
> 4 files changed, 9 insertions(+), 175 deletions(-)
> delete mode 100644 meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
> rename meta/recipes-devtools/pseudo/{pseudo_1.1.1.bb => pseudo_1.2.bb} (48%)
>
Merged to OE-Core
Thanks
Sau!
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2011-11-08 23:05 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-11-02 21:11 [PATCH 0/1] Uprev to pseudo 1.2 Mark Hatle
2011-11-02 21:11 ` [PATCH 1/1] pseudo: Uprev pseudo to version 1.2 Mark Hatle
2011-11-02 21:22 ` [PATCH 0/1] Uprev to pseudo 1.2 Martin Jansa
2011-11-02 21:39 ` Mark Hatle
2011-11-02 22:11 ` Martin Jansa
2011-11-02 22:31 ` Mark Hatle
2011-11-02 22:47 ` Martin Jansa
2011-11-03 1:52 ` Mark Hatle
2011-11-03 12:47 ` Richard Purdie
2011-11-08 22:58 ` Saul Wold
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox