* [PATCH 0/1] persist_data: reconnect when DB is locked
@ 2012-01-16 4:07 Lianhao Lu
2012-01-16 4:07 ` [PATCH 1/1] bitbake/persist_data: Reconnect " Lianhao Lu
0 siblings, 1 reply; 6+ messages in thread
From: Lianhao Lu @ 2012-01-16 4:07 UTC (permalink / raw)
To: bitbake-devel
Fixed yocto bug #1761.
The timeout value set in sqlite3 connection only applies to the very first
time when accessing a locked DB file.
By reconnect to the DB in 'database is locked' exception, we can leverage
the timeout value in each retry.
The following changes since commit 684c6ef2d08ad503ce68ac3cc273f1b6a8a56ac7:
Paul Eggleton (1):
bitbake/knotty: print task failure summary
are available in the git repository at:
git://git.pokylinux.org/poky-contrib llu/bug1761
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/bug1761
Lianhao Lu (1):
bitbake/persist_data: Reconnect when DB is locked
bitbake/lib/bb/persist_data.py | 12 +++++++-----
1 files changed, 7 insertions(+), 5 deletions(-)
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 1/1] bitbake/persist_data: Reconnect when DB is locked 2012-01-16 4:07 [PATCH 0/1] persist_data: reconnect when DB is locked Lianhao Lu @ 2012-01-16 4:07 ` Lianhao Lu 2012-01-19 0:16 ` Richard Purdie 2012-05-31 0:33 ` McClintock Matthew-B29882 0 siblings, 2 replies; 6+ messages in thread From: Lianhao Lu @ 2012-01-16 4:07 UTC (permalink / raw) To: bitbake-devel [YOCTO #1761] Reconnect to the backend Sqlite DB in 'database is locked' exception so the timeout can be leveraged in each time retry. Signed-off-by: Lianhao Lu <lianhao.lu@intel.com> --- bitbake/lib/bb/persist_data.py | 12 +++++++----- 1 files changed, 7 insertions(+), 5 deletions(-) diff --git a/bitbake/lib/bb/persist_data.py b/bitbake/lib/bb/persist_data.py index 17620ef..c4ea23b 100644 --- a/bitbake/lib/bb/persist_data.py +++ b/bitbake/lib/bb/persist_data.py @@ -47,9 +47,10 @@ if hasattr(sqlite3, 'enable_shared_cache'): @total_ordering class SQLTable(collections.MutableMapping): """Object representing a table/domain in the database""" - def __init__(self, cursor, table): - self.cursor = cursor + def __init__(self, cachefile, table): + self.cachefile = cachefile self.table = table + self.cursor = connect(self.cachefile) self._execute("CREATE TABLE IF NOT EXISTS %s(key TEXT, value TEXT);" % table) @@ -63,6 +64,8 @@ class SQLTable(collections.MutableMapping): except sqlite3.OperationalError as exc: if 'database is locked' in str(exc) and count < 500: count = count + 1 + self.cursor.close() + self.cursor = connect(self.cachefile) continue raise @@ -188,7 +191,7 @@ class PersistData(object): del self.data[domain][key] def connect(database): - return sqlite3.connect(database, timeout=30, isolation_level=None) + return sqlite3.connect(database, timeout=5, isolation_level=None) def persist(domain, d): """Convenience factory for SQLTable objects based upon metadata""" @@ -201,5 +204,4 @@ def persist(domain, d): bb.utils.mkdirhier(cachedir) cachefile = os.path.join(cachedir, "bb_persist_data.sqlite3") - connection = connect(cachefile) - return SQLTable(connection, domain) + return SQLTable(cachefile, domain) -- 1.7.0.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] bitbake/persist_data: Reconnect when DB is locked 2012-01-16 4:07 ` [PATCH 1/1] bitbake/persist_data: Reconnect " Lianhao Lu @ 2012-01-19 0:16 ` Richard Purdie 2012-05-31 0:33 ` McClintock Matthew-B29882 1 sibling, 0 replies; 6+ messages in thread From: Richard Purdie @ 2012-01-19 0:16 UTC (permalink / raw) To: Lianhao Lu; +Cc: bitbake-devel On Mon, 2012-01-16 at 12:07 +0800, Lianhao Lu wrote: > [YOCTO #1761] > Reconnect to the backend Sqlite DB in 'database is locked' exception so > the timeout can be leveraged in each time retry. > > Signed-off-by: Lianhao Lu <lianhao.lu@intel.com> > --- > bitbake/lib/bb/persist_data.py | 12 +++++++----- > 1 files changed, 7 insertions(+), 5 deletions(-) Merged to master, thanks. Good to finally have a handle on why this kept coming up! :) Cheers, Richard ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] bitbake/persist_data: Reconnect when DB is locked 2012-01-16 4:07 ` [PATCH 1/1] bitbake/persist_data: Reconnect " Lianhao Lu 2012-01-19 0:16 ` Richard Purdie @ 2012-05-31 0:33 ` McClintock Matthew-B29882 2012-05-31 17:54 ` Joshua Lock 1 sibling, 1 reply; 6+ messages in thread From: McClintock Matthew-B29882 @ 2012-05-31 0:33 UTC (permalink / raw) To: Lianhao Lu; +Cc: bitbake-devel@lists.openembedded.org This needs to be in edison-next. It cherry-pick's cleanly as an added plus (doing more testing now). Seems like it's already in denzil-next -M On Sun, Jan 15, 2012 at 10:07 PM, Lianhao Lu <lianhao.lu@intel.com> wrote: > [YOCTO #1761] > Reconnect to the backend Sqlite DB in 'database is locked' exception so > the timeout can be leveraged in each time retry. > > Signed-off-by: Lianhao Lu <lianhao.lu@intel.com> > --- > bitbake/lib/bb/persist_data.py | 12 +++++++----- > 1 files changed, 7 insertions(+), 5 deletions(-) > > diff --git a/bitbake/lib/bb/persist_data.py b/bitbake/lib/bb/persist_data.py > index 17620ef..c4ea23b 100644 > --- a/bitbake/lib/bb/persist_data.py > +++ b/bitbake/lib/bb/persist_data.py > @@ -47,9 +47,10 @@ if hasattr(sqlite3, 'enable_shared_cache'): > @total_ordering > class SQLTable(collections.MutableMapping): > """Object representing a table/domain in the database""" > - def __init__(self, cursor, table): > - self.cursor = cursor > + def __init__(self, cachefile, table): > + self.cachefile = cachefile > self.table = table > + self.cursor = connect(self.cachefile) > > self._execute("CREATE TABLE IF NOT EXISTS %s(key TEXT, value TEXT);" > % table) > @@ -63,6 +64,8 @@ class SQLTable(collections.MutableMapping): > except sqlite3.OperationalError as exc: > if 'database is locked' in str(exc) and count < 500: > count = count + 1 > + self.cursor.close() > + self.cursor = connect(self.cachefile) > continue > raise > > @@ -188,7 +191,7 @@ class PersistData(object): > del self.data[domain][key] > > def connect(database): > - return sqlite3.connect(database, timeout=30, isolation_level=None) > + return sqlite3.connect(database, timeout=5, isolation_level=None) > > def persist(domain, d): > """Convenience factory for SQLTable objects based upon metadata""" > @@ -201,5 +204,4 @@ def persist(domain, d): > > bb.utils.mkdirhier(cachedir) > cachefile = os.path.join(cachedir, "bb_persist_data.sqlite3") > - connection = connect(cachefile) > - return SQLTable(connection, domain) > + return SQLTable(cachefile, domain) > -- > 1.7.0.4 > > > _______________________________________________ > bitbake-devel mailing list > bitbake-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/bitbake-devel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] bitbake/persist_data: Reconnect when DB is locked 2012-05-31 0:33 ` McClintock Matthew-B29882 @ 2012-05-31 17:54 ` Joshua Lock 2012-05-31 19:45 ` McClintock Matthew-B29882 0 siblings, 1 reply; 6+ messages in thread From: Joshua Lock @ 2012-05-31 17:54 UTC (permalink / raw) To: McClintock Matthew-B29882; +Cc: bitbake-devel@lists.openembedded.org On 30/05/12 17:33, McClintock Matthew-B29882 wrote: > This needs to be in edison-next. It cherry-pick's cleanly as an added > plus (doing more testing now). Seems like it's already in denzil-next cherry-picked to my josh/edison branch on poky-contrib - I will try and ensure it makes it into the edison branch of the main repository promptly. Cheers, Joshua -- Joshua Lock Yocto Project Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] bitbake/persist_data: Reconnect when DB is locked 2012-05-31 17:54 ` Joshua Lock @ 2012-05-31 19:45 ` McClintock Matthew-B29882 0 siblings, 0 replies; 6+ messages in thread From: McClintock Matthew-B29882 @ 2012-05-31 19:45 UTC (permalink / raw) To: Joshua Lock Cc: McClintock Matthew-B29882, bitbake-devel@lists.openembedded.org On Thu, May 31, 2012 at 12:54 PM, Joshua Lock <josh@linux.intel.com> wrote: > On 30/05/12 17:33, McClintock Matthew-B29882 wrote: >> >> This needs to be in edison-next. It cherry-pick's cleanly as an added >> plus (doing more testing now). Seems like it's already in denzil-next > > > cherry-picked to my josh/edison branch on poky-contrib - I will try and > ensure it makes it into the edison branch of the main repository promptly. This has resolved my issue and I've seen no new issues after quite a bit of building. -M ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-05-31 20:11 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-01-16 4:07 [PATCH 0/1] persist_data: reconnect when DB is locked Lianhao Lu 2012-01-16 4:07 ` [PATCH 1/1] bitbake/persist_data: Reconnect " Lianhao Lu 2012-01-19 0:16 ` Richard Purdie 2012-05-31 0:33 ` McClintock Matthew-B29882 2012-05-31 17:54 ` Joshua Lock 2012-05-31 19:45 ` McClintock Matthew-B29882
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.