xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Ian Jackson <ian.jackson@eu.citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>,
	Ian Campbell <ian.campbell@citrix.com>
Subject: [OSSTEST PATCH 6/7] Database locking: Tcl: Retry only on DEADLOCK DETECTED
Date: Thu, 7 Jan 2016 19:38:15 +0000	[thread overview]
Message-ID: <1452195496-16016-7-git-send-email-ian.jackson@eu.citrix.com> (raw)
In-Reply-To: <1452195496-16016-1-git-send-email-ian.jackson@eu.citrix.com>

Use the new errorCode coming out of db-execute* to tell when the error
is that we got a database deadlock, which is the situation in which we
should retry.

This involves combining the two catch blocks, so that there is only
one error handling strategy.  Previously errors on COMMIT would be
retried and others would not.  Now errors anywhere might be retried
but only if the DB indicated deadlock.

We now unconditionally execute ROLLBACK.  This is more correct, since
we always previously executed BEGIN.

And, we pass the errorInfo and errorCode from the $body to the caller.

I have tested this with a test db instance, using contrived means to
generate a database deadlock, and it does actually retry.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 tcl/JobDB-Executive.tcl |   30 ++++++++++++++++--------------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl
index ed9abbb..63db4f0 100644
--- a/tcl/JobDB-Executive.tcl
+++ b/tcl/JobDB-Executive.tcl
@@ -283,25 +283,27 @@ proc transaction {tables script} {
 	    db-execute "SET TRANSACTION ISOLATION LEVEL SERIALIZABLE"
 	    lock-tables $tables
 	    uplevel 1 $script
+	    db-execute COMMIT
 	} result]
-	if {!$rc} {
-	    if {[catch {
-		db-execute COMMIT
-	    } emsg]} {
-		puts "commit failed: $emsg; retrying ..."
-		db-execute ROLLBACK
-		if {[incr retries -1] <= 0} {
-		    error \
- "commit failed, too many retries: $emsg\n$errorInfo\n$errorCode\n"
+	set ei $errorInfo
+	set ec $errorCode
+	if {$rc} {
+	    db-execute ROLLBACK
+	    switch -glob $errorCode {
+		{OSSTEST-PSQL * 40P01} {
+		    # DEADLOCK DETECTED
+		    puts "transaction deadlock ($result) retrying ..."
+		    if {[incr retries -1] <= 0} {
+			error \
+ "transaction failed, too many retries: $result\n$errorInfo\n$errorCode\n"
+		    }
+		    after 500
+		    continue
 		}
-		after 500
-		continue
 	    }
-	} else {
-	    db-execute ROLLBACK
 	}
         db-close
-	return -code $rc $result
+	return -code $rc -errorinfo $ei -errorcode $ec $result
     }
 }
 
-- 
1.7.10.4

  parent reply	other threads:[~2016-01-07 19:38 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-01-07 19:38 [OSSTEST PATCH 0/7] Better database handling in Tcl Ian Jackson
2016-01-07 19:38 ` [OSSTEST PATCH 1/7] Tcl database debugging: Actually work Ian Jackson
2016-01-07 19:38 ` [OSSTEST PATCH 2/7] Database locking: Tcl: Use db-execute-array Ian Jackson
2016-01-07 19:38 ` [OSSTEST PATCH 3/7] Database locking: Tcl: Use db-execute Ian Jackson
2016-01-08  9:28   ` Ian Campbell
2016-01-12 15:38     ` Ian Jackson
2016-01-07 19:38 ` [OSSTEST PATCH 4/7] Database locking: Tcl: Always use db-execute-array for SELECT Ian Jackson
2016-01-07 19:38 ` [OSSTEST PATCH 5/7] Database locking: Tcl: for errorCode, use pg_exec, not pg_execute Ian Jackson
2016-01-08  9:32   ` Ian Campbell
2016-01-12 15:39     ` Ian Jackson
2016-01-14 10:33       ` Ian Campbell
2016-01-07 19:38 ` Ian Jackson [this message]
2016-01-07 19:38 ` [OSSTEST PATCH 7/7] ms-ownerdaemon: Cope with db restart. Retry recording dead tasks Ian Jackson
2016-01-08  9:40 ` [OSSTEST PATCH 0/7] Better database handling in Tcl Ian Campbell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1452195496-16016-7-git-send-email-ian.jackson@eu.citrix.com \
    --to=ian.jackson@eu.citrix.com \
    --cc=ian.campbell@citrix.com \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).