From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hitoshi Mitake Subject: Re: [sheepdog] [PATCH] bs_sheepdog.c: fix io errors during writing to a snapshot Date: Tue, 07 Jan 2014 22:23:00 +0900 Message-ID: <87ppo4c4ej.wl%mitake.hitoshi@gmail.com> References: <1389099571-10567-1-git-send-email-konishi.ryusuke@lab.ntt.co.jp> Mime-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:message-id:from:to:cc:subject:in-reply-to:references :user-agent:mime-version:content-type; bh=QevI5aDkAbmL/DNnywXkcXGbDBQneyLwndwLckRkDzk=; b=Uye1dF2Pcb6vsG0dEHu7x+VGf609Xg7zD6RRuUUsnb+b7Kf6EuuiWWsBOASYFPxajZ p+gvkhd6z7lP7ulKZf7Hi3+/vpWEPqZc6By/Fwazp+KsZVYzFnFEfmeaWPQgTwzeggt2 QRtth9wJuUzftqb2LbLM3+KDWJID08zW6/IlO2GGjBQamM8jZiSVET85Sz7dyfrv9J+H RAG01jJriwKg++gcJX/yfp9rFu94bbLIbGj0J6AaC+mtNvZ2BYnPFr/RU5Wji7gqZZIo vkuwXsgTzvDJbuUsqCbYiCWAwDpMcFbKhbiU/Wa4U/5I5lJkguDWdBT8b7g+M8ukv6wN 3CEA== In-Reply-To: <1389099571-10567-1-git-send-email-konishi.ryusuke@lab.ntt.co.jp> Sender: stgt-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Ryusuke Konishi Cc: stgt@vger.kernel.org, Hitoshi Mitake , sheepdog@lists.wpkg.org At Tue, 7 Jan 2014 21:59:31 +0900, Ryusuke Konishi wrote: > > The sheepdog driver fails with io errors when write access is > requested for a snapshot vdi. The failure happens in create_branch > function: > > tgtd: read_write_object(684) No object found (oid: 8000000 > 000000000, old_oid: 0) > tgtd: create_branch(1160) reloading new inode object failed > tgtd: bs_sheepdog_request(1197) creating writable VDI from > snapshot failed > > sd 12:0:0:1: [sdb] Unhandled sense code > sd 12:0:0:1: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE > sd 12:0:0:1: [sdb] Sense Key : Medium Error [current] > sd 12:0:0:1: [sdb] Add. Sense: Unrecovered read error > sd 12:0:0:1: [sdb] CDB: Write(10): 2a 00 00 00 20 a8 00 00 08 00 > Buffer I/O error on device sdb1, logical block 1041 > lost page write due to I/O error on sdb1 > > This turned out to be caused by a race condition among multiple write > requests. When bs_sheepdog_request() receives a write request for the > snapshot vdi, it tries to change the snapshot to a writable vdi with > the create_branch function. However, the current implementation of > create_branch() cannot handle concurrent requests exclusively nor > protected from regular io routine (sd_io). > > This fixes the above io-error issue by serializing create_branch() > with a pthread reader/writer lock, and also fixes the race condition > between create_branch() and sd_io() with the lock. > > Signed-off-by: Ryusuke Konishi > Cc: Hitoshi Mitake > --- > usr/bs_sheepdog.c | 22 +++++++++++++++++----- > 1 file changed, 17 insertions(+), 5 deletions(-) Ooops, thanks a lot for your fix. It must be a very hard debug and sorry for annoying! Reviewed-by: Hitoshi Mitake Thanks, Hitoshi