From mboxrd@z Thu Jan 1 00:00:00 1970 From: Fernando Luis Vazquez Cao Subject: Re: [PATCH 4/9] fsfreeze: emergency thaw will deadlock on s_umount Date: Tue, 09 Oct 2012 14:07:52 +0900 Message-ID: <5073B128.6010107@lab.ntt.co.jp> References: <1349414653.7347.2.camel@nexus.lab.ntt.co.jp> <1349415353.7347.8.camel@nexus.lab.ntt.co.jp> <20121008135733.GC9243@quack.suse.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Al Viro , Josef Bacik , Eric Sandeen , Dave Chinner , Christoph Hellwig , linux-fsdevel@vger.kernel.org To: Jan Kara Return-path: Received: from tama500.ecl.ntt.co.jp ([129.60.39.148]:46546 "EHLO tama500.ecl.ntt.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752195Ab2JIFI2 (ORCPT ); Tue, 9 Oct 2012 01:08:28 -0400 In-Reply-To: <20121008135733.GC9243@quack.suse.cz> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On 2012/10/08 22:57, Jan Kara wrote: > On Fri 05-10-12 14:35:53, Fernando Luis V=E1zquez Cao wrote: >> The emergency thaw process uses iterate_super() which holds the >> sb->s_umount lock in read mode. The current thaw_super() code takes >> the sb->s_umount lock in write mode, hence leading to an instant >> deadlock. >> =20 >> Use the unlocked version of thaw_super() to do the thawing and repla= ce >> iterate_supers() with __iterate_supers() so that the unfreeze operat= ion can > ^^ iterate_supers_write() Good catch. >> be performed with s_umount held as the locking rules for fsfreeze in= dicate. >> >> As a bonus, by using thaw_super(), which does not nest, instead of t= haw_bdev() >> when can get rid of the ugly while loop. >> >> Jan Kara pointed out that with this approach we will leave the block= devices >> frozen, but this is a problem we have had since the introduction of = the >> superblock level API: if we thaw the filesystem using the superblock= level API >> (be it through the thaw ioctl or emergency thaw) the bdev level free= ze >> reference counter (bd_fsfreeze_count) will not be updated and even t= hough >> subsequent calls to thaw_bdev() will decrease it it will never get b= ack to 0 >> (if thaw_super() returns an error, and it will when the superblock i= s unfrozen, >> thaw_bdev() will return without decreasing the counter). The solutio= n I propose >> (and will be implementing in the followup patch "fsfreeze: freeze_su= per and >> thaw_bdev don't play well together") is letting bd_fsfreeze_count >> become zero when the superblock sitting on top of it is unfrozen, so= that >> future calls to freeze_bdev() actually try to freeze the superblock. >> >> Cc: Josef Bacik >> Cc: Eric Sandeen >> Cc: Christoph Hellwig >> Cc: Jan Kara >> Cc: Dave Chinner >> Signed-off-by: Fernando Luis Vazquez Cao >> --- >> >> diff -urNp linux-3.6.0-rc7-orig/fs/buffer.c linux-3.6.0-rc7/fs/buffe= r.c >> --- linux-3.6.0-rc7-orig/fs/buffer.c 2012-09-26 13:20:14.842365056 += 0900 >> +++ linux-3.6.0-rc7/fs/buffer.c 2012-09-26 15:02:22.630595704 +0900 >> @@ -513,15 +513,28 @@ repeat: >> =20 >> static void do_thaw_one(struct super_block *sb, void *unused) >> { >> - char b[BDEVNAME_SIZE]; >> - while (sb->s_bdev && !thaw_bdev(sb->s_bdev, sb)) >> - printk(KERN_WARNING "Emergency Thaw on %s\n", >> + int res; >> + >> + if (sb->s_bdev) { >> + char b[BDEVNAME_SIZE]; >> + printk(KERN_WARNING "Emergency Thaw on %s.\n", >> bdevname(sb->s_bdev, b)); >> + } >> + >> + /* We got here from __iterate_supers with the superblock lock take= n >> + * so we can call the lockless version of thaw_super() safely. */ >> + res =3D __thaw_super(sb); >> + /* If we are going to drop the final active reference call >> + * deactivate_locked_super to clean things up. In the general case >> + * we avoid calling deactivate_locked_super() because it would rel= ase >> + * the superblock lock, which is __iterate_supers()'s job. */ >> + if (!res && !atomic_add_unless(&sb->s_active, -1, 1)) >> + deactivate_locked_super(sb); > This just looks wrong. When we *do* end up calling > deactivate_locked_super() we will return with sb unlocked which makes > iterate_supers_write() unlock already unlocked lock. Thank you for the heads-up. I missed the fact that ->kill_sb() which gets called in=20 deactivate_locked_super() will unlock the superblock indirectly via generic_shutdown_super() or o= ne of the wrappers around it (kill_block_super(), kill_anon_super(),=20 kill_litter_super()). > What I would put here is: > if (!res) { > deactivate_locked_super(sb); > /* > * We have to re-acquire s_umount because > * iterate_supers_write() will unlock it. It still holds > * passive reference so sb cannot be freed under us. > */ > down_write(&sb->s_umount); > } > =09 > Is there any problem with this I miss? The reason I wrote the code as I did is that I did not want to re-acqu= ire s_umount in the normal case (s_active >=3D 2 entering the if statement)= =2E What about combining our approaches and doing something like this?: if (!res && !atomic_add_unless(&sb->s_active, -1, 1)) { deactivate_locked_super(sb); down_write(&sb->s_umount); } Thanks, =46ernando -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel= " in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html