From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vyacheslav Dubeyko Subject: Re: very large mount time after unxepected power down Date: Fri, 16 Nov 2012 10:53:55 +0400 Message-ID: <1353048835.2029.17.camel@slavad-ubuntu> References: <1351604965.2069.13.camel@slavad-ubuntu> <1351608774.2026.6.camel@slavad-ubuntu> <1351664002.2105.3.camel@slavad-ubuntu> <1352961172.2076.10.camel@slavad-ubuntu> <1353047197.2029.5.camel@slavad-ubuntu> Mime-Version: 1.0 Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dubeyko.com; s=default; h=Mime-Version:Content-Transfer-Encoding:Content-Type:References:In-Reply-To:Date:Cc:To:From:Subject:Message-ID; bh=xGydPi3IMFytDq6LKFrwld74uh21KZmJeVNTY99X0OQ=; b=I+3JbFPgNBwz8lIAbqc3byUe1OiT2X1NLUDRdCUTe0vLi0BA5tZM4nk/f1KIhNgirG4Hu4k63qp5y+5KLyRLACcUGO519br5HFlAfLjLJ1vTcAL1N2w0x5hHkgB1GnPS; In-Reply-To: Sender: linux-nilfs-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-ID: Content-Type: text/plain; charset="utf-8" To: =?UTF-8?Q?=D0=A1=D0=B5=D1=80=D0=B3=D0=B5=D0=B9_?= =?UTF-8?Q?=D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0=B4=D1=80=D0=BE?= =?UTF-8?Q?=D0=B2?= Cc: linux-nilfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org On Fri, 2012-11-16 at 09:40 +0300, =D0=A1=D0=B5=D1=80=D0=B3=D0=B5=D0=B9= =D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0=B4=D1=80=D0=BE=D0=B2 wro= te: > Sorry, but I didn't save top output this time.. > But for sure, it was "mount /dev/md0 /nfs/raid -o ...." process. The > CPU load was fully in kernel space. > So while the mount call, the kernel was doing something very both IO > and CPU intensive for almost 50 minutes. > As I have already written the load was about 80MB/s read IO according > to iotop, and about 60% of the first CPU core according to top. >=20 Ok. I see. I suspect currently that you can have some special corruption of the volume state that is resulted in so long recovery code working time. Bu= t if so, then you can have some warning messages in system log from recovery subsystem (maybe not, of course). As I know, Gentoo has specia= l log that keeps error and warning messages from the kernel. Could you check that shared by you the dmesg output contains error messages from kernel? Moreover, current functionality state of fsck.nilfs2 is not very useful yet. But it can check superblocks and segment summary headers validity. Maybe it makes sense to check your volume by fsck.nilfs2. Could you try to check your volume? With the best regards, Vyacheslav Dubeyko. > If this info is not sufficient I'll try to reproduce the case as soon > as possible. > -------------------------------------------------- > =D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0=B4=D1=80=D0=BE=D0=B2 =D0= =A1=D0=B5=D1=80=D0=B3=D0=B5=D0=B9 =D0=92=D0=B0=D1=81=D0=B8=D0=BB=D1=8C=D0= =B5=D0=B2=D0=B8=D1=87 >=20 >=20 > 2012/11/16 Vyacheslav Dubeyko : > > On Thu, 2012-11-15 at 16:08 +0300, =D0=A1=D0=B5=D1=80=D0=B3=D0=B5=D0= =B9 =D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0=B4=D1=80=D0=BE=D0=B2 = wrote: > >> lssu, lscp after mount. Actually I missed the moment and > >> nilfs_cleanerd has cleaned some data. > >> Mount took about 50 minutes. > >> > > > > Thank you for info. > > > > I have some additional questions after thinking about issue. As I > > remember, you wrote that you tried to understand what process eats = CPU > > time during issue. But you don't share details about it. Could you = share > > details of "top" and "ps ax" outputs for the case of issue reproduc= ing? > > > > With the best regards, > > Vyacheslav Dubeyko. > > > >> -------------------------------------------------- > >> =D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0=B4=D1=80=D0=BE=D0=B2= =D0=A1=D0=B5=D1=80=D0=B3=D0=B5=D0=B9 =D0=92=D0=B0=D1=81=D0=B8=D0=BB=D1= =8C=D0=B5=D0=B2=D0=B8=D1=87 > >> > >> -- To unsubscribe from this list: send the line "unsubscribe linux-nilfs" = in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html