Which version of nilfs do you use?

Which version of nilfs do you use?

[Elmer Zhang]

Hi konishi ryusuke,

I am trying to use NILFS2 to make MySQL cold backup, but  the MySQL tables often be crashed. It seems there is some problems with nilfs filesystem.

I have described how I use nilfs in this mail: http://www.mail-archive.com/linux-nilfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg01481.html

Can I know which version of nilfs do you use, and what do you use nilfs for?

Thanks,
Elmer Zhang--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[dexen deVries]

Hi Elmer,


On Thursday 27 of December 2012 17:47:34 you wrote:
> I am trying to use NILFS2 to make MySQL cold backup, but  the MySQL tables
> often be crashed. It seems there is some problems with nilfs filesystem.
> 
> I have described how I use nilfs in this mail:
> http://www.mail-archive.com/linux-nilfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg01481.html
> 
> Can I know which version of nilfs do you use, and what do you use nilfs for?

FWIW,  I have MySQL installation on NILFS2 and it works just fine. The server 
is only under light load, thou. MySQL version 5.5.27 (and 5.1.50 earlier on); 
NILFS2 from kernel v3.0 and later, up to v3.5.

Perhaps your problems have something to do with MySQL recovery of data from 
the cold backup? In MySQL Reference Manual chapter 7 -- Backup and Recovery 
covers the subject.

If I'm reading your other email correctly, it seems you copied MyISAM files 
without shutting MySQL down first. That's not supposed to be done with non-
transactional tables, AFAIK, and I'd suspect this as the main trouble source. 

On the other hand, InnoDB tables should survive being copied from working 
MySQL, as long as you observe correct order of (table, log files) copying -- 
not sure which goes first, it's in the docs.


Cheers,
-- 
dexen deVries

[[[↓][→]]]


Reality is just a convenient measure of complexity.
                -- Alvy Ray Smith
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[Elmer Zhang]

Hi,

There is more than 2TB of data and a large number of databases, tables in our MySQL service. The conventional cold backup methods can not meet our needs. So we are trying to find some other methods.

I copied the MySQL files after shut MySQL done. The same data runs well with ext4 file system on another server, so I can confirm that my original data is no problem.

Elmer Zhang


ÔÚ 2012-12-27£¬18:35£¬dexen deVries <dexen.devries-Re5JQEeQqe8@public.gmane.orgm> дµÀ£º

> Hi Elmer,
> 
> 
> On Thursday 27 of December 2012 17:47:34 you wrote:
>> I am trying to use NILFS2 to make MySQL cold backup, but  the MySQL tables
>> often be crashed. It seems there is some problems with nilfs filesystem.
>> 
>> I have described how I use nilfs in this mail:
>> http://www.mail-archive.com/linux-nilfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg01481.html
>> 
>> Can I know which version of nilfs do you use, and what do you use nilfs for?
> 
> FWIW,  I have MySQL installation on NILFS2 and it works just fine. The server 
> is only under light load, thou. MySQL version 5.5.27 (and 5.1.50 earlier on); 
> NILFS2 from kernel v3.0 and later, up to v3.5.
> 
> Perhaps your problems have something to do with MySQL recovery of data from 
> the cold backup? In MySQL Reference Manual chapter 7 -- Backup and Recovery 
> covers the subject.
> 
> If I'm reading your other email correctly, it seems you copied MyISAM files 
> without shutting MySQL down first. That's not supposed to be done with non-
> transactional tables, AFAIK, and I'd suspect this as the main trouble source. 
> 
> On the other hand, InnoDB tables should survive being copied from working 
> MySQL, as long as you observe correct order of (table, log files) copying -- 
> not sure which goes first, it's in the docs.
> 
> 
> Cheers,
> -- 
> dexen deVries
> 
> [[[¡ý][¡ú]]]
> 
> 
> Reality is just a convenient measure of complexity.
>                -- Alvy Ray Smith

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[ARAI Shun-ichi]

In <BB9A0F28-A902-4FE4-BA58-A66EA93290F8-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>;
   Elmer Zhang <freeboy6716-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote
   as Subject "Re: Which version of nilfs do you use?":

> There is more than 2TB of data and a large number of databases, tables in our MySQL service. The conventional cold backup methods can not meet our needs. So we are trying to find some other methods.

I posted report mail about the "bad btree node" error in 2012-11-30,
I told about VirtualBox.
When I met the error first, virtual disk file (approx. 16GB) was in
NILFS FS.

Zhang's DB is large. Perhaps there are files which has large size, I think.
And these files are updated frequently.

Can the common point (file size, update) cause inconsistency?
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[Vyacheslav Dubeyko]

Hi,

On Thu, 2012-12-27 at 17:47 +0800, Elmer Zhang wrote:
> Hi konishi ryusuke,
> 
> I am trying to use NILFS2 to make MySQL cold backup, but  the MySQL tables often be crashed. It seems there is some problems with nilfs filesystem.
> 

As I remember, you reported initially the error message about issue with
(inode number=321775) and (blocknr=710153406). If you have new MySQL
tables' crashes then it means that you have new NILFS error messages in
your syslog. Do you have error messages about new inode numbers and
block numbers or about the same?

With the best regards,
Vyacheslav Dubeyko.

> I have described how I use nilfs in this mail: http://www.mail-archive.com/linux-nilfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg01481.html
> 
> Can I know which version of nilfs do you use, and what do you use nilfs for?
> 
> Thanks,
> Elmer Zhang--
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[Vyacheslav Dubeyko]

On Fri, 2012-12-28 at 11:58 +0900, ARAI Shun-ichi wrote:
> In <BB9A0F28-A902-4FE4-BA58-A66EA93290F8-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>;
>    Elmer Zhang <freeboy6716-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote
>    as Subject "Re: Which version of nilfs do you use?":
> 
> > There is more than 2TB of data and a large number of databases, tables in our MySQL service. The conventional cold backup methods can not meet our needs. So we are trying to find some other methods.
> 
> I posted report mail about the "bad btree node" error in 2012-11-30,
> I told about VirtualBox.
> When I met the error first, virtual disk file (approx. 16GB) was in
> NILFS FS.
> 
> Zhang's DB is large. Perhaps there are files which has large size, I think.
> And these files are updated frequently.
> 
> Can the common point (file size, update) cause inconsistency?

I think that it can be a synchronization issue. But it needs to
investigate the situation more deeply. As I can see, it is using 8 CPUs
in the case of the issue and it takes place frequent update. So, it can
be a ground for good issue reproducibility, from my point of view.

With the best regards,
Vyacheslav Dubeyko.

> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[Elmer Zhang]

Hi,

There was three bad btree nodes:

Dec 20 16:12:08 yf237 kernel: NILFS: bad btree node (blocknr=710153406): level = 0, flags = 0x2, nchildren = 25088
Dec 20 16:12:08 yf237 kernel: NILFS error (device sdb2): nilfs_bmap_last_key: broken bmap (inode number=321775)
--
Dec 21 10:10:07 yf237 kernel: NILFS: bad btree node (blocknr=711343917): level = 199, flags = 0x60, nchildren = 16670
Dec 21 10:10:07 yf237 kernel: NILFS error (device sdb2): nilfs_bmap_last_key: broken bmap (inode number=324685)
--
Dec 21 10:13:41 yf237 kernel: NILFS: bad btree node (blocknr=14167806): level = 0, flags = 0x0, nchildren = 0
Dec 21 10:13:41 yf237 kernel: NILFS error (device sdb2): nilfs_bmap_last_key: broken bmap (inode number=253710)

I have not found new bad btree nodes in recent days. But the MyISAM tables still often be crashed and soon be ok without repairing.


ÔÚ 2012-12-28£¬14:06£¬Vyacheslav Dubeyko <slava-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org> дµÀ£º

> Hi,
> 
> On Thu, 2012-12-27 at 17:47 +0800, Elmer Zhang wrote:
>> Hi konishi ryusuke,
>> 
>> I am trying to use NILFS2 to make MySQL cold backup, but  the MySQL tables often be crashed. It seems there is some problems with nilfs filesystem.
>> 
> 
> As I remember, you reported initially the error message about issue with
> (inode number=321775) and (blocknr=710153406). If you have new MySQL
> tables' crashes then it means that you have new NILFS error messages in
> your syslog. Do you have error messages about new inode numbers and
> block numbers or about the same?
> 
> With the best regards,
> Vyacheslav Dubeyko.
> 
>> I have described how I use nilfs in this mail: http://www.mail-archive.com/linux-nilfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg01481.html
>> 
>> Can I know which version of nilfs do you use, and what do you use nilfs for?
>> 
>> Thanks,
>> Elmer Zhang--
>> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[Vyacheslav Dubeyko]

On Fri, 2012-12-28 at 15:12 +0800, Elmer Zhang wrote:
> Hi,
> 
> There was three bad btree nodes:
> 
> Dec 20 16:12:08 yf237 kernel: NILFS: bad btree node (blocknr=710153406): level = 0, flags = 0x2, nchildren = 25088
> Dec 20 16:12:08 yf237 kernel: NILFS error (device sdb2): nilfs_bmap_last_key: broken bmap (inode number=321775)
> --
> Dec 21 10:10:07 yf237 kernel: NILFS: bad btree node (blocknr=711343917): level = 199, flags = 0x60, nchildren = 16670
> Dec 21 10:10:07 yf237 kernel: NILFS error (device sdb2): nilfs_bmap_last_key: broken bmap (inode number=324685)
> --
> Dec 21 10:13:41 yf237 kernel: NILFS: bad btree node (blocknr=14167806): level = 0, flags = 0x0, nchildren = 0
> Dec 21 10:13:41 yf237 kernel: NILFS error (device sdb2): nilfs_bmap_last_key: broken bmap (inode number=253710)
> 

I think that it makes sense to investigate dumps of the segments that
contains these blocks. Could you make dumps of #346754, #347335, #6917
segments and send only on my personal e-mail for analysis?

Example:

dd if=<device> of=<file> bs=4096 skip=710152192 count=2048

(710152192 == 346754 * 2048)

With the best regards,
Vyacheslav Dubeyko.

> I have not found new bad btree nodes in recent days. But the MyISAM tables still often be crashed and soon be ok without repairing.
> 
> 
> 在 2012-12-28，14:06，Vyacheslav Dubeyko <slava@dubeyko.com> 写道：
> 
> > Hi,
> > 
> > On Thu, 2012-12-27 at 17:47 +0800, Elmer Zhang wrote:
> >> Hi konishi ryusuke,
> >> 
> >> I am trying to use NILFS2 to make MySQL cold backup, but  the MySQL tables often be crashed. It seems there is some problems with nilfs filesystem.
> >> 
> > 
> > As I remember, you reported initially the error message about issue with
> > (inode number=321775) and (blocknr=710153406). If you have new MySQL
> > tables' crashes then it means that you have new NILFS error messages in
> > your syslog. Do you have error messages about new inode numbers and
> > block numbers or about the same?
> > 
> > With the best regards,
> > Vyacheslav Dubeyko.
> > 
> >> I have described how I use nilfs in this mail: http://www.mail-archive.com/linux-nilfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg01481.html
> >> 
> >> Can I know which version of nilfs do you use, and what do you use nilfs for?
> >> 
> >> Thanks,
> >> Elmer Zhang--
> >> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> > 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[ARAI Shun-ichi]

In <1356675289.2126.10.camel@slavad-ubuntu>;
   Vyacheslav Dubeyko <slava-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org> wrote
   as Subject "Re: Which version of nilfs do you use?":

> I think that it can be a synchronization issue. But it needs to
> investigate the situation more deeply. As I can see, it is using 8 CPUs
> in the case of the issue and it takes place frequent update. So, it can
> be a ground for good issue reproducibility, from my point of view.

CPU of my PC is Core i7 3770S. It has 4 cores with HTT (8 threads).
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Which version of nilfs do you use?

[Vyacheslav Dubeyko]

On Fri, 2012-12-28 at 16:58 +0900, ARAI Shun-ichi wrote:
> In <1356675289.2126.10.camel@slavad-ubuntu>;
>    Vyacheslav Dubeyko <slava-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org> wrote
>    as Subject "Re: Which version of nilfs do you use?":
> 
> > I think that it can be a synchronization issue. But it needs to
> > investigate the situation more deeply. As I can see, it is using 8 CPUs
> > in the case of the issue and it takes place frequent update. So, it can
> > be a ground for good issue reproducibility, from my point of view.
> 
> CPU of my PC is Core i7 3770S. It has 4 cores with HTT (8 threads).

And you had the issue also. But I tried to reproduce on 2 cores CPU
(without success currently). So, in the case of synchronization issue a
count of CPUs' cores can be a important factor, I think. And
unfortunately, currently, we haven't clear and easy way of the issue
reproducing.

With the best regards,
Vyacheslav Dubeyko.

> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html