From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hin-Tak Leung Subject: Re: hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation. Date: Thu, 26 Jul 2012 06:53:59 +0100 (BST) Message-ID: <1343282039.10130.YahooMailClassic@web29406.mail.ird.yahoo.com> References: <1343198046.2025.38.camel@slavad-ubuntu-11> Reply-To: htl10@users.sourceforge.net Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: linux-fsdevel@vger.kernel.org, Till Kamppeter , Naohiro Aota , Matthew Garrett To: Vyacheslav Dubeyko Return-path: Received: from nm6-vm0.bullet.mail.ird.yahoo.com ([77.238.189.210]:31630 "HELO nm6-vm0.bullet.mail.ird.yahoo.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1750916Ab2GZFyB convert rfc822-to-8bit (ORCPT ); Thu, 26 Jul 2012 01:54:01 -0400 In-Reply-To: <1343198046.2025.38.camel@slavad-ubuntu-11> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: Hi Vyacheslav, --- On Wed, 25/7/12, Vyacheslav Dubeyko wrote: > Hi Hin-Tak, >=20 > Sorry, I can see from the hfsplus code about what you are > talking. You > are right. The situation is clear for me. It needs to work. > :-) >=20 > Thank you for bug report. >=20 > With the best regards, > Vyacheslav Dubeyko. I was going to confess that I might be using buggy code - it is rather = a long history, but I am testing a port of netgear's HFS+ journalling c= hange to current kernel. The long and short of the story is that Netgea= r contracted some company to add HFS+ support in 2010, which they relea= sed as a NAS product in 2011, and therefore posted a tar ball of modifi= ed 2.6.15 kernel source. This was worked on to port to 3.0-ish last sum= mer, and I am moving it forward to 3.4.x-ish. Updating the journal shou= ldn't cause the kind of things I saw - even if it is done wrongly, I w= as hoping.=20 I am using 3.4.5/3.4.6, and indeed on special code branch and on journa= lled hfs+. It is an MBR disk. I found that I can cause massive amount of problem by just untarring so= me big thing and then deleting the files one by one. (I did this with t= he netgear tar ball, which contains a kernel tree, and after comparing = against a git checkout of the relevant version, delete identical files = one by one from a list with a script). i.e. try something similar to this: tar -jxpvf kernel-tarball.tar.bz2 > somewhere_else/file-list cat somewhere_else/file-list | perl -ne 'chomp; if (-f $_) {unlink $_= ;}' On a separate but related note, - there are 6 netgear tarballs, with 2 implementations of hfs+ journall= ing. i.e. there is a slightly newer hfs+ journalling from them since w= e checked last summer. The newer code copied the whole of /jbd/ from ex= t3 and renamed everything from jbd to hfsplus_jbd, etc and added some s= mall HFS+-related things. Rather ugly, but possibly do something simili= ar as jbd does for ext3. - there is a commercial/free-for-personal-use implementation of hfs+ su= pport from a company Paragon, released last summer, for 2.6.38-. Doesn'= t build with current kernel; it is almost like nvidia/ATI's X 3D driver= s - small glue code to build a kernel module against a binary-only arch= ive library. (also Paragon's HFS+ product for windows seems to make a l= ot of mac users unhappy for corrupting data, but I might be doing some = selective reading there :-)) - I have been looking at some random HFS+ disk images I have as well. O= ne - the command line tools from xcode on apple's web site - is interes= ting in that the HFS+ volume header says it is over 400MB, but the file= itself is only 180MB so obviously there is no secondary volume header = (which is located at the end of volume). Or at least, not at the expect= ed location. But Mac OS X does not have problem mounting it - there is = only about 180MB worth of stuff inside. I am thinking maybe for disk images, and also read-only mount, one shou= ld not need to check the secondary volume header, as this seems to be w= hat Mac OS X is doing. The interesting disk image is one of the command-line-tools from xcode = on apple's web site (free registration). A few of them are compressed, = which is at about 150MB, but one is 180MB, and shows what I wrote about= , for those who wants to investigate. Hin-Tak =20 > On Mon, 2012-07-23 at 23:35 +0400, Vyacheslav Dubeyko > wrote: > > Hi, > >=20 > > On Jul 22, 2012, at 1:06 AM, Hin-Tak Leung wrote: > >=20 > > >=20 > > > Correct - two interesting bugs. I thought I had > supplied enough details for others to try to reproduce? The > first is simply about deleting files, the latter about > deleting files with extended attributes. So, to reproduce: > > >=20 > > > 1. Have a hfs+ volume (created under Mac OS X > would be better). Make sure it passed fsck.hfsplus (on > linux). > > > 2. copy the system Fonts folder (I think it is > under /Library/Fonts) to it from Mac OS X. Font files have > extended attributes, but other system files might do too. > > > 3. Try to delete some of those files from Linux. > > > 4. umount, run fsck.hfsplus on the volume. One > would see: > > >=20 > > >=A0=A0=A0Executing fsck_hfs (version > 540.1-Linux). > > >=A0=A0=A0... > > > ** Checking extents overflow file. > > >=A0=A0=A0Unused node is not erased (node =3D > 1) > > > ** Checking catalog file. > > >=A0=A0=A0Unused node is not erased (node =3D > 18) > > >=A0=A0=A0... > > > ** Checking extended attributes file. > > >=A0=A0=A0Incorrect number of extended > attributes > > >=A0=A0=A0(It should be 13 instead of 6) > > >=A0=A0=A0... > > > ** Repairing volume. > > > =A0=A0=A0 RepairAttributesCheckABT: No > matching catalog record found for id=3D438 > > >=A0=A0=A0... > > > ** Rechecking volume. > > >=A0=A0=A0... > > > ** The volume journalled was repaired > successfully. > >=20 > > Currently, I can't reproduce these bugs. I think that I > have not fully correct reproduction path. So, I have some > questions. > > What version of Linux kernel do you use? Or maybe do > you have hfsplus file system driver code from special > branch? > >=20 > > I tried to reproduce these bugs on non-journaled hfs+ > volume, firstly, but without success. As I can see you use > journaled hfs+ volume. Maybe the forced mount of journaled > hfs+ was the reason of these bugs? By the way do you use MBR > or GPT partitioned disk? > >=20 > > >=20 > > > 'Unused node is not erased' & 'Incorrect > number of extended attributes' were essentially what I wrote > earlier. BTW, I ran fsck.hfsplus with: > > >=20 > > > fsck.hfsplus -d -D 0x0033 -f=20 > > >=20 > > > - i.e. maximum amount of information, and have a > look even if it appears to be clean. (there are -l, -y and > -n switches to control whether fsck.hfsplus would actually > fix anything found or leave it alone and just output info). > > >=20 > > > The font folder is only about a few hundred MB, so > one should be able to experiment with a small pen drive and > do byte-level comparisons to see what fsk.hfsplus is not > happy about and what does it change - although I am sure > looking at diskdev_cmds's source would also be useful. I > just happen to want the font folder for other things I do. > > >=20 > > >=20 > > > -- > > > To unsubscribe from this list: send the line > "unsubscribe linux-fsdevel" in > > > the body of a message to majordomo@vger.kernel.org > > > More majordomo info at=A0 http://vger.kernel.org/majordomo-info.h= tml > >=20 > > With the best regards, > > Vyacheslav Dubeyko. > >=20 > > -- > > To unsubscribe from this list: send the line > "unsubscribe linux-fsdevel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at=A0 http://vger.kernel.org/majordomo-info.htm= l >=20 >=20 >=20 -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel= " in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html