[PATCH v3 00/15] hfsplus: introduce journal replay functionality

[PATCH v3 00/15] hfsplus: introduce journal replay functionality

[Vyacheslav Dubeyko]

Hi,

This is third version of patchset that implements journal replay
functionality in HFS+ file system driver.

v2->v3
* Fix unresolvable references to __udivdi3 and __umoddi3 on i386 (Andrew Morton)
* Fix printk mismatch warnings on i386 (Andrew Morton)
* Fix bisection holes (implicit declarations and warnings) on i386 (Andrew Morton)

v1->v2
* Fix error messages incorrectness (Hin-Tak Leung)
* Fix issue with remount "mount -o remount,rw <device> <folder>"
* Rework "force" and "norecovery" mount options support (Hin-Tak Leung)

Technical Note TN1150:
"The purpose of the journal is to ensure that when a group of
 related changes are being made, that either all of those changes
 are actually made, or none of them are made. This is done by
 gathering up all of the changes, and storing them in a separate
 place (in the journal). Once the journal copy of the changes is
 completely written to disk, the changes can actually be written
 to their normal locations on disk. If a failure happens at that
 time, the changes can simply be copied from the journal to their
 normal locations. If a failure happens when the changes are being
 written to the journal, but before they are marked complete, then
 all of those changes are ignored."

"A group of related changes is called a transaction. When all of
 the changes of a transaction have been written to their normal
 locations on disk, that transaction has been committed, and is
 removed from the journal. The journal may contain several
 transactions. Copying changes from all transactions to their
 normal locations on disk is called replaying the journal."

"In order to replay the journal, an implementation just loops
 over the transactions, copying each individual block in the
 transaction from the journal to its proper location on the
 volume. Once those blocks have been flushed to the media
 (not just the driver!), it may update the journal header to
 remove the transactions."

"Here are the steps to replay the journal:
  1. Read the volume header into variable vhb. The volume may
     have an HFS wrapper; if so, you will need to use it to
     determine the location of the volume header.
  2. Test the kHFSVolumeJournaledBit in the attributes field of
     the volume header. If it is not set, there is no journal
     to replay, and you are done.
  3. Read the journal info block from the allocation block number
     vhb.journalInfoBlock, into variable jib.
  4. If kJIJournalNeedsInitMask is set in jib.flags, the journal
     was never initialized, so there is no journal to replay.
  5. Verify that kJIJournalInFSMask is set and kJIJournalOnOtherDeviceMask
     is clear in jib.flags.
  6. Read the journal header at jib.offset bytes from the start
     of the volume, and place it in variable jhdr.
  7. If jhdr.start equals jhdr.end, the journal does not have
     any transactions, so there is nothing to replay.
  8. Set the current offset in the journal (typically a local
     variable) to the start of the journal buffer, jhdr.start.
  9. While jhdr.start does not equal jhdr.end, perform the
     following steps:
       1. Read a block list header of jhdr.blhdr_size bytes from
          the current offset in the journal into variable blhdr.
       2. For each block in bhdr.binfo[1] to bhdr.binfo[blhdr.num_blocks],
          inclusive, copy bsize bytes from the current offset in
          the journal to sector bnum on the volume (to byte offset
          bnum*jdhr.jhdr_size). Remember that jhdr_size is the
          size of a sector, in bytes.
       3. If bhdr.binfo[0].next is zero, you have completed the
          last block list of the current transaction; set jhdr.start
          to the current offset in the journal."

It was reworked concept of "force" mount option. It is used to force
write access to volumes that are marked as locked, only. This option
doesn't work for journaled volumes with non-empty journal. HFS+ drive
tries to replay journal and ignores forcing write access in the case of
journal replay failure. It is possible to access HFS+ volume in
read-only mode only for such case. The "force" option is ignored
for the case of "norecovery" option request also.

With the best regards,
Vyacheslav Dubeyko.
---
 Documentation/filesystems/hfsplus.txt |   16 +-
 fs/hfsplus/Makefile                   |    3 +-
 fs/hfsplus/hfsplus_fs.h               |   33 +-
 fs/hfsplus/hfsplus_raw.h              |   53 +-
 fs/hfsplus/journal.c                  | 1221 +++++++++++++++++++++++++++++++++
 fs/hfsplus/options.c                  |   17 +-
 fs/hfsplus/part_tbl.c                 |    4 +-
 fs/hfsplus/super.c                    |  105 ++-
 fs/hfsplus/wrapper.c                  |   28 +-
 9 files changed, 1441 insertions(+), 39 deletions(-)
-- 
1.7.9.5

Re: [PATCH v3 00/15] hfsplus: introduce journal replay functionality

[Sergei Antonov]

On 12 February 2014 15:24, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:
> Hi,
>
> This is third version of patchset that implements journal replay
> functionality in HFS+ file system driver.

Causes file system errors after trying to replay journal on iPod Nano
8GB with 4K sector size. Details follow. My implementation "[PATCH]
hfsplus: add journal replay" works fine.

fsck_hfs output:
Verifying volume disk2s2
Checking file system
Checking Journaled HFS Plus volume.
Checking extents overflow file.Checking catalog file.
Checking multi-linked files.Checking catalog hierarchy.
Checking extended attributes file.
Unused node is not erased (node = 53)
Checking volume bitmap.
Volume bitmap needs minor repair for under-allocation
Checking volume information.
Volume header needs minor repairThe volume Sergei Antonov’s iPod was
found corrupt and needs to be repaired.
Error: This disk needs to be repaired. Click Repair Disk.

In system log:
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: do_jnl_io:
strategy err 0x6
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2:
end_transaction: only wrote 0 of 147456 bytes to the journal!
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: hfs: unmount initiated on
Sergei Antonovs iPod on device disk2s2
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: close:
journal 0xffffff8015507a80, is invalid.  aborting outstanding
transactions
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v3 00/15] hfsplus: introduce journal replay functionality

[Vyacheslav Dubeyko]

On Mon, 2014-02-24 at 10:59 +0100, Sergei Antonov wrote:
> On 12 February 2014 15:24, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:
> > Hi,
> >
> > This is third version of patchset that implements journal replay
> > functionality in HFS+ file system driver.
> 
> Causes file system errors after trying to replay journal on iPod Nano
> 8GB with 4K sector size. Details follow. My implementation "[PATCH]
> hfsplus: add journal replay" works fine.
> 

If you have found some bug then you can report about it. And I'll fix it
after reproducing. But it doesn't mean that it needs to suggest the
patch instead of mine. It needs to cooperate. And nobody prevented you
from sharing your patch before mine. I think that suggesting a patch
after existing of another one is dishonest way and simply cheating.

Yesterday I published fourth version of the patch. Do you have the same
result for this version? Could you describe in more details how I can
reproduce the issue?

Thanks,
Vyacheslav Dubeyko.

> fsck_hfs output:
> Verifying volume disk2s2
> Checking file system
> Checking Journaled HFS Plus volume.
> Checking extents overflow file.Checking catalog file.
> Checking multi-linked files.Checking catalog hierarchy.
> Checking extended attributes file.
> Unused node is not erased (node = 53)
> Checking volume bitmap.
> Volume bitmap needs minor repair for under-allocation
> Checking volume information.
> Volume header needs minor repairThe volume Sergei Antonov’s iPod was
> found corrupt and needs to be repaired.
> Error: This disk needs to be repaired. Click Repair Disk.
> 
> In system log:
> Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: do_jnl_io:
> strategy err 0x6
> Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2:
> end_transaction: only wrote 0 of 147456 bytes to the journal!
> Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: hfs: unmount initiated on
> Sergei Antonovs iPod on device disk2s2
> Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: close:
> journal 0xffffff8015507a80, is invalid.  aborting outstanding
> transactions
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html