NILFS for a "chroot": a kind of a work-around

NILFS for a "chroot": a kind of a work-around

[Ivan Shmakov]

	It was mentioned recently [1] that NILFS doesn’t handle frequent
	file “creation-removal” cycles well, and these are typical to,
	in particular, # apt-get install (upgrade) operation.

	I’m currently using NILFS for “root” filesystems of a couple of
	chrooted environments I use to test new software, which implies
	that Debian packages are installed and upgraded quite often.

	As a work-around, I’ve made a copy of the filesystem on tmpfs,
	and run # apt-get from there, like:

# chroot /tmp/debian.UvYusUaj apt-get upgrade 

	Then, I propagate the changes back to the original NILFS root
	with rsync(1), like:

# rsync -x -a -v -rlOtH \
      -b --suffix=.~$(date +%s)~ --backup-dir=.rsync-backup \
      --exclude=/.rsync-backup/ --exclude=/.nilfs \
      --delete \
      -- /tmp/debian.UvYusUaj/ /srv/chroot/2012-07-06-unsafe/ 

	This obviously results in much less checkpoints, too, and thus,
	AIUI, less overall stress to the filesystem.

	(Sometimes, I’d also chcp(8) the latest of the newly-made
	checpoints into a snapshot.)

[1] http://permalink.gmane.org/gmane.comp.file-systems.nilfs.user/2397

-- 
FSF associate member #7257	http://sf-day.org/

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: NILFS for a "chroot": a kind of a work-around

[Vyacheslav Dubeyko]

Hi,

As I know, checkpointing is a fundamental technique of the NILFS. It is possible to imagine checkpointing as an act of final placing of file system blocks on disk storage. Another filesystems also writing on disk but, usually, without possibility to keep history of file system operations and manage by history. Checkpoint in NILFS is natural possibility to prevent from file system inconsistency and failure by means of Copy-On-Write policy. Of course, checkpoints keep duplicates of  metadata and data information but obsolete checkpoints can be deleted by nilfs_cleanerd. So, from my point of view, your approach is trying to overcome the main goal of the NILFS.

What about reliability in your approach? Everything can occur during operations under tmpfs partition (for example, sudden power off).

I think that it doesn't exist ideal file system that can be best in all possible workloads. So, if you have some use-case or workload in that NILFS is not so good, maybe it will be better to use another file system?

With the best regards,
Vyacheslav Dubeyko.

On Jul 14, 2012, at 8:31 PM, Ivan Shmakov wrote:

> 	It was mentioned recently [1] that NILFS doesn’t handle frequent
> 	file “creation-removal” cycles well, and these are typical to,
> 	in particular, # apt-get install (upgrade) operation.
> 
> 	I’m currently using NILFS for “root” filesystems of a couple of
> 	chrooted environments I use to test new software, which implies
> 	that Debian packages are installed and upgraded quite often.
> 
> 	As a work-around, I’ve made a copy of the filesystem on tmpfs,
> 	and run # apt-get from there, like:
> 
> # chroot /tmp/debian.UvYusUaj apt-get upgrade 
> 
> 	Then, I propagate the changes back to the original NILFS root
> 	with rsync(1), like:
> 
> # rsync -x -a -v -rlOtH \
>      -b --suffix=.~$(date +%s)~ --backup-dir=.rsync-backup \
>      --exclude=/.rsync-backup/ --exclude=/.nilfs \
>      --delete \
>      -- /tmp/debian.UvYusUaj/ /srv/chroot/2012-07-06-unsafe/ 
> 
> 	This obviously results in much less checkpoints, too, and thus,
> 	AIUI, less overall stress to the filesystem.
> 
> 	(Sometimes, I’d also chcp(8) the latest of the newly-made
> 	checpoints into a snapshot.)
> 
> [1] http://permalink.gmane.org/gmane.comp.file-systems.nilfs.user/2397
> 
> -- 
> FSF associate member #7257	http://sf-day.org/
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: NILFS for a "chroot": a kind of a work-around

[Ivan Shmakov]

>>>>> Vyacheslav Dubeyko writes:

 > As I know, checkpointing is a fundamental technique of the NILFS.  It
 > is possible to imagine checkpointing as an act of final placing of
 > file system blocks on disk storage.  Another filesystems also writing
 > on disk but, usually, without possibility to keep history of file
 > system operations and manage by history.  Checkpoint in NILFS is
 > natural possibility to prevent from file system inconsistency and
 > failure by means of Copy-On-Write policy.  Of course, checkpoints
 > keep duplicates of metadata and data information but obsolete
 > checkpoints can be deleted by nilfs_cleanerd.  So, from my point of
 > view, your approach is trying to overcome the main goal of the NILFS.

	Not at all.  The idea is to utilize NILFS snapshots to record
	consistent states of the chroot environment.  So, e. g., having
	found that certain software package did break, I can bisect the
	filesystem history to find the version of the package which have
	introduced the bug.

	Using rsync(1) makes this easier, as there's both much less work
	to the GC, and it takes less time to perform the changes, so
	there are only a few checkpoints to check.

 > What about reliability in your approach?  Everything can occur during
 > operations under tmpfs partition (for example, sudden power off).

	Then, the latest operation (# apt-get install something) won't
	be effective and will have to be repeated.  But the target
	chroot environment will still be in a consistent state.

[…]

-- 
FSF associate member #7257	http://sf-day.org/

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html