Re: Number of hard links limit

[Sami Liedes <sliedes@cc.hut.fi>]

[-- Attachment #1: Type: text/plain, Size: 1820 bytes --]

On Tue, Aug 03, 2010 at 12:22:14AM +0200, Oystein Viggen wrote:
> IIRC, the limit on hard links is per directory.  That is, if you put
> each hard link into its own directory, there's basically no limit to the
> amount of hard links you can make to one file.

Yes, that's always pointed out in these threads. Still, it seems to be
breaking real use cases also beyond backuppc (someone mentioned
installing some Gentoo package).

> Thus, many generations of backup with BackupPC shouldn't trigger the
> problem, as each generation is stored in its own directory tree.  The
> problem appears when your source data has many identical files in the
> same directory, since these would be deduplicated as hard links to the
> same file in the backup pool.

I think you are right. I looked at the errors I reported in

  https://bugzilla.kernel.org/show_bug.cgi?id=15762

and figured out what happens there. In dbus's source tree, under
doc/api/man/man3dbus, there are 119 files with the content ".so
man3dbus/DBusProtocol.3dbus". I think these are redirects to a single
man page.

Interestingly (and only somewhat off-topic), because of deduplication
backuppc can be used to explore the merits of deduplication in
filesystems too. I looked at the files I have most copies of. Many of
them seem perhaps a bit silly.

The most common file has a single line with the text "# dummy". I have
54991 copies of that file. They are some kind of dependency files
(perhaps by libtool or something?), always in a directory named .deps
and with a file extension of .Po or .Plo.

There are also tens of thousands of copies of a file with a single
line with the text "END". Subversion VCS seems to have two of these
for every file under version control. In fact a large portion of the
most common files seem to be owned by Subversion.

	Sami

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]