Number of hard links limit

Number of hard links limit

[Sami Liedes]

[-- Attachment #1: Type: text/plain, Size: 1300 bytes --]

Hi,

There's been discussion before on this list on the very small number
of hard links supported by btrfs.[1][2] In those threads, an often
asked question has been if there's a real world use case the limit
breaks. Also it has been pointed out that a fix for this would need a
disk format change.

As discussed in bug #15762 [3], there are certainly real-world use
cases this limitation breaks. I don't usually like to bring up my pet
bugs on mailing lists, and I'm sorry for doing it here, but if it
indeed needs a disk format change, I think this should be considered
before the format is set in stone. I won't personally lose my sleep if
this is not fixed - I can use other filesystems for backuppc and other
similar systems, although I'd be disappointed to see a production
backup system unexpectedly fail because of this - just that I think
it's better to think about this before things are set in stone.

I'd venture to guess that if I have hit this limit with the very small
amount of btrfs use I've done, thousands of others are going to hit it
when btrfs is the default filesystem of distributions.

	Sami


[1] http://comments.gmane.org/gmane.comp.file-systems.btrfs/4589
[2] http://thread.gmane.org/gmane.comp.file-systems.btrfs/3427
[3] https://bugzilla.kernel.org/show_bug.cgi?id=15762

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Number of hard links limit

[Xavier Nicollet]

Le 02 ao=FBt 2010 =E0 14:40, Sami Liedes a =E9crit:
> [BTRFS supports only 256 hard-links per directory ...] but if it
> indeed needs a disk format change, I think this should be considered
> before the format is set in stone. I won't personally lose my sleep i=
f
> this is not fixed - I can use other filesystems for backuppc and othe=
r
> similar systems,=20

Wouldn't it be even better to actually patch BackupPC to handle btrfs
snapshots and COW (bcp) ?

--=20
Xavier Nicollet
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Number of hard links limit

[Anthony Roberts]

On Mon, 2 Aug 2010 15:05:56 +0200, Xavier Nicollet <nicollet@jeru.org>
wrote:
> Le 02 ao=C3=BBt 2010 =C3=A0 14:40, Sami Liedes a =C3=A9crit:
>> [BTRFS supports only 256 hard-links per directory ...] but if it
>> indeed needs a disk format change, I think this should be considered
>> before the format is set in stone. I won't personally lose my sleep =
if
>> this is not fixed - I can use other filesystems for backuppc and oth=
er
>> similar systems,=20
>=20
> Wouldn't it be even better to actually patch BackupPC to handle btrfs
> snapshots and COW (bcp) ?

That's not the only application impacted by this.

Also, I think it's unrealistic to expect everyone else to code to
BTRFS-specific ioctls when there's other filesystems and other platform=
s to
worry about. It would also be nice if we could tar/rsync/whatever betwe=
en
BTRFS and something else like ext3 or some other OS entirely, without
archiving tools either blowing up or requiring application-specific
knowledge of how to convert dedups to hard links and back.

Also, I believe it's not strictly 256 links, it's dependent on the leng=
th
of the names.

I recall Chris posting something about being able to fix this without a
format change, though it wasn't a priority yet.

-Anthony
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Number of hard links limit

[Michael Niederle]

> Also, I believe it's not strictly 256 links, it's dependent on the length
> of the names.
> 
> I recall Chris posting something about being able to fix this without a
> format change, though it wasn't a priority yet.

As to my knowledge the limit is 64KB for all names of a single file and due to
Chris it will take a format change to fix this.

I ran into the limit some month ago while installing some Gentoo packages.

Greetings, Michael

Re: Number of hard links limit

[Roberto Ragusa]

Michael Niederle wrote:
>> Also, I believe it's not strictly 256 links, it's dependent on the length
>> of the names.
>>
>> I recall Chris posting something about being able to fix this without a
>> format change, though it wasn't a priority yet.
> 
> As to my knowledge the limit is 64KB for all names of a single file and due to
> Chris it will take a format change to fix this.
> 
> I ran into the limit some month ago while installing some Gentoo packages.

That means it would not work for my backup server.
At 4 backups per day, failure for filenames with 45 characters after just
one year.

-- 
   Roberto Ragusa    mail at robertoragusa.it

Re: Number of hard links limit

[Oystein Viggen]

* [Roberto Ragusa]=20

> That means it would not work for my backup server.
> At 4 backups per day, failure for filenames with 45 characters after =
just
> one year.

IIRC, the limit on hard links is per directory.  That is, if you put
each hard link into its own directory, there's basically no limit to th=
e
amount of hard links you can make to one file.

Thus, many generations of backup with BackupPC shouldn't trigger the
problem, as each generation is stored in its own directory tree.  The
problem appears when your source data has many identical files in the
same directory, since these would be deduplicated as hard links to the
same file in the backup pool.

=D8ystein
--=20
This message was generated by a horde of attack elephants armed with PR=
NGs.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Number of hard links limit

[Sami Liedes]

[-- Attachment #1: Type: text/plain, Size: 1820 bytes --]

On Tue, Aug 03, 2010 at 12:22:14AM +0200, Oystein Viggen wrote:
> IIRC, the limit on hard links is per directory.  That is, if you put
> each hard link into its own directory, there's basically no limit to the
> amount of hard links you can make to one file.

Yes, that's always pointed out in these threads. Still, it seems to be
breaking real use cases also beyond backuppc (someone mentioned
installing some Gentoo package).

> Thus, many generations of backup with BackupPC shouldn't trigger the
> problem, as each generation is stored in its own directory tree.  The
> problem appears when your source data has many identical files in the
> same directory, since these would be deduplicated as hard links to the
> same file in the backup pool.

I think you are right. I looked at the errors I reported in

  https://bugzilla.kernel.org/show_bug.cgi?id=15762

and figured out what happens there. In dbus's source tree, under
doc/api/man/man3dbus, there are 119 files with the content ".so
man3dbus/DBusProtocol.3dbus". I think these are redirects to a single
man page.

Interestingly (and only somewhat off-topic), because of deduplication
backuppc can be used to explore the merits of deduplication in
filesystems too. I looked at the files I have most copies of. Many of
them seem perhaps a bit silly.

The most common file has a single line with the text "# dummy". I have
54991 copies of that file. They are some kind of dependency files
(perhaps by libtool or something?), always in a directory named .deps
and with a file extension of .Po or .Plo.

There are also tens of thousands of copies of a file with a single
line with the text "END". Subversion VCS seems to have two of these
for every file under version control. In fact a large portion of the
most common files seem to be owned by Subversion.

	Sami

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Number of hard links limit

[Chris Mason]

On Fri, Aug 06, 2010 at 02:30:39PM +0300, Sami Liedes wrote:
> On Tue, Aug 03, 2010 at 12:22:14AM +0200, Oystein Viggen wrote:
> > IIRC, the limit on hard links is per directory.  That is, if you put
> > each hard link into its own directory, there's basically no limit to the
> > amount of hard links you can make to one file.
> 
> Yes, that's always pointed out in these threads. Still, it seems to be
> breaking real use cases also beyond backuppc (someone mentioned
> installing some Gentoo package).

Right, this will get fixed in a future btrfs update.  We're focusing on
stability with what we have right now, but I do agree we should have
done this link back reference design differently.

-chris

Re: Number of hard links limit

[Roberto Ragusa]

Chris Mason wrote:

> Right, this will get fixed in a future btrfs update.  We're focusing on
> stability with what we have right now, but I do agree we should have
> done this link back reference design differently.

I think people are complaining about this (apparently not too urgent)
limitation, because if the on-disk format has to be changed, it is better
to think about it as soon as possible, before back-compatibility becomes
an issue.

-- 
   Roberto Ragusa    mail at robertoragusa.it

Re: Number of hard links limit

[Martin Steigerwald]

Hello,

Am Freitag, 6. August 2010 schrieb Chris Mason:
> On Fri, Aug 06, 2010 at 02:30:39PM +0300, Sami Liedes wrote:
> > On Tue, Aug 03, 2010 at 12:22:14AM +0200, Oystein Viggen wrote:
> > > IIRC, the limit on hard links is per directory.  That is, if you
> > > put each hard link into its own directory, there's basically no
> > > limit to the amount of hard links you can make to one file.
> > 
> > Yes, that's always pointed out in these threads. Still, it seems to
> > be breaking real use cases also beyond backuppc (someone mentioned
> > installing some Gentoo package).
> 
> Right, this will get fixed in a future btrfs update.  We're focusing on
> stability with what we have right now, but I do agree we should have
> done this link back reference design differently.

What is the status of fixing the limits of hardlinks in BTRFS?

It is now easy to hit on Debian systems that have git package installed:

Preparing to replace git 1:1.7.6.3-1 (using .../git_1%3a1.7.7-1_amd64.deb) 
...
Unpacking replacement git ...
dpkg: error processing /var/cache/apt/archives/git_1%3a1.7.7-1_amd64.deb 
(--unpack):
 unable to make backup link of `./usr/lib/git-core/git-send-pack' before 
installing new version: Too many links
configured to not write apport reports
                                      dpkg-deb: error: subprocess paste 
was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/git_1%3a1.7.7-1_amd64.deb


also see:

git - too many hardlinks in /usr/lib/git-core
http://bugs.debian.org/642603


While above bug might be fixed by using symlinks or by some other 
workaround, I think this limit will be hit more likely as BTRFS matures. I 
think it should be fixed, before the experimental flag of BTRFS is removed.

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

Re: Number of hard links limit

[Sami Liedes]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Wed, Oct 12, 2011 at 07:34:06PM +0200, Martin Steigerwald wrote:
> What is the status of fixing the limits of hardlinks in BTRFS?
> 
> It is now easy to hit on Debian systems that have git package installed:

I too wonder if there still is an intention to fix this... I'd expect
to see much more breakage like this once some distribution adopts
btrfs as the default filesystem. If this is not going to be fixed in
btrfs, perhaps we should start considering it a possible bug in
programs that trigger it (that's apparently the way it was fixed in
Debian for git)?

	Sami

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]