isofs hardlink bug (inode numbers different)

isofs hardlink bug (inode numbers different)

[Volker Kuhlmann]

I am trying to back up directory trees on CD, preserving hard links.
newer versions of mkisofs are supposedly able to do this, but although
the data is written to the isofs only once, the resulting directory
entries have differing inode numbers thus making restore operations
impossible.

When I sent a bug report to the author of mkisofs, Jörg Schilling, I
got the reply

>>mkisofs 2.01a01 (i686-pc-linux-gnu)
>>mkisofs 2.0 (i686-pc-linux-gnu)
>>mkisofs 1.15a27 (i686-suse-linux)
>
>>Google shows no reference to anything which tells me that this is not
>>supposed to work, therefore I assume it's a bug.
>
>Nachdenken hilft wie in vielen Fällen auch hier:
>
>Der Bug auch hier ist da, wo es wegen schlechter SW Qualität wahrscheinlicher
>ist: Im Linux Kernel.

(Translation: thinking helps here too, like in many other cases: the bug
is in the linux kernel, where it is more likely to be due to lower
software quality.)

Insults aside, is it true that the kernel's isofs can't produce correct
inode numbers for hardlinked files? If that is the case it would
somewhat reduce the usefulness of isofs for backups.

Volker

-- 
Volker Kuhlmann			is possibly list0570 with the domain in header
http://volker.dnsalias.net/		Please do not CC list postings to me.

Re: isofs hardlink bug (inode numbers different)

[H. Peter Anvin]

Followup to:  <20030126235556.GA5560@paradise.net.nz>
By author:    Volker Kuhlmann <list0570@paradise.net.nz>
In newsgroup: linux.dev.kernel
>
> I am trying to back up directory trees on CD, preserving hard links.
> newer versions of mkisofs are supposedly able to do this, but although
> the data is written to the isofs only once, the resulting directory
> entries have differing inode numbers thus making restore operations
> impossible.
> 
> When I sent a bug report to the author of mkisofs, Jörg Schilling, I
> got the reply
> 
> >>mkisofs 2.01a01 (i686-pc-linux-gnu)
> >>mkisofs 2.0 (i686-pc-linux-gnu)
> >>mkisofs 1.15a27 (i686-suse-linux)
> >
> >>Google shows no reference to anything which tells me that this is not
> >>supposed to work, therefore I assume it's a bug.
> >
> >Nachdenken hilft wie in vielen Fällen auch hier:
> >
> >Der Bug auch hier ist da, wo es wegen schlechter SW Qualität wahrscheinlicher
> >ist: Im Linux Kernel.
> 
> (Translation: thinking helps here too, like in many other cases: the bug
> is in the linux kernel, where it is more likely to be due to lower
> software quality.)

[FWIW, Jörg is well-known for thinking that anything that isn't
Solaris is complete crap.  He's entitled to his opinions.]

> Insults aside, is it true that the kernel's isofs can't produce correct
> inode numbers for hardlinked files? If that is the case it would
> somewhat reduce the usefulness of isofs for backups.

It's sort of true.

There are inode numbers stored in RockRidge attributes, but using them
comes with some humongous caveats:

First: You have absolutely no guarantee that they are actually
unique.  Broken software could easily have written them with all
zeroes.

Second: If there are files on the CD-ROM *without* RockRidge
attributes, you can get collisions with the synthesized inode numbers
for non-RR files.

Third: If you actually rely on inode numbers to be able to find your
files, like most versions of Unix including old (but not current)
versions of Linux, then they are completely meaningless.

The basic problem is that the RR attributes are arbitrary numbers,
instead of any kind of pointer saying "I'm a hard link to this other
file over here."

There is another way to generate consistent inodes for hard links,
which is to use the data block pointer as the "inode number."  This,
however, has the problem that *ALL* zero-lenght files become "hard
links" to each other.

I have been trying to come up with a solution to this, and have pretty
much failed.  Hopefully the UDF filesystem doesn't have this
problem...

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: cris ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: isofs hardlink bug (inode numbers different)

[Frank van Maarseveen]

On Mon, Feb 03, 2003 at 07:48:06PM -0800, H. Peter Anvin wrote:
> 
> There are inode numbers stored in RockRidge attributes, but using them
> comes with some humongous caveats:
> 
> First: You have absolutely no guarantee that they are actually
> unique.  Broken software could easily have written them with all
> zeroes.
> 
> Second: If there are files on the CD-ROM *without* RockRidge
> attributes, you can get collisions with the synthesized inode numbers
> for non-RR files.

Suppose that the root of a iso9660 has entries such as '/', '.' and '..'
all with inode 2 and maybe that there are  some other indications that
the creator applied UNIX style inode numbering, wouldn't it be reasonable
to assume that inode numbers should be trusted?

In which case any wrong inode should be declared a bug in the program
that created the image?

A mount option to handle this could be useful as well.

> 
> Third: If you actually rely on inode numbers to be able to find your
> files, like most versions of Unix including old (but not current)
> versions of Linux, then they are completely meaningless.

No need to change that part, well, except maybe for knfsd if we cannot
safely export a CD-ROM but that's something I don't know.


It is annoying that wonderful zisofs backups need fixup scripts for the
hard links after a restore.

-- 
Frank

Re: isofs hardlink bug (inode numbers different)

[H. Peter Anvin]

Frank van Maarseveen wrote:
> 
> Suppose that the root of a iso9660 has entries such as '/', '.' and '..'
> all with inode 2 and maybe that there are  some other indications that
> the creator applied UNIX style inode numbering, wouldn't it be reasonable
> to assume that inode numbers should be trusted?
> 

The thing is, RockRidge attributes are a file-by-file thing.  You can
have it on half the files if you want.

Of course, since Linux isn't dependent on the inode numbers to find the
directory entries anymore (since we now have dentries), one could
rewrite the iso9660 filesystem so that it doesn't matter.  This is a
major rewrite, however.  It wouldn't be unwelcome; the current code is
pretty awful.

	-hpa

Re: isofs hardlink bug (inode numbers different)

[Kasper Dupont]

"H. Peter Anvin" wrote:
> 
> There are inode numbers stored in RockRidge attributes, but using them
> comes with some humongous caveats:
> 
> First: You have absolutely no guarantee that they are actually
> unique.  Broken software could easily have written them with all
> zeroes.

Maybe we can detect some of the cases with broken software, and
at least provide an option to turn the RockRidge inode numbers
off.

> 
> Second: If there are files on the CD-ROM *without* RockRidge
> attributes, you can get collisions with the synthesized inode numbers
> for non-RR files.

That can easily be solved. RockRidge inode numbers are multiplied
by two, and synthesized inode numbers are all odd. Of course if
the multiplication overflows a fallback to synthesized inode
numbers would be necesarry. Does any software produce inode
numbers large enough to make this a problem?

> 
> Third: If you actually rely on inode numbers to be able to find your
> files, like most versions of Unix including old (but not current)
> versions of Linux, then they are completely meaningless.

Agreed.

> 
> There is another way to generate consistent inodes for hard links,
> which is to use the data block pointer as the "inode number."  This,
> however, has the problem that *ALL* zero-lenght files become "hard
> links" to each other.

That problem can easily be solved. Simply use different methods
for zero-length files and all other files. But there might be
other problems with such an approach:

1) Could two different files have same data block pointer?
   (different sizes perhaps?)
2) Do we need a way to find metadata from the inode number?

-- 
Kasper Dupont -- der bruger for meget tid på usenet.
For sending spam use mailto:aaarep@daimi.au.dk
for(_=52;_;(_%5)||(_/=5),(_%5)&&(_-=2))putchar(_);

Re: isofs hardlink bug (inode numbers different)

[H. Peter Anvin]

Kasper Dupont wrote:
>> 
>>Second: If there are files on the CD-ROM *without* RockRidge
>>attributes, you can get collisions with the synthesized inode numbers
>>for non-RR files.
>  
> That can easily be solved. RockRidge inode numbers are multiplied
> by two, and synthesized inode numbers are all odd. Of course if
> the multiplication overflows a fallback to synthesized inode
> numbers would be necesarry. Does any software produce inode
> numbers large enough to make this a problem?
> 

We have no idea, and we will never be able to know.  They certainly 
*CAN*... they are just anonymous 32-bit values.

> 
>>Third: If you actually rely on inode numbers to be able to find your
>>files, like most versions of Unix including old (but not current)
>>versions of Linux, then they are completely meaningless.
> 
> Agreed.
> 
>>There is another way to generate consistent inodes for hard links,
>>which is to use the data block pointer as the "inode number."  This,
>>however, has the problem that *ALL* zero-lenght files become "hard
>>links" to each other.
> 
> That problem can easily be solved. Simply use different methods
> for zero-length files and all other files. But there might be
> other problems with such an approach:
> 
> 1) Could two different files have same data block pointer?
>    (different sizes perhaps?)

Theoretically yes.

> 2) Do we need a way to find metadata from the inode number?

Currently we do, but we could rewrite the code not to.

	-hpa

Re: isofs hardlink bug (inode numbers different)

[James Pearson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2635 bytes --]

>> I am trying to back up directory trees on CD, preserving hard links.
>> newer versions of mkisofs are supposedly able to do this, but although
>> the data is written to the isofs only once, the resulting directory
>> entries have differing inode numbers thus making restore operations
>> impossible.
>> 
>> When I sent a bug report to the author of mkisofs, Jörg Schilling, I
>> got the reply
>> 
>> >>mkisofs 2.01a01 (i686-pc-linux-gnu)
>> >>mkisofs 2.0 (i686-pc-linux-gnu)
>> >>mkisofs 1.15a27 (i686-suse-linux) 
>> >>Google shows no reference to anything which tells me that this is not
>> >>supposed to work, therefore I assume it's a bug.
>> >
>> >Nachdenken hilft wie in vielen Fällen auch hier:
>> >
>> >Der Bug auch hier ist da, wo es wegen schlechter SW Qualität wahrscheinlicher
>> >ist: Im Linux Kernel.
>> 
>> (Translation: thinking helps here too, like in many other cases: the bug
>> is in the linux kernel, where it is more likely to be due to lower
>> software quality.)
>
>[FWIW, Jörg is well-known for thinking that anything that isn't
>Solaris is complete crap.  He's entitled to his opinions.]
>
>> Insults aside, is it true that the kernel's isofs can't produce correct
>> inode numbers for hardlinked files? If that is the case it would
>> somewhat reduce the usefulness of isofs for backups.
>
>It's sort of true.
>
>There are inode numbers stored in RockRidge attributes, but using them
>comes with some humongous caveats:
>
>First: You have absolutely no guarantee that they are actually
>unique.  Broken software could easily have written them with all
>zeroes.
>
>Second: If there are files on the CD-ROM *without* RockRidge
>attributes, you can get collisions with the synthesized inode numbers
>for non-RR files.
>
>Third: If you actually rely on inode numbers to be able to find your
>files, like most versions of Unix including old (but not current)
>versions of Linux, then they are completely meaningless.
>
>The basic problem is that the RR attributes are arbitrary numbers,
>instead of any kind of pointer saying "I'm a hard link to this other
>file over here."
>
>There is another way to generate consistent inodes for hard links,
>which is to use the data block pointer as the "inode number."  This,
>however, has the problem that *ALL* zero-lenght files become "hard
>links" to each other.

In fact, for isofs file systems created with mkisofs, zero length files
would be hard links to the next file with data written to the CD.

You would also have the problem that there is nothing to prevent files
with different owner, permissions, dates, etc. sharing the same file data
...

James Pearson