Stable inode numbers

[Vito Caputo <vito.caputo@coreos.com>]

Hello list,

We're receiving bug reports from users simply untarring archives onto overlayfs
mounts (docker + overlayfs) where tar fails with "Directory renamed before its
status could be extracted" errors.

This is triggered in GNU tar when the inode number of an ancestor directory
changes unexpectedly in the deferred application of ancestral metadata.

It's not consistently reproducible, but in a slightly memory-constrained vm it
is very easy to reproduce.

What appears to be happening is the tar extraction activity is churning through
the kernel's cache so a subsequent lookup of the directory in question returns
a new inode number.  One can easily simulate this using `stat` and `echo 2 >
/proc/sys/vm/drop_caches` to observe the overlayfs inode number change across
the cache drop.

The seemingly erratic nature of this failure is particularly frustrating for
users, as it gives the impression things _should_ work and just randomly aren't
due to a bug.  Scrutiny of the overlayfs implementation reveals that unstable
inode numbers is intentional, and spurious failures of applications which
notice intersecting relevant cache evictions should be expected.

Are there any plans or strategies for changing this situation in overlayfs?

I can think of a few potential solutions, but they all tend to revolve around
persistently allocating inode numbers on first lookup, which of course requires
both space and time overlayfs currently elides.

Cheers,
Vito Caputo