[PATCH 0/2 V2] xfs_repair: improve caching in the face of ENOMEM

[Eric Sandeen <sandeen@redhat.com>]

V2:
- Flip the order of the patches; put the bugfix first and the
  cache expansion guard second, at which point it's just defensive.

- Be more explicit about an enomem return due to a real allocation
  failure; don't try to figure that out indirectly as before.

- Actually care about the return values as pointed out by djwong.

There are problems with how we react to cache allocation failures that
are more apparent before:
6b6239689e36 ("libxfs: unmap xmbuf pages to avoid disaster")
landed, but with enough pressure they show up again.

Two problems: cache_expand has no bounds checking, so call it enough
and c_maxcount will overflow, which typically results in an infinite
loop.

That can be triggered by the fact that today, an underlying ENOMEM
can look like a "cache too small" return, so we double when that
won't help at all. Do that enough times and kaboom.

I can trigger this with something like:

mkfs.xfs -d file,name=testfile.img,agcount=10000,size=1t
echo 1000 > /proc/sys/vm/max_map_count
xfs_repair -n -P -m 1280 testfile.img

to force an ENOMEM from the max map count, though the actual
max_map_count value needed varies depending on the machine I'm on,
for some reason.

These were quickly tested with ./check -g quick