open files limit

open files limit

[Pete Wyckoff]

Somebody at $work found this problem:

    $ git ls-files -s | wc
    error: packfile .git/objects/pack/pack-1627e77da82bbb36118762649c8aa88c05664b1e.pack cannot be accessed
    [..lots more similar errors..]

Turns out his shell's open file descriptor limit was 500.  And
there are 1600 pack files in the repo.

Increasing the descriptor limit to 1024 fixed it.  I could
probably get him to repack, which may also fix it.

Does it seem feasible to look for EMFILE errors and close
some packs?  Or at least spit out a more intuitive error?

		-- Pete

Re: open files limit

[Shawn Pearce]

On Fri, Aug 12, 2011 at 08:15, Pete Wyckoff <pw@padd.com> wrote:
> Somebody at $work found this problem:
>
>    $ git ls-files -s | wc
>    error: packfile .git/objects/pack/pack-1627e77da82bbb36118762649c8aa88c05664b1e.pack cannot be accessed
>    [..lots more similar errors..]
>
> Turns out his shell's open file descriptor limit was 500.  And
> there are 1600 pack files in the repo.
>
> Increasing the descriptor limit to 1024 fixed it.  I could
> probably get him to repack, which may also fix it.
>
> Does it seem feasible to look for EMFILE errors and close
> some packs?  Or at least spit out a more intuitive error?

What version of Git? I remember fixing this already.... :-)

-- 
Shawn.

Re: open files limit

[Pete Wyckoff]

spearce@spearce.org wrote on Fri, 12 Aug 2011 09:09 -0700:
> On Fri, Aug 12, 2011 at 08:15, Pete Wyckoff <pw@padd.com> wrote:
> > Somebody at $work found this problem:
> >
> >    $ git ls-files -s | wc
> >    error: packfile .git/objects/pack/pack-1627e77da82bbb36118762649c8aa88c05664b1e.pack cannot be accessed
> >    [..lots more similar errors..]
> >
> > Turns out his shell's open file descriptor limit was 500.  And
> > there are 1600 pack files in the repo.
> >
> > Increasing the descriptor limit to 1024 fixed it.  I could
> > probably get him to repack, which may also fix it.
> >
> > Does it seem feasible to look for EMFILE errors and close
> > some packs?  Or at least spit out a more intuitive error?
> 
> What version of Git? I remember fixing this already.... :-)

Initially 1.7.5.4.  Same problem on 1.7.6 and master.  

I have your "Limit file descriptors used by packs" (c793430, 28
feb 2011).

It fails here:

	if (!is_pack_valid(p)) {
		error("packfile %s cannot be accessed", p->pack_name);
		goto next;
	}

because p->pack_fd is -1, because an earlier git_open_noatime()
got EMFILE.  The function unuse_one_window() is never able to
find anything to close.

I'll do some more debugging this weekend.  Thanks for pointing
out that it _should_ be fixed.

		-- Pete

[RFC PATCH] test showing EMFILE error with too many packs

[Pete Wyckoff]

pw@padd.com wrote on Fri, 12 Aug 2011 10:26 -0700:
> spearce@spearce.org wrote on Fri, 12 Aug 2011 09:09 -0700:
> > On Fri, Aug 12, 2011 at 08:15, Pete Wyckoff <pw@padd.com> wrote:
> > > Somebody at $work found this problem:
> > >
> > >    $ git ls-files -s | wc
> > >    error: packfile .git/objects/pack/pack-1627e77da82bbb36118762649c8aa88c05664b1e.pack cannot be accessed
> > >    [..lots more similar errors..]
> > >
> > > Turns out his shell's open file descriptor limit was 500.  And
> > > there are 1600 pack files in the repo.
> > >
> > > Increasing the descriptor limit to 1024 fixed it.  I could
> > > probably get him to repack, which may also fix it.
> > >
> > > Does it seem feasible to look for EMFILE errors and close
> > > some packs?  Or at least spit out a more intuitive error?
> > 
> > What version of Git? I remember fixing this already.... :-)
> 
> Initially 1.7.5.4.  Same problem on 1.7.6 and master.  
> 
> I have your "Limit file descriptors used by packs" (c793430, 28
> feb 2011).
> 
> It fails here:
> 
> 	if (!is_pack_valid(p)) {
> 		error("packfile %s cannot be accessed", p->pack_name);
> 		goto next;
> 	}
> 
> because p->pack_fd is -1, because an earlier git_open_noatime()
> got EMFILE.  The function unuse_one_window() is never able to
> find anything to close.
> 
> I'll do some more debugging this weekend.  Thanks for pointing
> out that it _should_ be fixed.

Here's a test case (on master).  It is easy to repack the
repository to avoid this situation, so I'm not convinced this is
a bug.

But it would be nice at least to report what's going on when we
run out of file descriptors.

The current error is:

    $ ../git commit -m fileN
    error: packfile .git/objects/pack/pack-7bff59b72a19f371ccd3e4ffefaac6cd1e07c6ed.pack cannot be accessed
    error: invalid object 100644 a7d02de3336c72e6d2d24f382b82195dbf625404 for 'file47'
    error: Error building trees


		-- Pete

--------8<--------

From: Pete Wyckoff <pw@padd.com>
Date: Sat, 13 Aug 2011 15:57:55 -0400
Subject: [RFC PATCH] test showing EMFILE error with too many packs

In a repository with too many pack files, it is
possible to have too many open at once and exceed
the process file descriptor limit.

This test shows one failure with an artificially low limit
of 50.  The same failure happens at a "normal" value
like 1024, but takes longer.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/t5308-pack-emfile.sh |   29 +++++++++++++++++++++++++++++
 1 files changed, 29 insertions(+), 0 deletions(-)
 create mode 100755 t/t5308-pack-emfile.sh

diff --git a/t/t5308-pack-emfile.sh b/t/t5308-pack-emfile.sh
new file mode 100755
index 0000000..66c62a7
--- /dev/null
+++ b/t/t5308-pack-emfile.sh
@@ -0,0 +1,29 @@
+#!/bin/sh
+
+test_description='repos with too many packs should not fail'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	for i in {1..60}
+	do
+		echo "$i" >"file$i" &&
+		git add "file$i" &&
+		test_tick &&
+		git commit -q -m "$i" &&
+		git repack -q
+	done
+	git prune-packed -q
+'
+
+# run in a subshell since limits cannot be increased
+test_expect_failure 'commit fail with too many packs' '
+	(
+	ulimit -n 50 &&
+	>"fileN" &&
+	git add "fileN" &&
+	git commit -q -m "fileN"
+	)
+'
+
+test_done
-- 
1.7.5.4