Re: [Qemu-devel] [PULL v1 0/2] Merge IO 2017/11/06

["Daniel P. Berrange" <berrange@redhat.com>]

On Mon, Nov 06, 2017 at 05:58:41PM +0000, Peter Maydell wrote:
> On 6 November 2017 at 17:43, Daniel P. Berrange <berrange@redhat.com> wrote:
> > On Mon, Nov 06, 2017 at 04:11:56PM +0000, Peter Maydell wrote:
> >> On 6 November 2017 at 15:33, Daniel P. Berrange <berrange@redhat.com> wrote:
> >> > The following changes since commit ec7a8bf0b8f7dc7288fe8745464ee8217528cc6c:
> >> >
> >> >   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2017-11-06 10:04:16 +0000)
> >> >
> >> > are available in the git repository at:
> >> >
> >> >   git://github.com/berrange/qemu tags/pull-qio-2017-11-06-1
> >> >
> >> > for you to fetch changes up to 74c0035408d5b327eac5985d976bca3009e0c5d6:
> >> >
> >> >   tests: Add test-listen - a stress test for QEMU socket listen (2017-11-06 11:49:15 +0000)
> >> >
> >> > ----------------------------------------------------------------
> >> > Merge qio 2017-11-06 v1
> >> >
> >> > ----------------------------------------------------------------
> >>
> >> Hi. I'm afraid this fails on various hosts.
> >
> > Sigh, anything network related in unit tests is soo hard to get right :-(
> > I was hoping the previous fail you saw with this test was all due to the
> > file descriptor leak the first patch fixed, but I guess not....
> >
> >>
> >> arm 32 bit (in a chroot on a 64 bit box):
> >> TEST: tests/test-listen... (pid=319)
> >>   /socket/listen-serial/ipv4:                                          OK
> >>   /socket/listen-serial/ipv6:
> >> ** Failed to assign a port to
> >> thread 0 (errno = 98)
> >> ** Failed to assign a port to thread 1 (errno = 98)
> >> ** Failed to assign a port to thread 2 (errno = 98)
> >> ** Failed to assign a port to thread 3 (errno = 98)
> >> ** Failed to assign a port to thread 4 (errno = 98)
> >> [...repeats all the way up to...]
> >> ** Failed to assign a port to thread 197 (errno = 98)
> >> ** Failed to assign a port to thread 198 (errno = 98)
> >> ** Failed to assign a port to thread 199 (errno = 98)
> >> **
> >> ERROR:/home/peter.maydell/qemu/tests/test-listen.c:195:listen_compete_nthr:
> >> assertion failed (failed_listens == 0): (200 == 0)
> >
> > Hmm, every single thread failed, which suggests some fundamental config in
> > this environment that prevents any use of networking at all from tests.
> >
> > Anything interesting about this build env related to network availability
> > that would prevent use of even localhost ?
> 
> You'll notice that the IPV4 test worked fine. I assume this is an
> IPV6 specific problem.

Oh yes, so it is.  The strange thing is that the test does check if we have
a protocol available by doing a test bind() at start.

> >> Test failure on OpenBSD:
> >>
> >> TEST: tests/test-listen... (pid=9208)
> >>   /socket/listen-serial/ipv4:
> >> ** Failed to assign a port to
> >> thread 124 (errno = 48)
> >> ** Failed to assign a port to thread 125 (errno = 48)
> >> ** Failed to assign a port to thread 126 (errno = 48)
> >> ** Failed to assign a port to thread 127 (errno = 48)
> >> [repeats up to]
> >> ** Failed to assign a port to thread 197 (errno = 48)
> >> ** Failed to assign a port to thread 198 (errno = 48)
> >> ** Failed to assign a port to thread 199 (errno = 48)
> >> **
> >> ERROR:/home/qemu/tests/test-listen.c:195:listen_compete_nthr:
> >> assertion failed (failed_listens == 0): (76 == 0)
> >> FAIL
> >> GTester: last random seed: R02S17d50ab00329e514ffbe630191e76e83
> >> (pid=80652)
> >
> > Only a subset failed, which is more interesting - this is definitely
> > more like the problem seen when FD leaks. Could you mention what the
> > ulimits are for your OpenBSD host - I wondering if it hits max FDs
> > due to having a parallel builds / test run.
> 
> # ulimit -a
> time(cpu-seconds)    unlimited
> file(blocks)         unlimited
> coredump(blocks)     unlimited
> data(kbytes)         33554432
> stack(kbytes)        8192
> lockedmem(kbytes)    670784
> memory(kbytes)       2010588
> nofiles(descriptors) 128
> processes            1310
> 
> That nofiles limit looks quite close to the number where we
> start failing...

Ok, I think I'll just make the test skip unless nofiles >= 1024
so we have a good margin of safety when things run very parallel


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|