Use mongoose to test smart-http unconditionally?

Use mongoose to test smart-http unconditionally?

[Duy Nguyen]

I was thinking of an alternative to apache for testing smart-http so
that most of http tests could always run. Mongoose [1] looks like a
good candidate to bundle with git. Just one pair of source files,
mongoose.[ch], a mainloop wrapper and we have an http server. Just
wondering, do we rely on any apache-specific features? I'm not so
familiar with lib-httpd.sh..

[1] https://github.com/cesanta/mongoose
-- 
Duy

Re: Use mongoose to test smart-http unconditionally?

[Shawn Pearce]

On Wed, Dec 4, 2013 at 2:53 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> I was thinking of an alternative to apache for testing smart-http so
> that most of http tests could always run. Mongoose [1] looks like a
> good candidate to bundle with git. Just one pair of source files,
> mongoose.[ch], a mainloop wrapper and we have an http server. Just
> wondering, do we rely on any apache-specific features? I'm not so
> familiar with lib-httpd.sh..

I don't think we do anything Apache specific in the test suite. It
basically relies on CGI execution, being able to configure a URL to
serve a directory, and making some URLs 404 or 500 so we can emulate a
broken or failing server to test the client behavior in those
conditions. At worst that 404/500 forced failure mode could be handled
by a CGI.

> [1] https://github.com/cesanta/mongoose

Re: Use mongoose to test smart-http unconditionally?

[Jeff King]

On Wed, Dec 04, 2013 at 10:13:11AM -0800, Shawn Pearce wrote:

> On Wed, Dec 4, 2013 at 2:53 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> > I was thinking of an alternative to apache for testing smart-http so
> > that most of http tests could always run. Mongoose [1] looks like a
> > good candidate to bundle with git. Just one pair of source files,
> > mongoose.[ch], a mainloop wrapper and we have an http server. Just
> > wondering, do we rely on any apache-specific features? I'm not so
> > familiar with lib-httpd.sh..
> 
> I don't think we do anything Apache specific in the test suite. It
> basically relies on CGI execution, being able to configure a URL to
> serve a directory, and making some URLs 404 or 500 so we can emulate a
> broken or failing server to test the client behavior in those
> conditions. At worst that 404/500 forced failure mode could be handled
> by a CGI.

I don't think there's anything apache specific, but there's a fair bit
of config for handling various auth scenarios. It's stuff I'd expect any
decent server implementation to handle, but somebody actually needs to
go through and translate all of the config to mongoose.

I've been tempted to add lighttpd support, as I generally find its
config much more readable (and less prone to breaking during upgrades).
But I think it would be a mistake to support multiple servers, as it
would mean updates to the tests need to hit all of the servers. If
mongoose gives a sane lowest common denominator, that's fine with me.

I don't know if it is worth all that much effort, though. I suppose it
could get us more exposure to the httpd tests, but I do not know if it
would be a good idea to turn them on by default anyway. They touch
global machine resources (like ports) that can cause conflicts or test
failures. I assume that is the reason we do not turn on git-daemon tests
by default (though perhaps it would be better in both cases to have it
on by default and let people with special needs, like running multiple
test instances at once, turn it off).

-Peff

Re: Use mongoose to test smart-http unconditionally?

[Jonathan Nieder]

Jeff King wrote:

> I don't know if it is worth all that much effort, though. I suppose it
> could get us more exposure to the httpd tests, but I do not know if it
> would be a good idea to turn them on by default anyway. They touch
> global machine resources (like ports) that can cause conflicts or test
> failures. I assume that is the reason we do not turn on git-daemon tests
> by default

Yup, that's why I don't run them.

For what it's worth, when I build git and run tests I tend to be in an
environment with apache available, but I'm too lazy to configure git's
tests to pick the right port and make sure it is reserved and so on.
Perhaps there's some way to help lazy people in the same boat?  (E.g.,
picking a port randomly and skipping instead of failing a test when
it's taken or something)

Re: Use mongoose to test smart-http unconditionally?

[Duy Nguyen]

On Thu, Dec 5, 2013 at 6:28 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Jeff King wrote:
>
>> I don't know if it is worth all that much effort, though. I suppose it
>> could get us more exposure to the httpd tests, but I do not know if it
>> would be a good idea to turn them on by default anyway. They touch
>> global machine resources (like ports) that can cause conflicts or test
>> failures. I assume that is the reason we do not turn on git-daemon tests
>> by default
>
> Yup, that's why I don't run them.
>
> For what it's worth, when I build git and run tests I tend to be in an
> environment with apache available, but I'm too lazy to configure git's
> tests to pick the right port and make sure it is reserved and so on.
> Perhaps there's some way to help lazy people in the same boat?  (E.g.,
> picking a port randomly and skipping instead of failing a test when
> it's taken or something)

Sounds like good first steps.
-- 
Duy

Re: Use mongoose to test smart-http unconditionally?

[Jeff King]

On Wed, Dec 04, 2013 at 03:28:00PM -0800, Jonathan Nieder wrote:

> For what it's worth, when I build git and run tests I tend to be in an
> environment with apache available, but I'm too lazy to configure git's
> tests to pick the right port and make sure it is reserved and so on.
> Perhaps there's some way to help lazy people in the same boat?  (E.g.,
> picking a port randomly and skipping instead of failing a test when
> it's taken or something)

What level of magic do you need?

For most people, "apt-get install apache && make GIT_TEST_HTTPD=yes
test" is enough to run the tests. It uses ports in the 5000-range; this
_can_ conflict with other services the user is running, but it should
not usually (we don't match anything in a typical /etc/services file).
And each script uses its own port, so running the tests in parallel is
fine.

If you are planning on running "make test" from multiple git checkouts
at the same time, though, they will conflict.

The failure mode is reasonable. You should get something like:

  $ GIT_TEST_HTTPD=1 LIB_HTTPD_PORT=80 ./t5540-http-push.sh
  1..0 # SKIP skipping test, web server setup failed

We could do better by retrying with a range of ports, but unless
somebody really needs that, I'd prefer to avoid complicating it more.

The one thing I have occasionally run into is a stale apache hanging
around. But I think that only happens when I am abusing the test script
(e.g., sticking a test_pause in, poking around, and then not letting the
script complete and do its cleanup).

So is the current behavior good enough to meet your needs and you
didn't know it, or do we need more?

-Peff

Re: Use mongoose to test smart-http unconditionally?

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> I was thinking of an alternative to apache for testing smart-http
> so that most of http tests could always run.  Mongoose [1] looks
> like a good candidate to bundle with git. Just one pair of source
> files, mongoose.[ch], a mainloop wrapper and we have an http
> server.

Hmmmm.  How would the high-level integration look like?

 - we add contrib/mongoose/*;

 - in t/Makefile, we:

   . set GIT_TEST_HTTPD to yes, unless it is already set to another value;

   . set LIB_HTTPD_PATH to $GIT_BUILD_DIR/contrib/mongoose/mongoose,
     unless it is already set to another value;

   . if LIB_HTTPD_PATH is set to our mongoose and if it hasn't been
     built, go ../contrib/mongoose and build it.

 - we teach lib-httpd.sh to trigger the DEFAULT_HTTPD_PATH
   computation when LIB_HTTPD_PATH is set to 'system-apache', so
   that people can test with their installed apache if they choose
   to; and

 - we teach lib-httpd.sh to write out an appropriate configuration
   for the mongoose server.

That would force people to run http tests by turning it from an
opt-in option to an opt-out option.

Or were you thinking about embedding mongoose in the git executable?

Re: Use mongoose to test smart-http unconditionally?

[Jeff King]

On Wed, Dec 04, 2013 at 12:09:09PM -0800, Junio C Hamano wrote:

> Duy Nguyen <pclouds@gmail.com> writes:
> 
> > I was thinking of an alternative to apache for testing smart-http
> > so that most of http tests could always run.  Mongoose [1] looks
> > like a good candidate to bundle with git. Just one pair of source
> > files, mongoose.[ch], a mainloop wrapper and we have an http
> > server.
> 
> Hmmmm.  How would the high-level integration look like?
> 
>  - we add contrib/mongoose/*;
> 
>  - in t/Makefile, we:
> 
>    . set GIT_TEST_HTTPD to yes, unless it is already set to another value;
> 
>    . set LIB_HTTPD_PATH to $GIT_BUILD_DIR/contrib/mongoose/mongoose,
>      unless it is already set to another value;
> 
>    . if LIB_HTTPD_PATH is set to our mongoose and if it hasn't been
>      built, go ../contrib/mongoose and build it.

I think building it on-demand is probably too much effort. If it is
portable, then it should not be a problem to just build it along with
the rest of git. If it is not, then we should rethink whether it is
worth including.

>  - we teach lib-httpd.sh to trigger the DEFAULT_HTTPD_PATH
>    computation when LIB_HTTPD_PATH is set to 'system-apache', so
>    that people can test with their installed apache if they choose
>    to; and

I do not think we want to allow run-time switching between an embedded
solution and apache. That would mean that we have to keep two sets of
http-server config in sync.

> Or were you thinking about embedding mongoose in the git executable?

I don't think it makes sense to embed it in git, but it could easily be
test-httpd.

The rollout would be:

  1. add contrib/mongoose/*

  2. add test-httpd which links against mongoose, built by default in the
     Makefile

  3. convert lib-httpd/apache.conf into mongoose config as necessary

  4. convert lib-httpd.sh to run test-httpd instead of LIB_HTTPD_PATH

  5. delete apache.conf, LIB_HTTPD_PATH and any other apache remnants

  6. default GIT_TEST_HTTPD to yes

Step 3 is the part where I would anticipate trouble (i.e., finding out
that the new server does not do everything the tests expect).

-Peff

Re: Use mongoose to test smart-http unconditionally?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> The rollout would be:
>
>   1. add contrib/mongoose/*
>
>   2. add test-httpd which links against mongoose, built by default in the
>      Makefile
>
>   3. convert lib-httpd/apache.conf into mongoose config as necessary
>
>   4. convert lib-httpd.sh to run test-httpd instead of LIB_HTTPD_PATH
>
>   5. delete apache.conf, LIB_HTTPD_PATH and any other apache remnants
>
>   6. default GIT_TEST_HTTPD to yes
>
> Step 3 is the part where I would anticipate trouble (i.e., finding out
> that the new server does not do everything the tests expect).

If it involves making things not tested with apache, I'd actually be
less supportive for the whole plan.  I thought the primary objective
was to encourage people who currently are _not_ running httpd tests
by making a lightweight server available out of the box, robbing an
excuse "my box does not have apache installed" from them.

As long as a server supports bog standard CGI interface, smart-http
should work the same way with any such server.  For that reason, it
should be theoretically sufficient to test with one non-apache
server (i.e. mongoose) for the purpose of making sure _our_ end of
the set-up works, but still...

Re: Use mongoose to test smart-http unconditionally?

[Jeff King]

On Wed, Dec 04, 2013 at 02:53:13PM -0800, Junio C Hamano wrote:

> If it involves making things not tested with apache, I'd actually be
> less supportive for the whole plan.

I hadn't really considered that angle. Apache is a much more realistic
real-world deployment. We give advice for it in git-http-backend(1), and
the tests do check that that advice works (OTOH, we also give advice for
lighttpd, but that is not checked in the test scripts).

> I thought the primary objective was to encourage people who currently
> are _not_ running httpd tests by making a lightweight server available
> out of the box, robbing an excuse "my box does not have apache
> installed" from them.

Whether we get rid of apache or not, I think a new lightweight server
would fulfill that goal. I just did not want the maintenance burden of
managing multiple configs (and our test harness apache config has grown
non-trivial).

> As long as a server supports bog standard CGI interface, smart-http
> should work the same way with any such server.  For that reason, it
> should be theoretically sufficient to test with one non-apache
> server (i.e. mongoose) for the purpose of making sure _our_ end of
> the set-up works, but still...

There are definitely subtleties between servers. For example, when I
worked on fetching bundles over http a while back, there was a big
difference between lighttpd and apache. A request for
"http://example.com/foo.bundle/info/refs" would return the bundle under
lighttpd, but not under apache (for an apache server, we would have to
make a fallback request). The client needs to be able to handle both
scenarios gracefully.

That's a case where it would be nice to be able to test _both_ cases,
and that may be an argument for having multiple (or trying to configure
apache to do both behaviors). But it shows that there may be subtle
differences between a fake test server and a real deployment.

So thinking on it more, I'm somewhat less enthusiastic about mongoose.

-Peff