Kernel HTTP server: Patch for name based virtual domains

Kernel HTTP server: Patch for name based virtual domains

[Christoph Lameter]

This is a patch against 2.4.0 which enables the use of virtual domains. I
ma running multiple domains on my server and was not able to use the
kernel webserver. The machine is only a Cyrix 166 and since it also serves
lots of files using the kernel server is a tremendous boost.

Settings for khttpd:

echo 80 >/proc/sys/net/khttpd/serverport
echo 8080 >/proc/sys/net/khttpd/clientport
echo "(lameter,com,www.lameter.com)/home/clameter/lameter:(vdm.org)/home/clameter/vdm:/var/www" >/proc/sys/net/khttpd/documentroot
echo 1 >/proc/sys/net/khttpd/start

The docs are included in the source below.

diff -urN linux-orig/net/khttpd/rfc.c linux/net/khttpd/rfc.c
--- linux-orig/net/khttpd/rfc.c	Mon Aug 23 10:41:25 1999
+++ linux/net/khttpd/rfc.c	Wed Jan 10 19:48:04 2001
@@ -23,6 +23,34 @@
  *
  ****************************************************************/
 
+/*
+  Virtual Host parser by Christoph Lameter <christoph@lameter.com>, January 10, 2001
+  
+  The proc file in /proc/sys/net/khttpd/document can either be set to a single webroot or
+  can contain a specification of a pathlist of webroots. Each path is prefixed by the virtual
+  hostnames in parentheses. For example.
+  
+  echo "(home.lameter.com,localhost)/a/christoph/new:(virtual3.cc.com)/home/richard:/var/www" >documentroot
+  
+  would redirect requests for http://lameter.com and http://localhost to /a/christoph/new.
+  Requests for http://virtual3.cc.com will be served from /home/richard.
+  Requests for other domains will be served from /var/www.
+
+  The syntax for specifying virtual host mappings is:
+  
+  <virtual-hostmap< { , <virtual_hostmap> } : <defaultpath>
+  
+  where
+  
+  <defaultmap>		=	Unix path
+  <virtual-hostmap>	=	( <hostname> { , <hostname> } ) : <path>
+  <hostname>		=	legal DNS name of a host
+  <path>		=	Unix path
+  
+  An additional enhancement of khttpd is that no attempt is made to open directories.
+  Instead index.html is appended to a directory to simulate classic webserver behavior.
+  
+*/
 
 #include <linux/kernel.h>
 
@@ -280,6 +308,8 @@
 void ParseHeader(char *Buffer,const int length, struct http_request *Head)
 {
 	char *Endval,*EOL,*tmp;
+	char *filename=NULL;
+	int filenamelen;
 	
 	EnterFunction("ParseHeader");
 	Endval = Buffer + length;
@@ -311,7 +341,6 @@
 		
 		if (strncmp("GET ",Buffer,4)==0)
 		{
-			int PrefixLen;
 			Buffer+=4;
 			
 			tmp=strchr(Buffer,' ');
@@ -323,12 +352,9 @@
 				Head->HTTPVER = 10;
 			
 			if (tmp>Endval) continue;
-			
-			strncpy(Head->FileName,sysctl_khttpd_docroot,sizeof(Head->FileName));
-			PrefixLen = strlen(sysctl_khttpd_docroot);
-			Head->FileNameLength = min(255,tmp-Buffer+PrefixLen);		
-			
-			strncat(Head->FileName,Buffer,min(255-PrefixLen,tmp-Buffer));
+
+			filename=Buffer;
+			filenamelen=tmp-Buffer;
 					
 			Buffer=EOL+1;	
 #ifdef BENCHMARK
@@ -369,6 +395,67 @@
 		}
 #endif		
 		Buffer = EOL+1;  /* Skip line */
+	}
+	if (filename) {
+
+	    char PrefixLen;
+
+	    if (sysctl_khttpd_docroot[0]=='(') {
+
+		char *p=sysctl_khttpd_docroot;
+		char *q;
+
+		while (*p=='(') {
+			do {
+				p++;
+				q=p;	/* Mark position of virtual hostname */
+
+				while (*p && *p!=')' && *p!=',') p++;		/* Find the end of the hostname */
+
+				if (*p==0) { printk(KERN_ERR ") missing in virtual host string.\n");return; }
+
+				if (strnicmp(Head->Host,q,p-q)==0) {
+					/* virtual hostname match */
+					while (*p && *p!=')') p++;	/* Skip to the end of the vhostname list */
+					if (*p == 0) printk(KERN_ERR "path missing for virtual host");
+
+					p++;
+					goto rootfound;
+				}
+			} while (*p==',');
+
+			if (*p==')') { /* at end of virtual host names need to skip the webroot since nothing matched */
+
+				while (*p && *p!=':') p++;
+				if (*p==0) { printk(KERN_ERR "closing : of virtual host specification missing.\n");return; }
+				p++;
+			}
+
+		}
+	rootfound:
+		if (*p==0) { printk(KERN_ERR "Error in virtual host specification.\n");return; }
+
+		/* p is pointing to matching webroot for this virtual host or the default root */
+		q=p;while (*p && *p!=':') p++;		/* Find the end of the pathname */
+		PrefixLen=p-q;
+		strncpy(Head->FileName,q,PrefixLen);
+
+	    } else {
+
+	       strncpy(Head->FileName,sysctl_khttpd_docroot,sizeof(Head->FileName));
+	       PrefixLen = strlen(sysctl_khttpd_docroot);
+
+	    }
+
+	    Head->FileNameLength = min(255,filenamelen+PrefixLen);		
+	    strncat(Head->FileName,filename,min(255-PrefixLen,filenamelen));
+	    
+	    if (Head->FileName[Head->FileNameLength-1]=='/' && Head->FileNameLength<255-10) {
+	    
+	    	/* A directory! khttpd will not be able to successfully open it. But it might be able to access index.html */
+	    	memcpy(Head->FileName+Head->FileNameLength,"index.html",10);
+	    	Head->FileNameLength+=10;
+	    }
 	}
 	LeaveFunction("ParseHeader");
 }


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

khttpd beaten by boa

[Christoph Lameter]

I got into a bragging game whose webserver is the fastest with Jim Nelson
one of the authors of the boa webserver. We finally settled on the Zeus
test to decide the battle.

First boa won hands down because it supports persistant connections. Boa
on port 6000. Khttpd on port 80:
 
clameter@melchi:~$ ./zb localhost /index.html -k -c 215 -n 20000 -p 6000
 
---
Server:                 Boa/0.94.8.3
Doucment Length:        1666
Concurency Level:       215
Time taken for tests:   33.865 seconds
Complete requests:      20000
Failed requests:        0
Keep-Alive requests:    20001
Bytes transfered:       37882109
HTML transfered:        33321666
Requests per seconds:   590.58
Transfer rate:          1118.62 kb/s
 
Connnection Times (ms)
           min   avg   max
Connect:     0     2   346
Total:     258   360   485
---

clameter@melchi:~$ ./zb localhost /index.html -k -c 215 -n 20000
 
---
Server:                 kHTTPd/0.1.6
Doucment Length:        1666
Concurency Level:       215
Time taken for tests:   101.735 seconds
Complete requests:      20000
Failed requests:        0
Keep-Alive requests:    0
Bytes transfered:       37096378
HTML transfered:        33643204
Requests per seconds:   196.59
Transfer rate:          364.64 kb/s
 
Connnection Times (ms)
           min   avg   max
Connect:    36   438  1084
Total:     394  1070  2009
---


Then we decided to switch persistant connection off... But boa still wins.

clameter@melchi:~$ ./zb localhost /index.html -c 215 -n 20000 -p 6000
 
---
Server:                 Boa/0.94.8.3
Doucment Length:        1666
Concurency Level:       215
Time taken for tests:   88.040 seconds
Complete requests:      20000
Failed requests:        0
Bytes transfered:       37352000
HTML transfered:        33528250
Requests per seconds:   227.17
Transfer rate:          424.26 kb/s
 
Connnection Times (ms)
           min   avg   max
Connect:     1   305  3417
Total:     458   932  4232
---

This shows the following problems with khttpd:

1. Connect times are on average longer than boa. Why???

2. Transfers also take longer,


What is wrong here? I would expect transferates of a 3-4 megabytes over a
localhost interface. The file is certainly in some kind of cache.
 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: khttpd beaten by boa

[Lars Marowsky-Bree]

On 2001-01-11T22:20:56,
   Christoph Lameter <christoph@lameter.com> said:

> Then we decided to switch persistant connection off... But boa still wins.
> 
> What is wrong here? I would expect transferates of a 3-4 megabytes over a
> localhost interface. The file is certainly in some kind of cache.

This just goes on to show that khttpd is unnecessary kernel bloat and can be
"just as well" handled by a userspace application, minus some rather very
special cases which do not justify its inclusion into the main kernel.

Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Perfection is our goal, excellence will be tolerated. -- J. Yahl

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: khttpd beaten by boa

[David S. Miller]

Lars Marowsky-Bree writes:
 > This just goes on to show that khttpd is unnecessary kernel bloat
 > and can be "just as well" handled by a userspace application, minus
 > some rather very special cases which do not justify its inclusion
 > into the main kernel.

My take on this is that khttpd is unmaintained garbage.

TUX is evidence that khttpd can be done properly and
beat the pants off of anything done in userspace.

Later,
David S. Miller
davem@redhat.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

lies, lies and web benchmarks :-)

[Ingo Molnar]

On Thu, 11 Jan 2001, Christoph Lameter wrote:

> I got into a bragging game whose webserver is the fastest with Jim
> Nelson one of the authors of the boa webserver. We finally settled on
> the Zeus test to decide the battle.

may i add my (hopefully comparable) TUX 2.0 numbers to this bragging game
:-)

TUX had logging turned on, a 1-CPU 500 PIII MHz system (with enough RAM)
was used for the test. UP kernel, nohighmem. The used code is 2.4.0-ac4 +
DaveM's zerocopy patch + Jen's blk patch + TUX 2.0 patch.

(to make these results somewhat comparable, what system did you use?)

> First boa won hands down because it supports persistant connections. Boa
> on port 6000. Khttpd on port 80:
>
> clameter@melchi:~$ ./zb localhost /index.html -k -c 215 -n 20000 -p 6000

> Server:                 Boa/0.94.8.3
> Doucment Length:        1666
> Requests per seconds:   590.58

> Server:                 kHTTPd/0.1.6
> Doucment Length:        1666
> Requests per seconds:   196.59

well, TUX supports persistent (keepalive, ie. lightweight) HTTP
connections as well:

over localhost (like the above test) it gives:

 m:~> ./zb localhost /index.html -k -c 215 -n 20000

 Server:                 TUX
 Document Length:        1666
 Complete requests:      20000
 Failed requests:        0
 Requests per seconds:   12658.23

 Connnection Times (ms)
            min   avg   max
 Connect:     0     0    11
 Total:       5    16    31

Over 100mbit Ethernet (using eepro100) TUX does:

 h:~> ./zb m /index.html -k -c 215 -n 20000

 Server:                 TUX
 Document Length:        1666
 Complete requests:      20000
 Failed requests:        0
 Requests per seconds:   6033.18
 Transfer rate:          11002.97 kb/s

 Connnection Times (ms)
            min   avg   max
 Connect:     0     0    12
 Total:       3    32  3250

As visible from the 'Transfer rate', the 100 mbit link is fully saturated.

The eepro100 was not running in zerocopy mode, so all data was copied
once. As a comparison, over 1000 mbit Ethernet with a native zero-copy
driver (SysKonnect), TUX does:

 Server:                 TUX
 Document Length:        1666
 Complete requests:      20000
 Failed requests:        0
 Keep-Alive requests:    20094
 Requests per seconds:   12812.30

 Connnection Times (ms)
            min   avg   max
 Connect:     0     0    12
 Total:      10    16    29

(but the server is at 70% CPU utilization in the gigabit test - the
dual-PIII/500 client is 100% CPU utilized and thus not fast enough to
saturate the server. With two clients it does about 15000 reqs/sec.)

> Then we decided to switch persistant connection off... But boa still wins.
>
> clameter@melchi:~$ ./zb localhost /index.html -c 215 -n 20000 -p 6000

> Server:                 Boa/0.94.8.3
> Doucment Length:        1666
> Requests per seconds:   227.17

with normal, non-keepalive HTTP requests, TUX 2.0 over localhost does:

 m:~> ./zb localhost /index.html -c 215 -n 20000

 Server:                 TUX
 Document Length:        1666
 Complete requests:      20000
 Failed requests:        0
 Requests per seconds:   5154.64

 Connnection Times (ms)
            min   avg   max
 Connect:    11    19    23
 Total:      34    40    45

over 100mbit ethernet (eepro100) it does:

 h:~> ./zb m /index.html -c 215 -n 20000

 Server:                 TUX
 Document Length:        1666
 Complete requests:      20000
 Failed requests:        0
 Requests per seconds:   4435.57

 Connnection Times (ms)
            min   avg   max
 Connect:     1    15  3020
 Total:      18    47  3068

over gigabit SysKonnect zero-copy it does:

 h:~> ./zb mg /index.html -c 215 -n 20000

 Server:                 TUX
 Document Length:        1666
 Requests per seconds:   5327.65

 Connnection Times (ms)
            min   avg   max
 Connect:     0    11    16
 Total:      21    39    81

(but the nonpersistent test puts even more load on the client, the server
is only about 60% utilized - with two clients it does about 8000
reqs/sec.)

at this point i couldnt resist - i assembled a few TUX 2.0 CGI execution
benchmarks. The CGI used for this test is a real, standard Linux ELF CGI
executable which is exec()-ed for every HTTP request: it read()s the same
/index.html file the other tests were using, write()s a HTML header to
stdout, then write()s the /index.html file to stdout and finally write()s
a HTML trailer to stdout and exit()s. [A separate process is created for
every single HTTP request]. Over localhost, TUX 2.0 CGI does:

 m:~> ./zb localhost x?/index.html -c 215 -n 20000

 ---
 Server:                 TUX
 Document Length:        1876
 Complete requests:      20000
 Failed requests:        0
 Requests per seconds:   1227.14

 Connnection Times (ms)
           min   avg   max
 Connect:     1    12    32
 Total:      41   172   346
 ---

over 100 mbit Ethernet (eepro100) TUX 2.0 CGI does:

 Requests per seconds:   1336.18

(the 1000 mbit number is the same as the 100 mbit one, because the server
is saturated executing CGIs already, network bandwidth has no relevance.)

so to conclude the bragging game, (and keeping in mind that mine might be
stronger hardware) here comes a real benchmarketing statement (Microsoft
PR folks turn green with envy): "TUX 2.0 serves CGIs over a real network
faster than boa and khttp serves static content over localhost =B-)"
[ducking down]

but at this point i have to submit that localhost tests are deceptive
because the 'zb' client takes away about half of the CPU power, creates
scheduling storms and skews results in many other ways as well. I'm almost
never testing TUX over localhost, it changes performance characteristics
fundamentally.

The over- 100mbit and 1000mbit numbers are a bit more informative, but
this still does not show lots of other, much more important webserver
characteristics that make the true and visible difference on the real
Internet these days: packet generation efficiency, flow control, caching,
fairness under load, low latency under load, robustness, flexibility,
feature set. Bandwidth and scalability is only at a distant eighth and
nineth place in my list :-)

	Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: khttpd beaten by boa

[H. Peter Anvin]

Followup to:  <14942.48157.259491.78067@pizda.ninka.net>
By author:    "David S. Miller" <davem@redhat.com>
In newsgroup: linux.dev.kernel
> 
> Lars Marowsky-Bree writes:
>  > This just goes on to show that khttpd is unnecessary kernel bloat
>  > and can be "just as well" handled by a userspace application, minus
>  > some rather very special cases which do not justify its inclusion
>  > into the main kernel.
> 
> My take on this is that khttpd is unmaintained garbage.
> 
> TUX is evidence that khttpd can be done properly and
> beat the pants off of anything done in userspace.
> 

Then why don't we unload khttpd and put in Tux?

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: khttpd beaten by boa

[Alan Cox]

> > TUX is evidence that khttpd can be done properly and
> > beat the pants off of anything done in userspace.
> > 
> 
> Then why don't we unload khttpd and put in Tux?

Tux needs the zero copy patches I believe so zero copy has to precede it
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: khttpd beaten by boa

[Arjan van de Ven]

In article <20010112084259.B441@marowsky-bree.de> you wrote:
> On 2001-01-11T22:20:56,
>    Christoph Lameter <christoph@lameter.com> said:

>> Then we decided to switch persistant connection off... But boa still wins.
>> 
>> What is wrong here? I would expect transferates of a 3-4 megabytes over a
>> localhost interface. The file is certainly in some kind of cache.

> This just goes on to show that khttpd is unnecessary kernel bloat and can be
> "just as well" handled by a userspace application, minus some rather very
> special cases which do not justify its inclusion into the main kernel.

Well, this test only shows khttpd does badly for localhost, as it doesn't give
userspace a fair chance to schedule. It's too bad Christoph didn't test it 
with the persistent-connections patch that didn't get sent to Linus due to 
the "no more features" freeze (I don't exactly remember which of the freezes
though). 

Regarding wether either khttpd or TuX should be in the kernel: I take it
that it is your oppinion that neither should be in the kernel. I disagree
with that and I think having a http-server-engine  (or even a more generic
file-serving engine) in the kernel can make sense for high-end uses. The
average desktop-user doesn't profit from it, sure. But that also holds for
things like hardware-raid or even SCSI. We still want those in though.

Wether TuX or khttpd should be in... well. I agree with DaveM that TuX is 
certainly the "next and better" generation, and I look forward to working 
with Ingo and others on it. 

Greetings,
 	Arjan van de Ven
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: khttpd beaten by boa

[Ingo Oeser]

On Fri, Jan 12, 2001 at 02:36:41PM +0100, Arjan van de Ven wrote:
> > This just goes on to show that khttpd is unnecessary kernel bloat and can be
> > "just as well" handled by a userspace application, minus some rather very
> > special cases which do not justify its inclusion into the main kernel.
> Regarding wether either khttpd or TuX should be in the kernel: I take it
> that it is your oppinion that neither should be in the kernel. I disagree
> with that and I think having a http-server-engine  (or even a more generic
> file-serving engine) in the kernel can make sense for high-end uses. The
> average desktop-user doesn't profit from it, sure. But that also holds for
> things like hardware-raid or even SCSI. We still want those in though.

This thingie is nice for embedded use, where you need a webserver
for some stuff, but don't wan't to include one, because you need
it only to represent some machine values of a process
computer[1].

But we need a more generic one, which has the functionality of a
read only entry of /proc.

That would be _very_ useful.

Regards

Ingo Oeser

[1] Don't know the right translation for "Prozessrechner", Sorry ;-(
-- 
10.+11.03.2001 - 3. Chemnitzer LinuxTag <http://www.tu-chemnitz.de/linux/tag>
         <<<<<<<<<<<<       come and join the fun       >>>>>>>>>>>>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/