nbd request too big

nbd request too big

[Luc Robalo Marques]

hi,

i would like to setup a nbd but when I try to mke2fs
the device on the client side, the connection hangs
and /log/messages contains a entry for mthe server
telling that the request was too big.
Any idea what caused it.

Thanks you

Luc robalo Marques

___________________________________________________________
Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
Yahoo! Courrier : http://courrier.yahoo.fr

Re: nbd request too big

[Steven Whitehouse]

Hi,

I presume you are using Pavel's server from his web page ? You need to increase
the buffer size that the server uses. This happens as a result of the change
a little while ago of nbd to use the elevator to merge requests at the nbd
client end. If memory serves the maximum size of request is sizeof(header) +
128 * block_size, so you need to alter the server to use that size.

You'll find the buffer size in the main loop of the mainloop() function
char buf[20480]; needs increasing (to (131072 + 28) for 1k blocks for example)
and also you'll need to increase the value here (to 131072 in this example):

                if (len > 10240)
                        err("Request too big!");

Then it should work fine.

Steve.

> 
> hi,
> 
> i would like to setup a nbd but when I try to mke2fs
> the device on the client side, the connection hangs
> and /log/messages contains a entry for mthe server
> telling that the request was too big.
> Any idea what caused it.
> 
> Thanks you
> 
> Luc robalo Marques
> 
> ___________________________________________________________
> Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
> Yahoo! Courrier : http://courrier.yahoo.fr
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: nbd request too big

[Paul Clements]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 1272 bytes --]

Hi,

If you're using the nbd-server from Pavel Machek's web site, you'll need to patch 
nbd-server.c to make it work on ~2.4.3+ kernels. My patched version of nbd-server.c
and cliserv.h are attached. The patched files make the server's request buffer be
dynamically allocated, so any size request can be handled by the server. 
The patched version also fixes some other outdated code (e.g., llseek -> lseek64).


-- 
Paul Clements
Paul.Clements@SteelEye.com



On Mon, 7 Jan 2002, [iso-8859-1] Luc Robalo Marques wrote:

> hi,
> 
> i would like to setup a nbd but when I try to mke2fs
> the device on the client side, the connection hangs
> and /log/messages contains a entry for mthe server
> telling that the request was too big.
> Any idea what caused it.
> 
> Thanks you
> 
> Luc robalo Marques
> 
> ___________________________________________________________
> Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français !
> Yahoo! Courrier : http://courrier.yahoo.fr
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

[-- Attachment #2: Type: TEXT/PLAIN, Size: 9229 bytes --]

/*
 * Network Block Device - server
 *
 * Copyright 1996-1998 Pavel Machek, distribute under GPL
 *  <pavel@atrey.karlin.mff.cuni.cz>
 *
 * Version 1.0 - hopefully 64-bit-clean
 * Version 1.1 - merging enhancements from Josh Parsons, <josh@coombs.anu.edu.au>
 * Version 1.2 - autodetect size of block devices, thanx to Peter T. Breuer" <ptb@it.uc3m.es>
 */

#define VERSION "1.3"
#define GIGA (1*1024*1024*1024)

/* use lseek64 exclusively */
#define _LARGEFILE_SOURCE   // Some more functions for correct standard I/O.
#define _LARGEFILE64_SOURCE // Additional functionality from LFS for large files

#include <sys/socket.h>
#include <sys/stat.h>
#include <netinet/tcp.h>
#include <netinet/in.h>		/* sockaddr_in, htons, in_addr */
#include <netdb.h>		/* hostent, gethostby*, getservby* */
#include <syslog.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <arpa/inet.h>

#define _IO(a,b)
#define ISSERVER
#define MY_NAME "nbd_server"
#include "cliserv.h"
#undef _IO
/* Deep magic: ioctl.h defines _IO macro (at least on linux) */

#include <sys/ioctl.h>
#include <sys/mount.h>		/* For BLKGETSIZE */

// #define DODBG
// #define DEBUG( a... ) printf( a )
#define DEBUG( a... ) do { } while(0)


inline void readit(int f, void *buf, int len)
{
	int res;
	while (len > 0) {
		DEBUG("*");
		if ((res = read(f, buf, len)) <= 0)
			err("Read failed: %m");
		len -= res;
		buf += res;
	}
}

inline void writeit(int f, void *buf, int len)
{
	int res;
	while (len > 0) {
		DEBUG("+");
		if ((res = write(f, buf, len)) <= 0)
			err("Write failed: %m");
		len -= res;
		buf += res;
	}
}

int port;			/* Port I'm listening at */
char *exportname;		/* File I'm exporting */
u64 exportsize = ~0, hunksize = ~0;	/* ...and its length */
int flags = 0;
int export[1024];
#define F_READONLY 1
#define F_MULTIFILE 2 

void cmdline(int argc, char *argv[])
{
	int i;

	if (argc < 3) {
		printf("This is nbd-server version " VERSION "\n");	
		printf("Usage: port file_to_export [size][kKmM] [-r]\n"
		       "	-r read only\n"
		       "	if port is set to 0, stdin is used (for running from inetd)\n"
		       "	if file_to_export contains '%%s', it is substituted with IP\n"
		       "		address of machine trying to connect\n" );
		exit(0);
	}
	port = atoi(argv[1]);
	for (i = 3; i < argc; i++) {
		if (*argv[i] == '-') {
			switch (argv[i][1]) {
			case 'r':
				flags |= F_READONLY;
				break;
			case 'm':
				flags |= F_MULTIFILE;
				hunksize = 1*GIGA;
				break;
			}
		} else {
			u64 es;
			int last = strlen(argv[i])-1;
			char suffix = argv[i][last];
			if (suffix == 'k' || suffix == 'K' ||
			    suffix == 'm' || suffix == 'M')
				argv[i][last] = '\0';
			es = (u64)atol(argv[i]);
			switch (suffix) {
				case 'm':
				case 'M':  es <<= 10;
				case 'k':
				case 'K':  es <<= 10;
				default :  break;
			}
			exportsize = es;
		}
	}

	exportname = argv[2];
}

int connectme(int port)
{
	struct sockaddr_in addrin;
	int addrinlen = sizeof(addrin);
	int net, sock;
	int size = 1;

	if ((sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
		err("socket: %m");
	// SteelEye change - reuse the port number
	if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &size, sizeof(int)) < 0)
		err("setsockopt: %m");

	DEBUG("Waiting for connections... bind, ");
	addrin.sin_family = AF_INET;
	addrin.sin_port = htons(port);
	addrin.sin_addr.s_addr = 0;
	if (bind(sock, (struct sockaddr *) &addrin, addrinlen) < 0)
		err("bind: %m");
	DEBUG("listen, ");
	if (listen(sock, 1) < 0)
		err("listen: %m");
	DEBUG("accept, ");
	if ((net = accept(sock, (struct sockaddr *) &addrin, &addrinlen)) < 0)
		err("accept: %m");

	return net;
}

#define SEND writeit( net, &reply, sizeof( reply ));
#define ERROR { reply.error = htonl(-1); SEND; reply.error = 0; lastpoint = -1; }

u64 lastpoint = -1;

void maybeseek(int handle, u64 a)
{
	if (a > exportsize)
		err("Can not happen\n");
	if (lastpoint != a) {
		if (lseek64(handle, a, SEEK_SET) < 0)
			err("Can not seek locally!\n");
		lastpoint = a;
	} else {
		DEBUG("@");
	}
}

int expread(u64 a, char *buf, int len)
{
	maybeseek(export[a/hunksize], a%hunksize);
	return (read(export[a/hunksize], buf, len) != len);
}

int expwrite(u64 a, char *buf, int len)
{
	maybeseek(export[a/hunksize], a%hunksize);
	return (write(export[a/hunksize], buf, len) != len);
}

int mainloop(int net)
{
	struct nbd_request request;
	struct nbd_reply reply;
	char zeros[300];
	int i = 0;
	u64 size_host;

	bzero(zeros, 290);
	if (write(net, INIT_PASSWD, 8) < 0)
		err("Negotiation failed: %m");
	cliserv_magic = htonll(cliserv_magic);
	if (write(net, &cliserv_magic, sizeof(cliserv_magic)) < 0)
		err("Negotiation failed: %m");
	size_host = htonll(exportsize);
	if (write(net, &size_host, 8) < 0)
		err("Negotiation failed: %m");
	if (write(net, zeros, 128) < 0)
		err("Negotiation failed: %m");

	DEBUG("Entering request loop!\n");
	reply.magic = htonl(NBD_REPLY_MAGIC);
	reply.error = 0;
	while (1) {
		/* SteelEye change - need dynamic buffer to work with elevator */
		static long max_nbd_request=131072; /* 128K */
		static char *buf=NULL;
		int len;

#ifdef DODBG
		i++;
		printf("%d: ", i);
#endif

		readit(net, &request, sizeof(request));
		request.from = ntohll(request.from);
		request.type = ntohl(request.type);
		len = ntohl(request.len);

		if (request.magic != htonl(NBD_REQUEST_MAGIC))
			err("Not enough magic.");

		DEBUG("request len: %d bytes\n", len);

		while (len > max_nbd_request || !buf) {
			/* SteelEye change - (re)allocate the buffer */
			if (buf) 
				free(buf);
			if (len > max_nbd_request) 
				max_nbd_request = len;
			buf=malloc(max_nbd_request);
			if (!buf) 
				DEBUG("failed to allocate %d byte buffer\n", max_nbd_request);
		}
#ifdef DODBG
		printf("%s from %d (%d) len %d, ", (request.type ? "WRITE" : "READ"),
		       (int) request.from, (int) request.from / 512, len);
#endif
		memcpy(reply.handle, request.handle, sizeof(reply.handle));
		if (((request.from + len) > exportsize) ||
		    ((flags & F_READONLY) && request.type)) {
			DEBUG("[RANGE!]");
			ERROR;
			continue;
		}
		if (request.type) {	/* WRITE */
			DEBUG("wr: net->buf, ");
			readit(net, buf, len);
			DEBUG("buf->exp, ");
			if (expwrite(request.from, buf, len)) {
				DEBUG("Write failed: %m" );
				ERROR;
				continue;
			}
			lastpoint += len;
			SEND;
			continue;
		}
		/* READ */

		DEBUG("exp->buf, ");
		if (expread(request.from, buf + sizeof(struct nbd_reply), len)) {
		 	lastpoint = -1;
			DEBUG("Read failed: %m");
			ERROR;
			continue;
		}
		lastpoint += len;

		DEBUG("buf->net, ");
		memcpy(buf, &reply, sizeof(struct nbd_reply));
		writeit(net, buf, len + sizeof(struct nbd_reply));
		DEBUG("OK!\n");
	}
}

char exportname2[1024];

void set_peername(int net)
{
	struct sockaddr_in addrin;
	int addrinlen = sizeof( addrin );
	char *peername;

	if (getpeername( net, (struct sockaddr *) &addrin, &addrinlen ) < 0)
		err("getsockname failed: %m");
	peername = inet_ntoa(addrin.sin_addr);
	sprintf(exportname2, exportname, peername);

	syslog(LOG_INFO, "connect from %s, assigned file is %s", peername, exportname2);
}

u64 size_autodetect(int export)
{
	u64 es;
	DEBUG("looking for export size with lseek SEEK_END\n");
	if ((es = lseek64(export, 0, SEEK_END)) == MINUS_ONE_64 || es == 0) {
		struct stat stat_buf = { 0, } ;
		int error;
		DEBUG("looking for export size with fstat\n");
		if ((error = fstat(export, &stat_buf)) == -1 || stat_buf.st_size == 0 ) {
			DEBUG("looking for export size with ioctl BLKGETSIZE\n");
#ifdef BLKGETSIZE
			if(ioctl(export, BLKGETSIZE, &es) || es == 0) {
#else
			if(1){
#endif
				err("Could not find size of exported block device: %m");
			} else {
				es *= 512; /* assume blocksize 512 */
			}
		} else {
			es = stat_buf.st_size;
		}
	}
	return es;
}

int main(int argc, char *argv[])
{
	int net;
	u64 i;

	if (sizeof( struct nbd_request )!=28)
		err("Bad size of structure. Alignment problems?");

	logging();
	cmdline(argc, argv);
	
	if (port)
		net = connectme(port);
	else
		net = 0;
	set_peername(net);

	for (i=0; i<exportsize; i+=hunksize) {
		char exportname3[1024];

		sprintf(exportname3, exportname2, i/hunksize);
		printf( "Opening %s\n", exportname3 );
		if ((export[i/hunksize] = open(exportname3, (flags & F_READONLY) ? O_RDONLY : O_RDWR)) == -1)
			err("Could not open exported file: %m");
	}
	
	if (exportsize == ~0) {
		exportsize = size_autodetect(export[0]);
	}
	if (exportsize > (~0UL >> 1))
		if ((exportsize >> 10) > (~0UL >> 1))
			syslog(LOG_INFO, "size of exported file/device is %luMB",
				       (unsigned long)(exportsize >> 20));
		else
			syslog(LOG_INFO, "size of exported file/device is %luKB",
			       (unsigned long)(exportsize >> 10));
	else
		syslog(LOG_INFO, "size of exported file/device is %lu",
		       (unsigned long)exportsize);
	setmysockopt(net);

	mainloop(net);
	return 0;
}

[-- Attachment #3: Type: TEXT/PLAIN, Size: 2417 bytes --]

/* This header file is shared by client & server. They really have
 * something to share...
 * */

/* Client/server protocol is as follows:
   Send INIT_PASSWD
   Send 64-bit cliserv_magic
   Send 64-bit size of exported device
   Send 128 bytes of zeros (reserved for future use)
 */

#include "config.h"
#include <errno.h>
#include <string.h>
#include <netdb.h>

#if SIZEOF_UNSIGNED_SHORT_INT==4
typedef unsigned short u32;
#elif SIZEOF_UNSIGNED_INT==4
typedef unsigned int u32;
#elif SIZEOF_UNSIGNED_LONG_INT==4
typedef unsigned long u32;
#else
#error I need at least some 32-bit type
#endif

#if SIZEOF_UNSIGNED_INT==8
typedef unsigned int u64;
#define MINUS_ONE_64 (-1U)
#elif SIZEOF_UNSIGNED_LONG_INT==8
typedef unsigned long u64;
#define MINUS_ONE_64 (-1UL)
#elif SIZEOF_UNSIGNED_LONG_LONG_INT==8
typedef unsigned long long u64;
#define MINUS_ONE_64 (-1ULL)
#else
#error I need at least some 64-bit type
#endif

#include "nbd.h"

u64 cliserv_magic = 0x00420281861253LL;
#define INIT_PASSWD "NBDMAGIC"

#define INFO(a) do { } while(0)

void setmysockopt(int sock)
{
	int size = 1;
#if 0
	if (setsockopt(sock, SOL_SOCKET, SO_SNDBUF, &size, sizeof(int)) < 0)
		 INFO("(no sockopt/1: %m)");
#endif
	size = 1;
	if (setsockopt(sock, SOL_TCP, TCP_NODELAY, &size, sizeof(int)) < 0)
		 INFO("(no sockopt/2: %m)");
#if 0
	size = 1024;
	if (setsockopt(sock, SOL_TCP, TCP_MAXSEG, &size, sizeof(int)) < 0)
		 INFO("(no sockopt/3: %m)");
#endif
}

void err(const char *s)
{
	const int maxlen = 150;
	char s1[maxlen], *s2;
	int n = 0;

	strncpy(s1, s, maxlen);
	if (s2 = strstr(s, "%m")) {
		strcpy(s1 + (s2 - s), sys_errlist[errno]);
		s2 += 2;
		strcpy(s1 + strlen(s1), s2);
	}
	else if (s2 = strstr(s, "%h")) {
		strcpy(s1 + (s2 - s), hstrerror(h_errno));
		s2 += 2;
		strcpy(s1 + strlen(s1), s2);
	}

	s1[maxlen-1] = '\0';
#ifdef ISSERVER
	syslog(LOG_ERR, s1);
#else
	fprintf(stderr, "Error: %s\n", s1);
#endif
	exit(1);
}

void logging(void)
{
#ifdef ISSERVER
	openlog(MY_NAME, LOG_PID, LOG_DAEMON);
#endif
	setvbuf(stdout, NULL, _IONBF, 0);
	setvbuf(stderr, NULL, _IONBF, 0);
}

#ifdef WORDS_BIGENDIAN
u64 ntohll(u64 a)
{
	return a;
}
#else
u64 ntohll(u64 a)
{
	u32 lo = a & 0xffffffff;
	u32 hi = a >> 32U;
	lo = ntohl(lo);
	hi = ntohl(hi);
	return ((u64) lo) << 32U | hi;
}
#endif
#define htonll ntohll

Re: nbd request too big

[Pavel Machek]

Hi!

> If you're using the nbd-server from Pavel Machek's web site, you'll need to patch 
> nbd-server.c to make it work on ~2.4.3+ kernels. My patched version of nbd-server.c
> and cliserv.h are attached. The patched files make the server's request buffer be
> dynamically allocated, so any size request can be handled by the server. 
> The patched version also fixes some other outdated code (e.g., llseek -> lseek64).

You seem to be have used old version of nbd-server to do the
patch. Could you port it to new version (at nbd.sourceforge.net) and
mail me a patch?

									Pavel
-- 
(about SSSCA) "I don't say this lightly.  However, I really think that the U.S.
no longer is classifiable as a democracy, but rather as a plutocracy." --hpa

Re: nbd request too big

[Pavel Machek]

Hi!

> I presume you are using Pavel's server from his web page ? You need to increase
> the buffer size that the server uses. This happens as a result of the change
> a little while ago of nbd to use the elevator to merge requests at the nbd
> client end. If memory serves the maximum size of request is sizeof(header) +
> 128 * block_size, so you need to alter the server to use that size.
> 
> You'll find the buffer size in the main loop of the mainloop() function
> char buf[20480]; needs increasing (to (131072 + 28) for 1k blocks for example)
> and also you'll need to increase the value here (to 131072 in this example):
> 
>                 if (len > 10240)
>                         err("Request too big!");

Thanx for explanation and would-be patch. I'll apply it into CVS.

Perhaps have acount on sourceforge and you want write access to the CVS?

								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.