Deadlock with icmpv6fuzz

Deadlock with icmpv6fuzz

[Eric Sesterhenn]

Hi,

I can get my box into a deadlock when running "icmpv6fuzz -r 25668",
source code is attached

# CONFIG_CLASSIC_RCU is not set
CONFIG_TREE_RCU=y
# CONFIG_PREEMPT_RCU is not set
CONFIG_RCU_TRACE=y
CONFIG_RCU_FANOUT=32
# CONFIG_RCU_FANOUT_EXACT is not set
CONFIG_TREE_RCU_TRACE=y
# CONFIG_PREEMPT_RCU_TRACE is not set
CONFIG_RCU_TORTURE_TEST=m
CONFIG_RCU_CPU_STALL_DETECTOR=y

CONFIG_IPV6=m
CONFIG_IPV6_PRIVACY=y
# CONFIG_IPV6_ROUTER_PREF is not set
# CONFIG_IPV6_OPTIMISTIC_DAD is not set
# CONFIG_IPV6_MIP6 is not set
CONFIG_IPV6_SIT=m
CONFIG_IPV6_NDISC_NODETYPE=y
CONFIG_IPV6_TUNNEL=m
# CONFIG_IPV6_MULTIPLE_TABLES is not set
# CONFIG_IPV6_MROUTE is not set
CONFIG_NF_CONNTRACK_IPV6=m
CONFIG_IP6_NF_MATCH_IPV6HEADER=m

Kernel is current -git

[ 5459.100023] INFO: RCU detected CPU 0 stall (t=2500 jiffies)
[ 5459.100023] Pid: 25668, comm: icmpv6fuzz Not tainted 2.6.29-rc2 #92
[ 5459.100023] Call Trace:
[ 5459.100023]  [<c0629676>] ? printk+0x18/0x1a
[ 5459.100023]  [<c016db22>] __rcu_pending+0x62/0x250
[ 5459.100023]  [<c014c55b>] ? trace_hardirqs_off+0xb/0x10
[ 5459.100023]  [<c016dd22>] rcu_pending+0x12/0x40
[ 5459.100023]  [<c0132b8a>] update_process_times+0x2a/0x60
[ 5459.100023]  [<c014a115>] tick_sched_timer+0x55/0xc0
[ 5459.100023]  [<c014a0c0>] ? tick_sched_timer+0x0/0xc0
[ 5459.100023]  [<c01409b0>] __run_hrtimer+0x70/0xe0
[ 5459.100023]  [<c014a0c0>] ? tick_sched_timer+0x0/0xc0
[ 5459.100023]  [<c014118d>] hrtimer_interrupt+0xed/0x190
[ 5459.100023]  [<c010597b>] timer_interrupt+0x3b/0x50
[ 5459.100023]  [<c016ac19>] handle_IRQ_event+0x29/0x60
[ 5459.100023]  [<c016c9a5>] handle_level_irq+0x65/0xe0
[ 5459.100023]  [<c016c940>] ? handle_level_irq+0x0/0xe0
[ 5459.100023]  <IRQ>  [<c014e9b1>] ? trace_hardirqs_on_caller+0x151/0x1c0
[ 5459.100023]  [<c0103bac>] ? common_interrupt+0x2c/0x34
[ 5459.100023]  [<c014007b>] ? posix_cpu_clock_get+0xb/0x160
[ 5459.100023]  [<d18a6e87>] ? ipv6_push_exthdr+0x17/0x70 [ipv6]
[ 5459.100023]  [<c0570aaf>] ? skb_store_bits+0x10f/0x200
[ 5459.100023]  [<d18a6f03>] ? ipv6_push_frag_opts+0x23/0x30 [ipv6]
[ 5459.100023]  [<d18858ab>] ? ip6_push_pending_frames+0x21b/0x3e0 [ipv6]
[ 5459.100023]  [<d189b01a>] ? rawv6_sendmsg+0xbfa/0xc70 [ipv6]
[ 5459.100023]  [<c05f0494>] ? inet_sendmsg+0x34/0x60
[ 5459.100023]  [<c056a909>] ? sock_sendmsg+0xe9/0x110
[ 5459.100023]  [<c013d800>] ? autoremove_wake_function+0x0/0x50
[ 5459.100023]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5459.100023]  [<c018dbe1>] ? might_fault+0x91/0xa0
[ 5459.100023]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5459.100023]  [<c031aa65>] ? copy_from_user+0x35/0x130
[ 5459.100023]  [<c056ac55>] ? sys_sendto+0xa5/0xd0
[ 5459.100023]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5459.100023]  [<c0568fb0>] ? sock_ioctl+0x0/0x240
[ 5459.100023]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5459.100023]  [<c056b28b>] ? sys_socketcall+0x18b/0x2a0
[ 5459.100023]  [<c0103551>] ? sysenter_do_call+0x12/0x31
[ 5489.100020] INFO: RCU detected CPU 0 stall (t=10000 jiffies)
[ 5489.100020] Pid: 25668, comm: icmpv6fuzz Not tainted 2.6.29-rc2 #92
[ 5489.100020] Call Trace:
[ 5489.100020]  [<c0629676>] ? printk+0x18/0x1a
[ 5489.100020]  [<c016db22>] __rcu_pending+0x62/0x250
[ 5489.100020]  [<c014c55b>] ? trace_hardirqs_off+0xb/0x10
[ 5489.100020]  [<c016dd22>] rcu_pending+0x12/0x40
[ 5489.100020]  [<c0132b8a>] update_process_times+0x2a/0x60
[ 5489.100020]  [<c014a115>] tick_sched_timer+0x55/0xc0
[ 5489.100020]  [<c014a0c0>] ? tick_sched_timer+0x0/0xc0
[ 5489.100020]  [<c01409b0>] __run_hrtimer+0x70/0xe0
[ 5489.100020]  [<c014a0c0>] ? tick_sched_timer+0x0/0xc0
[ 5489.100020]  [<c014118d>] hrtimer_interrupt+0xed/0x190
[ 5489.100020]  [<c010597b>] timer_interrupt+0x3b/0x50
[ 5489.100020]  [<c016ac19>] handle_IRQ_event+0x29/0x60
[ 5489.100020]  [<c016c9a5>] handle_level_irq+0x65/0xe0
[ 5489.100020]  [<c016c940>] ? handle_level_irq+0x0/0xe0
[ 5489.100020]  <IRQ>  [<c014e9b1>] ? trace_hardirqs_on_caller+0x151/0x1c0
[ 5489.100020]  [<c0103bac>] ? common_interrupt+0x2c/0x34
[ 5489.100020]  [<c014007b>] ? posix_cpu_clock_get+0xb/0x160
[ 5489.100020]  [<d18a6e87>] ? ipv6_push_exthdr+0x17/0x70 [ipv6]
[ 5489.100020]  [<c0570aaf>] ? skb_store_bits+0x10f/0x200
[ 5489.100020]  [<d18a6f03>] ? ipv6_push_frag_opts+0x23/0x30 [ipv6]
[ 5489.100020]  [<d18858ab>] ? ip6_push_pending_frames+0x21b/0x3e0 [ipv6]
[ 5489.100020]  [<d189b01a>] ? rawv6_sendmsg+0xbfa/0xc70 [ipv6]
[ 5489.100020]  [<c05f0494>] ? inet_sendmsg+0x34/0x60
[ 5489.100020]  [<c056a909>] ? sock_sendmsg+0xe9/0x110
[ 5489.100020]  [<c013d800>] ? autoremove_wake_function+0x0/0x50
[ 5489.100020]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5489.100020]  [<c018dbe1>] ? might_fault+0x91/0xa0
[ 5489.100020]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5489.100020]  [<c031aa65>] ? copy_from_user+0x35/0x130
[ 5489.100020]  [<c056ac55>] ? sys_sendto+0xa5/0xd0
[ 5489.100020]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5489.100020]  [<c0568fb0>] ? sock_ioctl+0x0/0x240
[ 5489.100020]  [<c018db96>] ? might_fault+0x46/0xa0
[ 5489.100020]  [<c056b28b>] ? sys_socketcall+0x18b/0x2a0
[ 5489.100020]  [<c0103551>] ? sysenter_do_call+0x12/0x31

and so on

(gdb) l*(ipv6_push_exthdr+0x17)
0x4d7 is in ipv6_push_exthdr (net/ipv6/exthdrs.c:700).
695		*proto = NEXTHDR_ROUTING;
696	}
697	
698	static void ipv6_push_exthdr(struct sk_buff *skb, u8 *proto, u8 type, struct ipv6_opt_hdr *opt)
699	{
700		struct ipv6_opt_hdr *h = (struct ipv6_opt_hdr *)skb_push(skb, ipv6_optlen(opt));
701	
702		memcpy(h, opt, ipv6_optlen(opt));
703		h->nexthdr = *proto;
704		*proto = type;

(gdb) l*(skb_store_bits+0x10f)
0x282f is in skb_store_bits (net/core/skbuff.c:1567).
1562				vaddr = kmap_skb_frag(frag);
1563				memcpy(vaddr + frag->page_offset + offset - start,
1564				       from, copy);
1565				kunmap_skb_frag(vaddr);
1566	
1567				if ((len -= copy) == 0)
1568					return 0;
1569				offset += copy;
1570				from += copy;
1571			}

(gdb) l*(ipv6_push_frag_opts+0x23)
0x553 is in ipv6_push_frag_opts (net/ipv6/exthdrs.c:730).
725	
726	void ipv6_push_frag_opts(struct sk_buff *skb, struct ipv6_txoptions *opt, u8 *proto)
727	{
728		if (opt->dst1opt)
729			ipv6_push_exthdr(skb, proto, NEXTHDR_DEST, opt->dst1opt);
730	}
731	
732	struct ipv6_txoptions *
733	ipv6_dup_options(struct sock *sk, struct ipv6_txoptions *opt)
734	{




Running the same command on a different box with a different config

[11237.306574] skb_under_panic: text:c071d473 len:2361 put:864 head:ca7510c0 data:ca750f10 tail:0xca751270 end:0xca751280 dev:<NULL>
[11237.306909] ------------[ cut here ]------------
[11237.307002] kernel BUG at net/core/skbuff.c:147!
[11237.307045] invalid opcode: 0000 [#1] PREEMPT DEBUG_PAGEALLOC
[11237.307045] last sysfs file: /sys/class/vc/vcsa50/dev
[11237.307045] Modules linked in: [last unloaded: rcutorture]
[11237.307045] 
[11237.307045] Pid: 19951, comm: icmpv6fuzz Not tainted (2.6.29-rc2-00021-gd84d31c #216) System Name
[11237.307045] EIP: 0060:[<c06918e1>] EFLAGS: 00010246 CPU: 0
[11237.307045] EIP is at skb_under_panic+0x3f/0x46
[11237.307045] EAX: 00000088 EBX: c098dafb ECX: c012413f EDX: c012476a
[11237.307045] ESI: 00000000 EDI: ca5b5ccc EBP: ca5b5c68 ESP: ca5b5c3c
[11237.307045]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[11237.307045] Process icmpv6fuzz (pid: 19951, ti=ca5b5000 task=c2aa0000 task.ti=ca5b5000)
[11237.307045] Stack:
[11237.307045]  c09d321a c071d473 00000939 00000360 ca7510c0 ca750f10 ca751270 ca751280
[11237.307045]  c098dafb cce2e620 c724c0cc ca5b5c74 c0692b30 ca5b5cd3 ca5b5c90 c071d473
[11237.307045]  0000003c ca5b5c90 cce2e620 cedc11b4 ca5b5ccc ca5b5ca0 c071d4c5 c724c0cc
[11237.307045] Call Trace:
[11237.307045]  [<c071d473>] ? ipv6_push_exthdr+0x26/0x55
[11237.307045]  [<c0692b30>] ? skb_push+0x2c/0x35
[11237.307045]  [<c071d473>] ? ipv6_push_exthdr+0x26/0x55
[11237.307045]  [<c071d4c5>] ? ipv6_push_frag_opts+0x23/0x29
[11237.307045]  [<c0701a8a>] ? ip6_push_pending_frames+0x1b2/0x39b
[11237.307045]  [<c07145b1>] ? rawv6_sendmsg+0xa84/0xb17
[11237.307045]  [<c013ec6d>] ? put_lock_stats+0xd/0x21
[11237.307045]  [<c013ec6d>] ? put_lock_stats+0xd/0x21
[11237.307045]  [<c013ed20>] ? lock_release_holdtime+0x9f/0xa7
[11237.307045]  [<c06db84a>] ? inet_sendmsg+0x40/0x4d
[11237.307045]  [<c068db41>] ? sock_sendmsg+0xce/0xe5
[11237.307045]  [<c013ec6d>] ? put_lock_stats+0xd/0x21
[11237.307045]  [<c013416c>] ? autoremove_wake_function+0x0/0x35
[11237.307045]  [<c0142df7>] ? lock_release_non_nested+0xb0/0x1f8
[11237.307045]  [<c017ccb3>] ? might_fault+0x4f/0x8b
[11237.307045]  [<c017ccb3>] ? might_fault+0x4f/0x8b
[11237.307045]  [<c068e4a1>] ? sys_sendto+0xa9/0xc8
[11237.307045]  [<c013ec6d>] ? put_lock_stats+0xd/0x21
[11237.307045]  [<c013ed20>] ? lock_release_holdtime+0x9f/0xa7
[11237.307045]  [<c07b2557>] ? sub_preempt_count+0xc0/0xd1
[11237.307045]  [<c013ec6d>] ? put_lock_stats+0xd/0x21
[11237.307045]  [<c013ed20>] ? lock_release_holdtime+0x9f/0xa7
[11237.307045]  [<c0142df7>] ? lock_release_non_nested+0xb0/0x1f8
[11237.307045]  [<c017ccb3>] ? might_fault+0x4f/0x8b
[11237.307045]  [<c068ec3d>] ? sys_socketcall+0xeb/0x180
[11237.307045]  [<c0102ea1>] ? sysenter_do_call+0x12/0x31
[11237.307045] Code: 0f 45 de 53 ff b0 94 00 00 00 ff b0 90 00 00 00 ff b0 9c 00 00 00 ff b0 98 00 00 00 52 ff 70 50 51 68 1a 32 9d c0 e8 6f 2e a9 ff <0f> 0b 83 c4 24 eb fe 55 89 e5 56 53 0f 1f 44 00 00 8b 70 14 bb 
[11237.307045] EIP: [<c06918e1>] skb_under_panic+0x3f/0x46 SS:ESP 0068:ca5b5c3c
[11237.374079] ---[ end trace 7188985726996093 ]---

(gdb) l *(ipv6_push_exthdr+0x26)
0xc071d473 is in ipv6_push_exthdr (net/ipv6/exthdrs.c:700).
695		*proto = NEXTHDR_ROUTING;
696	}
697	
698	static void ipv6_push_exthdr(struct sk_buff *skb, u8 *proto, u8 type, struct ipv6_opt_hdr *opt)
699	{
700		struct ipv6_opt_hdr *h = (struct ipv6_opt_hdr *)skb_push(skb, ipv6_optlen(opt));
701	
702		memcpy(h, opt, ipv6_optlen(opt));
703		h->nexthdr = *proto;
704		*proto = type;


----------------8<------------------------

/*
 * ICMPv6 or ICMPv4 socket fuzzer.
 *
 * Copyright (c) 2006, Clément Lecigne
 */
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <sys/param.h>
#include <net/if.h>
//#include <net/if_var.h>
#include <sys/uio.h>
//#include <netinet6/ip6_mroute.h>
//#include <netinet6/in6_var.h>


#define SIOCGETMIFCNT_IN6       SIOCPROTOPRIVATE        /* IP protocol privates */
#define SIOCGETSGCNT_IN6        (SIOCPROTOPRIVATE+1)
#define SIOCGETRPF      (SIOCPROTOPRIVATE+2)


/* functions */
unsigned int randaddr(void);
void randsoopt(int);
void randgoopt(int);
void randioctl(int);
void usage(char *);

/*
* boucle until we hit a valid socket option
*/
void randsoopt(int sock)
{
	unsigned int optval;
	int optlen, optname, level, ret, on = rand() % 2;
	do
	{
		switch (rand() % 5)
		{
			case 0:
				level = IPPROTO_IPV6;
				break;
			case 1:
				level = SOL_SOCKET;
				break;
			case 2:
				level = IPPROTO_RAW;
				break;
			case 3:
				level = rand() & 0xFF;
				break;
			case 4:
				level = IPPROTO_IP;
				break;
		}
		
		if (rand() % 6)
		{
			optlen = rand();
			optval = (unsigned int)randaddr();
		}
		else
		{
		/* 
			* In some cases, kernel excepts that
			* optlen == sizeof (int) and that's
			* the first bound checking.
			*/
			optlen = sizeof (int);
			on = rand();
			optval = (unsigned int)&on;
		}
		
		if (rand() % 8)
			optname = rand() % 255;
		else
			optname = rand();
#if 0
		/*
		 * anti well know FreeBSD mbufs exhaustion.
		 */
		if (optname == 25 || optname == IPV6_IPSEC_POLICY || 
				optname == IPV6_FW_ADD || optname == IPV6_FW_FLUSH
				|| optname == IPV6_FW_DEL || optname == IPV6_FW_ZERO)
			continue;
		/*printf("level : %d - optname : %d - optlen : %d\n", 
				level, optname, optlen);*/
#endif
		ret = setsockopt(sock, level, optname, (void *)optval, optlen);

	}while(ret == -1);
	return;
}


/*
* ioctl ipv6 socket fuzzer.
*/
void randioctl(int sock)
{
	unsigned long reqs[] = { SIOCGETSGCNT_IN6, SIOCGETMIFCNT_IN6,
		SIOCGETRPF};
/*
		GSCOPE6DEF, SIOCGLIFADDR, SIOCSIFPHYADDR_IN6, SIOCGIFNETMASK_IN6,
		SIOCAIFADDR_IN6, SIOCGIFDSTADDR_IN6, SIOCSIFALIFETIME_IN6, 
		SIOCGIFADDR_IN6, SIOCGIFDSTADDR_IN6, SIOCGIFNETMASK_IN6, SIOCGIFAFLAG_IN6,
		SIOCGIFSTAT_IN6, SIOCGIFSTAT_ICMP6, SIOCGIFALIFETIME_IN6, SIOCSIFALIFETIME_IN6,
		SIOCAIFADDR_IN6, SIOCDIFADDR_IN6 }; */
	unsigned int arg;
	int ret;
	unsigned long request;
	
	if (rand() % 8)
		request = reqs[rand() % (sizeof (reqs) / sizeof (reqs[0]))];
	else
		request = rand() + rand();
	if (rand() % 2)
	{
		arg = randaddr();
		ret = ioctl(sock, request, (caddr_t)arg);
	}
	else
	{
		arg = rand();
		ret = ioctl(sock, request, (int)arg);
	}
}


/*
* return a random address
*/
unsigned int randaddr(void)
{
	char *p = malloc(1);
	unsigned int heap = (unsigned int)p;
	free(p);
	switch (rand() % 4)
	{
		case 0:
			return (heap + (rand() & 0xFFF));
		case 1:
			return ((unsigned int)&heap + (rand() & 0xFFF));
		case 2:
			return (0xc0000000 + (rand() & 0xFFFF));
		case 3:
			return (rand());
	}
	return (0);
}


int main(int ac, char **av) 
{
	int32_t cc, s, occ, i, j, a, try, count, opts;
	u_int32_t seed, maxsize;
	u_int8_t ip6;
	char c, *buf;
	struct addrinfo *res, hints;
	struct sockaddr_in6 from;
	socklen_t fromlen;
	struct msghdr msg;
	struct cmsghdr *cmsg = NULL;
	struct iovec iov;
	
	/* default values */
	seed = getpid();
	count = 50;
	occ = 10000;
	maxsize = 4096;
	opts = 50;
	ip6 = 1;
	fromlen = sizeof(from);
	
	if (getuid())
	{
		fprintf(stderr, " - you must be root.\n");
		exit(EXIT_FAILURE);
	}
	
	while ((c = getopt(ac, av, "r:n:c:m:o:46")) != EOF)
	{
		switch (c)
		{
			case '6':
				ip6 = 1;
				break;
			case '4':
				ip6 = 0;
				break;
			case 'r':
				seed = atoi(optarg);
				break;
			case 'n':
				occ = atoi(optarg);
				break;
			case 'c':
				count = atoi(optarg);
				break;
			case 'm':
				maxsize = atoi(optarg);
				break;
			case 'o':
				opts = atoi(optarg);
				break;
			default:
				usage(av[0]);
				break;
		}
	}
	
	printf("seeding with %u\n", seed);
	srand(seed);

	buf = malloc(maxsize);
	if (buf == NULL)
	{
		printf("%s: out of memory.\n", av[0]);
		exit(EXIT_FAILURE);
	}

	memset(&hints, 0, sizeof(hints));
	hints.ai_flags = AI_CANONNAME;
	hints.ai_socktype = SOCK_RAW;
	
	if(ip6)
	{
		hints.ai_family = AF_INET6;
		hints.ai_protocol = IPPROTO_ICMPV6;
		getaddrinfo("::1", NULL, &hints, &res); 
	}
	else
	{
		hints.ai_family = AF_INET;
		hints.ai_protocol = IPPROTO_ICMP;
		getaddrinfo("127.0.0.1", NULL, &hints, &res);
	}

	for (i = 0; i < occ; i++)
	{
		printf(".\n");
		s = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
		//cc = bind(s, res->ai_addr, res->ai_addrlen);


		for (j = 0; j < opts; j++)
		{
			randsoopt(s);
			//randgoopt(s);
			randioctl(s);

			for (a = 0; a < 32; a++)
				buf[a] = rand() % 255;

			try = 0;
			do
			{
				switch(rand() % 3)
				{
				case 0:
					cc = sendto(s, buf, rand() % maxsize, 0,
							(struct sockaddr *)res->ai_addr, res->ai_addrlen);
					break;
				case 1:
				case 2:
					msg.msg_controllen = (rand() % 2) ? rand() & maxsize : 0;
					if (msg.msg_controllen)
					{
						if (msg.msg_controllen < sizeof (struct cmsghdr))
							cmsg = (struct cmsghdr *)malloc(sizeof (struct cmsghdr));
						else
							cmsg = (struct cmsghdr *)malloc(msg.msg_controllen);
						if (cmsg == NULL) goto nocmsghdr;
						msg.msg_control = cmsg;
						cmsg->cmsg_level = (rand() % 2) ? IPPROTO_IPV6 : rand();
						cmsg->cmsg_type = (rand() % 2) ? rand() % 255 : rand();
						cmsg->cmsg_len = (rand() % 2) ? msg.msg_controllen : rand();
					}
					else
					{
nocmsghdr:
						msg.msg_control = (rand() % 5) ? NULL : (void*)randaddr();
						msg.msg_controllen = (rand() % 2) ? rand() : 0;
					}
					iov.iov_len = (rand() % 2) ? rand() : rand() & maxsize;
					iov.iov_base = (rand() % 2) ? (void*)randaddr() : &buf;
					msg.msg_iov = (rand() % 2) ? (void*)randaddr() : &iov;
					if (rand() % 5)
					{
						msg.msg_name = res->ai_addr;
						msg.msg_namelen = res->ai_addrlen;
					}
					else
					{
						msg.msg_name = (caddr_t)randaddr();
						msg.msg_namelen = rand();
					}
					msg.msg_flags = rand();
					cc = sendmsg (s, &msg, rand());
				}
				if (cmsg != NULL)
				{	
				//	free(cmsg);
				//	cmsg = NULL;
				}
				try++;
			} while(cc == -1 && try != count); 
			recvmsg(s, &msg, MSG_DONTWAIT);
		}
		close(s);
	}
	free(buf);
	freeaddrinfo(res);
	exit(EXIT_SUCCESS);
}

/* 
* usage
*/
void usage(char *prog)
{
	printf("usage: %s [-4] [-6] [-r seed] [-c sendto-timeout]\n"
		"          [-m maxsize] [-o maxsetsockopt] [-n occ]\n", prog);
	exit(EXIT_FAILURE);
}

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Eric Sesterhenn <snakebyte@gmx.de>
Date: Tue, 20 Jan 2009 21:47:43 +0100

> Kernel is current -git

Weird trace.

I can't figure out what would cause it.

Is the program counter on the skb_push() call
that is part of that:

	struct ipv6_opt_hdr *h = (struct ipv6_opt_hdr *)skb_push(skb, ipv6_optlen(opt));

line it seems to be stuck on?

Re: Deadlock with icmpv6fuzz

[Eric Sesterhenn]

* David Miller (davem@davemloft.net) wrote:
> From: Eric Sesterhenn <snakebyte@gmx.de>
> Date: Tue, 20 Jan 2009 21:47:43 +0100
> 
> > Kernel is current -git
> 
> Weird trace.
> 
> I can't figure out what would cause it.
> 
> Is the program counter on the skb_push() call
> that is part of that:
> 
> 	struct ipv6_opt_hdr *h = (struct ipv6_opt_hdr *)skb_push(skb, ipv6_optlen(opt));
> 
> line it seems to be stuck on?

With current -git i get a different issue (and the box stays alive)

[  233.207012] skb_under_panic: text:c071d3ab len:2361 put:864
head:cba29a40 data:cba29798 tail:0xcba29af8 end:0xcba29b00 dev:<NULL>
[  233.223482] ------------[ cut here ]------------
[  233.223660] kernel BUG at net/core/skbuff.c:143!
[  233.223789] invalid opcode: 0000 [#1] PREEMPT DEBUG_PAGEALLOC
[  233.224086] last sysfs file: /sys/block/ram9/range
[  233.224086] Modules linked in:
[  233.224086] 
[  233.224086] Pid: 5020, comm: icmpv6fuzz Not tainted
(2.6.29-rc2-00362-g884f64f #224) System Name
[  233.224086] EIP: 0060:[<c0691721>] EFLAGS: 00010246 CPU: 0
[  233.224086] EIP is at skb_under_panic+0x3f/0x46
[  233.224086] EAX: 00000088 EBX: c098dc65 ECX: 00000003 EDX: c0124782
[  233.224086] ESI: 00000000 EDI: cbb9accc EBP: cbb9ac68 ESP: cbb9ac3c
[  233.224086]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[  233.224086] Process icmpv6fuzz (pid: 5020, ti=cbb9a000 task=cbb25c50
task.ti=cbb9a000)
[  233.224086] Stack:
[  233.224086]  c09d333e c071d3ab 00000939 00000360 cba29a40 cba29798
cba29af8 cba29b00
[  233.224086]  c098dc65 cef549a0 cbb7903c cbb9ac74 c06929ce cbb9acd3
cbb9ac90 c071d3ab
[  233.224086]  0000003c cbb9ac90 cef549a0 cedc31b4 cbb9accc cbb9aca0
c071d3fd cbb7903c
[  233.224086] Call Trace:
[  233.224086]  [<c071d3ab>] ? ipv6_push_exthdr+0x26/0x55
[  233.224086]  [<c06929ce>] ? skb_push+0x2c/0x35
[  233.224086]  [<c071d3ab>] ? ipv6_push_exthdr+0x26/0x55
[  233.224086]  [<c071d3fd>] ? ipv6_push_frag_opts+0x23/0x29
[  233.224086]  [<c07019c2>] ? ip6_push_pending_frames+0x1b2/0x39b
[  233.224086]  [<c07144e9>] ? rawv6_sendmsg+0xa84/0xb17
[  233.224086]  [<c013eee5>] ? put_lock_stats+0xd/0x21
[  233.224086]  [<c013eee5>] ? put_lock_stats+0xd/0x21
[  233.224086]  [<c013ef98>] ? lock_release_holdtime+0x9f/0xa7
[  233.224086]  [<c06db73a>] ? inet_sendmsg+0x40/0x4d
[  233.224086]  [<c068d981>] ? sock_sendmsg+0xce/0xe5
[  233.224086]  [<c013eee5>] ? put_lock_stats+0xd/0x21
[  233.224086]  [<c0134344>] ? autoremove_wake_function+0x0/0x35
[  233.224086]  [<c014306f>] ? lock_release_non_nested+0xb0/0x1f8
[  233.224086]  [<c017cf8d>] ? might_fault+0x4f/0x8b
[  233.224086]  [<c017cf8d>] ? might_fault+0x4f/0x8b
[  233.224086]  [<c068e2e1>] ? sys_sendto+0xa9/0xc8
[  233.224086]  [<c013eee5>] ? put_lock_stats+0xd/0x21
[  233.224086]  [<c013ef98>] ? lock_release_holdtime+0x9f/0xa7
[  233.224086]  [<c07b2507>] ? sub_preempt_count+0xc0/0xd1
[  233.224086]  [<c013eee5>] ? put_lock_stats+0xd/0x21
[  233.224086]  [<c013ef98>] ? lock_release_holdtime+0x9f/0xa7
[  233.224086]  [<c014306f>] ? lock_release_non_nested+0xb0/0x1f8
[  233.224086]  [<c017cf8d>] ? might_fault+0x4f/0x8b
[  233.224086]  [<c068ea7d>] ? sys_socketcall+0xeb/0x180
[  233.224086]  [<c0102ea1>] ? sysenter_do_call+0x12/0x31
[  233.224086] Code: 0f 45 de 53 ff b0 94 00 00 00 ff b0 90 00 00 00 ff
b0 9c 00 00 00 ff b0 98 00 00 00 52 ff 70 50 51 68 3e 33 9d c0 e8 6f 30
a9 ff <0f> 0b 83 c4 24 eb fe 55 89 e5 56 53 0f 1f 44 00 00 8b 70 14 bb 
[  233.224086] EIP: [<c0691721>] skb_under_panic+0x3f/0x46 SS:ESP
0068:cbb9ac3c
[  233.346932] ---[ end trace a3c25240b047560e ]---

But the callsite stays the same

0xc071d3ab is in ipv6_push_exthdr (net/ipv6/exthdrs.c:700).
695		*proto = NEXTHDR_ROUTING;
696	}
697	
698	static void ipv6_push_exthdr(struct sk_buff *skb, u8 *proto, u8
type, struct ipv6_opt_hdr *opt)
699	{
700		struct ipv6_opt_hdr *h = (struct ipv6_opt_hdr
*)skb_push(skb, ipv6_optlen(opt));
701	
702		memcpy(h, opt, ipv6_optlen(opt));
703		h->nexthdr = *proto;
704		*proto = type;

Greetings, Eric

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Tue, Jan 27, 2009 at 07:53:56AM +0000, Eric Sesterhenn wrote:
> 
> With current -git i get a different issue (and the box stays alive)

I think I see the problem (though I must say that this code is
really hairy, it's almost like I'm reading your average device
driver!).

When a protocol such as raw or UDP parses extension headers it
stores a pointer to the parsed option in ipv6_txoptions instead
of copying the option.

So as long as you only use it during the system call nothing bad
happens, but once you put the cork on, the next send is going to
go boom.

Any volunteers to fix this?

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Wed, 28 Jan 2009 20:35:07 +1100

> Any volunteers to fix this?

I'll try to take a stab at it later tonight.

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Thu, Jan 29, 2009 at 05:49:54PM -0800, David Miller wrote:
> From: Herbert Xu <herbert@gondor.apana.org.au>
> Date: Wed, 28 Jan 2009 20:35:07 +1100
> 
> > Any volunteers to fix this?
> 
> I'll try to take a stab at it later tonight.

I took a stab at it.

ipv6: Copy cork options in ip6_append_data

As the options passed to ip6_append_data may be ephemeral, we need
to duplicate it for corking.  This patch applies the simplest fix
which is to memdup all the relevant bits.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 4b15938..9fb49c3 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1105,6 +1105,18 @@ static inline int ip6_ufo_append_data(struct sock *sk,
 	return err;
 }
 
+static inline struct ipv6_opt_hdr *ip6_opt_dup(struct ipv6_opt_hdr *src,
+					       gfp_t gfp)
+{
+	return src ? kmemdup(src, (src->hdrlen + 1) * 8, gfp) : NULL;
+}
+
+static inline struct ipv6_rt_hdr *ip6_rthdr_dup(struct ipv6_rt_hdr *src,
+						gfp_t gfp)
+{
+	return src ? kmemdup(src, (src->hdrlen + 1) * 8, gfp) : NULL;
+}
+
 int ip6_append_data(struct sock *sk, int getfrag(void *from, char *to,
 	int offset, int len, int odd, struct sk_buff *skb),
 	void *from, int length, int transhdrlen,
@@ -1130,17 +1142,37 @@ int ip6_append_data(struct sock *sk, int getfrag(void *from, char *to,
 		 * setup for corking
 		 */
 		if (opt) {
-			if (np->cork.opt == NULL) {
-				np->cork.opt = kmalloc(opt->tot_len,
-						       sk->sk_allocation);
-				if (unlikely(np->cork.opt == NULL))
-					return -ENOBUFS;
-			} else if (np->cork.opt->tot_len < opt->tot_len) {
-				printk(KERN_DEBUG "ip6_append_data: invalid option length\n");
+			if (WARN_ON(np->cork.opt))
 				return -EINVAL;
-			}
-			memcpy(np->cork.opt, opt, opt->tot_len);
-			inet->cork.flags |= IPCORK_OPT;
+
+			np->cork.opt = kmalloc(opt->tot_len, sk->sk_allocation);
+			if (unlikely(np->cork.opt == NULL))
+				return -ENOBUFS;
+
+			np->cork.opt->tot_len = opt->tot_len;
+			np->cork.opt->opt_flen = opt->opt_flen;
+			np->cork.opt->opt_nflen = opt->opt_nflen;
+
+			np->cork.opt->dst0opt = ip6_opt_dup(opt->dst0opt,
+							    sk->sk_allocation);
+			if (opt->dst0opt && !np->cork.opt->dst0opt)
+				return -ENOBUFS;
+
+			np->cork.opt->dst1opt = ip6_opt_dup(opt->dst1opt,
+							    sk->sk_allocation);
+			if (opt->dst1opt && !np->cork.opt->dst1opt)
+				return -ENOBUFS;
+
+			np->cork.opt->hopopt = ip6_opt_dup(opt->hopopt,
+							   sk->sk_allocation);
+			if (opt->hopopt && !np->cork.opt->hopopt)
+				return -ENOBUFS;
+
+			np->cork.opt->srcrt = ip6_rthdr_dup(opt->srcrt,
+							    sk->sk_allocation);
+			if (opt->srcrt && !np->cork.opt->srcrt)
+				return -ENOBUFS;
+
 			/* need source address above miyazawa*/
 		}
 		dst_hold(&rt->u.dst);
@@ -1167,8 +1199,7 @@ int ip6_append_data(struct sock *sk, int getfrag(void *from, char *to,
 	} else {
 		rt = (struct rt6_info *)inet->cork.dst;
 		fl = &inet->cork.fl;
-		if (inet->cork.flags & IPCORK_OPT)
-			opt = np->cork.opt;
+		opt = np->cork.opt;
 		transhdrlen = 0;
 		exthdrlen = 0;
 		mtu = inet->cork.fragsize;
@@ -1407,9 +1438,15 @@ error:
 
 static void ip6_cork_release(struct inet_sock *inet, struct ipv6_pinfo *np)
 {
-	inet->cork.flags &= ~IPCORK_OPT;
-	kfree(np->cork.opt);
-	np->cork.opt = NULL;
+	if (np->cork.opt) {
+		kfree(np->cork.opt->dst0opt);
+		kfree(np->cork.opt->dst1opt);
+		kfree(np->cork.opt->hopopt);
+		kfree(np->cork.opt->srcrt);
+		kfree(np->cork.opt);
+		np->cork.opt = NULL;
+	}
+
 	if (inet->cork.dst) {
 		dst_release(inet->cork.dst);
 		inet->cork.dst = NULL;

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[Eric Sesterhenn]

* Herbert Xu (herbert@gondor.apana.org.au) wrote:
> On Thu, Jan 29, 2009 at 05:49:54PM -0800, David Miller wrote:
> > From: Herbert Xu <herbert@gondor.apana.org.au>
> > Date: Wed, 28 Jan 2009 20:35:07 +1100
> > 
> > > Any volunteers to fix this?
> > 
> > I'll try to take a stab at it later tonight.
> 
> I took a stab at it.
> 
> ipv6: Copy cork options in ip6_append_data
> 
> As the options passed to ip6_append_data may be ephemeral, we need
> to duplicate it for corking.  This patch applies the simplest fix
> which is to memdup all the relevant bits.

Thanks, this fixes the issue, I've been running icmpv6fuzz for a while
again and the only issue i saw so far was a page allocation failure:

Kernel is only dirty from your patch

[ 2880.044328] icmpv6fuzz: page allocation failure. order:9, mode:0x40d0
[ 2880.044495] Pid: 10968, comm: icmpv6fuzz Not tainted
2.6.29-rc3-00580-ga2fe994-dirty #239
[ 2880.044694] Call Trace:
[ 2880.044802]  [<c016886a>] __alloc_pages_internal+0x38e/0x3aa
[ 2880.044954]  [<c016889a>] __get_free_pages+0x14/0x24
[ 2880.071336]  [<c018412c>] __kmalloc+0x2e/0x122
[ 2880.071466]  [<c06fa1f9>] ? ipv6_flowlabel_opt+0x1b2/0x7b1
[ 2880.071589]  [<c06fa227>] ipv6_flowlabel_opt+0x1e0/0x7b1
[ 2880.071689]  [<c013e32f>] ? mark_held_locks+0x43/0x5a
[ 2880.071818]  [<c0125ecf>] ? local_bh_enable+0xa1/0xba
[ 2880.071910]  [<c013e4f1>] ? trace_hardirqs_on_caller+0x10d/0x14b
[ 2880.092665]  [<c066cfb6>] ? lock_sock_nested+0xb2/0xbd
[ 2880.092800]  [<c06e831d>] ? ipv6_setsockopt+0x8e/0xb89
[ 2880.092922]  [<c06e8c9e>] ipv6_setsockopt+0xa0f/0xb89
[ 2880.093098]  [<c013fce5>] ? __lock_acquire+0x6a8/0x6fe
[ 2880.093192]  [<c013fce5>] ? __lock_acquire+0x6a8/0x6fe
[ 2880.093323]  [<c0106d8d>] ? native_sched_clock+0x41/0x68
[ 2880.093420]  [<c013be58>] ? lock_release_holdtime+0x9f/0xa7
[ 2880.093541]  [<c0106d8d>] ? native_sched_clock+0x41/0x68
[ 2880.093634]  [<c013bda5>] ? put_lock_stats+0xd/0x21
[ 2880.093748]  [<c013be58>] ? lock_release_holdtime+0x9f/0xa7
[ 2880.093847]  [<c06edf93>] rawv6_setsockopt+0x78/0xe9
[ 2880.093963]  [<c066c9dd>] sock_common_setsockopt+0x13/0x18
[ 2880.094257]  [<c066b098>] sys_setsockopt+0x59/0x77
[ 2880.094424]  [<c066c58a>] sys_socketcall+0x12a/0x17b
[ 2880.094631]  [<c0102dc1>] sysenter_do_call+0x12/0x31
[ 2880.094797] Mem-Info:
[ 2880.094961] DMA per-cpu:
[ 2880.095200] CPU    0: hi:    0, btch:   1 usd:   0
[ 2880.095363] Normal per-cpu:
[ 2880.095536] CPU    0: hi:   90, btch:  15 usd:   5
[ 2880.095703] Active_anon:15024 active_file:510 inactive_anon:36637
[ 2880.095707]  inactive_file:761 unevictable:0 dirty:13 writeback:457
unstable:0
[ 2880.095712]  free:1798 slab:3367 mapped:300 pagetables:387 bounce:0
[ 2880.096242] DMA free:300kB min:120kB low:148kB high:180kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB present:15752kB pages_scanned:0 all_unreclaimable? no
[ 2880.096523] lowmem_reserve[]: 0 238 238 238
[ 2880.096838] Normal free:6892kB min:1912kB low:2388kB high:2868kB
active_anon:60096kB inactive_anon:146548kB active_file:2040kB
inactive_file:3044kB unevictable:0kB present:243824kB pages_scanned:0
all_unreclaimable? no
[ 2880.097479] lowmem_reserve[]: 0 0 0 0
[ 2880.097749] DMA: 1*4kB 1*8kB 0*16kB 1*32kB 2*64kB 1*128kB 0*256kB
0*512kB 0*1024kB 0*2048kB 0*4096kB = 300kB
[ 2880.098406] Normal: 415*4kB 130*8kB 10*16kB 4*32kB 5*64kB 2*128kB
1*256kB 2*512kB 2*1024kB 0*2048kB 0*4096kB = 6892kB
[ 2880.098994] 24927 total pagecache pages
[ 2880.130231] 23693 pages in swap cache
[ 2880.130343] Swap cache stats: add 424475, delete 400782, find
17623/61701
[ 2880.130979] Free swap  = 311644kB
[ 2880.131249] Total swap = 746980kB
[ 2880.225394] 65532 pages RAM
[ 2880.225551] 0 pages HighMem
[ 2880.225677] 4932 pages reserved
[ 2880.225772] 1554 pages shared
[ 2880.225889] 57445 pages non-shared

Greetings, Eric

Re: Deadlock with icmpv6fuzz

[Roland Dreier]

 > [ 2880.044328] icmpv6fuzz: page allocation failure. order:9, mode:0x40d0
 > [ 2880.044495] Pid: 10968, comm: icmpv6fuzz Not tainted
 > 2.6.29-rc3-00580-ga2fe994-dirty #239
 > [ 2880.044694] Call Trace:
 > [ 2880.044802]  [<c016886a>] __alloc_pages_internal+0x38e/0x3aa
 > [ 2880.044954]  [<c016889a>] __get_free_pages+0x14/0x24
 > [ 2880.071336]  [<c018412c>] __kmalloc+0x2e/0x122
 > [ 2880.071589]  [<c06fa227>] ipv6_flowlabel_opt+0x1e0/0x7b1
 > [ 2880.092922]  [<c06e8c9e>] ipv6_setsockopt+0xa0f/0xb89

>From a quick scan of the code, it looks as if optlen is never sanity
checked in the case of setsockopt(IPV6_FLOWLABEL_MGR), and
ipv6_flowlabel_opt() calls into fl_create() with whatever value
userspace passes in, which then pretty much does kmalloc(optlen).
So if icmpv6fuzz passes some big random value, it can cause this failure.

I don't know what the appropriate limit should be, so no patch, sorry.

 - R.

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 6 Feb 2009 00:01:49 +1100

> ipv6: Copy cork options in ip6_append_data
> 
> As the options passed to ip6_append_data may be ephemeral, we need
> to duplicate it for corking.  This patch applies the simplest fix
> which is to memdup all the relevant bits.
> 
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Great work, applied.

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Thu, Feb 05, 2009 at 02:24:02PM -0800, Roland Dreier wrote:
>
> >From a quick scan of the code, it looks as if optlen is never sanity
> checked in the case of setsockopt(IPV6_FLOWLABEL_MGR), and
> ipv6_flowlabel_opt() calls into fl_create() with whatever value
> userspace passes in, which then pretty much does kmalloc(optlen).
> So if icmpv6fuzz passes some big random value, it can cause this failure.
> 
> I don't know what the appropriate limit should be, so no patch, sorry.

Well it is legal (though unlikely) to pass very long options.
So the real fix would be to avoid copying the control message
at all and modify all the code involved to read user pointers
directly.  Someone with a lot of patience is required for this :)

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 6 Feb 2009 10:43:18 +1100

> On Thu, Feb 05, 2009 at 02:24:02PM -0800, Roland Dreier wrote:
> >
> > >From a quick scan of the code, it looks as if optlen is never sanity
> > checked in the case of setsockopt(IPV6_FLOWLABEL_MGR), and
> > ipv6_flowlabel_opt() calls into fl_create() with whatever value
> > userspace passes in, which then pretty much does kmalloc(optlen).
> > So if icmpv6fuzz passes some big random value, it can cause this failure.
> > 
> > I don't know what the appropriate limit should be, so no patch, sorry.
> 
> Well it is legal (though unlikely) to pass very long options.
> So the real fix would be to avoid copying the control message
> at all and modify all the code involved to read user pointers
> directly.  Someone with a lot of patience is required for this :)

That's a little bit overboard I think.

We already impose limits for pktoptions specified by the user
directly, so we should impose the same limit for this pktoption
blob sitting after the flowlabel entry.

I'll add the following to net-2.6

--------------------
ipv6: Disallow rediculious flowlabel option sizes.

Just like PKTINFO, limit the options area to 64K.

Based upon report by Eric Sesterhenn and analysis by
Roland Dreier.

Signed-off-by: David S. Miller <davem@davemloft.net>
---
 net/ipv6/ip6_flowlabel.c |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/net/ipv6/ip6_flowlabel.c b/net/ipv6/ip6_flowlabel.c
index c62dd24..7712578 100644
--- a/net/ipv6/ip6_flowlabel.c
+++ b/net/ipv6/ip6_flowlabel.c
@@ -323,17 +323,21 @@ static struct ip6_flowlabel *
 fl_create(struct net *net, struct in6_flowlabel_req *freq, char __user *optval,
 	  int optlen, int *err_p)
 {
-	struct ip6_flowlabel *fl;
+	struct ip6_flowlabel *fl = NULL;
 	int olen;
 	int addr_type;
 	int err;
 
+	olen = optlen - CMSG_ALIGN(sizeof(*freq));
+	err = -EINVAL;
+	if (olen > 64 * 1024)
+		goto done;
+
 	err = -ENOMEM;
 	fl = kzalloc(sizeof(*fl), GFP_KERNEL);
 	if (fl == NULL)
 		goto done;
 
-	olen = optlen - CMSG_ALIGN(sizeof(*freq));
 	if (olen > 0) {
 		struct msghdr msg;
 		struct flowi flowi;
-- 
1.6.1.2.253.ga34a

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Fri, Feb 06, 2009 at 12:50:34AM -0800, David Miller wrote:
>
> ipv6: Disallow rediculious flowlabel option sizes.
> 
> Just like PKTINFO, limit the options area to 64K.
> 
> Based upon report by Eric Sesterhenn and analysis by
> Roland Dreier.
> 
> Signed-off-by: David S. Miller <davem@davemloft.net>

This should eliminate the backtrace as reported, but it'll just
show up somewhere else.  In particular, the control message (see
sys_sendmsg) itself is copied to kernel memory via kmalloc and
it's limited to INT_MAX :)

(The option we're discussing here is stored in the control message).

We either need to vmalloc it or have everyone read the user
memory directly.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 6 Feb 2009 19:54:06 +1100

> This should eliminate the backtrace as reported, but it'll just
> show up somewhere else.  In particular, the control message (see
> sys_sendmsg) itself is copied to kernel memory via kmalloc and
> it's limited to INT_MAX :)

Can you be more specific, do you mean this is happening
via datagram_send_ctl()?

If we limit how big these txoption blobs can be, at all
sites where we parse instances the user gives us, I can't
see any such problems.

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Fri, Feb 06, 2009 at 01:05:59AM -0800, David Miller wrote:
> From: Herbert Xu <herbert@gondor.apana.org.au>
> Date: Fri, 6 Feb 2009 19:54:06 +1100
> 
> > This should eliminate the backtrace as reported, but it'll just
> > show up somewhere else.  In particular, the control message (see
> > sys_sendmsg) itself is copied to kernel memory via kmalloc and
> > it's limited to INT_MAX :)
> 
> Can you be more specific, do you mean this is happening
> via datagram_send_ctl()?

Well the data that led to the back trace came from a cmsg, I was
merely pointing out that the cmsg itself is kmalloced and copied
by sys_sendmsg and it's only limited to INT_MAX.

So even if you throw away the message in IPv6, sys_sendmsg will
still do an even bigger allocation on that message which is just
as likely to fail.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 6 Feb 2009 20:22:18 +1100

> Well the data that led to the back trace came from a cmsg, I was
> merely pointing out that the cmsg itself is kmalloced and copied
> by sys_sendmsg and it's only limited to INT_MAX.

Actually, it's limited to sysctl_optmem_max :-)

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Fri, Feb 06, 2009 at 01:27:33AM -0800, David Miller wrote:
>
> Actually, it's limited to sysctl_optmem_max :-)

Hiding the check in sock_kmalloc, clever :) In that case we don't
have a problem.

But we should probably bring the check down to sysctl_optmem_max
in ip6_flowlabel.c too since allocating 64K is still quite likely
to fail and warn.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Fri, Feb 06, 2009 at 09:27:10PM +1100, Herbert Xu wrote:
>
> But we should probably bring the check down to sysctl_optmem_max
> in ip6_flowlabel.c too since allocating 64K is still quite likely
> to fail and warn.

In fact, we should probably just use sock_kmalloc in these places.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 6 Feb 2009 21:27:10 +1100

> On Fri, Feb 06, 2009 at 01:27:33AM -0800, David Miller wrote:
> >
> > Actually, it's limited to sysctl_optmem_max :-)
> 
> Hiding the check in sock_kmalloc, clever :) In that case we don't
> have a problem.
> 
> But we should probably bring the check down to sysctl_optmem_max
> in ip6_flowlabel.c too since allocating 64K is still quite likely
> to fail and warn.

Good idea.

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 6 Feb 2009 21:34:43 +1100

> On Fri, Feb 06, 2009 at 09:27:10PM +1100, Herbert Xu wrote:
> >
> > But we should probably bring the check down to sysctl_optmem_max
> > in ip6_flowlabel.c too since allocating 64K is still quite likely
> > to fail and warn.
> 
> In fact, we should probably just use sock_kmalloc in these places.

Very likely we could get away with that.

But some apps might start breaking since this would now
charge the socket and we could hit the limits whereas
before we wouldn't.

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

David Miller <davem@davemloft.net> wrote:
>
> But some apps might start breaking since this would now
> charge the socket and we could hit the limits whereas
> before we wouldn't.

Yes we should definitely test this carefully.

However, I think it is the right thing to do since we shouldn't
leave holes where the user can allocate kernel memory that is
uncapped.

In fact, a number of spots in IPv6 already use sock_kmalloc
for these objects anyway (e.g., ipv6_dup_options in exthdrs.c)
so there is no fundamental reason why this limit can't be imposed.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Mon, 9 Feb 2009 16:03:54 +1100

> In fact, a number of spots in IPv6 already use sock_kmalloc
> for these objects anyway (e.g., ipv6_dup_options in exthdrs.c)
> so there is no fundamental reason why this limit can't be imposed.

Indeed, we are consistently inconsistent here.

Re: Deadlock with icmpv6fuzz

[David Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Mon, 9 Feb 2009 16:03:54 +1100

> David Miller <davem@davemloft.net> wrote:
> >
> > But some apps might start breaking since this would now
> > charge the socket and we could hit the limits whereas
> > before we wouldn't.
> 
> Yes we should definitely test this carefully.
> 
> However, I think it is the right thing to do since we shouldn't
> leave holes where the user can allocate kernel memory that is
> uncapped.
> 
> In fact, a number of spots in IPv6 already use sock_kmalloc
> for these objects anyway (e.g., ipv6_dup_options in exthdrs.c)
> so there is no fundamental reason why this limit can't be imposed.

Actually it turns out we can't do that here.  This flow label object
is global, rather than specifically associated with a given socket.

See net/ipv6/ip6_flowlabel.c:fl_{create,intern}() and how those
are used.

Re: Deadlock with icmpv6fuzz

[Herbert Xu]

On Tue, Feb 24, 2009 at 11:44:01PM -0800, David Miller wrote:
>
> Actually it turns out we can't do that here.  This flow label object
> is global, rather than specifically associated with a given socket.
> 
> See net/ipv6/ip6_flowlabel.c:fl_{create,intern}() and how those
> are used.

They're global but that doesn't stop us from charging it once
for each socket that uses it (isn't that what Microsoft does
for Windows :)

The fact is right now we have an entity that a user can create
willy nilly with no limits.  That can't be good.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt