* Bad performance when CUBIC HyStart enabled
From: Lucas Nussbaum @ 2011-03-03 14:22 UTC (permalink / raw)
To: netdev; +Cc: Sangtae Ha
Hi,
I'm experiencing bad performance transferring data over a large BDP
network with CUBIC when HyStart is enabled. Disabling HyStart fixes the
problem.
Test setup:
I'm transfering a 2-GB file between two machines using netcat. On both
machines, /dev/shm is used to store the data.
On the sending side, I use:
/dev/shm# while true; do dd if=foo bs=10M | nc -l -p 2222 -q 0; done
On the receiving side, I use:
/dev/shm# /usr/bin/time nc 131.254.203.6 2222 > foo
Both machines are on Gigabit ethernet networks, and the RTT between them
is 11.3ms.
The default sysctl values are used.
I use nload -i 1000000 -o 1000000 -u m eth0 to visualize the transfer
rate.
With a 2.6.26 kernel, everything is fine. The bandwidth quickly gets to
~940 Mbps, and I get the following in nload:
..................................
.##################################
###################################
###################################
###################################
####################################
####################################
####################################
####################################
####################################
####################################
####################################
####################################
####################################
The transfer time is 18.2s, which is reasonable for a 2-GB file.
With a 2.6.32 kernel (Debian's) or with a 2.6.37 kernel (on the sending
side only), with the default HyStart parameters, I get the following:
..........................
|##########################.
|############################
|#############################
|##############################
|###############################
.#################################
|##################################|
|#####################################
..||#######################################
....|##############################################
.#####################################################
The transfer time is 26.8s. (There's one character every 0.5s on the
graph above)
Disabling hystart on the sending machine with:
echo 0 > /sys/module/tcp_cubic/parameters/hystart
Fixes the problem.
It seems that waiting for some time (or rebooting) fixes the problem for
some time, so I'm suspecting that the problem is caused by a cache of
some sort. However, doing:
echo 1 > /proc/sys/net/ipv4/route/flush
doesn't fix the problem.
Ideas? I could arrange to give access to the testbed if needed.
(https://www.grid5000.fr/)
--
| Lucas Nussbaum MCF Université Nancy 2 |
| lucas.nussbaum@loria.fr LORIA / AlGorille |
| http://www.loria.fr/~lnussbau/ +33 3 54 95 86 19 |
^ permalink raw reply
* Re: Bug inkvm_set_irq
From: Michael S. Tsirkin @ 2011-03-03 14:47 UTC (permalink / raw)
To: Jean-Philippe Menil; +Cc: netdev, kvm, virtualization
In-Reply-To: <4D6D0510.4060300@univ-nantes.fr>
On Tue, Mar 01, 2011 at 03:39:12PM +0100, Jean-Philippe Menil wrote:
> so this time the bug is:
>
> [17882.612303] BUG: unable to handle kernel paging request at
> 0000000000002458
> [17882.612342] IP: [<ffffffffa03898a0>] kvm_set_irq+0x30/0x140 [kvm]
>
> markup_oops give me this:
>
> root@ayrshire:~# cat bug-0103.txt | perl markup_oops.pl -m
> /lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
> /boot/vmlinuz-2.6.37.2-dsiun-110105+
> vmaoffset = 18446744072102621184 ffffffffa0389871: 48 89 e5 mov
> %rsp,%rbp
> ffffffffa0389874: 41 57 push %r15
> ffffffffa0389876: 41 89 cf mov %ecx,%r15d | %r15
> => 1 %ecx = 1
> ffffffffa0389879: 41 56 push %r14 | %r14
> => ffffffffa038aad0
> ffffffffa038987b: 41 55 push %r13
> ffffffffa038987d: 49 89 fd mov %rdi,%r13 | %edi
> = 0 %r13 => 0
> ffffffffa0389880: 41 54 push %r12 | %r12 => 0
> ffffffffa0389882: 53 push %rbx
> ffffffffa0389883: 89 d3 mov %edx,%ebx | %ebx => 1a
> ffffffffa0389885: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
> ffffffffa038988c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx
> # ffffffffa0389892 <kvm_set_irq+0x22>
> ffffffffa0389892: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp) |
> %esi = 0
> ffffffffa0389898: 85 d2 test %edx,%edx | %edx => 0
> ffffffffa038989a: 0f 85 d5 00 00 00 jne ffffffffa0389975
> <kvm_set_irq+0x105>
> *ffffffffa03898a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax |
> %eax = 0 %r13 = 0 <--- faulting instruction
> ffffffffa03898a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
> ffffffffa03898ad: 73 61 jae ffffffffa0389910
> <kvm_set_irq+0xa0>
> ffffffffa03898af: 89 db mov %ebx,%ebx
> ffffffffa03898b1: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax
> ffffffffa03898b8: 00
> ffffffffa03898b9: 48 85 c0 test %rax,%rax
> ffffffffa03898bc: 74 52 je ffffffffa0389910
> <kvm_set_irq+0xa0>
> ffffffffa03898be: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
> ffffffffa03898c5: 31 db xor %ebx,%ebx
> ffffffffa03898c7: 48 8b 08 mov (%rax),%rcx
> ffffffffa03898ca: 83 c3 01 add $0x1,%ebx
> ffffffffa03898cd: 0f 18 09 prefetcht0 (%rcx)
> ffffffffa03898d0: 48 8b 48 e0 mov -0x20(%rax),%rcx
> ffffffffa03898d4: 48 89 0a mov %rcx,(%rdx)
> ffffffffa03898d7: 48 8b 48 e8 mov -0x18(%rax),%rcx
> ffffffffa03898db: 48 89 4a 08 mov %rcx,0x8(%rdx)
> ffffffffa03898df: 48 8b 48 f0 mov -0x10(%rax),%rcx
> ffffffffa03898e3: 48 89 4a 10 mov %rcx,0x10(%rdx)
> ffffffffa03898e7: 48 8b 48 f8 mov -0x8(%rax),%rcx
> ffffffffa03898eb: 48 89 4a 18 mov %rcx,0x18(%rdx)
>
> wich correspond to offset 68a0 (from objdump):
>
> kvm_set_irq():
> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
> 68a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax
> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
> 68a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
>
> root@ayrshire:~# addr2line -e
> /lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
> 0x68a0
> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
>
> So here kvm->irq_routing is null.
>
> How can it be?
>
> Regards.
Not null, this seems to be invalid.
I suspect use after free where the kvm pointer is
pointing at some random memory. Use after free?
Could you please try enabling a slab debugger,
recompile and rerun the test?
--
MST
^ permalink raw reply
* Re: Bad performance when CUBIC HyStart enabled
From: Eric Dumazet @ 2011-03-03 15:03 UTC (permalink / raw)
To: Lucas Nussbaum; +Cc: netdev, Sangtae Ha
In-Reply-To: <20110303142229.GA9262@xanadu.blop.info>
Le jeudi 03 mars 2011 à 15:22 +0100, Lucas Nussbaum a écrit :
> Hi,
>
> I'm experiencing bad performance transferring data over a large BDP
> network with CUBIC when HyStart is enabled. Disabling HyStart fixes the
> problem.
>
> Test setup:
> I'm transfering a 2-GB file between two machines using netcat. On both
> machines, /dev/shm is used to store the data.
> On the sending side, I use:
> /dev/shm# while true; do dd if=foo bs=10M | nc -l -p 2222 -q 0; done
> On the receiving side, I use:
> /dev/shm# /usr/bin/time nc 131.254.203.6 2222 > foo
> Both machines are on Gigabit ethernet networks, and the RTT between them
> is 11.3ms.
> The default sysctl values are used.
> I use nload -i 1000000 -o 1000000 -u m eth0 to visualize the transfer
> rate.
>
> With a 2.6.26 kernel, everything is fine. The bandwidth quickly gets to
> ~940 Mbps, and I get the following in nload:
> ..................................
> .##################################
> ###################################
> ###################################
> ###################################
> ####################################
> ####################################
> ####################################
> ####################################
> ####################################
> ####################################
> ####################################
> ####################################
> ####################################
> The transfer time is 18.2s, which is reasonable for a 2-GB file.
>
> With a 2.6.32 kernel (Debian's) or with a 2.6.37 kernel (on the sending
> side only), with the default HyStart parameters, I get the following:
> ..........................
> |##########################.
> |############################
> |#############################
> |##############################
> |###############################
> .#################################
> |##################################|
> |#####################################
> ..||#######################################
> ....|##############################################
> .#####################################################
> The transfer time is 26.8s. (There's one character every 0.5s on the
> graph above)
>
> Disabling hystart on the sending machine with:
> echo 0 > /sys/module/tcp_cubic/parameters/hystart
> Fixes the problem.
>
> It seems that waiting for some time (or rebooting) fixes the problem for
> some time, so I'm suspecting that the problem is caused by a cache of
> some sort. However, doing:
> echo 1 > /proc/sys/net/ipv4/route/flush
> doesn't fix the problem.
>
> Ideas? I could arrange to give access to the testbed if needed.
> (https://www.grid5000.fr/)
eventually, after fresh boot of both sender/receiver, it would be nice
to have "netstat -s" results after one transfert with said problems.
^ permalink raw reply
* Re: [PATCH] sched: QFQ - quick fair queue scheduler (v2)
From: Fabio Checconi @ 2011-03-03 15:07 UTC (permalink / raw)
To: Eric Dumazet
Cc: Stephen Hemminger, David Miller, Luigi Rizzo, netdev,
Paolo Valente
In-Reply-To: <1299148820.2983.56.camel@edumazet-laptop>
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Thu, Mar 03, 2011 11:40:20AM +0100
>
> Le jeudi 03 mars 2011 à 09:27 +0100, Eric Dumazet a écrit :
>
> > I am going to test an updated version, thanks for all these hints !
> >
>
> Same problem in qfq_qlen_notify()
>
> Should use :
>
> struct qfq_sched *q = qdisc_priv(sch);
>
Absolutely right, as you're right about qfq_drop(). I'll be able to take
a look when I'm back from work this evening.
A question on the data below:
> After a stress (and no more trafic) I still have some packets in
> backlog:
>
> (8 packets here)
>
...
> class qfq 11:10 root weight 1600 maxpkt 2048
> Sent 200 bytes 1 pkt (dropped 18181, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
Any other reason, apart from a bug in QFQ, why there could be 200b and
0p of backlog? A packet count out of sync could explain also why you
were experiencing problems with pfifo in presence of drops...
Thanks for your help, fixes, and testing,
fabio
^ permalink raw reply
* Re: [PATCH] sched: QFQ - quick fair queue scheduler (v2)
From: Eric Dumazet @ 2011-03-03 15:18 UTC (permalink / raw)
To: Fabio Checconi
Cc: Stephen Hemminger, David Miller, Luigi Rizzo, netdev,
Paolo Valente
In-Reply-To: <1299148820.2983.56.camel@edumazet-laptop>
Le jeudi 03 mars 2011 à 11:40 +0100, Eric Dumazet a écrit :
> Le jeudi 03 mars 2011 à 09:27 +0100, Eric Dumazet a écrit :
>
> > I am going to test an updated version, thanks for all these hints !
> >
>
> Same problem in qfq_qlen_notify()
>
> Should use :
>
> struct qfq_sched *q = qdisc_priv(sch);
>
>
> After a stress (and no more trafic) I still have some packets in
> backlog:
>
> (8 packets here)
>
> # tc -s -d qdisc show dev dummy0
> qdisc cbq 1: root refcnt 2 rate 1000Mbit cell 8b (bounded,isolated) prio no-transmit/8 weight 1000Mbit allot 1514b
> level 2 ewma 5 avpkt 1000b maxidle 0us
> Sent 109916000 bytes 549580 pkt (dropped 350412, overlimits 1180608 requeues 0)
> backlog 0b 8p requeues 0
> borrowed 0 overactions 0 avgidle 125 undertime 0
> qdisc qfq 11: parent 1:11
> Sent 109916000 bytes 549580 pkt (dropped 350412, overlimits 0 requeues 0)
> rate 2848bit 2pps backlog 0b 8p requeues 0
> # tc -s -d class show dev dummy0
> class cbq 1:11 parent 1:1 leaf 11: rate 40000Kbit cell 128b mpu 64b (bounded) prio 2/2 weight 40000Kbit allot 20000b
> level 0 ewma 5 avpkt 1400b maxidle 0us
> Sent 109916000 bytes 549580 pkt (dropped 350412, overlimits 675712 requeues 0)
> rate 1516Kbit 947pps backlog 0b 8p requeues 0
> borrowed 0 overactions 285254 avgidle -22 undertime -24
> class cbq 1: root rate 1000Mbit cell 8b (bounded,isolated) prio no-transmit/8 weight 1000Mbit allot 1514b
> level 2 ewma 5 avpkt 1000b maxidle 0us
> Sent 109916000 bytes 549580 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
> borrowed 0 overactions 0 avgidle 125 undertime 0
> class cbq 1:1 parent 1: rate 1000Mbit cell 64b mpu 64b (bounded) prio 1/1 weight 1000Mbit allot 10000b
> level 1 ewma 5 avpkt 1500b maxidle 0us
> Sent 109916000 bytes 549580 pkt (dropped 0, overlimits 0 requeues 0)
> rate 1516Kbit 947pps backlog 0b 0p requeues 0
> borrowed 0 overactions 0 avgidle 125 undertime 0
> class qfq 11:10 root weight 1600 maxpkt 2048
> Sent 200 bytes 1 pkt (dropped 18181, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:1 root weight 100 maxpkt 1100
> Sent 2731200 bytes 13656 pkt (dropped 40889, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:2 root weight 200 maxpkt 1200
> Sent 10899800 bytes 54499 pkt (dropped 47, overlimits 0 requeues 0)
> rate 24bit 0pps backlog 0b 0p requeues 0
> class qfq 11:3 root weight 300 maxpkt 1300
> Sent 12714400 bytes 63572 pkt (dropped 65, overlimits 0 requeues 0)
> rate 32bit 0pps backlog 0b 0p requeues 0
> class qfq 11:4 root weight 400 maxpkt 1400
> Sent 400 bytes 2 pkt (dropped 36362, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:5 root weight 500 maxpkt 1500
> Sent 400 bytes 2 pkt (dropped 45452, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:6 root weight 600 maxpkt 1600
> Sent 800 bytes 4 pkt (dropped 27269, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:7 root weight 700 maxpkt 1700
> Sent 600 bytes 3 pkt (dropped 45452, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:8 root weight 800 maxpkt 1800
> Sent 1000 bytes 5 pkt (dropped 72722, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:9 root weight 900 maxpkt 1900
> Sent 1200 bytes 6 pkt (dropped 63631, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 200b 0p requeues 0
> class qfq 11:a root weight 1000 maxpkt 2000
> Sent 14535200 bytes 72676 pkt (dropped 51, overlimits 0 requeues 0)
> rate 32bit 0pps backlog 0b 0p requeues 0
> class qfq 11:b root weight 1100 maxpkt 2048
> Sent 14531200 bytes 72656 pkt (dropped 72, overlimits 0 requeues 0)
> rate 32bit 0pps backlog 0b 0p requeues 0
> class qfq 11:c root weight 1200 maxpkt 2048
> Sent 10905800 bytes 54529 pkt (dropped 16, overlimits 0 requeues 0)
> rate 24bit 0pps backlog 0b 0p requeues 0
> class qfq 11:d root weight 1300 maxpkt 2048
> Sent 14538200 bytes 72691 pkt (dropped 36, overlimits 0 requeues 0)
> rate 32bit 0pps backlog 0b 0p requeues 0
> class qfq 11:e root weight 1400 maxpkt 2048
> Sent 18162200 bytes 90811 pkt (dropped 96, overlimits 0 requeues 0)
> rate 40bit 0pps backlog 0b 0p requeues 0
> class qfq 11:f root weight 1500 maxpkt 2048
> Sent 10895000 bytes 54475 pkt (dropped 71, overlimits 0 requeues 0)
> rate 24bit 0pps backlog 0b 0p requeues 0
>
>
>
With debugging, I see dequeue() returns early because bitmaps[ER] is
null
if (!q->bitmaps[ER]) {
pr_err("qfq_dequeue bitmaps[ER] is null, return NULL\n");
return NULL;
}
[ 3319.540984] qfq calc_index: W = 1, L = 2048, I = 19
[ 3319.544098] qfq calc_index: W = 1, L = 2048, I = 19
[ 3319.547184] qfq calc_index: W = 1, L = 2048, I = 19
[ 3319.550272] qfq calc_index: W = 1, L = 2048, I = 19
[ 3319.553354] qfq calc_index: W = 1, L = 2048, I = 19
[ 3319.556435] qfq calc_index: W = 1, L = 2048, I = 19
[ 3319.559533] qfq calc_index: W = 1, L = 2048, I = 19
[ 3319.562627] qfq calc_index: W = 1, L = 2048, I = 19
[ 3322.993412] qfq_enqueue: cl = 110004
[ 3322.993457] qfq enqueue: new state 0 0x80000 S 0 F 214748364800 V 0
[ 3322.993493] qfq dequeue: cl ffff880077d45380 len 200 F 214748364800 now 1638400
[ 3322.993543] qfq slot_scan: grp 19 full 0x0
[ 3322.993575] qfq_enqueue: cl = 110007
[ 3322.993603] qfq enqueue: new state 0 0x80000 S 1638400 F 214750003200 V 1638400
[ 3322.993653] qfq_enqueue: cl = 110006
[ 3322.993684] qfq dequeue: cl ffff880077d45100 len 200 F 214750003200 now 3276800
[ 3322.993735] qfq slot_scan: grp 19 full 0x1
[ 3322.993766] qfq_enqueue: cl = 110003
[ 3322.993797] qfq dequeue: cl ffff88011d10e080 len 200 F 214751641600 now 4915200
[ 3322.993847] qfq slot_scan: grp 19 full 0x1
[ 3322.993877] qfq_enqueue: cl = 110002
[ 3322.993906] qfq dequeue: cl ffff880077d45c00 len 200 F 214753280000 now 6553600
[ 3322.997895] qfq slot_scan: grp 19 full 0x1
[ 3322.997925] qfq_enqueue: cl = 110003
[ 3322.997956] qfq_enqueue: cl = 110004
[ 3322.997989] qfq dequeue: cl ffff880077d45380 len 200 F 429496729600 now 8192000
[ 3322.998040] qfq slot_scan: grp 19 full 0x1
[ 3322.998070] qfq_enqueue: cl = 110006
[ 3322.998099] qfq_enqueue: cl = 110008
[ 3322.998130] qfq_enqueue: cl = 110002
[ 3322.998160] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998192] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998224] qfq_enqueue: cl = 110002
[ 3322.998249] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998276] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998305] qfq_enqueue: cl = 110007
[ 3322.998333] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998363] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998395] qfq_enqueue: cl = 110001
[ 3322.998421] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998449] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998478] qfq_enqueue: cl = 110004
[ 3322.998502] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998530] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998558] qfq_enqueue: cl = 110006
[ 3322.998585] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998616] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998649] qfq_enqueue: cl = 110004
[ 3322.998677] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998708] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998739] qfq_enqueue: cl = 110007
[ 3322.998763] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998790] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998832] qfq_enqueue: cl = 110004
[ 3322.998875] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998907] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998937] qfq_enqueue: cl = 110001
[ 3322.998964] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.998997] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999029] qfq_enqueue: cl = 110003
[ 3322.999057] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999087] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999120] qfq_enqueue: cl = 110007
[ 3322.999147] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999178] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999210] qfq_enqueue: cl = 110001
[ 3322.999234] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999261] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999290] qfq_enqueue: cl = 110005
[ 3322.999316] qfq_dequeue bitmaps[ER] is null, return NULL
[ 3322.999343] qfq_dequeue bitmaps[ER] is null, return NULL
[...]
Only 5 packets sent ...
qdisc cbq 1: root refcnt 2 rate 1000Mbit cell 8b (bounded,isolated) prio no-transmit/8 weight 1000Mbit allot 1514b
level 2 ewma 5 avpkt 1000b maxidle 0us
Sent 1000 bytes 5 pkt (dropped 0, overlimits 82 requeues 0)
backlog 0b 85p requeues 0
borrowed 0 overactions 0 avgidle 125 undertime 0
qdisc qfq 11: parent 1:11
Sent 1000 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
rate 0bit 0pps backlog 0b 85p requeues 0
qdisc pfifo 8011: parent 11:1 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2600b 13p requeues 0
qdisc pfifo 8012: parent 11:2 limit 30p
Sent 200 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2400b 12p requeues 0
qdisc pfifo 8013: parent 11:3 limit 30p
Sent 200 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2200b 11p requeues 0
qdisc pfifo 8014: parent 11:4 limit 30p
Sent 400 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2400b 12p requeues 0
qdisc pfifo 8015: parent 11:5 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2000b 10p requeues 0
qdisc pfifo 8016: parent 11:6 limit 30p
Sent 200 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2000b 10p requeues 0
qdisc pfifo 8017: parent 11:7 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2400b 12p requeues 0
qdisc pfifo 8018: parent 11:8 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 1000b 5p requeues 0
^ permalink raw reply
* A misbehavior of handling TCP out of band data ?
From: Li Yu @ 2011-03-03 15:24 UTC (permalink / raw)
To: netdev
Hi,
In my words, if we did not turn on SO_OOBINLINE at receiver,
any OOB bytes do not have to be read from regular TCP byte stream.
But in below extreme case: the sender process have a loop to
send 1 OOB byte 'O' and 1 byte normal data 'D', the receiver ignore
all OOB data, so if OOB feature works well, the receiver should
receive 'D' only, However, in fact, the receiver still can see OOB
bytes by recv() syscall without MSG_OOB flag.
I found that we may send "ODODODODODOD...." in a tcp segment,
and tcp_sendmsg() has no special code to avoid this.
Well, I know that it seem that above case is too extreme to
use in actual environment, but above is simplified from our actual
scenario.
Anyway, I think that overwriting or skip previous OOB bytes is
acceptable, but leave them in regular TCP byte stream likely is a bug.
En, probably, the disabled SO_OOBINLINE just is a feature of receiver
side, sender do not need to have to guarantee it?
Thanks.
Yu
^ permalink raw reply
* Re: [PATCH] sched: QFQ - quick fair queue scheduler (v2)
From: Eric Dumazet @ 2011-03-03 15:25 UTC (permalink / raw)
To: Fabio Checconi
Cc: Stephen Hemminger, David Miller, Luigi Rizzo, netdev,
Paolo Valente
In-Reply-To: <20110303150744.GB29685@gandalf.sssup.it>
Le jeudi 03 mars 2011 à 16:07 +0100, Fabio Checconi a écrit :
> A question on the data below:
>
> > After a stress (and no more trafic) I still have some packets in
> > backlog:
> >
> > (8 packets here)
> >
> ...
> > class qfq 11:10 root weight 1600 maxpkt 2048
> > Sent 200 bytes 1 pkt (dropped 18181, overlimits 0 requeues 0)
> > rate 0bit 0pps backlog 200b 0p requeues 0
>
> Any other reason, apart from a bug in QFQ, why there could be 200b and
> 0p of backlog? A packet count out of sync could explain also why you
> were experiencing problems with pfifo in presence of drops...
My packets are all 200 bytes (from my packet generator)
qfq_dump_class_stats() probably miss one line :
scl->bstats.qlen = cl->qdisc->q.qlen;
^ permalink raw reply
* Re: Bug inkvm_set_irq
From: Jean-Philippe Menil @ 2011-03-03 15:26 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: netdev, kvm, virtualization
In-Reply-To: <20110303144715.GA12400@redhat.com>
Le 03/03/2011 15:47, Michael S. Tsirkin a écrit :
> On Tue, Mar 01, 2011 at 03:39:12PM +0100, Jean-Philippe Menil wrote:
>> so this time the bug is:
>>
>> [17882.612303] BUG: unable to handle kernel paging request at
>> 0000000000002458
>> [17882.612342] IP: [<ffffffffa03898a0>] kvm_set_irq+0x30/0x140 [kvm]
>>
>> markup_oops give me this:
>>
>> root@ayrshire:~# cat bug-0103.txt | perl markup_oops.pl -m
>> /lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
>> /boot/vmlinuz-2.6.37.2-dsiun-110105+
>> vmaoffset = 18446744072102621184 ffffffffa0389871: 48 89 e5 mov
>> %rsp,%rbp
>> ffffffffa0389874: 41 57 push %r15
>> ffffffffa0389876: 41 89 cf mov %ecx,%r15d | %r15
>> => 1 %ecx = 1
>> ffffffffa0389879: 41 56 push %r14 | %r14
>> => ffffffffa038aad0
>> ffffffffa038987b: 41 55 push %r13
>> ffffffffa038987d: 49 89 fd mov %rdi,%r13 | %edi
>> = 0 %r13 => 0
>> ffffffffa0389880: 41 54 push %r12 | %r12 => 0
>> ffffffffa0389882: 53 push %rbx
>> ffffffffa0389883: 89 d3 mov %edx,%ebx | %ebx => 1a
>> ffffffffa0389885: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
>> ffffffffa038988c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx
>> # ffffffffa0389892<kvm_set_irq+0x22>
>> ffffffffa0389892: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp) |
>> %esi = 0
>> ffffffffa0389898: 85 d2 test %edx,%edx | %edx => 0
>> ffffffffa038989a: 0f 85 d5 00 00 00 jne ffffffffa0389975
>> <kvm_set_irq+0x105>
>> *ffffffffa03898a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax |
>> %eax = 0 %r13 = 0<--- faulting instruction
>> ffffffffa03898a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
>> ffffffffa03898ad: 73 61 jae ffffffffa0389910
>> <kvm_set_irq+0xa0>
>> ffffffffa03898af: 89 db mov %ebx,%ebx
>> ffffffffa03898b1: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax
>> ffffffffa03898b8: 00
>> ffffffffa03898b9: 48 85 c0 test %rax,%rax
>> ffffffffa03898bc: 74 52 je ffffffffa0389910
>> <kvm_set_irq+0xa0>
>> ffffffffa03898be: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
>> ffffffffa03898c5: 31 db xor %ebx,%ebx
>> ffffffffa03898c7: 48 8b 08 mov (%rax),%rcx
>> ffffffffa03898ca: 83 c3 01 add $0x1,%ebx
>> ffffffffa03898cd: 0f 18 09 prefetcht0 (%rcx)
>> ffffffffa03898d0: 48 8b 48 e0 mov -0x20(%rax),%rcx
>> ffffffffa03898d4: 48 89 0a mov %rcx,(%rdx)
>> ffffffffa03898d7: 48 8b 48 e8 mov -0x18(%rax),%rcx
>> ffffffffa03898db: 48 89 4a 08 mov %rcx,0x8(%rdx)
>> ffffffffa03898df: 48 8b 48 f0 mov -0x10(%rax),%rcx
>> ffffffffa03898e3: 48 89 4a 10 mov %rcx,0x10(%rdx)
>> ffffffffa03898e7: 48 8b 48 f8 mov -0x8(%rax),%rcx
>> ffffffffa03898eb: 48 89 4a 18 mov %rcx,0x18(%rdx)
>>
>> wich correspond to offset 68a0 (from objdump):
>>
>> kvm_set_irq():
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
>> 68a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
>> 68a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
>>
>> root@ayrshire:~# addr2line -e
>> /lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
>> 0x68a0
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
>>
>> So here kvm->irq_routing is null.
>>
>> How can it be?
>>
>> Regards.
> Not null, this seems to be invalid.
> I suspect use after free where the kvm pointer is
> pointing at some random memory. Use after free?
> Could you please try enabling a slab debugger,
> recompile and rerun the test?
>
Hi,
I'm not sure to activate the right thing.
Is that what you want?
CONFIG_SLAB=y
CONFIG_SLABINFO=y
CONFIG_DEBUG_SLAB=y
CONFIG_DEBUG_SLAB_LEAK=y
Regards.
--
Jean-Philippe Menil - Pôle réseau Service IRTS
DSI Université de Nantes
jean-philippe.menil@univ-nantes.fr
Tel : 02.53.48.49.27 - Fax : 02.53.48.49.09
^ permalink raw reply
* Re: [PATCH] sched: QFQ - quick fair queue scheduler (v2)
From: Eric Dumazet @ 2011-03-03 15:31 UTC (permalink / raw)
To: Fabio Checconi
Cc: Stephen Hemminger, David Miller, Luigi Rizzo, netdev,
Paolo Valente
In-Reply-To: <1299165940.2983.111.camel@edumazet-laptop>
Le jeudi 03 mars 2011 à 16:25 +0100, Eric Dumazet a écrit :
> qfq_dump_class_stats() probably miss one line :
>
> scl->bstats.qlen = cl->qdisc->q.qlen;
>
>
Hmm,
cl->qdisc->qstats.qlen = cl->qdisc->q.qlen;
It should be better ;)
^ permalink raw reply
* Re: [PATCH] sched: QFQ - quick fair queue scheduler (v2)
From: Eric Dumazet @ 2011-03-03 15:39 UTC (permalink / raw)
To: Fabio Checconi
Cc: Stephen Hemminger, David Miller, Luigi Rizzo, netdev,
Paolo Valente
In-Reply-To: <1299166292.2983.112.camel@edumazet-laptop>
Le jeudi 03 mars 2011 à 16:31 +0100, Eric Dumazet a écrit :
> Le jeudi 03 mars 2011 à 16:25 +0100, Eric Dumazet a écrit :
>
> > qfq_dump_class_stats() probably miss one line :
> >
> > scl->bstats.qlen = cl->qdisc->q.qlen;
> >
> >
>
> Hmm,
>
> cl->qdisc->qstats.qlen = cl->qdisc->q.qlen;
>
> It should be better ;)
>
>
Indeed, 'packet count' is now correct in qfq classes
# tc -s -d qdisc show dev dummy0
qdisc cbq 1: root refcnt 2 rate 1000Mbit cell 8b (bounded,isolated) prio no-transmit/8 weight 1000Mbit allot 1514b
level 2 ewma 5 avpkt 1000b maxidle 0us
Sent 800 bytes 4 pkt (dropped 0, overlimits 83 requeues 0)
backlog 0b 86p requeues 0
borrowed 0 overactions 0 avgidle 125 undertime 0
qdisc qfq 11: parent 1:11
Sent 800 bytes 4 pkt (dropped 0, overlimits 0 requeues 0)
rate 360bit 0pps backlog 0b 86p requeues 0
qdisc pfifo 8001: parent 11:1 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2400b 12p requeues 0
qdisc pfifo 8002: parent 11:2 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2000b 10p requeues 0
qdisc pfifo 8003: parent 11:3 limit 30p
Sent 200 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 3400b 17p requeues 0
qdisc pfifo 8004: parent 11:4 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 1200b 6p requeues 0
qdisc pfifo 8005: parent 11:5 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 2400b 12p requeues 0
qdisc pfifo 8006: parent 11:6 limit 30p
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 1400b 7p requeues 0
qdisc pfifo 8007: parent 11:7 limit 30p
Sent 400 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)
backlog 1400b 7p requeues 0
qdisc pfifo 8008: parent 11:8 limit 30p
Sent 200 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
backlog 3000b 15p requeues 0
^ permalink raw reply
* Re: Bug inkvm_set_irq
From: Michael S. Tsirkin @ 2011-03-03 15:55 UTC (permalink / raw)
To: Jean-Philippe Menil; +Cc: netdev, kvm, virtualization
In-Reply-To: <4D6FB313.1010400@univ-nantes.fr>
On Thu, Mar 03, 2011 at 04:26:11PM +0100, Jean-Philippe Menil wrote:
> Le 03/03/2011 15:47, Michael S. Tsirkin a écrit :
> >On Tue, Mar 01, 2011 at 03:39:12PM +0100, Jean-Philippe Menil wrote:
> >>so this time the bug is:
> >>
> >>[17882.612303] BUG: unable to handle kernel paging request at
> >>0000000000002458
> >>[17882.612342] IP: [<ffffffffa03898a0>] kvm_set_irq+0x30/0x140 [kvm]
> >>
> >>markup_oops give me this:
> >>
> >>root@ayrshire:~# cat bug-0103.txt | perl markup_oops.pl -m
> >>/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
> >>/boot/vmlinuz-2.6.37.2-dsiun-110105+
> >>vmaoffset = 18446744072102621184 ffffffffa0389871: 48 89 e5 mov
> >>%rsp,%rbp
> >> ffffffffa0389874: 41 57 push %r15
> >> ffffffffa0389876: 41 89 cf mov %ecx,%r15d | %r15
> >>=> 1 %ecx = 1
> >> ffffffffa0389879: 41 56 push %r14 | %r14
> >>=> ffffffffa038aad0
> >> ffffffffa038987b: 41 55 push %r13
> >> ffffffffa038987d: 49 89 fd mov %rdi,%r13 | %edi
> >>= 0 %r13 => 0
> >> ffffffffa0389880: 41 54 push %r12 | %r12 => 0
> >> ffffffffa0389882: 53 push %rbx
> >> ffffffffa0389883: 89 d3 mov %edx,%ebx | %ebx => 1a
> >> ffffffffa0389885: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
> >> ffffffffa038988c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx
> >># ffffffffa0389892<kvm_set_irq+0x22>
> >> ffffffffa0389892: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp) |
> >>%esi = 0
> >> ffffffffa0389898: 85 d2 test %edx,%edx | %edx => 0
> >> ffffffffa038989a: 0f 85 d5 00 00 00 jne ffffffffa0389975
> >><kvm_set_irq+0x105>
> >>*ffffffffa03898a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax |
> >>%eax = 0 %r13 = 0<--- faulting instruction
> >> ffffffffa03898a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
> >> ffffffffa03898ad: 73 61 jae ffffffffa0389910
> >><kvm_set_irq+0xa0>
> >> ffffffffa03898af: 89 db mov %ebx,%ebx
> >> ffffffffa03898b1: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax
> >> ffffffffa03898b8: 00
> >> ffffffffa03898b9: 48 85 c0 test %rax,%rax
> >> ffffffffa03898bc: 74 52 je ffffffffa0389910
> >><kvm_set_irq+0xa0>
> >> ffffffffa03898be: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
> >> ffffffffa03898c5: 31 db xor %ebx,%ebx
> >> ffffffffa03898c7: 48 8b 08 mov (%rax),%rcx
> >> ffffffffa03898ca: 83 c3 01 add $0x1,%ebx
> >> ffffffffa03898cd: 0f 18 09 prefetcht0 (%rcx)
> >> ffffffffa03898d0: 48 8b 48 e0 mov -0x20(%rax),%rcx
> >> ffffffffa03898d4: 48 89 0a mov %rcx,(%rdx)
> >> ffffffffa03898d7: 48 8b 48 e8 mov -0x18(%rax),%rcx
> >> ffffffffa03898db: 48 89 4a 08 mov %rcx,0x8(%rdx)
> >> ffffffffa03898df: 48 8b 48 f0 mov -0x10(%rax),%rcx
> >> ffffffffa03898e3: 48 89 4a 10 mov %rcx,0x10(%rdx)
> >> ffffffffa03898e7: 48 8b 48 f8 mov -0x8(%rax),%rcx
> >> ffffffffa03898eb: 48 89 4a 18 mov %rcx,0x18(%rdx)
> >>
> >>wich correspond to offset 68a0 (from objdump):
> >>
> >>kvm_set_irq():
> >>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
> >> 68a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax
> >>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
> >> 68a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
> >>
> >>root@ayrshire:~# addr2line -e
> >>/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
> >>0x68a0
> >>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
> >>
> >>So here kvm->irq_routing is null.
> >>
> >>How can it be?
> >>
> >>Regards.
> >Not null, this seems to be invalid.
> >I suspect use after free where the kvm pointer is
> >pointing at some random memory. Use after free?
> >Could you please try enabling a slab debugger,
> >recompile and rerun the test?
> >
> Hi,
>
> I'm not sure to activate the right thing.
> Is that what you want?
>
> CONFIG_SLAB=y
> CONFIG_SLABINFO=y
> CONFIG_DEBUG_SLAB=y
> CONFIG_DEBUG_SLAB_LEAK=y
>
> Regards.
Yes, maybe disable SLAB_LEAK.
> --
> Jean-Philippe Menil - Pôle réseau Service IRTS
> DSI Université de Nantes
> jean-philippe.menil@univ-nantes.fr
> Tel : 02.53.48.49.27 - Fax : 02.53.48.49.09
^ permalink raw reply
* Re: bonding...
From: Andy Gospodarek @ 2011-03-03 16:04 UTC (permalink / raw)
To: Neil Horman; +Cc: David Miller, fubar, netdev
In-Reply-To: <20110303114539.GB5146@hmsreliant.think-freely.org>
On Thu, Mar 03, 2011 at 06:45:39AM -0500, Neil Horman wrote:
> On Wed, Mar 02, 2011 at 09:49:10PM -0800, David Miller wrote:
> >
> > Hey, if someone could step up and help with bonding maintainence
> > in some tangible way, I'd really appreciate it.
> >
> > Currently the situation is that many people work on bonding patches,
> > and whilst I do try and wait for some ACKs to arrive, I am the person
> > who has to sort out when changes are ready, decide to apply them, and
> > poke for review when things fall through the cracks.
> >
> > Sometimes patches go for weeks without ACKs, and in that situation
> > I have to either try to understand the changes myself, or wait
> > potentially forever for someone with bonding knowledge to take a
> > good look at the patch and properly review it.
> >
> > It was nearly 2 weeks before Oleg V. Ukhno's 802.3ad round-robin patch
> > got looked at by anyone with bonding knowledge. And it only happened
> > because I got tired of seeing his poor patch rot in patchwork
> > and had to explicitly asked for review the other day.
> >
> > This is unacceptable, people are submitting multiple bonding patches
> > every single day now. It needs a clueful bonding person looking at
> > these submissions on a constant basis.
> >
> > This is a serious problem and is backlogging the netdev patch queue.
> >
> > So if someone would become an active bonding patch-accumulator, and
> > send me sets of patches that are ready to apply, I would really
> > appreciate it.
> >
> > Thanks.
> I nominate gospo. If he doesn't want to, I can do it
> Neil
>
I would be willing to do it, but my one of my goals would be to prevent
some of the feature creep we are currently seeing with bonding (as Ben
has suggested). That doesn't mean I want to stop all new features, but
at this point things are starting to get out of control. I suspect this
is why Jay has struggled to keep up with the patches.
I would also want to look at a restructuring of the configuration. The
lines are starting to blur between some of the modes and output port
selection for other modes and that needs to be cleared up.
How does that sound?
-andy
^ permalink raw reply
* Re: [PATCH] sched: QFQ - quick fair queue scheduler (v2)
From: Eric Dumazet @ 2011-03-03 16:03 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: David Miller, Fabio Checconi, Luigi Rizzo, netdev
In-Reply-To: <20110228171738.2cc8c9a0@nehalam>
Le lundi 28 février 2011 à 17:17 -0800, Stephen Hemminger a écrit :
> +static int qfq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
> + struct gnet_dump *d)
> +{
> + struct qfq_class *cl = (struct qfq_class *)arg;
> + struct tc_qfq_stats xstats;
> +
> + memset(&xstats, 0, sizeof(xstats));
> +
> + xstats.weight = ONE_FP/cl->inv_w;
> + xstats.lmax = cl->lmax;
> +
cl->qdisc->qstats.qlen = cl->qdisc->q.qlen;
> + if (gnet_stats_copy_basic(d, &cl->bstats) < 0 ||
> + gnet_stats_copy_rate_est(d, NULL, &cl->rate_est) < 0 ||
> + gnet_stats_copy_queue(d, &cl->qdisc->qstats) < 0)
> + return -1;
> +
> + return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
> +}
> +
using
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0
is better : We avoid dumping null rate estimation, if admin never asked
a rate estimation...
^ permalink raw reply
* [BUG net-next-2.6] dst_release() crash
From: Eric Dumazet @ 2011-03-03 16:28 UTC (permalink / raw)
To: David Miller; +Cc: netdev
Just had this while working on latest net-next-2.6
[11483.697210] BUG: unable to handle kernel NULL pointer dereference at
0000002a
[11483.697233] IP: [<c12b0638>] dst_release+0x18/0x60
[11483.697252] *pdpt = 0000000034917001 *pde = 0000000000000000
[11483.697270] Oops: 0002 [#1] SMP
[11483.697283] last sysfs
file: /sys/devices/virtual/net/bond0/bonding/active_slave
[11483.697299] Modules linked in: pktgen ipmi_si ipmi_msghandler hpilo
tg3 bonding ipv6
[11483.697335]
[11483.697340] Pid: 5490, comm: ntpd Not tainted
2.6.38-rc5-02884-g0853e6c-dirty #974 HP ProLiant BL460c G1
[11483.697366] EIP: 0060:[<c12b0638>] EFLAGS: 00010282 CPU: 4
[11483.697392] EIP is at dst_release+0x18/0x60
[11483.697416] EAX: ffffffea EBX: ffffffea ECX: 00000000 EDX: 6e14a8c0
[11483.697444] ESI: ffffffff EDI: ffffffea EBP: f445fd18 ESP: f445fd10
[11483.697471] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
[11483.697497] Process ntpd (pid: 5490, ti=f445e000 task=f3542490
task.ti=f445e000)
[11483.697540] Stack:
[11483.697562] f3401400 ffffffea f445fdc0 c12fc9d2 00000030 f445fd3c
c12a1c01 00000282
[11483.697619] 00000008 00000030 0dfeaa0a 6e14a8c0 7b001000 00000000
0dfeaa0a f445fda0
[11483.697680] 00000000 00000000 c12dbd90 00000000 f445ff38 00000000
00000000 00000000
[11483.697741] Call Trace:
[11483.697764] [<c12fc9d2>] udp_sendmsg+0x282/0x6e0
[11483.697790] [<c12a1c01>] ? memcpy_toiovec+0x51/0x70
[11483.697818] [<c12dbd90>] ? ip_generic_getfrag+0x0/0xb0
[11483.697848] [<c1184e00>] ? timerqueue_add+0x30/0xc0
[11483.697874] [<c1304ae5>] inet_sendmsg+0x45/0xa0
[11483.697901] [<c10617ab>] ? __hrtimer_start_range_ns+0x1ab/0x500
[11483.697930] [<c1298543>] sock_sendmsg+0xc3/0xf0
[11483.697957] [<c100b2b4>] ? save_i387_fxsave+0x74/0x80
[11483.697984] [<c1189022>] ? _copy_from_user+0x32/0x50
[11483.698010] [<c1299702>] sys_sendto+0xb2/0xe0
[11483.698035] [<c100bd50>] ? restore_i387_xstate+0xd0/0x1f0
[11483.698062] [<c129a07d>] sys_socketcall+0x18d/0x270
[11483.698088] [<c1002cd0>] sysenter_do_call+0x12/0x26
[11483.698123] Code: 53 c1 e8 ac 0b 0a 00 5b c9 c3 89 f6 8d bc 27 00 00
00 00 55 89 e5 83 ec 08 85 c0 89 1c 24 89 74 24 04 89 c3 74 11 be ff ff
ff ff <f0> 0f c1 70 40 4e 78 2f 85 f6 74 0c 8b 1c 24 8b 74 24 04 c9 c3
[11483.698319] EIP: [<c12b0638>] dst_release+0x18/0x60 SS:ESP
0068:f445fd10
[11483.698350] CR2: 000000000000002a
[11483.698594] ---[ end trace cb654a6638801e35 ]---
crash in
c12b0638: f0 0f c1 70 40 lock xadd %esi,0x40(%eax)
EAX = -EINVAL
commit b23dd4fe42b455af (ipv4: Make output route lookup return rtable
directly) introduced a regression ...
We could add a sanity test in dst_release(), or fix callers ?
What do you think ?
^ permalink raw reply
* Re: bonding...
From: Stephen Hemminger @ 2011-03-03 16:31 UTC (permalink / raw)
To: Ben Hutchings; +Cc: David Miller, fubar, netdev
In-Reply-To: <1299159991.4277.45.camel@localhost>
On Thu, 03 Mar 2011 13:46:31 +0000
Ben Hutchings <bhutchings@solarflare.com> wrote:
> On Wed, 2011-03-02 at 21:49 -0800, David Miller wrote:
> [...]
> > This is unacceptable, people are submitting multiple bonding patches
> > every single day now. It needs a clueful bonding person looking at
> > these submissions on a constant basis.
> [...]
>
> And preferably saying 'no' to most new features...
Agreed. It seems bonding has diverged from the standards and wants
to support every packet flow some user can think up.
^ permalink raw reply
* Re: bonding...
From: Eric Dumazet @ 2011-03-03 16:52 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Ben Hutchings, David Miller, fubar, netdev
In-Reply-To: <20110303083110.4e7f020c@nehalam>
Le jeudi 03 mars 2011 à 08:31 -0800, Stephen Hemminger a écrit :
> On Thu, 03 Mar 2011 13:46:31 +0000
> Ben Hutchings <bhutchings@solarflare.com> wrote:
>
> > On Wed, 2011-03-02 at 21:49 -0800, David Miller wrote:
> > [...]
> > > This is unacceptable, people are submitting multiple bonding patches
> > > every single day now. It needs a clueful bonding person looking at
> > > these submissions on a constant basis.
> > [...]
> >
> > And preferably saying 'no' to most new features...
>
> Agreed. It seems bonding has diverged from the standards and wants
> to support every packet flow some user can think up.
A simple setup that was working in 2.6.35 doesnt work anymore here
vlan.103 - bond0 - eth3 (tg3 nic)
With a bnx2 nic, it works
Not sure its a bond problem, more probably a vlan one...
^ permalink raw reply
* [PATCH net-next-2.6] udp: udp_sendmsg() fix
From: Eric Dumazet @ 2011-03-03 17:07 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <1299169686.2983.129.camel@edumazet-laptop>
Le jeudi 03 mars 2011 à 17:28 +0100, Eric Dumazet a écrit :
> Just had this while working on latest net-next-2.6
>
> [11483.697210] BUG: unable to handle kernel NULL pointer dereference at
> 0000002a
> [11483.697233] IP: [<c12b0638>] dst_release+0x18/0x60
> [11483.697252] *pdpt = 0000000034917001 *pde = 0000000000000000
> [11483.697270] Oops: 0002 [#1] SMP
> [11483.697283] last sysfs
> file: /sys/devices/virtual/net/bond0/bonding/active_slave
> [11483.697299] Modules linked in: pktgen ipmi_si ipmi_msghandler hpilo
> tg3 bonding ipv6
> [11483.697335]
> [11483.697340] Pid: 5490, comm: ntpd Not tainted
> 2.6.38-rc5-02884-g0853e6c-dirty #974 HP ProLiant BL460c G1
> [11483.697366] EIP: 0060:[<c12b0638>] EFLAGS: 00010282 CPU: 4
> [11483.697392] EIP is at dst_release+0x18/0x60
> [11483.697416] EAX: ffffffea EBX: ffffffea ECX: 00000000 EDX: 6e14a8c0
> [11483.697444] ESI: ffffffff EDI: ffffffea EBP: f445fd18 ESP: f445fd10
> [11483.697471] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
> [11483.697497] Process ntpd (pid: 5490, ti=f445e000 task=f3542490
> task.ti=f445e000)
> [11483.697540] Stack:
> [11483.697562] f3401400 ffffffea f445fdc0 c12fc9d2 00000030 f445fd3c
> c12a1c01 00000282
> [11483.697619] 00000008 00000030 0dfeaa0a 6e14a8c0 7b001000 00000000
> 0dfeaa0a f445fda0
> [11483.697680] 00000000 00000000 c12dbd90 00000000 f445ff38 00000000
> 00000000 00000000
> [11483.697741] Call Trace:
> [11483.697764] [<c12fc9d2>] udp_sendmsg+0x282/0x6e0
> [11483.697790] [<c12a1c01>] ? memcpy_toiovec+0x51/0x70
> [11483.697818] [<c12dbd90>] ? ip_generic_getfrag+0x0/0xb0
> [11483.697848] [<c1184e00>] ? timerqueue_add+0x30/0xc0
> [11483.697874] [<c1304ae5>] inet_sendmsg+0x45/0xa0
> [11483.697901] [<c10617ab>] ? __hrtimer_start_range_ns+0x1ab/0x500
> [11483.697930] [<c1298543>] sock_sendmsg+0xc3/0xf0
> [11483.697957] [<c100b2b4>] ? save_i387_fxsave+0x74/0x80
> [11483.697984] [<c1189022>] ? _copy_from_user+0x32/0x50
> [11483.698010] [<c1299702>] sys_sendto+0xb2/0xe0
> [11483.698035] [<c100bd50>] ? restore_i387_xstate+0xd0/0x1f0
> [11483.698062] [<c129a07d>] sys_socketcall+0x18d/0x270
> [11483.698088] [<c1002cd0>] sysenter_do_call+0x12/0x26
> [11483.698123] Code: 53 c1 e8 ac 0b 0a 00 5b c9 c3 89 f6 8d bc 27 00 00
> 00 00 55 89 e5 83 ec 08 85 c0 89 1c 24 89 74 24 04 89 c3 74 11 be ff ff
> ff ff <f0> 0f c1 70 40 4e 78 2f 85 f6 74 0c 8b 1c 24 8b 74 24 04 c9 c3
> [11483.698319] EIP: [<c12b0638>] dst_release+0x18/0x60 SS:ESP
> 0068:f445fd10
> [11483.698350] CR2: 000000000000002a
> [11483.698594] ---[ end trace cb654a6638801e35 ]---
>
>
> crash in
>
> c12b0638: f0 0f c1 70 40 lock xadd %esi,0x40(%eax)
>
> EAX = -EINVAL
>
>
>
> commit b23dd4fe42b455af (ipv4: Make output route lookup return rtable
> directly) introduced a regression ...
>
> We could add a sanity test in dst_release(), or fix callers ?
>
> What do you think ?
>
>
Here is the patch I cooked after an audit
[PATCH net-next-2.6] udp: udp_sendmsg() fix
commit b23dd4fe42b455af (ipv4: Make output route lookup return rtable
directly) introduced a regression in udp_sendmsg()
If IP route lookup fails, we should not try to release an non existent
route.
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 95e0c2c..f6c0c00 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -927,7 +927,7 @@ int udp_sendmsg(struct kiocb *iocb, struct sock *sk, struct msghdr *msg,
err = PTR_ERR(rt);
if (err == -ENETUNREACH)
IP_INC_STATS_BH(net, IPSTATS_MIB_OUTNOROUTES);
- goto out;
+ goto out2;
}
err = -EACCES;
@@ -991,6 +991,7 @@ do_append_data:
out:
ip_rt_put(rt);
+out2:
if (free)
kfree(ipc.opt);
if (!err)
^ permalink raw reply related
* Re: bonding...
From: Jay Vosburgh @ 2011-03-03 17:24 UTC (permalink / raw)
To: Andy Gospodarek; +Cc: Neil Horman, David Miller, netdev
In-Reply-To: <20110303160404.GO11864@gospo.rdu.redhat.com>
Andy Gospodarek <andy@greyhouse.net> wrote:
>On Thu, Mar 03, 2011 at 06:45:39AM -0500, Neil Horman wrote:
>> On Wed, Mar 02, 2011 at 09:49:10PM -0800, David Miller wrote:
>> >
>> > Hey, if someone could step up and help with bonding maintainence
>> > in some tangible way, I'd really appreciate it.
>> >
>> > Currently the situation is that many people work on bonding patches,
>> > and whilst I do try and wait for some ACKs to arrive, I am the person
>> > who has to sort out when changes are ready, decide to apply them, and
>> > poke for review when things fall through the cracks.
>> >
>> > Sometimes patches go for weeks without ACKs, and in that situation
>> > I have to either try to understand the changes myself, or wait
>> > potentially forever for someone with bonding knowledge to take a
>> > good look at the patch and properly review it.
>> >
>> > It was nearly 2 weeks before Oleg V. Ukhno's 802.3ad round-robin patch
>> > got looked at by anyone with bonding knowledge. And it only happened
>> > because I got tired of seeing his poor patch rot in patchwork
>> > and had to explicitly asked for review the other day.
>> >
>> > This is unacceptable, people are submitting multiple bonding patches
>> > every single day now. It needs a clueful bonding person looking at
>> > these submissions on a constant basis.
>> >
>> > This is a serious problem and is backlogging the netdev patch queue.
>> >
>> > So if someone would become an active bonding patch-accumulator, and
>> > send me sets of patches that are ready to apply, I would really
>> > appreciate it.
>> >
>> > Thanks.
>> I nominate gospo. If he doesn't want to, I can do it
>> Neil
>>
>
>I would be willing to do it, but my one of my goals would be to prevent
>some of the feature creep we are currently seeing with bonding (as Ben
>has suggested). That doesn't mean I want to stop all new features, but
>at this point things are starting to get out of control. I suspect this
>is why Jay has struggled to keep up with the patches.
My main problem keeping up at the moment is another demand on my
time that should run its course in a couple of weeks. My time for
community activity is somewhat limited until then.
As far as the features go, yes, I agree that things are getting
out of hand. The two pending patches for special cases related to
802.3ad (Oleg's patch to permit round robining, the other to enable
spanning aggregators across switches), for example, are fine
functionality, but there really needs to be a better, generic, way to do
this sort of niche case activity without adding more knobs.
I personally like Stephen's suggestion to hook bonding into a
netfilter gadget, similar to ebtables for bridge. Done properly, such a
gadget should handle (or be extended to handle) the niche cases that
right now end up as new knobs in the driver.
>I would also want to look at a restructuring of the configuration. The
>lines are starting to blur between some of the modes and output port
>selection for other modes and that needs to be cleared up.
What did you have in mind here?
-J
---
-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com
^ permalink raw reply
* Re: [BUG] VPN broken in net-next
From: Stephen Hemminger @ 2011-03-03 17:32 UTC (permalink / raw)
To: Julian Anastasov; +Cc: David Miller, netdev
In-Reply-To: <alpine.LFD.2.00.1103031454060.2113@ja.ssi.bg>
> Hello,
>
> On Thu, 3 Mar 2011, Julian Anastasov wrote:
>
> > May be the problem is in inet_hash_insert(), it should
> > hash ifa_local, not ifa_address. May be they are equal for
>
> ... and of course the new __ip_dev_find should use
> ifa_local too.
>
> > the common case but not for peer addresses. In devinet_ioctl()
> > we can see they are equal initially:
> >
> > ifa->ifa_address = ifa->ifa_local = sin->sin_addr.s_addr;
> >
> > but later SIOCSIFDSTADDR can change ifa_address which
> > is destination address from the same prefix:
> >
> > ifa->ifa_address = sin->sin_addr.s_addr;
>
> While checking for ifa_address usage I see other
> two places that look suspicious:
>
> - inet_gifconf() exposes address from ifa_local but then
> devinet_ioctl() matches by ifa_address in the
> 'if (tryaddrmatch)' block. I think, we should use ifa_local.
>
> - IN_DEV_ARP_NOTIFY: announces ifa_address instead of ifa_local.
Julian you are on the right track with the ifa_local being the issue.
1. Bring up VPN
(no errors)
2. Ping one address gets connect error
3. Instrumentation triggers. I added code so if __ip_dev_find failed,
it walked the hash table.
net=ffffffff81a12d80
[ 393.224228] __ip_dev_find(ffffffff81a12d80, 10.250.0.104) hash=108 failed
[ 393.224232] 138: ffffffff81a12d80 ifa_addr=127.0.0.1 ifa_local=127.0.0.1
[ 393.224236] 150: ffffffff81a12d80 ifa_addr=192.168.1.11 ifa_local=192.168.1.11
[ 393.224239] 249: ffffffff81a12d80 ifa_addr=192.168.100.1 ifa_local=192.168.100.1
[ 393.224242] 254: ffffffff81a12d80 ifa_addr=192.168.99.1 ifa_local=192.168.99.1
[ 393.224245] 255: ffffffff81a12d80 ifa_addr=10.255.254.0 ifa_local=10.250.0.10
^ permalink raw reply
* Re: [PATCH 2/2] netlink: kill eff_cap from struct netlink_skb_parms
From: Chris Wright @ 2011-03-03 17:32 UTC (permalink / raw)
To: Patrick McHardy
Cc: NetDev, dm-devel, David S. Miller, Chris Wright,
linux-security-module@vger.kernel.org, drbd-dev
In-Reply-To: <4D6F6180.5030903@trash.net>
* Patrick McHardy (kaber@trash.net) wrote:
> commit 8ff259625f0ab295fa085b0718eed13093813fbc
> Author: Patrick McHardy <kaber@trash.net>
> Date: Thu Mar 3 10:17:31 2011 +0100
>
> netlink: kill eff_cap from struct netlink_skb_parms
>
> Netlink message processing in the kernel is synchronous these days,
> capabilities can be checked directly in security_netlink_recv() from
> the current process.
>
> Signed-off-by: Patrick McHardy <kaber@trash.net>
Thanks for doing that Patrick. I looked at this earlier and thought
there was still an async path, but I guess that's just to another
userspace process.
BTW, I think you missed a couple connector based callers:
drivers/staging/pohmelfs/config.c: if (!cap_raised(nsp->eff_cap, CAP_SYS_AD
drivers/video/uvesafb.c: if (!cap_raised(nsp->eff_cap, CAP_SYS_ADMIN))
Fix those and:
Acked-by: Chris Wright <chrisw@sous-sol.org>
Ideally, we'd consolidate those into a variant of security_netlink_recv().
However the issue is with types. Inside connector callback we only have
netlink_skb_params (seems inapproriate to cast back out to skb).
We could change the lsm hook to only pass nsp, but SELinux actually
cares about the netlink type. Any ideas?
^ permalink raw reply
* Re: Bad performance when CUBIC HyStart enabled
From: Lucas Nussbaum @ 2011-03-03 17:37 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev, Sangtae Ha
In-Reply-To: <1299164580.2983.99.camel@edumazet-laptop>
[-- Attachment #1: Type: text/plain, Size: 3694 bytes --]
On 03/03/11 at 16:03 +0100, Eric Dumazet wrote:
> Le jeudi 03 mars 2011 à 15:22 +0100, Lucas Nussbaum a écrit :
> > Hi,
> >
> > I'm experiencing bad performance transferring data over a large BDP
> > network with CUBIC when HyStart is enabled. Disabling HyStart fixes the
> > problem.
> >
> > Test setup:
> > I'm transfering a 2-GB file between two machines using netcat. On both
> > machines, /dev/shm is used to store the data.
> > On the sending side, I use:
> > /dev/shm# while true; do dd if=foo bs=10M | nc -l -p 2222 -q 0; done
> > On the receiving side, I use:
> > /dev/shm# /usr/bin/time nc 131.254.203.6 2222 > foo
> > Both machines are on Gigabit ethernet networks, and the RTT between them
> > is 11.3ms.
> > The default sysctl values are used.
> > I use nload -i 1000000 -o 1000000 -u m eth0 to visualize the transfer
> > rate.
> >
> > With a 2.6.26 kernel, everything is fine. The bandwidth quickly gets to
> > ~940 Mbps, and I get the following in nload:
> > ..................................
> > .##################################
> > ###################################
> > ###################################
> > ###################################
> > ####################################
> > ####################################
> > ####################################
> > ####################################
> > ####################################
> > ####################################
> > ####################################
> > ####################################
> > ####################################
> > The transfer time is 18.2s, which is reasonable for a 2-GB file.
> >
> > With a 2.6.32 kernel (Debian's) or with a 2.6.37 kernel (on the sending
> > side only), with the default HyStart parameters, I get the following:
> > ..........................
> > |##########################.
> > |############################
> > |#############################
> > |##############################
> > |###############################
> > .#################################
> > |##################################|
> > |#####################################
> > ..||#######################################
> > ....|##############################################
> > .#####################################################
> > The transfer time is 26.8s. (There's one character every 0.5s on the
> > graph above)
> >
> > Disabling hystart on the sending machine with:
> > echo 0 > /sys/module/tcp_cubic/parameters/hystart
> > Fixes the problem.
> >
> > It seems that waiting for some time (or rebooting) fixes the problem for
> > some time, so I'm suspecting that the problem is caused by a cache of
> > some sort. However, doing:
> > echo 1 > /proc/sys/net/ipv4/route/flush
> > doesn't fix the problem.
> >
> > Ideas? I could arrange to give access to the testbed if needed.
> > (https://www.grid5000.fr/)
>
> eventually, after fresh boot of both sender/receiver, it would be nice
> to have "netstat -s" results after one transfert with said problems.
Hi,
Attached.
I had 6 successful transfers. There's the netstat -s output after the 5
first successful transfers. Then the 7th transfer showed the problem,
as well as the 8th and the 9th. There's the netstat -s output after each
of them.
--
| Lucas Nussbaum MCF Université Nancy 2 |
| lucas.nussbaum@loria.fr LORIA / AlGorille |
| http://www.loria.fr/~lnussbau/ +33 3 54 95 86 19 |
[-- Attachment #2: netstat-s.tgz --]
[-- Type: application/x-gtar, Size: 2275 bytes --]
^ permalink raw reply
* Re: Bad performance when CUBIC HyStart enabled
From: Eric Dumazet @ 2011-03-03 17:46 UTC (permalink / raw)
To: Lucas Nussbaum; +Cc: netdev, Sangtae Ha
In-Reply-To: <20110303173733.GB8028@xanadu.blop.info>
Le jeudi 03 mars 2011 à 18:37 +0100, Lucas Nussbaum a écrit :
> I had 6 successful transfers. There's the netstat -s output after the 5
> first successful transfers. Then the 7th transfer showed the problem,
> as well as the 8th and the 9th. There's the netstat -s output after each
> of them.
Its a bit strange : my 'netstat -s' are more verbose than yours :
netstat -s
Ip:
1033944 total packets received
0 forwarded
0 incoming packets discarded
1033705 incoming packets delivered
701852 requests sent out
4 dropped because of missing route
120 reassemblies required
60 packets reassembled ok
60 fragments received ok
120 fragments created
Icmp:
704 ICMP messages received
0 input ICMP message failed.
ICMP input histogram:
destination unreachable: 177
timeout in transit: 117
echo requests: 103
echo replies: 307
693 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 177
echo request: 413
echo replies: 103
IcmpMsg:
InType0: 307
InType3: 177
InType8: 103
InType11: 117
OutType0: 103
OutType3: 177
OutType8: 413
Tcp:
10526 active connections openings
212 passive connection openings
78 failed connection attempts
300 connection resets received
26 connections established
969786 segments received
690213 segments send out
894 segments retransmited
0 bad segments received.
305 resets sent
Udp:
43123 packets received
4 packets to unknown port received.
0 packet receive errors
9570 packets sent
UdpLite:
TcpExt:
877 TCP sockets finished time wait in fast timer
23798 delayed acks sent
10 delayed acks further delayed because of locked socket
Quick ack mode was activated 598 times
542574 packets directly queued to recvmsg prequeue.
140834 bytes directly in process context from backlog
45033318 bytes directly received in process context from prequeue
646384 packet headers predicted
81439 packets header predicted and directly queued to user
24971 acknowledgments not containing data payload received
189383 predicted acknowledgments
2 times recovered from packet loss by selective acknowledgements
3 congestion windows recovered without slow start by DSACK
308 congestion windows recovered without slow start after partial
ack
2 TCP data loss events
3 timeouts after SACK recovery
2 fast retransmits
1 retransmits in slow start
632 other TCP timeouts
9 times receiver scheduled too late for direct processing
587 DSACKs sent for old packets
2 DSACKs sent for out of order packets
417 DSACKs received
130 connections reset due to unexpected data
72 connections aborted due to timeout
TCPDSACKIgnoredOld: 176
TCPDSACKIgnoredNoUndo: 56
TCPSackMerged: 5
TCPSackShiftFallback: 4
TCPDeferAcceptDrop: 106
IpExt:
InNoRoutes: 5
InMcastPkts: 35832
OutMcastPkts: 1403
InBcastPkts: 19536
OutBcastPkts: 89
InOctets: 787953535
OutOctets: 113825814
InMcastOctets: 4098542
OutMcastOctets: 211245
InBcastOctets: 1906912
OutBcastOctets: 13625
^ permalink raw reply
* [PATCH 1/3] KEYS: Add a new keyctl op to reject a key with a specified error code
From: David Howells @ 2011-03-03 17:49 UTC (permalink / raw)
To: linux-security-module, keyrings; +Cc: linux-kernel, netdev, David Howells
Add a new keyctl op to reject a key with a specified error code. This works
much the same as negating a key, and so keyctl_negate_key() is made a special
case of keyctl_reject_key(). The difference is that keyctl_negate_key()
selects ENOKEY as the error to be reported.
Typically the key would be rejected with EKEYEXPIRED, EKEYREVOKED or
EKEYREJECTED, but this is not mandatory.
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/keys-request-key.txt | 9 +++++---
Documentation/keys.txt | 10 +++++++--
include/linux/key-type.h | 11 +++++++++-
include/linux/key.h | 1 +
include/linux/keyctl.h | 1 +
security/keys/compat.c | 3 +++
security/keys/internal.h | 1 +
security/keys/key.c | 19 ++++++++++-------
security/keys/keyctl.c | 40 ++++++++++++++++++++++++++++++++++--
security/keys/keyring.c | 4 ++--
security/keys/request_key.c | 2 +-
11 files changed, 81 insertions(+), 20 deletions(-)
diff --git a/Documentation/keys-request-key.txt b/Documentation/keys-request-key.txt
index 09b55e4..69686ad 100644
--- a/Documentation/keys-request-key.txt
+++ b/Documentation/keys-request-key.txt
@@ -127,14 +127,15 @@ This is because process A's keyrings can't simply be attached to
of them, and (b) it requires the same UID/GID/Groups all the way through.
-======================
-NEGATIVE INSTANTIATION
-======================
+====================================
+NEGATIVE INSTANTIATION AND REJECTION
+====================================
Rather than instantiating a key, it is possible for the possessor of an
authorisation key to negatively instantiate a key that's under construction.
This is a short duration placeholder that causes any attempt at re-requesting
-the key whilst it exists to fail with error ENOKEY.
+the key whilst it exists to fail with error ENOKEY if negated or the specified
+error if rejected.
This is provided to prevent excessive repeated spawning of /sbin/request-key
processes for a key that will never be obtainable.
diff --git a/Documentation/keys.txt b/Documentation/keys.txt
index cf68d1f..a6a97fd 100644
--- a/Documentation/keys.txt
+++ b/Documentation/keys.txt
@@ -657,6 +657,8 @@ The keyctl syscall functions are:
long keyctl(KEYCTL_NEGATE, key_serial_t key,
unsigned timeout, key_serial_t keyring);
+ long keyctl(KEYCTL_REJECT, key_serial_t key,
+ unsigned timeout, unsigned error, key_serial_t keyring);
If the kernel calls back to userspace to complete the instantiation of a
key, userspace should use this call mark the key as negative before the
@@ -669,6 +671,10 @@ The keyctl syscall functions are:
that keyring, however all the constraints applying in KEYCTL_LINK apply in
this case too.
+ If the key is rejected, future searches for it will return the specified
+ error code until the rejected key expires. Negating the key is the same
+ as rejecting the key with ENOKEY as the error code.
+
(*) Set the default request-key destination keyring.
@@ -1240,8 +1246,8 @@ example, the KDE desktop manager).
The program (or whatever it calls) should finish construction of the key by
calling KEYCTL_INSTANTIATE, which also permits it to cache the key in one of
the keyrings (probably the session ring) before returning. Alternatively, the
-key can be marked as negative with KEYCTL_NEGATE; this also permits the key to
-be cached in one of the keyrings.
+key can be marked as negative with KEYCTL_NEGATE or KEYCTL_REJECT; this also
+permits the key to be cached in one of the keyrings.
If it returns with the key remaining in the unconstructed state, the key will
be marked as being negative, it will be added to the session keyring, and an
diff --git a/include/linux/key-type.h b/include/linux/key-type.h
index fc8525e8..9efd081 100644
--- a/include/linux/key-type.h
+++ b/include/linux/key-type.h
@@ -105,11 +105,20 @@ extern int key_instantiate_and_link(struct key *key,
size_t datalen,
struct key *keyring,
struct key *instkey);
-extern int key_negate_and_link(struct key *key,
+extern int key_reject_and_link(struct key *key,
unsigned timeout,
+ unsigned error,
struct key *keyring,
struct key *instkey);
extern void complete_request_key(struct key_construction *cons, int error);
+static inline int key_negate_and_link(struct key *key,
+ unsigned timeout,
+ struct key *keyring,
+ struct key *instkey)
+{
+ return key_reject_and_link(key, timeout, ENOKEY, keyring, instkey);
+}
+
#endif /* CONFIG_KEYS */
#endif /* _LINUX_KEY_TYPE_H */
diff --git a/include/linux/key.h b/include/linux/key.h
index a6b1edc..b2bb017 100644
--- a/include/linux/key.h
+++ b/include/linux/key.h
@@ -170,6 +170,7 @@ struct key {
struct list_head link;
unsigned long x[2];
void *p[2];
+ int reject_error;
} type_data;
/* key data
diff --git a/include/linux/keyctl.h b/include/linux/keyctl.h
index bd383f1..7022974 100644
--- a/include/linux/keyctl.h
+++ b/include/linux/keyctl.h
@@ -53,5 +53,6 @@
#define KEYCTL_ASSUME_AUTHORITY 16 /* assume request_key() authorisation */
#define KEYCTL_GET_SECURITY 17 /* get key security label */
#define KEYCTL_SESSION_TO_PARENT 18 /* apply session keyring to parent process */
+#define KEYCTL_REJECT 19 /* reject a partially constructed key */
#endif /* _LINUX_KEYCTL_H */
diff --git a/security/keys/compat.c b/security/keys/compat.c
index 07a5f35..17c99d0 100644
--- a/security/keys/compat.c
+++ b/security/keys/compat.c
@@ -85,6 +85,9 @@ asmlinkage long compat_sys_keyctl(u32 option,
case KEYCTL_SESSION_TO_PARENT:
return keyctl_session_to_parent();
+ case KEYCTL_REJECT:
+ return keyctl_reject_key(arg2, arg3, arg4, arg5);
+
default:
return -EOPNOTSUPP;
}
diff --git a/security/keys/internal.h b/security/keys/internal.h
index a52aa7c..286c095 100644
--- a/security/keys/internal.h
+++ b/security/keys/internal.h
@@ -214,6 +214,7 @@ extern long keyctl_assume_authority(key_serial_t);
extern long keyctl_get_security(key_serial_t keyid, char __user *buffer,
size_t buflen);
extern long keyctl_session_to_parent(void);
+extern long keyctl_reject_key(key_serial_t, unsigned, unsigned, key_serial_t);
/*
* Debugging key validation
diff --git a/security/keys/key.c b/security/keys/key.c
index 8e315ef..f7f9d93 100644
--- a/security/keys/key.c
+++ b/security/keys/key.c
@@ -511,26 +511,29 @@ int key_instantiate_and_link(struct key *key,
EXPORT_SYMBOL(key_instantiate_and_link);
/**
- * key_negate_and_link - Negatively instantiate a key and link it into the keyring.
+ * key_reject_and_link - Negatively instantiate a key and link it into the keyring.
* @key: The key to instantiate.
* @timeout: The timeout on the negative key.
+ * @error: The error to return when the key is hit.
* @keyring: Keyring to create a link in on success (or NULL).
* @authkey: The authorisation token permitting instantiation.
*
* Negatively instantiate a key that's in the uninstantiated state and, if
- * successful, set its timeout and link it in to the destination keyring if one
- * is supplied. The key and any links to the key will be automatically garbage
- * collected after the timeout expires.
+ * successful, set its timeout and stored error and link it in to the
+ * destination keyring if one is supplied. The key and any links to the key
+ * will be automatically garbage collected after the timeout expires.
*
* Negative keys are used to rate limit repeated request_key() calls by causing
- * them to return -ENOKEY until the negative key expires.
+ * them to return the stored error code (typically ENOKEY) until the negative
+ * key expires.
*
* If successful, 0 is returned, the authorisation token is revoked and anyone
* waiting for the key is woken up. If the key was already instantiated,
* -EBUSY will be returned.
*/
-int key_negate_and_link(struct key *key,
+int key_reject_and_link(struct key *key,
unsigned timeout,
+ unsigned error,
struct key *keyring,
struct key *authkey)
{
@@ -556,6 +559,7 @@ int key_negate_and_link(struct key *key,
atomic_inc(&key->user->nikeys);
set_bit(KEY_FLAG_NEGATIVE, &key->flags);
set_bit(KEY_FLAG_INSTANTIATED, &key->flags);
+ key->type_data.reject_error = -error;
now = current_kernel_time();
key->expiry = now.tv_sec + timeout;
key_schedule_gc(key->expiry + key_gc_delay);
@@ -585,8 +589,7 @@ int key_negate_and_link(struct key *key,
return ret == 0 ? link_ret : ret;
}
-
-EXPORT_SYMBOL(key_negate_and_link);
+EXPORT_SYMBOL(key_reject_and_link);
/*
* Garbage collect keys in process context so that we don't have to disable
diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c
index 31a0fd8..0d7b194 100644
--- a/security/keys/keyctl.c
+++ b/security/keys/keyctl.c
@@ -1013,12 +1013,42 @@ error:
*/
long keyctl_negate_key(key_serial_t id, unsigned timeout, key_serial_t ringid)
{
+ return keyctl_reject_key(id, timeout, ENOKEY, ringid);
+}
+
+/*
+ * Negatively instantiate the key with the given timeout (in seconds) and error
+ * code and link the key into the destination keyring if one is given.
+ *
+ * The caller must have the appropriate instantiation permit set for this to
+ * work (see keyctl_assume_authority). No other permissions are required.
+ *
+ * The key and any links to the key will be automatically garbage collected
+ * after the timeout expires.
+ *
+ * Negative keys are used to rate limit repeated request_key() calls by causing
+ * them to return the specified error code until the negative key expires.
+ *
+ * If successful, 0 will be returned.
+ */
+long keyctl_reject_key(key_serial_t id, unsigned timeout, unsigned error,
+ key_serial_t ringid)
+{
const struct cred *cred = current_cred();
struct request_key_auth *rka;
struct key *instkey, *dest_keyring;
long ret;
- kenter("%d,%u,%d", id, timeout, ringid);
+ kenter("%d,%u,%u,%d", id, timeout, error, ringid);
+
+ /* must be a valid error code and mustn't be a kernel special */
+ if (error <= 0 ||
+ error >= MAX_ERRNO ||
+ error == ERESTARTSYS ||
+ error == ERESTARTNOINTR ||
+ error == ERESTARTNOHAND ||
+ error == ERESTART_RESTARTBLOCK)
+ return -EINVAL;
/* the appropriate instantiation authorisation key must have been
* assumed before calling this */
@@ -1038,7 +1068,7 @@ long keyctl_negate_key(key_serial_t id, unsigned timeout, key_serial_t ringid)
goto error;
/* instantiate the key and link it into a keyring */
- ret = key_negate_and_link(rka->target_key, timeout,
+ ret = key_reject_and_link(rka->target_key, timeout, error,
dest_keyring, instkey);
key_put(dest_keyring);
@@ -1492,6 +1522,12 @@ SYSCALL_DEFINE5(keyctl, int, option, unsigned long, arg2, unsigned long, arg3,
case KEYCTL_SESSION_TO_PARENT:
return keyctl_session_to_parent();
+ case KEYCTL_REJECT:
+ return keyctl_reject_key((key_serial_t) arg2,
+ (unsigned) arg3,
+ (unsigned) arg4,
+ (key_serial_t) arg5);
+
default:
return -EOPNOTSUPP;
}
diff --git a/security/keys/keyring.c b/security/keys/keyring.c
index 5620f08..cdd2f3f 100644
--- a/security/keys/keyring.c
+++ b/security/keys/keyring.c
@@ -352,7 +352,7 @@ key_ref_t keyring_search_aux(key_ref_t keyring_ref,
goto error_2;
if (key->expiry && now.tv_sec >= key->expiry)
goto error_2;
- key_ref = ERR_PTR(-ENOKEY);
+ key_ref = ERR_PTR(key->type_data.reject_error);
if (kflags & (1 << KEY_FLAG_NEGATIVE))
goto error_2;
goto found;
@@ -401,7 +401,7 @@ descend:
/* we set a different error code if we pass a negative key */
if (kflags & (1 << KEY_FLAG_NEGATIVE)) {
- err = -ENOKEY;
+ err = key->type_data.reject_error;
continue;
}
diff --git a/security/keys/request_key.c b/security/keys/request_key.c
index a3dc0d4..df3c041 100644
--- a/security/keys/request_key.c
+++ b/security/keys/request_key.c
@@ -585,7 +585,7 @@ int wait_for_key_construction(struct key *key, bool intr)
if (ret < 0)
return ret;
if (test_bit(KEY_FLAG_NEGATIVE, &key->flags))
- return -ENOKEY;
+ return key->type_data.reject_error;
return key_validate(key);
}
EXPORT_SYMBOL(wait_for_key_construction);
^ permalink raw reply related
* [PATCH 2/3] KEYS: Define an error code to indicate an authentication service was unavailable
From: David Howells @ 2011-03-03 17:49 UTC (permalink / raw)
To: linux-security-module, keyrings; +Cc: linux-kernel, netdev, David Howells
In-Reply-To: <20110303174934.21635.18601.stgit@warthog.procyon.org.uk>
As the kernel has or will have filesystems (and possibly other services) that
want to obtain authentication tokens and/or encryption data on demand (via
GSSAPI for example), it would seem useful to provide an additional error code
to indicate a problem with the lookup, rather than overloading some other error
code.
We already have EKEYREJECTED, EKEYREVOKED and EKEYEXPIRED to indicate problems
with a token that we already have, but what if the authentication server just
isn't available?
Define ENOAUTHSERVICE to indicate "Authentication service unavailable". This
can be used to indicate, for example, that an attempt was made by request_key()
to retrieve a key, but the authentication server (e.g. a KDC) it is supposed to
contact didn't answer or that it couldn't determine the location of a suitable
server.
One way this can be used is that the user of a network filesystem can get a TGT
from the KDC and stash it in their session keyring, then the filesystem can
attempt to automatically get a ticket for accessing the filesystem - but if the
server is uncontactable then the ticket can be negatively instantiated with
KEYCTL_REJECT, giving the error to be handed to future requests as
ENOAUTHSERVICE and a small timeout so that the key will expire from the cache
and allow a retry after a short while to prevent thrashing.
Signed-off-by: David Howells <dhowells@redhat.com>
---
arch/alpha/include/asm/errno.h | 1 +
arch/mips/include/asm/errno.h | 1 +
arch/parisc/include/asm/errno.h | 1 +
arch/sparc/include/asm/errno.h | 1 +
include/asm-generic/errno.h | 1 +
5 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/arch/alpha/include/asm/errno.h b/arch/alpha/include/asm/errno.h
index 98099bd..519663c 100644
--- a/arch/alpha/include/asm/errno.h
+++ b/arch/alpha/include/asm/errno.h
@@ -121,5 +121,6 @@
#define ENOTRECOVERABLE 137 /* State not recoverable */
#define ERFKILL 138 /* Operation not possible due to RF-kill */
+#define ENOAUTHSERVICE 139 /* Authentication service not available */
#endif
diff --git a/arch/mips/include/asm/errno.h b/arch/mips/include/asm/errno.h
index a0efc73..5fa7fed 100644
--- a/arch/mips/include/asm/errno.h
+++ b/arch/mips/include/asm/errno.h
@@ -120,6 +120,7 @@
#define ENOTRECOVERABLE 166 /* State not recoverable */
#define ERFKILL 167 /* Operation not possible due to RF-kill */
+#define ENOAUTHSERVICE 168 /* Authentication service not available */
#define EDQUOT 1133 /* Quota exceeded */
diff --git a/arch/parisc/include/asm/errno.h b/arch/parisc/include/asm/errno.h
index 9992abd..6a19346 100644
--- a/arch/parisc/include/asm/errno.h
+++ b/arch/parisc/include/asm/errno.h
@@ -121,5 +121,6 @@
#define ENOTRECOVERABLE 255 /* State not recoverable */
#define ERFKILL 256 /* Operation not possible due to RF-kill */
+#define ENOAUTHSERVICE 257 /* Authentication service not available */
#endif
diff --git a/arch/sparc/include/asm/errno.h b/arch/sparc/include/asm/errno.h
index 4e2bc49..ab3a26a 100644
--- a/arch/sparc/include/asm/errno.h
+++ b/arch/sparc/include/asm/errno.h
@@ -111,5 +111,6 @@
#define ENOTRECOVERABLE 133 /* State not recoverable */
#define ERFKILL 134 /* Operation not possible due to RF-kill */
+#define ENOAUTHSERVICE 135 /* Authentication service not available */
#endif
diff --git a/include/asm-generic/errno.h b/include/asm-generic/errno.h
index 28cc03b..bcf1908 100644
--- a/include/asm-generic/errno.h
+++ b/include/asm-generic/errno.h
@@ -107,5 +107,6 @@
#define ENOTRECOVERABLE 131 /* State not recoverable */
#define ERFKILL 132 /* Operation not possible due to RF-kill */
+#define ENOAUTHSERVICE 133 /* Authentication service not available */
#endif
^ permalink raw reply related
* [PATCH 3/3] KEYS: Define error codes to indicate a name service errors
From: David Howells @ 2011-03-03 17:49 UTC (permalink / raw)
To: linux-security-module, keyrings; +Cc: linux-kernel, netdev, David Howells
In-Reply-To: <20110303174934.21635.18601.stgit@warthog.procyon.org.uk>
Now that the kernel has filesystems (and possibly other services) that want to
look up internet addresses corresponding to arbitrary hostnames retrieved from
the server, it would seem useful to provide a couple of error codes to indicate
problems with the look up, rather than overloading some other error code.
Define ENONAMESERVICE to indicate "Network name service unavailable". This can
be used to indicate, for example, that an attempt was made by dns_query() to
make a query, but the name server (e.g. a DNS server) it is supposed to contact
didn't answer or that it couldn't determine the location of a suitable server.
Define ENAMEUNKNOWN to indicate "Network name unknown". This can be used to
indicate, for example, that an attempt was made by dns_query() to make a query,
but the name server (e.g. a DNS server) replied indicating that it had no
matching records.
The DNS query upcall program can report these to keyctl_reject() so that cached
failed queries will respond with these errors until they expire.
Signed-off-by: David Howells <dhowells@redhat.com>
---
arch/alpha/include/asm/errno.h | 2 ++
arch/mips/include/asm/errno.h | 2 ++
arch/parisc/include/asm/errno.h | 2 ++
arch/sparc/include/asm/errno.h | 2 ++
include/asm-generic/errno.h | 2 ++
5 files changed, 10 insertions(+), 0 deletions(-)
diff --git a/arch/alpha/include/asm/errno.h b/arch/alpha/include/asm/errno.h
index 519663c..ee8a3f0 100644
--- a/arch/alpha/include/asm/errno.h
+++ b/arch/alpha/include/asm/errno.h
@@ -122,5 +122,7 @@
#define ERFKILL 138 /* Operation not possible due to RF-kill */
#define ENOAUTHSERVICE 139 /* Authentication service not available */
+#define ENONAMESERVICE 140 /* Network name service unavailable */
+#define ENAMEUNKNOWN 141 /* Network name unknown */
#endif
diff --git a/arch/mips/include/asm/errno.h b/arch/mips/include/asm/errno.h
index 5fa7fed..faed965 100644
--- a/arch/mips/include/asm/errno.h
+++ b/arch/mips/include/asm/errno.h
@@ -121,6 +121,8 @@
#define ERFKILL 167 /* Operation not possible due to RF-kill */
#define ENOAUTHSERVICE 168 /* Authentication service not available */
+#define ENONAMESERVICE 169 /* Network name service unavailable */
+#define ENAMEUNKNOWN 170 /* Network name unknown */
#define EDQUOT 1133 /* Quota exceeded */
diff --git a/arch/parisc/include/asm/errno.h b/arch/parisc/include/asm/errno.h
index 6a19346..63ee0bc 100644
--- a/arch/parisc/include/asm/errno.h
+++ b/arch/parisc/include/asm/errno.h
@@ -122,5 +122,7 @@
#define ERFKILL 256 /* Operation not possible due to RF-kill */
#define ENOAUTHSERVICE 257 /* Authentication service not available */
+#define ENONAMESERVICE 258 /* Network name service unavailable */
+#define ENAMEUNKNOWN 259 /* Network name unknown */
#endif
diff --git a/arch/sparc/include/asm/errno.h b/arch/sparc/include/asm/errno.h
index ab3a26a..f40d5ae 100644
--- a/arch/sparc/include/asm/errno.h
+++ b/arch/sparc/include/asm/errno.h
@@ -112,5 +112,7 @@
#define ERFKILL 134 /* Operation not possible due to RF-kill */
#define ENOAUTHSERVICE 135 /* Authentication service not available */
+#define ENONAMESERVICE 136 /* Network name service unavailable */
+#define ENAMEUNKNOWN 137 /* Network name unknown */
#endif
diff --git a/include/asm-generic/errno.h b/include/asm-generic/errno.h
index bcf1908..25878f1 100644
--- a/include/asm-generic/errno.h
+++ b/include/asm-generic/errno.h
@@ -108,5 +108,7 @@
#define ERFKILL 132 /* Operation not possible due to RF-kill */
#define ENOAUTHSERVICE 133 /* Authentication service not available */
+#define ENONAMESERVICE 134 /* Network name service unavailable */
+#define ENAMEUNKNOWN 135 /* Network name unknown */
#endif
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox