[BUG?] the first bind() of an AF_PACKET socket to an interface is slow

[BUG?] the first bind() of an AF_PACKET socket to an interface is slow

[Tom Gundersen]

[-- Attachment #1: Type: text/plain, Size: 1353 bytes --]

Hi,

I'm observing some strange behavior using bind().

For any given process, and any given interface, the first time I bind
a socket to the interface it takes a very long time. Subsequent binds
to the same interface in the same process are instantaneous.

The time it takes seems independent of the interface, so also affects
the loopback interface, but binding to all interfaces (ifindex=0) is
instantaneous.

It is also peculiar to note that the time it takes to bind is
seemingly randomly chosen on my machine from 9ms, 19ms, 29ms, 39ms,
49ms and 59 ms, but I never observed any other values.

The attached test program illustrates the problem, binding to
ifindex=0 (all) and ifindex=1 (loopback).

Sample run:
# strace -f -T -ebind ./bind
bind(3, {sa_family=AF_PACKET, proto=0x800, if0, pkttype=0x40 /* ? */,
addr(6)={1328, ffffffffffff}, 20) = 0 <0.000036>
bind(3, {sa_family=AF_PACKET, proto=0x800, if0, pkttype=0x40 /* ? */,
addr(6)={1328, ffffffffffff}, 20) = 0 <0.000033>
bind(3, {sa_family=AF_PACKET, proto=0x800, if1, pkttype=0x40 /* ? */,
addr(6)={1328, ffffffffffff}, 20) = 0 <0.059706>
bind(3, {sa_family=AF_PACKET, proto=0x800, if1, pkttype=0x40 /* ? */,
addr(6)={1328, ffffffffffff}, 20) = 0 <0.000111>


Can anyone enlighten me to what is going on? Is this expected behavior
or a bug? Any suggestions for a work-around?

Cheers,

Tom

[-- Attachment #2: bind.c --]
[-- Type: text/x-csrc, Size: 664 bytes --]

#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <string.h>
#include <linux/if_packet.h>
#include <net/ethernet.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <arpa/inet.h>

int main() {
        struct sockaddr_ll ll;
        int s;
	int i;

	for (i = 0; i < 2; i++) {
		s = socket(AF_PACKET, SOCK_DGRAM, htons(ETH_P_IP));

	        ll.sll_family = AF_PACKET;
	        ll.sll_protocol = htons(ETH_P_IP);
	        ll.sll_ifindex = i;

	        bind(s, (const struct sockaddr *)&ll, sizeof(ll));
	        bind(s, (const struct sockaddr *)&ll, sizeof(ll));
		close(s);
	}

        return 0;
}

Re: [BUG?] the first bind() of an AF_PACKET socket to an interface is slow

[Daniel Borkmann]

On 03/31/2014 08:43 PM, Tom Gundersen wrote:
> Hi,
>
> I'm observing some strange behavior using bind().
>
> For any given process, and any given interface, the first time I bind
> a socket to the interface it takes a very long time. Subsequent binds
> to the same interface in the same process are instantaneous.
>
> The time it takes seems independent of the interface, so also affects
> the loopback interface, but binding to all interfaces (ifindex=0) is
> instantaneous.
>
> It is also peculiar to note that the time it takes to bind is
> seemingly randomly chosen on my machine from 9ms, 19ms, 29ms, 39ms,
> 49ms and 59 ms, but I never observed any other values.
>
> The attached test program illustrates the problem, binding to
> ifindex=0 (all) and ifindex=1 (loopback).
>
> Sample run:
> # strace -f -T -ebind ./bind
> bind(3, {sa_family=AF_PACKET, proto=0x800, if0, pkttype=0x40 /* ? */,
> addr(6)={1328, ffffffffffff}, 20) = 0 <0.000036>
> bind(3, {sa_family=AF_PACKET, proto=0x800, if0, pkttype=0x40 /* ? */,
> addr(6)={1328, ffffffffffff}, 20) = 0 <0.000033>
> bind(3, {sa_family=AF_PACKET, proto=0x800, if1, pkttype=0x40 /* ? */,
> addr(6)={1328, ffffffffffff}, 20) = 0 <0.059706>
> bind(3, {sa_family=AF_PACKET, proto=0x800, if1, pkttype=0x40 /* ? */,
> addr(6)={1328, ffffffffffff}, 20) = 0 <0.000111>
>
>
> Can anyone enlighten me to what is going on? Is this expected behavior
> or a bug? Any suggestions for a work-around?

Expected behaviour as you can save in some cases a synchronize_net() call:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=902fefb82ef72a50c78cb4a20cc954b037a98d1c

> Cheers,
>
> Tom
>

Re: [BUG?] the first bind() of an AF_PACKET socket to an interface is slow

[Tom Gundersen]

On Mon, Mar 31, 2014 at 10:16 PM, Daniel Borkmann <dborkman@redhat.com> wrote:
> On 03/31/2014 08:43 PM, Tom Gundersen wrote:
>>
>> Hi,
>>
>> I'm observing some strange behavior using bind().
>>
>> For any given process, and any given interface, the first time I bind
>> a socket to the interface it takes a very long time. Subsequent binds
>> to the same interface in the same process are instantaneous.
>>
>> The time it takes seems independent of the interface, so also affects
>> the loopback interface, but binding to all interfaces (ifindex=0) is
>> instantaneous.
>>
>> It is also peculiar to note that the time it takes to bind is
>> seemingly randomly chosen on my machine from 9ms, 19ms, 29ms, 39ms,
>> 49ms and 59 ms, but I never observed any other values.
>>
>> The attached test program illustrates the problem, binding to
>> ifindex=0 (all) and ifindex=1 (loopback).
>>
>> Sample run:
>> # strace -f -T -ebind ./bind
>> bind(3, {sa_family=AF_PACKET, proto=0x800, if0, pkttype=0x40 /* ? */,
>> addr(6)={1328, ffffffffffff}, 20) = 0 <0.000036>
>> bind(3, {sa_family=AF_PACKET, proto=0x800, if0, pkttype=0x40 /* ? */,
>> addr(6)={1328, ffffffffffff}, 20) = 0 <0.000033>
>> bind(3, {sa_family=AF_PACKET, proto=0x800, if1, pkttype=0x40 /* ? */,
>> addr(6)={1328, ffffffffffff}, 20) = 0 <0.059706>
>> bind(3, {sa_family=AF_PACKET, proto=0x800, if1, pkttype=0x40 /* ? */,
>> addr(6)={1328, ffffffffffff}, 20) = 0 <0.000111>
>>
>>
>> Can anyone enlighten me to what is going on? Is this expected behavior
>> or a bug? Any suggestions for a work-around?
>
>
> Expected behaviour as you can save in some cases a synchronize_net() call:
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=902fefb82ef72a50c78cb4a20cc954b037a98d1c

Great, that explains what I need to do. Thanks!

Cheers,

Tom