Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[David Miller]

From: Gil Beniamini <gil.beniamini@gmail.com>
Date: Mon, 8 Jun 2009 21:05:25 +0300

> Dear Linux Experts,
> 
> My application is using PF_PACKET and SOCK_RAW in order to receive all
> Ethernet packets for the machine-unicast MAC plus all broadcast
> packets (like ARP broadcast).

Please post networking questions and reports to netdev@vger.kernel.org
which is where the networking experts are subscribed.

I am also fully aware of your problem as Linus Torvalds forwarded
your report to me last week, you just need to be patient as I simply
haven't had a chance to look into it yet.

> That worked OK with kernel 2.6.20 (Ubuntu 7.04 with or without VLAN).
> 
> Now with kernel 2.6.28 (Ubuntu 9.04) it works OK only without-VLAN,
> but with VLAN the application receive only packets for the
> machine-unicast MAC, but no broadcast (like ARP broadcast) packets,
> only if I set the socket to PROMISC mode, I start receive also
> broadcast packets.
> 
> To me it seems a bug, as SOCK_RAW should pass all ‘received’ packet
> (including broadcast) to the application, without the need for PROMISC
> (which cause my machine to receive other machine unicast  packets, and
> filter those by software)!
> 
> The original application:
> 
> sock=socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
> memset((char *) &sockad, 0, sizeof(sockad));
> sockad.sll_family = PF_PACKET;
> sockad.sll_protocol = htons(ETH_P_ALL);
> sockad.sll_ifindex = ethreq.ifr_ifindex;//the relevant unit number
> 
> 
> 
> Now the workaround is to set (unwanted) PROMISC mode:
> 
> ioctl(sock,SIOCGIFFLAGS,&ethreq);
> ethreq.ifr_flags|=IFF_PROMISC; // why does 9.04 need this (+0x100) at
> least once?!
> ioctl(sock,SIOCSIFFLAGS,&ethreq);
> 
> 
> Many thanks in advance,
> Gil Beniamini
> gil.beniamini@gmail.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Patrick McHardy]

David Miller wrote:
> From: Gil Beniamini <gil.beniamini@gmail.com>
> Date: Mon, 8 Jun 2009 21:05:25 +0300
> 
>> My application is using PF_PACKET and SOCK_RAW in order to receive all
>> Ethernet packets for the machine-unicast MAC plus all broadcast
>> packets (like ARP broadcast).
> 
>> That worked OK with kernel 2.6.20 (Ubuntu 7.04 with or without VLAN).
>>
>> Now with kernel 2.6.28 (Ubuntu 9.04) it works OK only without-VLAN,
>> but with VLAN the application receive only packets for the
>> machine-unicast MAC, but no broadcast (like ARP broadcast) packets,
>> only if I set the socket to PROMISC mode, I start receive also
>> broadcast packets.

Which driver are you using? Are you sniffing on the VLAN device
or on the underlying device?

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Gil Beniamini]

9Jun09, The problem was seen with eth1 using driver e1000e, but also
was reproduced on 3c59x, to me it seems that the problem is not
related to a specific nic driver.
VLAN is running for other NICs, but on the application specific nic
(eth1) I use PF_PACKET (SOCK_RAW) , in order to get  the raw-packet
from the underlying device, and this start working again (like in
older kernel) as soon as the application set the socket to PROMISC
mode.
Thanks Gil

On Tue, Jun 9, 2009 at 1:59 PM, Patrick McHardy<kaber@trash.net> wrote:
> David Miller wrote:
>>
>> From: Gil Beniamini <gil.beniamini@gmail.com>
>> Date: Mon, 8 Jun 2009 21:05:25 +0300
>>
>>> My application is using PF_PACKET and SOCK_RAW in order to receive all
>>> Ethernet packets for the machine-unicast MAC plus all broadcast
>>> packets (like ARP broadcast).
>>
>>> That worked OK with kernel 2.6.20 (Ubuntu 7.04 with or without VLAN).
>>>
>>> Now with kernel 2.6.28 (Ubuntu 9.04) it works OK only without-VLAN,
>>> but with VLAN the application receive only packets for the
>>> machine-unicast MAC, but no broadcast (like ARP broadcast) packets,
>>> only if I set the socket to PROMISC mode, I start receive also
>>> broadcast packets.
>
> Which driver are you using? Are you sniffing on the VLAN device
> or on the underlying device?
>
>

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Patrick McHardy]

Gil Beniamini wrote:
> 9Jun09, The problem was seen with eth1 using driver e1000e, but also
> was reproduced on 3c59x, to me it seems that the problem is not
> related to a specific nic driver.
> VLAN is running for other NICs, but on the application specific nic
> (eth1) I use PF_PACKET (SOCK_RAW) , in order to get  the raw-packet
> from the underlying device, and this start working again (like in
> older kernel) as soon as the application set the socket to PROMISC
> mode.

OK, that might explain it. e1000e performs VLAN filtering in hardware
and unless there are VLANs configured locally, all VLANs are filtered
out. Are you seeing seeing only unicast VLAN traffic or no VLAN traffic
at all? Please post a dump of the packets with and without promiscous
mode.

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Gil Beniamini]

Patrick,
On the specific NIC (eth1) no vlan is defined, and in the new Linux
the application receive NOTHING at all. In order to debug, I start
"wireshark as root" and "wireshark" set PROMISC on, and the
application start receive the raw packets as expected. Later I started
setting PROMISC by my application, and it can receive the packets even
when "wireshark" is not running.
The problem that I have with PROMISC mode, is that I need to do the
irelevant  unicast filtering in software rather than get it from the
hardware (as it works in old kernel 2.6.20).
Thanks a lot, Gil

On Tue, Jun 9, 2009 at 2:40 PM, Patrick McHardy<kaber@trash.net> wrote:
> Gil Beniamini wrote:
>>
>> 9Jun09, The problem was seen with eth1 using driver e1000e, but also
>> was reproduced on 3c59x, to me it seems that the problem is not
>> related to a specific nic driver.
>> VLAN is running for other NICs, but on the application specific nic
>> (eth1) I use PF_PACKET (SOCK_RAW) , in order to get  the raw-packet
>> from the underlying device, and this start working again (like in
>> older kernel) as soon as the application set the socket to PROMISC
>> mode.
>
> OK, that might explain it. e1000e performs VLAN filtering in hardware
> and unless there are VLANs configured locally, all VLANs are filtered
> out. Are you seeing seeing only unicast VLAN traffic or no VLAN traffic
> at all? Please post a dump of the packets with and without promiscous
> mode.
>
>

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Patrick McHardy]

Please don't top-post.

Gil Beniamini wrote:
> On the specific NIC (eth1) no vlan is defined, and in the new Linux
> the application receive NOTHING at all. In order to debug, I start
> "wireshark as root" and "wireshark" set PROMISC on, and the
> application start receive the raw packets as expected. Later I started
> setting PROMISC by my application, and it can receive the packets even
> when "wireshark" is not running.
> The problem that I have with PROMISC mode, is that I need to do the
> irelevant  unicast filtering in software rather than get it from the
> hardware (as it works in old kernel 2.6.20).

in older kernels the hardware filtering was inconsistent. Some drivers
behaved as today, meaning they filtered out all VLANs unless locally
configured, some filtered nothing until the first VLAN was added,
at which point they would filter all other VLANs (and maybe enable
header stripping).

The behaviour you describe is intended, only locally configured VLANs
are received unless you switch to promiscous mode.

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Brent Cook]

On Tuesday 09 June 2009 08:08:01 am Gil Beniamini wrote:
> Patrick,
> On the specific NIC (eth1) no vlan is defined, and in the new Linux
> the application receive NOTHING at all. In order to debug, I start
> "wireshark as root" and "wireshark" set PROMISC on, and the
> application start receive the raw packets as expected. Later I started
> setting PROMISC by my application, and it can receive the packets even
> when "wireshark" is not running.
> The problem that I have with PROMISC mode, is that I need to do the
> irelevant  unicast filtering in software rather than get it from the
> hardware (as it works in old kernel 2.6.20).
> Thanks a lot, Gil

One solution is to attach a BPF filter to the socket. Then you will only 
receive packets for whatever you are filtering. This is still in software, but 
it is higher up in the abstraction, and undoubtedly more efficient than doing 
it at the app level.

Simply compile the BPF program you wish to filter, e.g:

# tcpdump -dd vlan 4
{ 0x28, 0, 0, 0x0000000c },
{ 0x15, 0, 4, 0x00008100 },
{ 0x28, 0, 0, 0x0000000e },
{ 0x54, 0, 0, 0x00000fff },
{ 0x15, 0, 1, 0x00000004 },
{ 0x6, 0, 0, 0x00000060 },
{ 0x6, 0, 0, 0x00000000 },

Then attach it to your socket:

    struct sock_filter filter[] = {
      { 0x28, 0, 0, 0x0000000c },
      { 0x15, 0, 4, 0x00008100 },
      { 0x28, 0, 0, 0x0000000e },
      { 0x54, 0, 0, 0x00000fff },
      { 0x15, 0, 1, 0x00000004 },
      { 0x6, 0, 0, 0x00000060 },
      { 0x6, 0, 0, 0x00000000 },
    };

    struct sock_fprog inbound_filter = {
        .len = 7, .filter = filter
    };

    if (setsockopt(fd, SOL_SOCKET, SO_ATTACH_FILTER,
        &inbound_filter, sizeof(inbound_filter)) < 0) {
        return -1;
    }

Also, note, you can easily set promiscuous mode directly with socket opts too. 
If you're more curious, just look at libpcap source.

 - Brent

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Gil Beniamini]

Patrick,
My application is a "proxy" and need to support few VLANs (for example vlan 24).
I define all vlans on interface 'eth0' which is my "out i/f", while
the other interface 'eth1' (used as "in i/f") I use with PF_PACKET /
SOCK_RAW in order to handle VLANs by my application (starting with
"arp broadcast").
When I try to define the same (relevant VLAN) on eth1 (inorder to
check your suggestion), I get "Error: trying to add VLAN #24 to IF
-:eth1:- error: file exist".
It seems that linux does not allow to define the same VLAN # on more
than a single i/f?!
Thanks Gil

Brent,
Thanks I am aware of BPF and if I don't find a better solution, this
will be my filter.
Thanks Gil

On Tue, Jun 9, 2009 at 10:30 PM, Brent Cook<bcook@bpointsys.com> wrote:
> On Tuesday 09 June 2009 08:08:01 am Gil Beniamini wrote:
>> Patrick,
>> On the specific NIC (eth1) no vlan is defined, and in the new Linux
>> the application receive NOTHING at all. In order to debug, I start
>> "wireshark as root" and "wireshark" set PROMISC on, and the
>> application start receive the raw packets as expected. Later I started
>> setting PROMISC by my application, and it can receive the packets even
>> when "wireshark" is not running.
>> The problem that I have with PROMISC mode, is that I need to do the
>> irelevant  unicast filtering in software rather than get it from the
>> hardware (as it works in old kernel 2.6.20).
>> Thanks a lot, Gil
>
> One solution is to attach a BPF filter to the socket. Then you will only
> receive packets for whatever you are filtering. This is still in software, but
> it is higher up in the abstraction, and undoubtedly more efficient than doing
> it at the app level.
>
> Simply compile the BPF program you wish to filter, e.g:
>
> # tcpdump -dd vlan 4
> { 0x28, 0, 0, 0x0000000c },
> { 0x15, 0, 4, 0x00008100 },
> { 0x28, 0, 0, 0x0000000e },
> { 0x54, 0, 0, 0x00000fff },
> { 0x15, 0, 1, 0x00000004 },
> { 0x6, 0, 0, 0x00000060 },
> { 0x6, 0, 0, 0x00000000 },
>
> Then attach it to your socket:
>
>    struct sock_filter filter[] = {
>      { 0x28, 0, 0, 0x0000000c },
>      { 0x15, 0, 4, 0x00008100 },
>      { 0x28, 0, 0, 0x0000000e },
>      { 0x54, 0, 0, 0x00000fff },
>      { 0x15, 0, 1, 0x00000004 },
>      { 0x6, 0, 0, 0x00000060 },
>      { 0x6, 0, 0, 0x00000000 },
>    };
>
>    struct sock_fprog inbound_filter = {
>        .len = 7, .filter = filter
>    };
>
>    if (setsockopt(fd, SOL_SOCKET, SO_ATTACH_FILTER,
>        &inbound_filter, sizeof(inbound_filter)) < 0) {
>        return -1;
>    }
>
> Also, note, you can easily set promiscuous mode directly with socket opts too.
> If you're more curious, just look at libpcap source.
>
>  - Brent
>

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Patrick McHardy]

Again, please don't top post.

Gil Beniamini wrote:
> My application is a "proxy" and need to support few VLANs (for example vlan 24).
> I define all vlans on interface 'eth0' which is my "out i/f", while
> the other interface 'eth1' (used as "in i/f") I use with PF_PACKET /
> SOCK_RAW in order to handle VLANs by my application (starting with
> "arp broadcast").
> When I try to define the same (relevant VLAN) on eth1 (inorder to
> check your suggestion), I get "Error: trying to add VLAN #24 to IF
> -:eth1:- error: file exist".
> It seems that linux does not allow to define the same VLAN # on more
> than a single i/f?!

No, that has got to be a mistake. You can use the same ID once
per underlying device:

# ip -d link list

4: vlan0@dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN
     link/ether d6:1f:c0:bd:b2:ec brd ff:ff:ff:ff:ff:ff
     vlan id 1000 <REORDER_HDR>
6: vlan1@dummy1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN
     link/ether 8a:17:13:5d:87:1a brd ff:ff:ff:ff:ff:ff
     vlan id 1000 <REORDER_HDR>

If it doesn't work for you, please post the full commands
and ip -d link list output you used.

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Gil Beniamini]

Dear Patrick,
Below is my try to add vlan-id 24 on eth1, followed by "ip -d link list":
root@StudentsHPA:/home/hpa# vconfig add eth1 24
ERROR: trying to add VLAN #24 to IF -:eth1:-  error: File exists

root@StudentsHPA:/home/hpa# ip -d link list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UP qlen 100
    link/ether 00:15:17:1e:27:fa brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,NOARP,PROMISC,UP,LOWER_UP> mtu 1500
qdisc pfifo_fast state UP qlen 100
    link/ether 00:15:17:1e:e8:46 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UP qlen 100
    link/ether 00:15:17:1e:e8:47 brd ff:ff:ff:ff:ff:ff
5: vlan24@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP
    link/ether 00:15:17:1e:27:fa brd ff:ff:ff:ff:ff:ff
    vlan id 24 <REORDER_HDR>
6: vlan31@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP
    link/ether 00:15:17:1e:27:fa brd ff:ff:ff:ff:ff:ff
    vlan id 31 <REORDER_HDR>
7: vlan32@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP
    link/ether 00:15:17:1e:27:fa brd ff:ff:ff:ff:ff:ff
    vlan id 32 <REORDER_HDR>
8: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether 42:fe:3a:c7:dd:96 brd ff:ff:ff:ff:ff:ff
Thanks Gil


On Wed, Jun 10, 2009 at 5:32 PM, Patrick McHardy<kaber@trash.net> wrote:
> Again, please don't top post.
>
> Gil Beniamini wrote:
>>
>> My application is a "proxy" and need to support few VLANs (for example
>> vlan 24).
>> I define all vlans on interface 'eth0' which is my "out i/f", while
>> the other interface 'eth1' (used as "in i/f") I use with PF_PACKET /
>> SOCK_RAW in order to handle VLANs by my application (starting with
>> "arp broadcast").
>> When I try to define the same (relevant VLAN) on eth1 (inorder to
>> check your suggestion), I get "Error: trying to add VLAN #24 to IF
>> -:eth1:- error: file exist".
>> It seems that linux does not allow to define the same VLAN # on more
>> than a single i/f?!
>
> No, that has got to be a mistake. You can use the same ID once
> per underlying device:
>
> # ip -d link list
>
> 4: vlan0@dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN
>    link/ether d6:1f:c0:bd:b2:ec brd ff:ff:ff:ff:ff:ff
>    vlan id 1000 <REORDER_HDR>
> 6: vlan1@dummy1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN
>    link/ether 8a:17:13:5d:87:1a brd ff:ff:ff:ff:ff:ff
>    vlan id 1000 <REORDER_HDR>
>
> If it doesn't work for you, please post the full commands
> and ip -d link list output you used.
>

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Patrick McHardy]

Gil Beniamini wrote:
> Dear Patrick,
> Below is my try to add vlan-id 24 on eth1, followed by "ip -d link list":
> root@StudentsHPA:/home/hpa# vconfig add eth1 24
> ERROR: trying to add VLAN #24 to IF -:eth1:-  error: File exists
> 
> 5: vlan24@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP

You're apparently using the "VLAN_NAME_TYPE_PLUS_VID_NO_PAD" naming
policy, which causes name clashes when you create two VLANs with the
same ID.

I'd suggest to use iproute to create the devices, which allows you to
specify the desired name manually. Alternatively you can use vconfig
and a different naming policy.

Re: SOCK_RAW does not receive broadcast (with VLAN unless PROMISC)

[Gil Beniamini]

Paul, Patrick,

THANKS!
using "vconfig set_name_type" DEV_PLUS_VID_NO_PAD" enable successful
"vconfig add eth1 <same-id as on eth0>". And now finally without
PROMISC the application receive BROADCASTs on this VLAN.

Many Thanks, Gil

2009/6/10 Paul Rolland <rol@witbe.net>:
> Hello,
>
> On Wed, 10 Jun 2009 17:48:40 +0300
> Gil Beniamini <gil.beniamini@gmail.com> wrote:
>
>> Dear Patrick,
>> Below is my try to add vlan-id 24 on eth1, followed by "ip -d link list":
>> root@StudentsHPA:/home/hpa# vconfig add eth1 24
>> ERROR: trying to add VLAN #24 to IF -:eth1:-  error: File exists
>
> Here is an output from my vconfig :
> bash-2.05$ /usr/local/witbe/sbin/vconfig
> Expecting argc to be 3-5, inclusive.  Was: 1
>
> Usage: add             [interface-name] [vlan_id]
>       rem             [vlan-name]
>       set_flag        [interface-name] [flag-num]       [0 | 1]
>       set_egress_map  [vlan-name]      [skb_priority]   [vlan_qos]
>       set_ingress_map [vlan-name]      [skb_priority]   [vlan_qos]
>       set_name_type   [name-type]
>
> ...
> * bind-type:  PER_DEVICE  # Allows vlan 5 on eth0 and eth1 to be unique.
>              PER_KERNEL  # Forces vlan 5 to be unique across all devices.
>
> But the documentation is lacking information on how to set this :(
>
> Have a look at the source of vconfig should help...
>
> Paul
>
> --
> Paul Rolland                                E-Mail : rol(at)witbe.net
> CTO - Witbe.net SA                          Tel. +33 (0)1 47 67 77 77
> Les Collines de l'Arche                     Fax. +33 (0)1 47 67 77 99
> F-92057 Paris La Defense                    RIPE : PR12-RIPE
>
> This is dedicated to all the ones who want to control Internet, its
> content or its usage :
>
> "I worry about my child and the Internet all the time, even though she's
> too young to have logged on yet. Here's what I worry about. I worry that 10
> or 15 years from now, she will come to me and say 'Daddy, where were you
> when they took freedom of the press away from the Internet?'"
> --Mike Godwin, Electronic Frontier Foundation
>
>
>