Re: Putting Into Account Packet End (ctx->data_end)

[Jesper Dangaard Brouer <brouer@redhat.com>]

On Wed, 03 Mar 2021 11:51:47 +0100
Toke Høiland-Jørgensen <toke@redhat.com> wrote:

> Srivats P <pstavirs@gmail.com> writes:
> 
> > On Mon, Mar 1, 2021 at 9:40 PM Christian Deacon <gamemann@gflclan.com> wrote:  
> >>
> >> Hey everyone,
> >>
> >> I wasn't sure if this belonged on the BPF mailing list or XDP Newbies.
> >> However, I figured I'd send it to the XDP Newbies list first since the
> >> project I'm making involves XDP.
> >>
> >> In my project, I'm trying to create a pointer that puts in account the
> >> ctx->data_end pointer. The new pointer is an unsigned 32-bit integer
> >> that is suppose to represent an IPv4 address. Here's an example of the code.
> >>
> >> ```
> >> void *data_end = (void *)(long)ctx->data_end;
> >>
> >> //uint32_t *icmpdata = data_end - sizeof(uint32_t);
> >> uint32_t *icmpdata = data_end;
> >> icmpdata -= sizeof(uint32_t);
> >>
> >> if (icmpdata + sizeof(uint32_t) > (uint32_t *)data_end)
> >> {
> >>      return XDP_DROP;
> >> }
> >> ```
> >>
> >> I'm trying to replace the last four bytes of the packet with this IPv4
> >> address. When I do this, I receive the following BPF verifier error when
> >> running the XDP program.
> >>
> >> ```
> >> R7 invalid mem access 'pkt_end'
> >> processed 909 insns (limit 100000000) max_states_per_insn 3 total_states
> >> 30 peak_states 30 mark_read 25
> >> ```
> >>
> >> To my understanding, this is due to accessing the packet end (data_end).
> >> However, I'm curious why this is prohibited if we're trying to go back
> >> four bytes into memory.
> >>
> >> I've also tried calculating the length of the packet and using ctx->data
> >> like the following.
> >>
> >> ```
> >> void *data = (void *)(long)ctx->data;
> >>
> >> unsigned int len = (ctx->data_end - ctx->data);
> >>
> >> uint32_t *icmpdata = data + len;
> >> icmpdata -= sizeof(uint32_t);
> >>
> >> if (icmpdata + sizeof(uint32_t) > (uint32_t *)data_end)
> >> {
> >>      return XDP_DROP;
> >> }
> >> ```
> >>
> >> However, this states the offset is outside of the packet.
> >>
> >> ```
> >> invalid access to packet, off=-16 size=4, R2(id=56,off=-16,r=0)
> >> R2 offset is outside of the packet
> >> processed 931 insns (limit 100000000) max_states_per_insn 3 total_states
> >> 29 peak_states 29 mark_read 24
> >> ```
> >>
> >> I'm sure there is something I'm doing wrong with the check. With that
> >> said, I believe I found the verifier check it's running into below.
> >>
> >> https://github.com/torvalds/linux/blob/master/kernel/bpf/verifier.c#L2882
> >>
> >> It looks like the `mem_size` argument is 0 and offset is below 0 which
> >> is causing it to fail. I'm not sure why, but I'd assume it's because the
> >> verifier believes `len` could be negative. Though, I tried adding checks
> >> for `len` and ran into the same issue.
> >>
> >> The XDP project I'm working on is a basic layer 3/4 forwarding program
> >> that does source port mapping when forwarding the packets. I have it
> >> working for TCP/UDP packets. However, for ICMP, I have nothing to keep
> >> track of within the headers. Therefore, I'm trying to add four bytes to
> >> the packet and appending the client's IPv4 address to the end of the
> >> packet before forwarding. When the packet comes back, I parse the last
> >> four bytes of the packet which is suppose to indicate the client IP
> >> address and remove the last four bytes of the packet. Below is the
> >> source code at the moment.
> >>
> >> https://github.com/gamemann/XDP-Forwarding/blob/master/src/xdp_prog.c#L181
> >>
> >> I hope this is enough information, but if isn't, please let me know. I
> >> also apologize if this is something silly I'm missing/not understanding.
> >>
> >> Thank you for your time!
> >>  
> >
> > See
> > https://lore.kernel.org/bpf/CANzUK5-g9wLiwUF88em4uVzMja_aR4xj9yzMS_ZObNKjvX6C6g@mail.gmail.com/  
> 
> Since this is a question that gets asked a lot: Would you care to submit
> this as an example to https://github.com/xdp-project/bpf-examples -
> we're trying to collect useful examples there, and I think this fits the
> bill.

Good idea!

Is this the solution to my slightly similar attempts here:
 https://github.com/xdp-project/xdp-tutorial/tree/master/experiment01-tailgrow

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer