Re: [RFC net-next 2/3] ipv4: storing and retrieval of per-nexthop encap

Robert Shearman
2015-06-02T16:36:05Z

On 02/06/15 17:01, roopa wrote:
> On 6/1/15, 9:46 AM, Robert Shearman wrote:
>> Parse RTA_ENCAP attribute for one path and multipath routes. The encap
>> length is stored in a newly added field to fib_nh, nh_encap_len,
>> although this is added to a padding hole in the structure so that it
>> doesn't increase the size at all. The encap data itself is stored at
>> the end of the array of nexthops. Whilst this means that retrieval
>> isn't optimal, especially if there are multiple nexthops, this avoids
>> the memory cost of an extra pointer, as well as any potential change
>> to the cache or instruction layout that could cause a performance
>> impact.
>>
>> Currently, the dst structure allocated to represent the destination of
>> the packet and used for retrieving the encap by the encap-type
>> interface has been grown through the addition of the rt_encap_len and
>> rt_encap fields. This isn't desirable and could be fixed by defining a
>> new destination type with operations copied from the normal case,
>> other than the addition of the get_encap operation.
>>
>> Signed-off-by: Robert Shearman <rshearma@brocade.com>
...
>> @@ -434,6 +445,83 @@ static int fib_detect_death(struct fib_info *fi,
>> int order,
>> return 1;
>> }
>> +static int fib_total_encap(struct fib_config *cfg)
>> +{
>> + struct net *net = cfg->fc_nlinfo.nl_net;
>> + int total_encap_len = 0;
>> +
>> + if (cfg->fc_mp) {
>> + int remaining = cfg->fc_mp_len;
>> + struct rtnexthop *rtnh = cfg->fc_mp;
>> +
>> + while (rtnh_ok(rtnh, remaining)) {
>> + struct nlattr *nla, *attrs = rtnh_attrs(rtnh);
>> + int attrlen;
>> +
>> + attrlen = rtnh_attrlen(rtnh);
>> + nla = nla_find(attrs, attrlen, RTA_ENCAP);
>> + if (nla) {
>> + struct net_device *dev;
>> + int len;
>> +
>> + dev = __dev_get_by_index(net,
>> + rtnh->rtnh_ifindex);
>> + if (!dev)
>> + return -EINVAL;
>> +
>> + /* Determine space required */
>> + len = rtnl_parse_encap(dev, nla, NULL);
>> + if (len < 0)
>> + return len;
>> +
>> + total_encap_len += len;
>> + }
>> +
>> + rtnh = rtnh_next(rtnh, &remaining);
>> + }
>> + } else {
>> + if (cfg->fc_encap) {
>> + struct net_device *dev;
>> + int len;
>> +
>> + dev = __dev_get_by_index(net, cfg->fc_oif);
>> + if (!dev)
>> + return -EINVAL;
>> +
>> + /* Determine space required */
>> + len = rtnl_parse_encap(dev, cfg->fc_encap, NULL);
>> + if (len < 0)
>> + return len;
>> +
>> + total_encap_len += len;
>> + }
>> + }
>> +
>> + return total_encap_len;
>> +}
> we could avoid parsing and finding this device twice, if fib_nh just
> held a pointer to the encap_info
> (or tunnel info) ?. And the encap_info/tun_info could be refcounted and
> shared between
> nexthops ?. In my implementation i have just a pointer to parsed encap
> state
> in fib_nh

Right - I took the approach here to avoid any extra memory use if encap
isn't in use for the nexthop/route, and to avoid any potential
performance penalty caused by extra cache misses. It would certainly
make things simpler if those weren't concerns. I'd appreciate input from
the community on this.

>> @@ -1403,6 +1418,15 @@ static void rt_set_nexthop(struct rtable *rt,
>> __be32 daddr,
>> #ifdef CONFIG_IP_ROUTE_CLASSID
>> rt->dst.tclassid = nh->nh_tclassid;
>> #endif
>> +
>> + nh_encap = fib_get_nh_encap(fi, nh);
>> + if (unlikely(nh_encap)) {
>> + rt->rt_encap = kmemdup(nh_encap, nh->nh_encap_len,
>> + GFP_KERNEL);
>> + if (rt->rt_encap)
>> + rt->rt_encap_len = nh->nh_encap_len;
>> + }
>> +
>
> And..., you could make the rtable point to the same encap info.

Ack.

Thanks,
Rob