Re: [RFC PATCH v2 0/7] VRF support in FIB library

["Medvedkin, Vladimir" <vladimir.medvedkin@intel.com>]

Hi Stephen,

On 10/14/2026 11:55 PM, Stephen Hemminger wrote:
> On Wed, 14 Oct 2026 18:39:14 +0000
> Vladimir Medvedkin <vladimir.medvedkin@intel.com> wrote:
>
>> This series adds multi-VRF support to the IPv4 FIB, allowing a single FIB
>> instance to host multiple isolated routing tables.
>>
>> Today a FIB instance is a single routing table, so an application serving
>> several VRFs needs one FIB per VRF. In a burst oriented datapath packets of
>> one burst can belong to different VRFs, and the application either does
>> per-packet lookups in different FIBs or regroups the burst by VRF before
>> the lookup. Both are expensive. With this series all the VRFs live in one
>> FIB, the bulk lookup takes a VRF ID per address, and the existing API
>> operates on VRF 0.
>>
>> v1 replicated the 16M entry tbl24 per VRF. That kept the lookup on par with
>> the single VRF case, but memory grew linearly with the number of VRFs,
>> which was the main concern raised on v1. v2 resolves the issue by adding
>> of two dataplane algorithms with different memory/lookup trade-offs.
>>
>> DIR8-16-8 is a new three level variant of DIR-24-8 with 8, 16 and 8 bit
>> strides. Only the 256 entry root table (2 KB) is replicated per VRF; the
>> tbl16 and tbl8 groups come from pools shared by all the VRFs, so memory
>> follows the routes actually installed rather than the number of VRFs. The
>> cost is an extra memory access for prefixes longer than /8.
>>
>> DIR-24-8 gets multi-VRF support by resolving the VRF after the longest
>> prefix match, along the lines Konstantin suggested on v1. Instead of a
>> vector of next hops per prefix, a single tbl24_8 shared by all the VRFs
>> returns an internal prefix ID, and a hash table keyed by {VRF, prefix ID}
>> returns the next hop, a miss meaning the VRF default next hop. The aim is
>> to resolve the VRF with a single memory access, so buckets hold 8 records
>> and there is no probing beyond the bucket.
>
> The problem with existing FIB lookups is they are space intensive.
comparing to what?
> They tradeoff memory for cache misses. Every level of indirection adds
> another cache miss.
This is not always a case. Also that's why we are using prefetching.
>
> Not sure they work well for VRF case at all.
I suspect this objection is applied for v1? v2 should resolve the issue
>   Normal LPM is more compact.
  more compact compared to what? Under "Normal LPM" I assume rte_lpm 
library, if so, then comparing to existing FIB, which uses DIR24_8 with 
4 byte nexthops (apples to apples), the memory footprint is exactly the 
same.
> For normal mortal size machines having multiple VRF's is going to be huge.
> How big are you envisioning this?

 From my experience when I was working as a network engineer in network 
operators, an average PE router could have 2 big VRFs (about 800k 
prefixes for v4) and thousands small VRFs with about hundreds prefixes each.

With this input an average size of a dataplane struct with:

DIR8_16_8 could be around 2Kb*1000 (VRFs) + 2*256*64k*8b(TBL16 for 2 
VRFs with full view) + 1000*4*64k*8b(small VRFs needs around 4 TBL16 - 3 
for RFC1918 + 1 customer public net) + 1000*100(prefixes more specific 
than /24, off the top of my head)*256*8b. This corresponds to TBL0: 2Mb, 
TBL16: ~2.5Gb, TBL8:~200Mb, so under 3Gb of RAM.

DIR24_8: single DIR24_8 with joint prefixes for all VRFs and 4b entries 
- 64mb for TBL24 + a few Mb for TBL8 (since all prefixes are joint) + 
hash table of size from 8Mb to 1Gb.

This sounds like mortal machine RAM usage to me.

-- 
Regards,
Vladimir