...
...
RARE project objective is to provide a routing platform proposing various solutions addressing multiple use cases in the R&E landscape. In the picture below you see in purple the different use casecases:
As you can notice, each use case will run on different hardware that potentially can have different dataplanes. As we were starting from a clean slate environment without not that much choice, especially with P4 programmability - the first dataplane or P4 target considered was BMv2. BMv2 is an excellent way to learn P4, it is also the first target we use in order to program and validate new features. After 6 months of practising our "P4-fu" we developed:
...
Our initial work, considering FreeRouter's Java nature, was to wrote write a Java P4Runtime GRPC client that would be able to program the entries in the tables exposed by BMv2 via the P4Info file. However, this would have intimately tied FreeRouter code to P4Runtime GRPC gRPC code. Even if this is it's more natural to choose this solution, going into in that direction implied that dataplanes other dataplane than BMv2 would be compliant to P4Runtime. Which It turns out that this is not the case. We then opted for a simple message API via a bi-directional raw UNIX socket. We will see what 's this means later in this blog.
Motivated by this the successful experience with BMv2, we then decided to move forward and started to study TOFINO as a target. We were greedy and eager to apply our P4 code against multi-terabits traffic. After a few P4 program compilations, our the first impression from my personal perspective was ... mind blowing ! INTEL/BAREFOOT TOFINO effectively opened to us the door to multi-terabits packet processing... Just the idea to be have at the tip of your finger the possibility to process traffic at these traffic levels was exciting !
As a side note, the journey was not without suffering and painspain... 
We had to port our BMv2 code - and to port to TOFINO and this was not "Une lettre à la poste"... It is not that TOFINO programming is gratuitously painful. It is just that it is p4c-tofino's job to make sure that our packets are processed at silicon lighting speed. Imagine you are asked to convey parcels by driving from Paris to Amsterdam with a car that has a truck with an infinite sizean infinitely sized trunk, with an infinite gaz gas tank and no particular speed constraint along the road. And then you are asked to to do the same trip, but with an actual real car that has a truck trunk with a fixed size and with a 50 litre gaz gas tank, and of course you'll have to follow speed signs along the road.
In the first case, you would put as many parcels as parcel you would want like and you even won't bother looking at your gaz gas tank level and maybe you'd set the speed to 200 Km/h. The second case forces you to carefully think about how many parcel parcels you can put in your trucktrunk, check to see if one completely full tank can be sufficient for the trip and of course, you would have to follow the speed signs.
...
But, this pain was not in vain, it was for the greater good... You can't imagine this the inherent joy when you see the TOFINO compiler displaying the DONE word ! For the veteran veterans who can remember, it is the same feeling when you manage to compile your first program in the ADA language. The compiler is not so strict that compiling an ADA program is in itself a feat. No wonder why this language is used in Spatial rocket (Ariane).
Back to our dataplane interface story, even TOFINO and BMv2 share some roots, while BMv2 had P4Runtime as a northnound interface, INTEL/BAREFOOT pushed into TOFINO platform with P4_16 their GRPC gRPC interface counterpart: BfRuntime.
Our best bet , paid off as FreeRouter message API was unchanged and without not very hard much effort we could add a new dataplane "wingman" to the FreeRouter control plane.
As a To recap:
- For BMv2: Our interface yield yields P4Runtime RPC calls. This program is called: forwarder.py
- For TOFINO: Our interface yield yields BfRuntime RPC calls. This program is called witout too much originality: bf_forwader.py
...
But we still had nothing at the EDGE/AGGREGATION layer in term terms of a solution proposal, deploying P4 hardware might be way too expensive in specific context sich contexts such as small R&E institutions like primary schools or small R&E labs. To that purapose purpose, we started to study new targets such as VMWARE XDP and a very promising projetproject: T4P4S ELTE. While we could not use XDP without a lot of P4 code rewriting and compromise, T4P4S ELTE was from our perpective very promising. But due to a compilation issue, we could not move forward.
...
As a result, we were a little bit bitter and started to read the DPDK library. And we started to play with DPDK examples... These examples were tremendously useful as it sparked some DPDK development into the RARE team. Csaba, the FreeRouter lead developer, step by step came up with this GENIUS idea: why don't we just use emulate P4 RARE P4 dataplane program ? We can still revert to using T4P4S ELTE when it will be ready ?
...