Skip to content

Commit 76616a0

Browse files
ekoopsekoops
committed
Add pcn-k8sdispatcher service
Signed-off-by: Leonardo Di Giovanna <[email protected]>
1 parent bf98773 commit 76616a0

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+6600
-2
lines changed

‎AUTHORS

+1
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ off on commits in the Polycube repository:
1010
Gianluca Scopelliti [email protected]
1111
Giuseppe Ognibebe [email protected]
1212
Jianwen Pi [email protected]
13+
Leonardo Di Giovanna [email protected]
1314
Matteo Bertrone [email protected]
1415
Mauricio Vásquez Bernal [email protected]
1516
Nico Caprioli [email protected]

‎Documentation/services/pcn-k8sdispatcher/egress.png

52.8 KB
Loading

‎Documentation/services/pcn-k8sdispatcher/ingress.png

62.4 KB
Loading

‎Documentation/services/pcn-k8sdispatcher/k8sdispatcher.md

+40
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# K8dispatcher
2+
3+
The ``pcn-k8sdispatcher`` service is specifically designed as part of our Kubernetes networking solution (please see [polykube](https://github.com/polycube-network/polykube) to get more information about it). The service provides an eBPF implementation of a custom NAT: it performs different actions depending on the type and on the direction of the traffic.
4+
5+
For Egress Traffic, the following flow chart can be used to explain the functioning of the service:
6+
7+
![K8sdispatcher egress flow chart](egress.png)
8+
9+
The Egress Traffic is the traffic generated by Pods and directed to the external world.
10+
This traffic can be generated by an internal Pod that wants to contact the external world or as a response to an
11+
external world request. For this traffic, the service maintains an egress session table containing information
12+
about the active egress sessions. The first time a Pod wants to contact the external world, no active egress session
13+
will be present in the table: in this scenario, the service performs SNAT, replacing the address of the Pod
14+
with the address of the node, and creates entries in the ingress and egress session table accordingly.
15+
If the outgoing traffic is generated as a response to an external request, it can only be originated as a response to
16+
a request made to a NodePort Service. For traffic related to NodePort Services with a CLUSTER ExternalTrafficPolicy,
17+
if an egress session table hit happens, the destination IP address and port are replaced accordingly to the session data.
18+
The traffic related to NodePort Services with a LOCAL ExternalTrafficPolicy is forwarded as it is to the next cube.
19+
20+
For Ingress Traffic, the following flow chart can be used to explain the functioning of the service:
21+
22+
![K8sdispatcher ingress flow chart](ingress.png)
23+
24+
The Ingress Traffic can be differentiated in traffic directed to the host (either directly or because it needs VxLAN
25+
processing) and traffic directed to Pods. The traffic directed to Pods can be the traffic generated by an external host
26+
trying to contact a NodePort service or the return traffic generated by an external host providing a response to an
27+
internal Pod request. The service uses an ingress session table containing all the active ingress sessions.
28+
If a session table hit happens, the service apply NAT according to the session data. If no session table entry is
29+
associated with the incoming packet, the service tries to determine if a NodePort rule matches the packet
30+
characteristics. In case of no NodePort rule matching, the packet is sent to the Linux stack for further processing.
31+
In case of NodePort rule matching, different actions are applied according to the ExternalTrafficPolicy of the
32+
Kubernetes NodePort Service associated to the rule. If the policy is LOCAL, the traffic is allowed to reach only
33+
backend Pods located on the current node: in this case the packet can proceed towards the Pod without modifications.
34+
In case the policy is CLUSTER, the packet can also reach backend Pods located on other nodes: since later in
35+
the chain the packet will be processed by a load balancer and the return packet will have to transit through
36+
the same load balancer, SNAT is applied by replacing the source IP address with a specific reserved address belonging
37+
to the Pod CIDR of the node on which the k8sdispatcher is deployed. In this way the two nodes (the one that
38+
receives the request and the one running the selected backend Pod) will exchange the packets of the flow over
39+
the VxLAN interconnect. In this latter case, corresponding session entries are stored into the ingress and egress
40+
sessions tables.

‎scripts/install.sh

+4-2
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,8 @@ if [ "$MODE" == "pcn-iptables" ]; then
125125
-DENABLE_SERVICE_SIMPLEFORWARDER=OFF \
126126
-DENABLE_SERVICE_TRANSPARENTHELLOWORLD=OFF \
127127
-DENABLE_SERVICE_SYNFLOOD=OFF \
128-
-DENABLE_SERVICE_PACKETCAPTURE=OFF
128+
-DENABLE_SERVICE_PACKETCAPTURE=OFF \
129+
-DENABLE_SERVICE_K8SDISPATCHER=OFF
129130
elif [ "$MODE" == "pcn-k8s" ]; then
130131
cmake .. -DENABLE_SERVICE_BRIDGE=OFF \
131132
-DENABLE_SERVICE_DDOSMITIGATOR=ON \
@@ -143,7 +144,8 @@ elif [ "$MODE" == "pcn-k8s" ]; then
143144
-DENABLE_SERVICE_SIMPLEFORWARDER=OFF \
144145
-DENABLE_SERVICE_TRANSPARENTHELLOWORLD=OFF \
145146
-DENABLE_SERVICE_SYNFLOOD=OFF \
146-
-DENABLE_SERVICE_PACKETCAPTURE=ON
147+
-DENABLE_SERVICE_PACKETCAPTURE=ON \
148+
-DENABLE_SERVICE_K8SDISPATCHER=OFF
147149
else
148150
cmake .. -DENABLE_PCN_IPTABLES=ON
149151
fi

‎src/services/CMakeLists.txt

+1
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ add_service(transparenthelloworld pcn-transparent-helloworld)
3737
add_service(synflood pcn-synflood)
3838
add_service(packetcapture pcn-packetcapture)
3939
add_service(dynmon pcn-dynmon)
40+
add_service(k8sdispatcher pcn-k8sdispatcher)
4041

4142
# save string to create code that load the services
4243
SET_PROPERTY(GLOBAL PROPERTY LOAD_SERVICES_ ${LOAD_SERVICES})

‎src/services/pcn-k8sdispatcher/.swagger-codegen-ignore

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Swagger Codegen Ignore
2+
# Generated by swagger-codegen https://github.com/swagger-api/swagger-codegen
3+
4+
# Use this file to prevent files from being overwritten by the generator.
5+
6+
.swagger-codegen-ignore
7+
8+
src/*.cpp
9+
src/*.h
10+
11+
!src/*Interface.h
12+
!src/*JsonObject.h
13+
!src/*JsonObject.cpp

‎src/services/pcn-k8sdispatcher/CMakeLists.txt

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
cmake_minimum_required (VERSION 3.2)
2+
3+
set (CMAKE_CXX_STANDARD 11)
4+
5+
add_subdirectory(src)

‎src/services/pcn-k8sdispatcher/datamodel/k8sdispatcher.yang

+173
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
module k8sdispatcher {
2+
yang-version 1.1;
3+
namespace "http://polycube.network/k8sdispatcher";
4+
prefix "k8sdispatcher";
5+
6+
import polycube-base { prefix "polycube-base"; }
7+
import polycube-standard-base { prefix "polycube-standard-base"; }
8+
9+
import ietf-inet-types { prefix "inet"; }
10+
11+
organization "Polycube open source project";
12+
description "YANG data model for the Polycube K8s Dispatcher";
13+
14+
polycube-base:service-description "K8s Dispatcher Service";
15+
polycube-base:service-version "2.0.0";
16+
polycube-base:service-name "k8sdispatcher";
17+
polycube-base:service-min-kernel-version "4.14.0";
18+
19+
typedef l4-proto {
20+
type enumeration {
21+
enum "TCP" {
22+
value 6;
23+
description "The TCP protocol type";
24+
}
25+
enum "UDP" {
26+
value 17;
27+
description "The UDP protocol type";
28+
}
29+
enum "ICMP" {
30+
value 1;
31+
description "The ICMP protocol type";
32+
}
33+
}
34+
description "L4 protocol";
35+
}
36+
37+
uses "polycube-standard-base:standard-base-yang-module" {
38+
augment ports {
39+
leaf type {
40+
type enumeration {
41+
enum BACKEND { description "Port connected to the internal CNI topology"; }
42+
enum FRONTEND { description "Port connected to the node NIC"; }
43+
}
44+
description "Type of the K8s Dispatcher cube port (e.g. BACKEND or FRONTEND)";
45+
mandatory true;
46+
polycube-base:init-only-config;
47+
}
48+
leaf ip {
49+
type inet:ipv4-address;
50+
description "IP address of the node interface (only for FRONTEND port)";
51+
polycube-base:cli-example "10.10.1.1";
52+
polycube-base:init-only-config;
53+
}
54+
}
55+
}
56+
57+
leaf internal-src-ip {
58+
type inet:ipv4-address;
59+
description "Internal source IP address used for natting incoming packets directed to Kubernetes Services with a CLUSTER external traffic policy";
60+
mandatory true;
61+
polycube-base:cli-example "10.10.1.1";
62+
polycube-base:init-only-config;
63+
}
64+
65+
leaf nodeport-range {
66+
type string;
67+
description "Port range used for NodePort Services";
68+
default "30000-32767";
69+
polycube-base:cli-example "30000-32767";
70+
}
71+
72+
list session-rule {
73+
key "direction src-ip dst-ip src-port dst-port proto";
74+
description "Session entry related to a specific traffic direction";
75+
config false;
76+
77+
leaf direction {
78+
type enumeration {
79+
enum INGRESS {
80+
description "Direction of traffic going from the internal topology to the external world";
81+
}
82+
enum EGRESS {
83+
description "Direction of traffic going from the external world to the internal CNI topology";
84+
}
85+
}
86+
description "Session entry direction (e.g. INGRESS or EGRESS)";
87+
}
88+
leaf src-ip {
89+
type inet:ipv4-address;
90+
description "Session entry source IP address";
91+
}
92+
leaf dst-ip {
93+
type inet:ipv4-address;
94+
description "Session entry destination IP address";
95+
}
96+
leaf src-port {
97+
type inet:port-number;
98+
description "Session entry source L4 port number";
99+
}
100+
leaf dst-port {
101+
type inet:port-number;
102+
description "Session entry destination L4 port number";
103+
}
104+
leaf proto {
105+
type l4-proto;
106+
description "Session entry L4 protocol";
107+
polycube-base:cli-example "TCP, UDP, ICMP";
108+
}
109+
110+
leaf new-ip {
111+
type inet:ipv4-address;
112+
description "Translated IP address";
113+
config false;
114+
}
115+
leaf new-port {
116+
type inet:port-number;
117+
description "Translated L4 port number";
118+
config false;
119+
}
120+
leaf operation {
121+
type enumeration {
122+
enum XLATE_SRC { description "The source IP and port are replaced"; }
123+
enum XLATE_DST { description "The destination IP and port are replaced"; }
124+
}
125+
description "Operation applied on the original packet";
126+
config false;
127+
}
128+
leaf originating-rule {
129+
type enumeration {
130+
enum POD_TO_EXT {
131+
description "Traffic related to communication between a Pod and the external world";
132+
}
133+
enum NODEPORT_CLUSTER {
134+
description "Traffic related to communication involving a NodePort Service with having a CLUSTER external traffic policy";
135+
}
136+
}
137+
description "Rule originating the session entry";
138+
config false;
139+
}
140+
}
141+
142+
list nodeport-rule {
143+
key "nodeport-port proto";
144+
description "NodePort rule associated with a Kubernetes NodePort Service";
145+
146+
leaf nodeport-port {
147+
type inet:port-number;
148+
description "NodePort rule nodeport port number";
149+
polycube-base:cli-example "30500";
150+
}
151+
leaf proto {
152+
type l4-proto;
153+
description "NodePort rule L4 protocol";
154+
polycube-base:cli-example "TCP, UDP, ICMP";
155+
}
156+
157+
leaf external-traffic-policy {
158+
type enumeration {
159+
enum LOCAL { description "Incoming traffic is allowed to be served only by local backends"; }
160+
enum CLUSTER { description "Incoming traffic is allowed to be served by any backend of the cluster"; }
161+
}
162+
default CLUSTER;
163+
description "The external traffic policy of the Kubernetes NodePort Service";
164+
}
165+
leaf rule-name {
166+
type string;
167+
description "An optional name for the NodePort rule";
168+
polycube-base:cli-example "my-nodeport-rule";
169+
polycube-base:init-only-config;
170+
}
171+
}
172+
}
173+

‎src/services/pcn-k8sdispatcher/src/CMakeLists.txt

+50
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
include(${PROJECT_SOURCE_DIR}/cmake/LoadFileAsVariable.cmake)
2+
3+
aux_source_directory(serializer SERIALIZER_SOURCES)
4+
aux_source_directory(api API_SOURCES)
5+
aux_source_directory(base BASE_SOURCES)
6+
7+
include_directories(serializer)
8+
9+
if (NOT DEFINED POLYCUBE_STANDALONE_SERVICE OR POLYCUBE_STANDALONE_SERVICE)
10+
find_package(PkgConfig REQUIRED)
11+
pkg_check_modules(POLYCUBE libpolycube)
12+
include_directories(${POLYCUBE_INCLUDE_DIRS})
13+
endif (NOT DEFINED POLYCUBE_STANDALONE_SERVICE OR POLYCUBE_STANDALONE_SERVICE)
14+
15+
# Needed to load files as variables
16+
include_directories(${CMAKE_CURRENT_BINARY_DIR})
17+
18+
add_library(pcn-k8sdispatcher SHARED
19+
${SERIALIZER_SOURCES}
20+
${API_SOURCES}
21+
${BASE_SOURCES}
22+
K8sdispatcher.cpp
23+
NodeportRule.cpp
24+
Ports.cpp
25+
SessionRule.cpp
26+
K8sdispatcher-lib.cpp
27+
Utils.cpp)
28+
29+
# load ebpf datapath code a variable
30+
load_file_as_variable(pcn-k8sdispatcher
31+
K8sdispatcher_dp.c
32+
k8sdispatcher_code)
33+
34+
# load datamodel in a variable
35+
load_file_as_variable(pcn-k8sdispatcher
36+
../datamodel/k8sdispatcher.yang
37+
k8sdispatcher_datamodel)
38+
39+
target_link_libraries(pcn-k8sdispatcher ${POLYCUBE_LIBRARIES})
40+
41+
# Specify shared library install directory
42+
43+
set(CMAKE_INSTALL_LIBDIR /usr/lib)
44+
45+
install(
46+
TARGETS
47+
pcn-k8sdispatcher
48+
DESTINATION
49+
"${CMAKE_INSTALL_LIBDIR}"
50+
)

‎src/services/pcn-k8sdispatcher/src/HashTuple.h

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
#include <tuple>
2+
// function has to live in the std namespace
3+
// so that it is picked up by argument-dependent name lookup (ADL).
4+
namespace std {
5+
namespace {
6+
// Code from boost
7+
// Reciprocal of the golden ratio helps spread entropy
8+
// and handles duplicates.
9+
// See Mike Seymour in magic-numbers-in-boosthash-combine:
10+
// https://stackoverflow.com/questions/4948780
11+
12+
template <class T>
13+
inline void hash_combine(std::size_t &seed, T const &v) {
14+
seed ^= hash<T>()(v) + 0x9e3779b9 + (seed << 6) + (seed >> 2);
15+
}
16+
17+
// Recursive template code derived from Matthieu M.
18+
template <class Tuple, size_t Index = std::tuple_size<Tuple>::value - 1>
19+
struct HashValueImpl {
20+
static void apply(size_t &seed, Tuple const &tuple) {
21+
HashValueImpl<Tuple, Index - 1>::apply(seed, tuple);
22+
hash_combine(seed, get<Index>(tuple));
23+
}
24+
};
25+
26+
template <class Tuple>
27+
struct HashValueImpl<Tuple, 0> {
28+
static void apply(size_t &seed, Tuple const &tuple) {
29+
hash_combine(seed, get<0>(tuple));
30+
}
31+
};
32+
}
33+
34+
template <typename... TT>
35+
struct hash<std::tuple<TT...>> {
36+
size_t operator()(std::tuple<TT...> const &tt) const {
37+
size_t seed = 0;
38+
HashValueImpl<std::tuple<TT...>>::apply(seed, tt);
39+
return seed;
40+
}
41+
};
42+
}

‎src/services/pcn-k8sdispatcher/src/K8sdispatcher-lib.cpp

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
/**
2+
* k8sdispatcher API generated from k8sdispatcher.yang
3+
*
4+
* NOTE: This file is auto generated by polycube-codegen
5+
* https://github.com/polycube-network/polycube-codegen
6+
*/
7+
8+
9+
/* Do not edit this file manually */
10+
11+
#include "api/K8sdispatcherApiImpl.h"
12+
#include "../datamodel/k8sdispatcher.h" // generated from datamodel
13+
14+
#define SERVICE_PYANG_GIT ""
15+
#define SERVICE_SWAGGER_CODEGEN_GIT "GIT_REPO_ID"
16+
17+
#include <polycube/services/shared_library.h>
18+
19+
extern "C" const char *data_model() {
20+
return k8sdispatcher_datamodel.c_str();
21+
}

0 commit comments

Comments
 (0)