firewall batch operations + batch update + changed default action policy + removed "transactional" + docs and tests (#323)

This commit aims to introduce batch operation in firewall, to optimize network usage under certain
situations. For example, it would be both time and resource consuming performing 1000 requests to add/remove
rules, which was still required in the previous "transaction" mode. So I decided to introduce an endpoint which can accept a list of operations to be performed.

Moreover, I have also changed the default action policy, previously set to DROP. I set it to FORWARD, to avoid that
a remote client which interacts with Polycube is completely cut out from the system, since the DROP policy would
block ALL the incoming traffic.

Signed-off-by: Simone Magnani <[email protected]>

Author: smagnani96

Documentation/services/pcn-firewall/firewall.rst

@@ -43,17 +43,40 @@ Rule insertion
 
 Rule insertion is guaranteed to be *atomic*: during the computation of the new datapath, the old rule set is used until the new rule set is ready, and only at that moment the new policies will be applied.
 
-Rule insertion is an expensive operation. For this reason, there are two modes that can be used based on the needs:
+Rule insertion is an expensive operation. For this reason, we have thought about different endpoints to optimize expensive operations:
 
-  - ``Interactive mode:`` this is the default mode. It makes the commands to modify policies synchronous, so that they return only when the modification is reflected in the datapath. This is the slowest mode, as it requires to recompute the datapath for each command, but it has the advantage that a command returns only when the operation is completed.
-  - ``Transaction mode:`` in this mode commands on the policies are chained and asynchronously applied to the datapath altogether when the user asks it. The performance gain is sensible when commands have to be issued together (e.g. a set of rules), as it requires only one interaction with the datapath at the end. To switch in the transaction mode, it is necessary to issue the command ``polycubectl firewall fwname set interactive=false``. In this way, rules can be inserted normally, but to apply them the command ``polycubectl firewall fwname chain INGRESS apply-rules`` has to be issued. Notice: this command is specific for the chain, and updates the specified chain only.
+  - ``/insert``, ``/delete``, ``/append`` and ``PUT`` on ``rule/<id>`` (update): these endpoints are used to perform a single operation on a rule. As soon as the rule-set is updated, it is compiled and all the modifications are immediately inserted in the datapath.
+  - ``/batch``: as suggested by the name, this endpoint is used to perform multiple operation on a single HTTP request. Instead of compiling the new rule-set as soon as a single operation is fulfilled, it waits for all the actions described in the request to be executed. Finally, a single compilation is performed and the datapath is updated once.
 
+Concerning the batch endpoint, it accepts a JSON list of rules like:
+
+.. code-block:: bash
+
+    {
+      "rules": [
+        {"operation": "insert", "id": 0, "l4proto":"TCP", "src":"192.168.1.1/32", "dst":"192.168.1.10/24", "action":"drop"},
+        {"operation": "append", "l4proto": "ICMP", "src":"192.168.1.100/32", "dst":"192.168.1.100/24", "action":"drop"},
+        {"operation": "update", "id": 0, "l4proto":"TCP", "src":"192.168.1.2/32", "dst":"192.168.1.20/24", "action":"forward"},
+        {"operation": "delete", "id": 0},
+        {"operation": "delete", "l4proto":"ICMP", "src":"192.168.1.100/32", "dst":"192.168.1.100/24", "action":"drop"}
+      ]
+    }
+
+
+As you can see, every element of the ``rules`` array MUST contain an operation (insert, append, update, delete) plus a rule/id which is the actual target.
+All the listed operation are performed sequentially, meaning that the user must sent the operation already ordered as he wants. Pay attention when sending some DELETE with other INSERT, you have to take in mind that during such operations IDs may vary (increase or decrease).
+
+This features is also available from the ``polycubectl`` command line. It is strongly suggested to create a JSON file containing the batch of rules and then type:
+
+``polycubectl firewall <fwname> chain <chainname> batch rules= < filename.json``
+
+Using the redirection diamond you are able to insert the file content in the body of the HTTP POST request generated from the command.
 
 Default action
 ^^^^^^^^^^^^^^
 
-The default action if no rule is matched is drop. This can be changed for each chain independently by issuing the command
-``polycubectl firewall fwname chain INGRESS set default=FORWARD`` or ``polycubectl firewall fwname chain EGRESS set default=FORWARD``.
+The default action if no rule is matched is forward. This can be changed for each chain independently by issuing the command
+``polycubectl firewall fwname chain INGRESS set default=DROP`` or ``polycubectl firewall fwname chain EGRESS set default=DROP``.
 
 Statistics
 ^^^^^^^^^^
@@ -65,6 +88,7 @@ Examples
 
 The `examples source folder <https://github.com/polycube-network/polycube/tree/master/src/services/pcn-firewall/examples/>`_ contains some simple scripts to show how to configure the service.
 
+Also under the test directory, there are plenty of scripts that test the firewall using both single and batch rule insertion/deletion.
 
 
 Implementation details

src/services/pcn-firewall/datamodel/firewall.yang

@@ -22,7 +22,7 @@ module firewall {
       enum LOG;
       enum FORWARD;
     }
-    default DROP;
+    default FORWARD;
   }
 
   typedef conntrackstatus {
@@ -34,6 +34,14 @@ module firewall {
     }
   }
 
+  typedef operation {
+    type enumeration {
+      enum INSERT;
+      enum APPEND;
+      enum DELETE;
+      enum UPDATE;
+    }
+  }
 
   grouping rule-fields {
     leaf src {
@@ -81,7 +89,6 @@ module firewall {
       description "Connection status (NEW, ESTABLISHED, RELATED, INVALID)";
     }
 
-
     leaf action {
       type action;
       polycube-base:init-only-config;
@@ -113,12 +120,6 @@ module firewall {
     description "If Connection Tracking is enabled, all packets belonging to ESTABLISHED connections will be forwarded automatically. Default is ON.";
   }
 
-  leaf interactive {
-    type boolean;
-    description "Interactive mode applies new rules immediately; if 'false', the command 'apply-rules' has to be used to apply all the rules at once. Default is TRUE.";
-    default true;
-  }
-
   list session-table {
     key "src dst l4proto sport dport";
     config false;
@@ -250,18 +251,23 @@ module firewall {
       }
     }
 
-    action reset-counters {
-      description "Reset the counters to 0 for the chain.";
-      output {
-        leaf result {
-          type boolean;
-          description "True if the operation is successful";
+    action batch {
+      input {
+        list rules {
+          key "id";
+          leaf id {
+            type uint32;
+          }
+          leaf operation {
+            type operation;
+          }
+          uses "firewall:rule-fields";
         }
       }
     }
 
-    action apply-rules {
-      description "Applies the rules when in batch mode (interactive==false)";
+    action reset-counters {
+      description "Reset the counters to 0 for the chain.";
       output {
         leaf result {
           type boolean;
@@ -270,4 +276,4 @@ module firewall {
       }
     }
   }
-}
+}

src/services/pcn-firewall/examples/README.md

@@ -4,6 +4,7 @@ The example folder contains a set of simple scripts to understand how the firewa
 ## Prerequisites
 All scripts assume that polycubed has been already launched, and that there is a standard cube running with two ports belonging to two namespaces already created and configured. Moreover, a firewall instance should be running and attached to one of the standard cube's port.
 To set up all the needed components, please execute the script [setup_env.sh](./setup_env.sh).
+Moreover, these examples contain a set of rules to allow some traffic, while denying the rest of it. To make this happen, since the default policy is FORWARD, the setup script automatically patches the default rule to DROP.
 
 ## Examples:
 - [Ping](./allow_ping.sh): Connects the firewall to one of the standard cube's port, and allows only the ICMP echo requests/responses. In order to test that the configuration succeeded, you can launch the script [test_ping.sh](./test_ping.sh).
@@ -14,7 +15,7 @@ To set up all the needed components, please execute the script [setup_env.sh](./
 - [Transactions](./use_transactions.sh): This example is like the Ping one, as the rule set is the same, but it shows how to use transactions instead of the interactive mode. This mode is strongly suggested when more than one rule has to be inserted, like in the example. **For each chain**, after the rules have been inserted, the command `polycubectl firewall fw chain INGRESS apply-rules` (*for the ingress chain*) is issued to apply the rule set, requiring a single interaction with the datapath.
 - [Host Mode](./host_mode.sh): This example shows how to use the firewall in the host mode, intercepting the traffic **from the outside to the host**. At the moment it is not possible to intercept traffic in the other direction. This example considers the physical interface connected to the internet.
 
-Please note that some example does not volountarly delete used resources like firewall or network namespace, since a user can play with multiple rules (e.g. allow IP and TCP). Thus, the behaviour of some tests may change depending on the allowed scripts run.
+Please note that some example does not voluntarily delete used resources like firewall or network namespace, since a user can play with multiple rules (e.g. allow IP and TCP). Thus, the behaviour of some tests may change depending on the allowed scripts run.
 
 To cleanup the entire environment or only the firewall's rules, refer to the following sections.
 

src/services/pcn-firewall/examples/setup_env.sh

@@ -43,6 +43,9 @@ polycubectl firewall add fw
 
 polycubectl attach fw br:port1
 
+polycubectl firewall fw chain INGRESS set default=DROP
+polycubectl firewall fw chain EGRESS set default=DROP
+
 # EGRESS_CHAIN and INGRESS_CHAIN are now considered independently
 
 # br:port1 <---- EGRESS ----< br:port2