Skip to content

Commit e075448

Browse files
committed
Fixes #1826 - Add a cross-network dynamic address capability
Fixes #1826 - Added a test for the cross-network feature Fixes #1826 - Fixed asan issue Fixes #1826 - Added support for edge-style cross-net dynamic addresses. Fixes #1826 - Make the test less resource-intensive for the CI Fixes #1826 - Introduced the inter-network role and inter-network-specific statistics Fixes #1826 - Renamed entity managedRouter => network; Added a doc file describing the feature. Fixes #1826 - Removed the backward-compatibility features to simplify the patch. This will need to be carried separately until the next major release. For inter-network deliveries, strip only the origin and trace annotations. Added network configuration to the 'skstat -g' output Delay the effect of network-id changes to allow the UPDATE operation to complete.
1 parent cf57e42 commit e075448

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+860
-215
lines changed

docs/notes/cross-network.md

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
<!-- Licensed to the Apache Software Foundation (ASF) under one -->
2+
<!-- or more contributor license agreements. See the NOTICE file -->
3+
<!-- distributed with this work for additional information -->
4+
<!-- regarding copyright ownership. The ASF licenses this file -->
5+
<!-- to you under the Apache License, Version 2.0 (the -->
6+
<!-- "License"); you may not use this file except in compliance -->
7+
<!-- with the License. You may obtain a copy of the License at -->
8+
9+
<!-- http://www.apache.org/licenses/LICENSE-2.0 -->
10+
11+
<!-- Unless required by applicable law or agreed to in writing, -->
12+
<!-- software distributed under the License is distributed on an -->
13+
<!-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
14+
<!-- KIND, either express or implied. See the License for the -->
15+
<!-- specific language governing permissions and limitations -->
16+
<!-- under the License. -->
17+
18+
# Cross (Inter) Network Communication
19+
20+
Cross-network communication is message transfer between endpoints connected to disjoint Skupper/AMQP networks. The connectivity between the networks is achieved using connections with the 'inter-network' role and auto-links established over those connections.
21+
22+
The easy part is creating connectivity for a particular mobile address, the address of a service. More challenging is the need to provide reply-to connectivity using a router-assigned dynamic address on the client-side (for request/response traffic patterns).
23+
24+
## Use Cases
25+
26+
There are a couple of use cases that drive the requirements for cross-network communication.
27+
28+
### Central Management of Networks
29+
30+
In the case where an enterprise maintains a large number of virtual application networks, a good way to provide connectivity for a management plane is to create a management network that maintains cross-network connectivity to each of the managed networks.
31+
32+
In such a network, there is a inter-network connection between at least one router in the managed network and at lease one router in the management network. Multiple connections may be desired for redundancy and availability. All message connectivity is strictly between the management network and the individual managed network. No communication will be possible between two managed networks via the management network, thus preserving the isolation provided by an application network.
33+
34+
### Inter-Network Federation
35+
36+
In the case where an application running on a virtual application network wishes to expose a service for use from within another VAN without creating a general-use ingress, cross-network connectivity can be used to create a secure tunnel between VANs for specifically configured services.
37+
38+
## Setting up Cross-Network Connectivity
39+
40+
To illustrate the configuration of inter-network communication, we'll use the inter-network federation use case with two networks, net-A and net-B.
41+
42+
Both networks are assumed to be composed of multiple interior routers and possibly some edge routers. The inter-network connection must be between two interior routers, one in each network.
43+
44+
### Router Configuration
45+
46+
All routers (interior and edge) must be configured with the name of the network using the `network` configuration:
47+
48+
```
49+
network {
50+
networkId: net-A
51+
}
52+
```
53+
### Listener and Connector Configuration
54+
55+
An interior router in each network must be designated to be endpoints of the inter-network connection. For example, routers A3 (in net-A) and B2 (in net-B) are designated. A3 is configured with a `listener` and B2 is configured with a `connector`, both using role `inter-network`. B2 is configured to connect to A3 using standard host/port addressing and security. Both the listener and connector must be named because the configurations will include `autoLink` entities that refer to those connections.
56+
57+
### Exposing Services Cross-Network
58+
59+
Since the `inter-network` connection does not provide the network-joining functions of an `inter-router` connection, the two networks are not joined and do not share any topology or routing information. Nothing will flow over the `inter-network` connection until an `autoLink` is created which establishes a unidirectional link between the networks.
60+
61+
The full functionality of auto-links is available for `inter-network` connections. They behave similarly to `route-container` connections. However, it is recommended for inter-network use cases that all auto-links be configured as `direction: in` and "pull" from the network hosting a particular destination.
62+
63+
For example, if net-A hosts a service using the address `service-45` anywhere in its network, an auto-link should be configured in the designated router (A3) as follows:
64+
65+
```
66+
autoLink {
67+
connection: <name of listener/connector>
68+
address: service-45
69+
direction: in
70+
}
71+
```
72+
73+
This will cause the `service-45` address to be reachable from all routers in net-B.
74+
75+
Each service that is to be reachable cross-network must have an auto-link created for it in the network in which the service is hosted.
76+
77+
Important Note: Destinations cannot be load-balanced across different networks. In other words, there must not be cross-network services in different networks with the same address. If auto-links are created in both directions for an address, a loop is created that will almost certainly forward deliveries to that address to the wrong places.
78+
79+
### Using Dynamic Addresses
80+
81+
Almost always, service traffic is not a one-way affair. A user of a service expects to receive a response from the service when a request is sent. Such a client will first create a receiving link using a dynamic terminus. The dynamically-allocated address is then placed in the request's `reply-to` header for the server to use as the destination address of the reply or replies.
82+
83+
If a network in a federation relationship is going to host clients using dynamic addresses, it must create an auto-link to carry those replies back from the serving network. The auto-link looks like this:
84+
85+
```
86+
autoLink {
87+
connection: <name of listener/connector>
88+
direction: in
89+
externalAddress: _topo/<local-network-id>
90+
}
91+
```
92+
93+
There are a couple of notable characteristics of this auto-link. It does not have an `address` attribute, making it "anonymous". This is important because it must issue credit to the peer network regardless of the existance of a particular destination locally.
94+
95+
The other important aspect of this auto-link is that its remote address is interpreted at the other end as a "remote-network" address which will match any dynamic address created on the local network. This means that only one auto-link needs to be created to handle the return traffic of any number of exposed services.
96+
97+
## A Complete Example
98+
99+
The following example shows how to create a simple two-network federation on a single host where each network has one router.
100+
101+
Router A:
102+
```
103+
router {
104+
id: A
105+
mode: interior
106+
}
107+
108+
network {
109+
networkId: net-A
110+
}
111+
112+
listener {
113+
port: 10001
114+
role: normal
115+
}
116+
117+
listener {
118+
name: federation
119+
port: 11001
120+
role: inter-network
121+
}
122+
123+
autoLink {
124+
connection: federation
125+
direction: in
126+
externalAddress: _topo/net-A
127+
}
128+
129+
autoLink {
130+
connection: federation
131+
direction: in
132+
address: service-A-to-B
133+
}
134+
```
135+
136+
Router B:
137+
```
138+
router {
139+
id: B
140+
mode: interior
141+
}
142+
143+
network {
144+
networkId: net-B
145+
}
146+
147+
listener {
148+
port: 10002
149+
role: normal
150+
}
151+
152+
connector {
153+
name: federation
154+
port: 11001
155+
role: inter-network
156+
}
157+
158+
autoLink {
159+
connection: federation
160+
direction: in
161+
externalAddress: _topo/net-B
162+
}
163+
164+
autoLink {
165+
connection: federation
166+
direction: in
167+
address: service-B-to-A
168+
}
169+
```
170+
171+
## Using Cross-Network Connectivity as an Endpoint
172+
173+
It is the intent of the implementation that cross-network operations are transparent to the endpoints involved. No changes to endpoint (producer, consumer, client, server) code need to be made to use this feature.
174+
175+
## Backward Compatibility and Breaking Changes
176+
177+
This feature represents an incompatible change from the version 3 router because the format of dynamic addresses has been changed to include a field for the network ID. This feature must be introduced in a major release (version 4). Until that time, the project team shall maintain a development branch that tracks main and carries a patch for this feature.

include/qpid/dispatch/iterator.h

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ typedef struct qd_iterator_t qd_iterator_t;
4949
*/
5050
#define QD_ITER_HASH_PREFIX_TOPOLOGICAL 'T'
5151
#define QD_ITER_HASH_PREFIX_LOCAL 'L'
52+
#define QD_ITER_HASH_PREFIX_NETWORK 'N'
5253
#define QD_ITER_HASH_PREFIX_AREA 'A'
5354
#define QD_ITER_HASH_PREFIX_ROUTER 'R'
5455
#define QD_ITER_HASH_PREFIX_MOBILE 'M'
@@ -137,6 +138,11 @@ void qd_iterator_finalize(void);
137138
*/
138139
void qd_iterator_set_address(bool edge_mode, const char *area, const char *router);
139140

141+
/**
142+
* Set the network ID for the local router. This can be updated repeatedly during run-time.
143+
*/
144+
void qd_iterator_set_network(const char *network);
145+
140146
/**
141147
* Add and delete peer-edge router identities. When in edge mode, peer edge routers
142148
* result in different hash results than remote edge routers. These functions are used

include/qpid/dispatch/protocol_adaptor.h

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -347,6 +347,7 @@ typedef enum {
347347
QDR_ROLE_EDGE_CONNECTION,
348348
QDR_ROLE_INTER_ROUTER_DATA,
349349
QDR_ROLE_INTER_EDGE,
350+
QDR_ROLE_INTER_NETWORK,
350351
} qdr_connection_role_t;
351352

352353
typedef void (*qdr_connection_bind_context_t) (qdr_connection_t *context, void *token);
@@ -424,6 +425,7 @@ void qdr_connection_set_tracing(qdr_connection_t *conn, bool enable_tracing);
424425
void qdr_core_close_connection(qdr_connection_t *conn);
425426

426427
bool qdr_connection_route_container(qdr_connection_t *conn);
428+
bool qdr_connection_inter_network(qdr_connection_t *conn);
427429

428430
/**
429431
* qdr_connection_set_context
@@ -446,6 +448,7 @@ void *qdr_connection_get_context(const qdr_connection_t *conn);
446448
* Retrieve the role of the connection object.
447449
*/
448450
qdr_connection_role_t qdr_connection_role(const qdr_connection_t *conn);
451+
qdr_connection_role_t qdr_link_connection_role(const qdr_link_t *link);
449452

450453

451454
/**

include/qpid/dispatch/router_core.h

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,12 @@ ENUM_DECLARE(qd_router_mode);
4949
/**
5050
* Allocate and start an instance of the router core module.
5151
*/
52-
qdr_core_t *qdr_core(qd_dispatch_t *qd, qd_router_mode_t mode, const char *area, const char *id, const char *van_id);
52+
qdr_core_t *qdr_core(qd_dispatch_t *qd, qd_router_mode_t mode, const char *area, const char *id, const char *tenant_id);
53+
54+
/**
55+
* Set the network id
56+
*/
57+
void qdr_core_set_network_id(qdr_core_t *core, const char *network_id);
5358

5459
/**
5560
* Stop and deallocate an instance of the router core.
@@ -75,12 +80,12 @@ qd_dispatch_t *qdr_core_dispatch(qdr_core_t *core);
7580
void qdr_process_tick(qdr_core_t *core);
7681

7782
/**
78-
* @brief Return the text of the router's virtual application network ID, or 0.
83+
* @brief Return the text of the router's virtual application network tenant ID, or 0.
7984
*
8085
* @param core Pointer to the core object returned by qdr_core()
81-
* @return const char* null-terminated text of the van-id or 0 if there's no id.
86+
* @return const char* null-terminated text of the tenant-id or 0 if there's no id.
8287
*/
83-
const char *qdr_core_van_id(const qdr_core_t *core);
88+
const char *qdr_core_tenant_id(const qdr_core_t *core);
8489

8590

8691
/**

python/skupper_router/management/client.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -101,9 +101,9 @@ def connect(cls, url=None, router=None, timeout=10, ssl_domain=None, sasl=None,
101101
if url_.path is not None:
102102
path = url_.path
103103
elif router:
104-
path = '_topo/0/%s/$management' % router
104+
path = '_topo/0/0/%s/$management' % router
105105
elif edge_router:
106-
path = '_edge/%s/$management' % edge_router
106+
path = '_edge/0/%s/$management' % edge_router
107107
else:
108108
path = '$management'
109109
connection = BlockingConnection(url,

python/skupper_router/management/skrouter.json

Lines changed: 38 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -462,6 +462,16 @@
462462
"description": "Number of deliveries that were sent to route container connections.",
463463
"graph": true
464464
},
465+
"deliveriesIngressInterNetwork": {
466+
"type": "integer",
467+
"description": "Number of deliveries that were received from a connected network.",
468+
"graph": true
469+
},
470+
"deliveriesEgressInterNetwork": {
471+
"type": "integer",
472+
"description": "Number of deliveries that were sent to a connected network.",
473+
"graph": true
474+
},
465475
"residentMemoryUsage": {
466476
"type": "integer",
467477
"graph": true,
@@ -511,7 +521,7 @@
511521
"create": true
512522
},
513523
"vanId": {
514-
"description":"Deprecated: See the managedRouter entity definition.",
524+
"description":"Deprecated: See the network tenantId attribute.",
515525
"type": "string",
516526
"required": false,
517527
"create": true
@@ -633,13 +643,21 @@
633643
}
634644
},
635645

636-
"managedRouter": {
637-
"description": "This optional entity holds configuration needed for the router to be managed by a central manager",
646+
"network": {
647+
"description": "This optional entity holds configuration needed for the router to participate in inter-network communication",
638648
"extends": "configurationEntity",
639649
"singleton": true,
650+
"operations": ["UPDATE"],
640651
"attributes": {
641-
"vanId": {
642-
"description":"The unique ID of the Virtual Application Network that this router is a member of.",
652+
"networkId": {
653+
"description":"The unique ID of the AMQP Network that this router is a member of.",
654+
"type": "string",
655+
"required": false,
656+
"create": true,
657+
"update": true
658+
},
659+
"tenantId": {
660+
"description":"The unique VAN tenant ID for routers that are connected to a multi-tenant service backbone.",
643661
"type": "string",
644662
"required": false,
645663
"create": true
@@ -648,7 +666,7 @@
648666
},
649667

650668
"site": {
651-
"description":"Optional. If present, this entity causes the router to emit a SITE record for VanFLow in the case that there is not an external controller to emit this record.",
669+
"description":"Optional. If present, this entity causes the router to emit a SITE record for VanFlow in the case that there is not an external controller to emit this record.",
652670
"extends": "configurationEntity",
653671
"attributes": {
654672
"location": {
@@ -784,7 +802,8 @@
784802
"inter-router",
785803
"route-container",
786804
"edge",
787-
"inter-edge"
805+
"inter-edge",
806+
"inter-network"
788807
],
789808
"default": "normal",
790809
"description": "The role of an established connection. In the normal role, the connection is assumed to be used for AMQP clients that are doing normal message delivery over the connection. In the inter-router role, the connection is assumed to be to another router in the network. Inter-router discovery and routing protocols can only be used over inter-router connections. route-container role can be used for router-container connections, for example, a router-broker connection. In the edge role, the connection is assumed to be between an edge router and an interior router.",
@@ -958,7 +977,8 @@
958977
"inter-router",
959978
"route-container",
960979
"edge",
961-
"inter-edge"
980+
"inter-edge",
981+
"inter-network"
962982
],
963983
"default": "normal",
964984
"description": "The role of an established connection. In the normal role, the connection is assumed to be used for AMQP clients that are doing normal message delivery over the connection. In the inter-router role, the connection is assumed to be to another router in the network. Inter-router discovery and routing protocols can only be used over inter-router connections. route-container role can be used for router-container connections, for example, a router-broker connection. In the edge role, the connection is assumed to be between and edge router and an interior router.",
@@ -1598,6 +1618,16 @@
15981618
"description": "Number of deliveries that were sent to a route-container address.",
15991619
"graph": true
16001620
},
1621+
"deliveriesIngressInterNetwork": {
1622+
"type": "integer",
1623+
"description": "Number of deliveries that were received from a connected network.",
1624+
"graph": true
1625+
},
1626+
"deliveriesEgressInterNetwork": {
1627+
"type": "integer",
1628+
"description": "Number of deliveries that were sent to a connected network.",
1629+
"graph": true
1630+
},
16011631
"key": {
16021632
"description": "Internal unique (to this router) key to identify the address",
16031633
"type": "string"

python/skupper_router_internal/dispatch.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,8 @@ def __init__(self) -> None:
103103
self._prototype(self.qd_error_code, c_long, [], check=False)
104104
self._prototype(self.qd_error_message, c_char_p, [], check=False)
105105
self._prototype(self.qd_log_entity, c_long, [py_object])
106-
self._prototype(self.qd_dispatch_configure_managed_router, None, [self.qd_dispatch_p, py_object])
106+
self._prototype(self.qd_dispatch_configure_network, None, [self.qd_dispatch_p, py_object])
107+
self._prototype(self.qd_dispatch_update_network, None, [self.qd_dispatch_p, py_object])
107108
self._prototype(self.qd_dispatch_configure_router, None, [self.qd_dispatch_p, py_object])
108109
self._prototype(self.qd_dispatch_configure_site, None, [self.qd_dispatch_p, py_object])
109110
self._prototype(self.qd_dispatch_prepare, None, [self.qd_dispatch_p])

python/skupper_router_internal/management/agent.py

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -280,15 +280,18 @@ def __str__(self):
280280
return super(RouterEntity, self).__str__().replace("Entity(", "RouterEntity(")
281281

282282

283-
class ManagedRouterEntity(EntityAdapter):
283+
class NetworkEntity(EntityAdapter):
284284
def __init__(self, agent, entity_type, attributes=None):
285-
super(ManagedRouterEntity, self).__init__(agent, entity_type, attributes, validate=False)
285+
super(NetworkEntity, self).__init__(agent, entity_type, attributes, validate=False)
286286

287287
def create(self):
288-
self._qd.qd_dispatch_configure_managed_router(self._dispatch, self)
288+
self._qd.qd_dispatch_configure_network(self._dispatch, self)
289+
290+
def _update(self):
291+
self._qd.qd_dispatch_update_network(self._dispatch, self)
289292

290293
def __str__(self):
291-
return super(ManagedRouterEntity, self).__str__().replace("Entity(", "ManagedRouterEntity(")
294+
return super(NetworkEntity, self).__str__().replace("Entity(", "NetworkEntity(")
292295

293296

294297
class SiteEntity(EntityAdapter):

0 commit comments

Comments
 (0)