Skip to content

IPv6 Hop-by-Hop & Destination Option #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rsivakolundu wants to merge 14 commits into
base: master
Choose a base branch
from
Open

IPv6 Hop-by-Hop & Destination Option #56

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
3771848
Update INT.mdk
rsivakolundu Jun 21, 2018
10355fd
Update INT.mdk
rsivakolundu Jun 21, 2018
fe346e4
Update INT.mdk
rsivakolundu Sep 6, 2018
d4972e5
Revert "Slides from 2018-05-17 P4 Memory Abstraction Use Cases"
rsivakolundu Sep 28, 2018
703fd49
Update INT.mdk
rsivakolundu Oct 2, 2018
cc70169
Update INT.mdk
rsivakolundu Oct 3, 2018
800c506
Update INT.mdk
rsivakolundu Oct 4, 2018
7456957
Update INT.mdk
rsivakolundu Oct 4, 2018
c5c1a52
Update INT.mdk
rsivakolundu Oct 4, 2018
9e75a1f
Update INT.mdk
rsivakolundu Dec 6, 2019
06a5ea0
Update INT.mdk
rsivakolundu Dec 6, 2019
4638a0f
Update INT.mdk
rsivakolundu Jan 16, 2020
4601746
Update INT.mdk
rsivakolundu Feb 11, 2020
5ff83a8
Merge branch 'master' into IPv6-HbyH-Option
rsivakolundu Feb 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed meeting_slides/2018_05_17_P4_Memory_Use_Cases.pdf
View file
Open in desktop
Binary file not shown.
137 changes: 129 additions & 8 deletions telemetry/specs/INT.mdk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -494,19 +494,140 @@ corruption at preceding hops.

## Header Location

We describe three encapsulation formats in this specification, covering
We describe five encapsulation formats in this specification, covering
different deployment scenarios, with and without network virtualization:

1. *INT over TCP/UDP* - A shim header is inserted following TCP/UDP
2. "INT over IPv6" - INT Headers are carried in the IPv6 packets as Hop-by-Hop option.
3. *INT over TCP/UDP* - A shim header is inserted following TCP/UDP
header. INT Headers are carried between this shim header and TCP/UDP payload.
This approach doesn’t rely on any tunneling/virtualization mechanism and is
versatile to apply INT to both native and virtualized traffic.
2. *INT over VXLAN* - VXLAN generic protocol extensions
4. *INT over VXLAN* - VXLAN generic protocol extensions
(draft-ietf-nvo3-vxlan-gpe) are used to carry INT Headers between
the VXLAN header and the encapsulated VXLAN payload.
3. *INT over Geneve* - Geneve is an extensible tunneling framework, allowing
5. *INT over Geneve* - Geneve is an extensible tunneling framework, allowing
Geneve options to be defined for INT Headers.

### INT over IPv6
Copy link
Contributor

@mhira1 mhira1 Oct 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add INT over IPv6 after the other three encaps? Specially because the text in the paragraph is referring to scenarios where "INT over VXLAN or Geneve is not helpful"

Copy link
Contributor Author

@rsivakolundu rsivakolundu Oct 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed what was done earlier. TCP/UDP was listed first and it referenced encaps. I just stuck to that. I am fine with changing the order.

Copy link

@jafingerhut jafingerhut Oct 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be woefully out of date on IPv6 extension header behavior, but regarding the option '"INT over IPv6" - INT Headers are carried in the IPv6 packets as Hop-by-Hop option.', I had thought that switches in practice have to punt packets with an IPv6 Hop-by-Hop extension header to the slow path, e.g. software forwarding on a general purpose CPU.

I did a quick search and found that RFC 7045 (published Dec 2013) says this in Section 2.2 "Hop-by-Hop Options":

The IPv6 Hop-by-Hop Options header SHOULD be processed by
intermediate forwarding nodes as described in [RFC2460]. However, it
is to be expected that high-performance routers will either ignore it
or assign packets containing it to a slow processing path. Designers
planning to use a hop-by-hop option need to be aware of this likely
behaviour.

Is there really a desire to put INT data into a header that will likely result in slow path processing in the network?


In case the traffic being monitored is not encapsulated by any virtualization
header, INT over VXLAN or INT over Geneve is not helpful. Instead,
the INT metadata can be carried as Hop-by-Hop option data.

IPv6 Hop-by-Hop Option format for carrying INT Header and
Metadata:

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| Nxt HDR = UL | HbyH Ext Len | Padding|(MBZ) | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ I
| Option Type | Opt Data Len | Reserved (MBZ) | N
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ T
| Variable Option Data (INT Metadata Headers and Metadata) | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+


Nxt Hdr: 8-bit selector. Identifies the type of header immediately following the
Hop-by-Hop or Destination Options header. Uses the same values as the IPv4 Protocol
Field [IANA-PN]

HDR Ext Len: 8-bit unsigned integer. Length of the Hop-by-Hop or Destination
Options header in 8-octet units, not including the first 8 octets.
Copy link

@mickeyspiegel mickeyspiegel Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above it only says Hop-by-Hop Options, not Destination Options. We can go either way, but it should be consistent.

Copy link
Contributor Author

@rsivakolundu rsivakolundu Feb 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


Option Type: 8-bit identifier of the type of option.

001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_HOP_BY_HOP_OPTION_IPV6.
001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_DESTINATION_OPTION_IPV6.
Copy link

@mickeyspiegel mickeyspiegel Oct 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the IANA registry, there are a total of 32 code points of which 17 have already been allocated. The registration procedure is IESG Approval, IETF Review or Standards Action. IOAM is asking for 4 code points, which seems unlikely. The chances for INT to get any code points are not high.

Copy link
Contributor Author

@rsivakolundu rsivakolundu Oct 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.

Copy link

@mickeyspiegel mickeyspiegel Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see two options:

  1. Go with the single experimental use hop-by-hop-options codepoint (0b11110), and split the "Reserved (MBZ)" field into 8 bits reserved followed by 8 bits of INT type.
  2. Wait for IOAM to get a codepoint and use that. Split the "Reserved (MBZ)" field into 8 bits reserved followed by 8 bits of IOAM type. Assign relatively high IOAM type codepoints for INT hop-by-hop option and INT destination option.

Copy link

@mickeyspiegel mickeyspiegel Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see another problem with the corresponding IETF IOAM IPv6 draft. The text says that "a router MUST drop packets which contain extension headers carrying IOAM data-fields", to "ensure that the IOAM data does not unintentionally get forwarded outside the IOAM domain." However, they asked for an Option Type codepoint starting with "00", which means when the option type is unrecognized, "skip over this option and continue processing the header". If the text is correct, then they should ask for any of the other codepoint prefixes "01" (discard the packet), "10" (discard and send ICMP parameter problem, code 2, back to the packet's source address), or "11" (discard and send ICMP only if the packet's destination address was not a multicast address).

Copy link
Contributor Author

@rsivakolundu rsivakolundu Jan 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will close loop with IETF and address this comment.

Copy link

@mickeyspiegel mickeyspiegel Feb 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whatever we do, two codepoints will not fly. At a minimum we would have to go with TBD_IANA_INT_OPTION_IPV6 (not distinguishing between INT hop-by-hop and INT destination), which would later get resolved to either experimental hop-by-hop options codeopint or whatever IOAM has assigned. If we go with IOAM then the INT Type values might need to be shifted to avoid conflicts.

I also wonder if we should use xxx or yyy for the first 3 bits as well given the other open issue I stated above.


Opt Data Len: 8-bit unsigned integer. Length of the Reserved and Option Data field of this
option, in octets.

Reserved (MBZ): 16 bit field, must be filled with zeroes.
Variable Opt Data: INT Header and Metadata, multiple of four octets in length.

Padding: 16-bit pad. Needed to ensure that the variable length of the complete
Hop-by-Hop or Destination Options Header is an integer multiple of 8 octets long.
Copy link

@mickeyspiegel mickeyspiegel Oct 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the padding is added explicitly. If there are multiple options in the same extension header, then padding is only inserted to meet the next option's alignment constraint. The padding to 8 bytes is done at the end, after all options have been added to the extension header.

What I suggested for IOAM is that the Hop ML must be even so that padding is not required. However, I did not consider that the INT header is included. For v1 and with the second padding option described above, if Hop ML is even no padding would be required. For v2 due to the odd number of octets before the per hop data, this would not work and padding would have to be added in some cases.

Copy link

@mickeyspiegel mickeyspiegel Jan 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For v2, it looks like we have 4 octets shim header and 12 octets fixed part of INT header before the variable metadata. There are two options with respect to padding:

  1. Go with 4n alignment. When INT is the only option present in the hop-by-hop options, then the "Padding|(MBZ)" above would be inserted in front of this option, and if Hop ML is even then there would be an additional 4 octets of padding after this option. If Hop ML is odd, then the additional 4 octets of padding after this option would only be present at even hops, so each hop either strips the incoming 4 octets of padding after this option, or adds 4 octets of padding after this option. If there are other hop-by-hop options present, padding may differ both before and after this option.
  2. Go with 8n alignment and require that Hop ML must be even. When INT is the only option present in the hop-by-hop options, then the "Padding|(MBZ)" in front of this option would be 6 octets rather than 2 octets. No padding would be added after this option. If there are other hop-by-hop options present, then the padding may differ both before and after this option, but it would not need to be adjusted at each hop.

IOAM has a similar issue, but the header size is different so the 4n alignment implications are different. When IOAM incremental trace is the only option present in the hop-by-hop options, and if hop ML is even then there would be no padding after this option. 8n alignment would not work since the total size of the fixed headers is not a multiple of 8 octets.



The format of the IPv6 packet with Hop-by-Hop option is shown below:

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| Traffic Class | Flow Label |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Length |Nxt HDR = HbyH | Hop Limit |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source IPv6 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination IPv6 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| Nxt HDR = UL | HbyH Data Len | Padding | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ I
| Option Type | Opt Data Len | Reserved (MBZ) | N
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ T
| Variable Option Data (INT Data) | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| Payload + Padding (L4/ESP/….) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The format of the IPv6 packet with Hop-by-Hop option for INT-MD (Embedded Metadata) is shown below:

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| Traffic Class | Flow Label |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Length | Nxt HDR = HbyH| Hop Limit |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (Outer) Source IPv6 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (Outer) Destination IPv6 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| Nxt HDR = IPv6| HbyH Ext Len | Padding|(MBZ) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len | Reserved (MBZ) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| INT-Type - MD | Length | Reserved | Next Protocol | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| |
|Ver = 2|Rep|C|E|M| Reserved | Hop ML |Rmainig HopCnt | I
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ N
| Instruction Bitmap | Domain Specific ID | T
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| DS Flags | DS Instruction | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| Variable Option Data (INT DATA) | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| Payload Original Packet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The format of the IPv6 packet with Hop-by-Hop option for INT-MX (Direct Export) is shown below:

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| Traffic Class | Flow Label |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Length | Nxt HDR = HbyH| Hop Limit |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (Outer) Source IPv6 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| (Outer) Destination IPv6 Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| Nxt HDR = IPv6| HbyH Ext Len | Padding|(MBZ) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Opt Data Len | Reserved (MBZ) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| INT-Type - MX | Length | Reserved | Next Protocol | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| |
|Ver = 2|Rep|C|E|M| Reserved | Hop ML |Rmainig HopCnt | I
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ N
| Instruction Bitmap | Domain Specific ID | T
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| DS Flags | DS Instruction | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
| Payload Original Packet |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

### INT over TCP/UDP

In case the traffic being monitored is not encapsulated by any virtualization
Expand Down Expand Up @@ -752,12 +873,12 @@ hop-by-hop INT header must fit in a single Geneve option.
In this section, we define the format for INT hop-by-hop metadata headers,
and the metadata itself.

INT Metadata Header and Metadata Stack:
INT Metadata Header and Metadata Stack (Version = 1):
`
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ver |Rep|C|E|M| Reserved | Hop ML |RemainingHopCnt|
|Ver = 1|Rep|C|E|M| Reserved | Hop ML |RemainingHopCnt|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Instruction Bitmap | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Expand Down Expand Up @@ -821,7 +942,7 @@ The original packet must have C bit set to 0.
switch(es) set the M bit based on knowledge of the network topology
and "Switch ID, Ingress port ID, Egress port ID" tuples in the INT
metadata stack.
- R: Reserved bits.
- R (10b): Reserved bits.
- Hop ML (5b): Per-hop Metadata Length, the length of metadata in 4-Byte words
to be inserted at each INT hop.
- While the largest value of Per-hop Metadata Length is 31, an INT-capable
Expand Down