Extending REST API to manipulate active/pending dataset

[agners]

This is OTBR posix specific, but I am starting this discussion here as the ot-br-posix has no GitHub Discussions.

I've recently added a REST API to set the active dataset by adding a REST API which takes a dataset in TLV form, see openthread/ot-br-posix#1658. This added the following endpoint:

• /node/active-dataset-tlvs [GET, PUT]

We realized we need more capabilities for our needs. In particular we'd like to have an API to form a new network from individual parameters (network name, channel etc.).

I've looked into openthread/ot-br-posix#537, a previous attempt to extend the REST API. The previous attempt implements the following REST endpoints:

• /networks [GET, POST (see Resource::PostNetworksCallback())
• /networks/current [GET, PUT (see Resource::PutCurrentNetworkCallback()), OPTIONS]
• /networks/current/commission
• /networks/current/prefix

This previous attempts uses APIs such as otThreadSetExtendedPanId.

Looking at the code two questions came up which I think I know the answer, but prefer to have confirmation on my assumptions:

1. I've noticed that calling APIs like otThreadSetExtendedPanId clear the active and pending dataset. Mixing these APIs with dataset APIs seems not a good idea. Is my assumption correct that these are backwards compatibility APIs and probably shouldn't be used for a new implementation?
2. What is the exact role of a pending dataset? Is it just a means to keep a persistent state of a second, non-active dataset on the OT device?

The dataset CLI essentially implements a set of commands to manipulation a dataset. The CLI is stateful. Only one dataset can be manipulated at a time. With init new a new dataset with sensible default values (some are random like the network keys etc.) and with init active a copy of the current active dataset is initialized for CLI manipulation. The commit active stores it as active dataset.

To implement a dataset API in REST, two routes could be followed:

• Similar to CLI: Allow to create new datasets, allow to store datasets as active/pending
• More resource oriented: Have pending/active endpoints and allow to manipulate those

I tend to lean towards resource oriented (as this feels more RESTful to me)

• /node/dataset/active [GET, POST, PUT]
• /node/dataset/pending [GET, POST, PUT]

POST would initialize a new dataset, and update that dataset with whatever data is provided as body. PUT would not initialize a new dataset, just update the existing active/pending dataset with the new values. If no active dataset iss present PUT would return with an error.

Also DELETE could be implemented to clear the dataset.

We also need a way to start the Thread network via REST API. The previous attempt seems to start the interface/Thread network on PUT or POST. We could do the same. However, this would not reflect what the CLI is doing (commit active does not start the OTBR). But then, at startup, the OTBR is starting the interface/Thread network implicitly (via TryResumeNetwork()) at startup as well...

If an implicit state change on manipulating the active dataset is not acceptable, we'd need to have a REST API to manipulate the node state. I have no good idea how that could look like. I tend to lean to a implicit start. From what I understand manipulating the active dataset also implicitly activates changes (in otDatasetSetActive), so implicitly starting doesn't feel far off.

Any thoughts/guidance welcome!

[wgtdkp]

I've noticed that calling APIs like otThreadSetExtendedPanId clear the active and pending dataset. Mixing these APIs with dataset APIs seems not a good idea. Is my assumption correct that these are backwards compatibility APIs and probably shouldn't be used for a new implementation?

Correct, we should use otDatasetSetActive/otDatasetSetpending in new code.

What is the exact role of a pending dataset? Is it just a means to keep a persistent state of a second, non-active dataset on the OT device?

Active Operational Dataset includes the Thread Network Key and parameters which are currently effective in the Thread network. Pending Operational Dataset is a operational dataset which will take effect in a near future which is specified by the Delay Timer fields. This is useful because if you change directly the Network Key or Channel of the Active Operational Dataset on the BR, it will effectively disconnected from its previous network and forms its own partition with the new Active Operational Dataset, leaving it's children orphaned. But if you set the new Network Key in a Pending Operational Dataset, it will be registered to the LEADER and propagated to all Thread devices and all Thread devices will turn the Pending Operational Dataset to Active Operational Dataset in a near future which is specified by the Delay Timer of the Pending Operational Dataset.

We suggest changing Active Operational Dataset directly only for testing purpose or when there is no existing network (e.g. the initial DISABLED state). You can use a small Delay Timer to update the dataset faster but at the risk of leaving some sleepy devices disconnected.

I tend to lean towards resource oriented (as this feels more RESTful to me)

• /node/dataset/active [GET, POST, PUT]
• /node/dataset/pending [GET, POST, PUT]

It sounds good to me.

If an implicit state change on manipulating the active dataset is not acceptable, we'd need to have a REST API to manipulate the node state. I have no good idea how that could look like. I tend to lean to a implicit start. From what I understand manipulating the active dataset also implicitly activates changes (in otDatasetSetActive), so implicitly starting doesn't feel far off.

I think providing a API for the user to controller the Thread network state is useful. If the Thread device state is "DISABLED" or "DETACHED", we allow direct modification to Active Operational Dataset. Otherwise, we accepts only changes via the Pending Operational Dataset. So if the user doesn't care about the rest of the device in the network (usually in tests or demo), he/she can disable the network before changing the Active Operational Dataset.

[wgtdkp]

What is the recommended delay for such an operation? I guess it depends on how long it usually takes for the new settings to propagate? Probably mostly depending on how often battery operated devices check in?

Yes, it depends on how long the sleepy end device will wake up again. The Thread specification has some suggestions:

Constant Name	Description	Default
MAX_DELAY_TIMER	Maximum Delay Timer value for a Pending Operational Dataset.	72 hours
DELAY_TIMER_MINIMAL	Minimum Delay Timer value for a Pending Operational Dataset (as set by the Commissioner and enforced by the Leader).	30 seconds
DELAY_TIMER_DEFAULT	Default Delay Timer value for a Pending Operational Dataset generated by the Leader. Minimum value enforced by the Leader when updating the Master Key.	5 minutes
DELAY_TIMER_EXTENDED	Recommended minimum Delay Timer value for a Pending Operational Dataset set by a Commissioner when updating the Network Master Key.	8 hours

A smart system can evaluate the minimum Delay Timer value based on the Max Child Timeout values collected in Network Diagnostic responses.

[wgtdkp]

For real use cases, the Active Operational Dataset will generally be set only once when the BR device is being set up. A user/commissioner may rotate the Network Key monthly (or longer) for security.

[jwhui]

Setting the Active Operational Dataset is OK as long as the parameters changing do not affect connectivity. The parameters that affect connectivity include:

• Channel
• Mesh-Local Prefix
• Network Key
• PAN ID

Thread devices compare Operational Datasets using the respective Timestamp values. Active Timestamp for comparing Active Operational Datasets and Pending Timestamp for comparing Pending Operational Datasets. The Pending Timestamp is useful when a pending change already exists in the network and you want to make an update to that pending change.

Another use case for setting the Active Operational Dataset is when you want to merge separate Thread networks into one. Today, the best way to do that is by taking the Active Operational Dataset from one network an applying that to another.

I prefer to separate configuring Operational Datasets with changing state of Thread interface. One use case for configuring Operational Datasets separately is simply to provision the Thread device without actually enabling Thread.

[agners]

Interesting information. I guess we could reject a REST PUT operation to the Active Operational Dataset based on what fields are being provided.

Thread devices compare Operational Datasets using the respective Timestamp values. Active Timestamp for comparing Active Operational Datasets

So when the pending is applied to active, it is this timer which is used, correct?

Should the REST API maybe simply set those timestamps by default?

Another use case for setting the Active Operational Dataset is when you want to merge separate Thread networks into one. Today, the best way to do that is by taking the Active Operational Dataset from one network an applying that to another.

If that Thread network already has devices connected, would changing the Active Operational Dataset leave them behind in their own partition? Can this use case also fullfilled through changing the Pending Operational Dataset?

[jwhui]

So when the pending is applied to active, it is this timer which is used, correct?

The Pending Operational Dataset includes both Active Timestamp and Pending Timestamp. The Active Timestamp is used to compare when applying Pending to Active. The Pending's Active Timestamp must be greater, unless the Network Key changes, in which case the Active Timestamp can be anything.

Should the REST API maybe simply set those timestamps by default?

I think that could work if the Timestamp values are not provided.

If that Thread network already has devices connected, would changing the Active Operational Dataset leave them behind in their own partition? Can this use case also fullfilled through changing the Pending Operational Dataset?

By "applying" I mean taking the Active Operational Dataset from one network, wrapping that in a Pending Operational Dataset by adding Pending Timestamp and Delay Timer values and applying the resulting Pending Operational Dataset to another network.