Add mechanism to fetch tags associated with metric/component data

[tiyash-basu-frequenz]

What's needed?

We need a mechanism to fetch tags associated with metric/component data while reading from our APIs. Such tags are represented by arbitrary payloads (JSONs) that have a one-to-one relation with sample timestamp.

This could be used to indicate the last activity or last command that was issued to an API, e.g., a charge command to the Dispatch API or Microgrid API. (such tags could be set on these APIs through their own RPCs).

As of now, the tag is intended to be present in only one outgoing data sample, and not in the succesive data samples.

Proposed solution

Solution 1

One way of doing this would be to add a tag field in the MetricSample message. This field would store a JSON to be able to represent arbotrary payloads:

google.protobuf.Struct tag = 6;

(also, suggesting tag as the field name because "payload" would generally refer to the whole message, so could be confusing in this context)

This would also affect the contents ComponentData message. If this solution is accepted, then the message would look like the following:

{
  "component_id": 13,
  "metric_samples": [
    {
      "sampled_at": "2023-10-01T00:00:00Z",
      "tag": { "command_issued_by": "dispatch_actor_123" },
      "metric": "DC_VOLTAGE",
      "sample": {},
      "bounds": {}
    },
    {
      "sampled_at": "2023-10-01T00:00:00Z",
      "tag": { "command_issued_by": "dispatch_actor_123" },
      "metric": "DC_CURRENT",
      "sample": {},
      "bounds": {}
    },
    {
      "sampled_at": "2023-10-01T00:00:01Z",
      "tag": { "command_issued_by": "dispatch_actor_123" },
      "metric": "DC_VOLTAGE_V",
      "sample": {},
      "bounds": {}
    },
    {
      "sampled_at": "2023-10-01T00:00:01Z",
      "tag": { "command_issued_by": "dispatch_actor_123" },
      "metric": "DC_CURRENT",
      "sample": {},
      "bounds": {}
    }
  ],
  "states": [
    {
      "sampled_at": "2023-10-01T00:00:00Z",
      "tag": { "command_issued_by": "dispatch_actor_123" },
      "states": [],
      "warnings": [],
      "errors": []
    },
    {
      "sampled_at": "2023-10-01T00:00:01Z",
      "tag": { "command_issued_by": "dispatch_actor_123" },
      "states": [],
      "warnings": [],
      "errors": []
    }
  ]
}

One downside of this solution is that we replicate the tag multiple times for each metric, given the way we have organize the data structure.

Solution 2

Similar to solution1, but structured a bit differently:

{
  "component_id": 13,
  "tags": [
    {"sampled_at": "2023-10-01T00:00:00Z", "tag": { "command_issued_by": "dispatch_actor_123" }},
    {"sampled_at": "2023-10-01T00:00:01Z", "tag": { "command_issued_by": "dispatch_actor_123" }}
  ],
  "metric_samples": [
    {
      "sampled_at": "2023-10-01T00:00:00Z",
      "metric": "DC_VOLTAGE",
      "sample": {},
      "bounds": {}
    },
    {
      "sampled_at": "2023-10-01T00:00:00Z",
      "metric": "DC_CURRENT",
      "sample": {},
      "bounds": {}
    },
    {
      "sampled_at": "2023-10-01T00:00:01Z",
      "metric": "DC_VOLTAGE_V",
      "sample": {},
      "bounds": {}
    },
    {
      "sampled_at": "2023-10-01T00:00:01Z",
      "metric": "DC_CURRENT",
      "sample": {},
      "bounds": {}
    }
  ],
  "states": [
    {
      "sampled_at": "2023-10-01T00:00:00Z",
      "states": [],
      "warnings": [],
      "errors": []
    },
    {
      "sampled_at": "2023-10-01T00:00:01Z",
      "states": [],
      "warnings": [],
      "errors": []
    }
  ]
}

This would need a protobuf message:

message SampleTimestampTag {
    google.protobuf.Timestamp sampled_at = 1;
    google.protobuf.Struct tag = 2;
}

Use cases

In addition to the broader use-case of tagging inverter activity, there are some cases where the AC electricity data is not enough to identify what's going on. E.g., in case of the FCR app, this could help determine working-point-offset (assuming the app uses it properly). In this example, obtaining the working-point-offset just from AC data would be difficult, if not impossible.

Alternatives and workarounds

No response

Additional context

No response

[llucax]

Not trying to start a big bike-shed discussion, but what do you think about metadata instead of tag?

[tiyash-basu-frequenz]

but what do you think about metadata instead of tag?

Sounds good to me!

[tiyash-basu-frequenz]

I moved it to the backlog, since this has been de-prioritised for now.

[tiyash-basu-frequenz]

I added the following to the issue description:

As of now, the tag is intended to be present in only one outgoing data sample, and not in the succesive data samples.