Validate unit types in flex context schema

[nhoening]

For each field, we can note if the field needs a power, energy, currency or energy price unit.

Then, we can amend the FlexContextSchema with a check for that (checking the fixed value or the sensor's unit). If the check fails throw a validation error with a helpful message (including an example, e.g. kW for power, or EUR/kWh for energy price).

The util functions are already in place in flexmeasures/utils/unit_utils.py, e.g. is_power_unit().

Let's check both the field values and sensor for fitting units.

My take on what should be checked for what field:

• inflexible device sensors: should all be power or energy sensors
• consumption-price: energy price
• production-price: energy price
• site-power-capacity: power
• site-consumption-capacity: power
• site-consumption-breach-price: energy price
• site-production-capacity: power
• site-production-breach-price: energy price
• site-peak-consumption: power
• site-peak-consumption-price: energy price
• site-peak-production: power
• site-peak-production-price: energy price

[Flix6x]

Just a quick note: what about scheduling against emission intensity?

[nhoening]

Just a quick note: what about scheduling against emission intensity?

Ah, so everything that we call "price" here could be "CO2/some energy", as well? What is the unit for CO2?

We could add another util function is_emission_intensity_unit()?

[Flix6x]

I suggest implementing your suggestions for the DBFlexContextSchema only. That way, API users can still request schedules that minimize costs expressed in arbitrary units, such as kg of emissions.

[Flix6x]

Also related: we've encountered practical use cases for expressing SoC units in arbitrary units, such as temperature units or percentages (something we don't currently support). Correspondingly, that would necessitate arbitrary conversion efficiencies between power and SoC deltas.

[nhoening]

It seems like that will be relevant for the flex-model right?
I believe only need power units for the capacities in the context. But maybe I'm wrong?

[Flix6x]

One more note: the breach prices and peak prices are capacity prices (EUR/MW rather than EUR/MWh).

It seems like that will be relevant for the flex-model right?

Yes, you are right.

I believe only need power units for the capacities in the context.

Please clarify.

[nhoening]

One more note: the breach prices and peak prices are capacity prices (EUR/MW rather than EUR/MWh).

So we might need another util function to check for these.

I believe only need power units for the capacities in the context.

Please clarify.

is_power_unit() should suffice as check for everything in the context not called "price".

[joshuaunity]

Following the discussions, the changes/modifications this issue suggested should be done in the DBFlexContextSchema, right? Or just in the FlexContextSchema?

[nhoening]

I believe the FlexContextSchema would be right for this, or do you see a reason otherwise?

I'm not sure why forbid_time_series_specs() and forbid_fixed_prices() only apply to the DB actually, maybe I'm out of the loop there.

[joshuaunity]

ithink we did that to avoid clashes with other parts of the code that are using the FlexContextSchema with these values we are forbiding

[joshuaunity]

@nhoening i just came across something. ltts use the prices field for this example (the prices field expect the first two that only takes sensors for now)

The other four values can take a sensor. That will then mean that this validation to check for valid energy_price unit will only take place if we confirm the value is not a sensor but rather a fixed value.

Or if its a sensor we go check if the sensor is of that unit.

This also applies to the power fields as well

[nhoening]

I assume "Itts" means "Let's"?

Yes, as the issue description says, we need to check either the fixed value or the sensor's unit.

Or did I miss another question?

[Flix6x]

I believe the FlexContextSchema would be right for this, or do you see a reason otherwise?

One reason is that scheduling against emission intensity should remain supported:

I suggest implementing your suggestions for the DBFlexContextSchema only. That way, API users can still request schedules that minimize costs expressed in arbitrary units, such as kg of emissions.

So in my opinion, the unit validation on price fields should be limited to the DBFlexContextSchema.

I'm not sure why forbid_time_series_specs() and forbid_fixed_prices() only apply to the DB actually, maybe I'm out of the loop there.

• For forbid_time_series_specs() one (the?) reason is that I considered having time series specs stored as attributes a nonsensical use case. If I should consider it longer, I might conjure up a use case, but at the time it wasn't worth the trouble of going into the direction of also adding UI support for defining an arbitrary time series.
• For forbid_fixed_prices(), I mention one (the?) reason here: it added unnecessary complexity to the scope of the PR that created the DBFlexContextSchema, which was to move some flex-context fields out of their own db columns and into the flex-context db column. To clarify: the former columns also only supported setting prices by referring to a sensor, rather than fixed quantities.

[nhoening]

Thanks for the explanations.

Considerations about this (not relevant for this PR): We might live with some differences between expressiveness in DB fields and what the API enables for some time.

• We should probably make a list of these differences
• The documentation of setting flex-context and flex-model values will be challenging, but also a high-value issue. Not just due to these differences, but that adds to the complexity.