Optimizer returns Infeasible for a single outlier value in household demand

[melle]

Problem

The optimizer suddenly stopped working for me and the only clue was the log message "Optimizer: Infeasible". This went on for a few days. There was no configuration change or anything that could explain the changed behavior.

My token-burning robot pointed out that a single outlier in time_series.gt makes the optimizer return Infeasible, even when everything else is sound. The model's M variable is initialized with a default value of 1e6. Once a slot needs more than 1_000_000 Wh of grid import, the solve fails.

Of course this value is absurdly high for a household. The source of the value is my EMS / inverter; in the past 6 months, I've found 4 abnormal values. Unfortunately, this is something I as a user cannot control. A single transient glitch (counter rollover, reset, spurious sample) inflates one value, and because the profile is a multi-day average, it then breaks every run until the bad sample ages out (up to ~30 days). This is a hard, persistent outage from an invisible fault.

Cause

There is the flow-direction constraint in src/optimizer/optimizer.py, where the grid import is implicitly capped by M:

self.problem += self.variables['n'][t] <= self.M * (1 - self.variables['y'][t])

With the energy balance … + n[t] == … + gt[t], any gt[t] > M is unsatisfiable, so the whole solve is Infeasible.

Reproduce

I sent a 192-slot request with two batteries, one slot contained the bad value: gt[42] = 1590737.6 (Wh, ~6.4 MW/15 min). It repeats at slot 138 because the 96-slot day profile is tiled across the 192-slot horizon.

• sent as is -> Infeasible
• gt clipped to <= 1e6 -> Optimal

I observed this both on a local build and on optimizer.evcc.io.

Request

{"strategy": {"charging_strategy": "none", "discharging_strategy": "discharge_before_import"}, "eta_c": 0.9, "eta_d": 0.9, "grid": {}, "batteries": [{"charge_from_grid": true, "c_min": 1380, "c_max": 11040, "d_max": 0, "s_min": 0, "s_max": 60000, "s_initial": 31800, "p_a": 0.000228987}, {"c_min": 0, "c_max": 5000, "d_max": 5000, "s_capacity": 10000, "s_min": 2000, "s_max": 9500, "s_initial": 7500, "p_a": 0.000228987}], "time_series": {"dt": [836, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900], "ft": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 39.977802, 123.51256, 216.441, 333.3839, 478.5338, 637.41595, 796.4402, 948.4937, 1096.6368, 1248.511, 1400.7466, 1539.8947, 1520.6185, 1486.0309, 1702.6925, 1909.8389, 2005.7045, 2001.2222, 1932.3643, 1969.2408, 2016.4004, 2070.2344, 2300.875, 2409.5298, 2457.2327, 2396.935, 2397.1355, 2536.1482, 2576.002, 2534.3125, 2168.6165, 1962.9708, 2095.5, 2118.5166, 2306.8047, 2739.605, 2994.0627, 2787.2693, 2354.0488, 2353.7446, 2564.1147, 2564.2925, 2629.989, 2745.5928, 2728.518, 2632.6313, 2524.6736, 2426.664, 2274.182, 2121.2832, 1849.8931, 1452.3761, 1119.7511, 931.13806, 846.5902, 1114.2628, 1333.3256, 1096.2434, 839.7334, 655.83136, 450.3288, 274.88586, 162.47734, 82.114944, 23.716476, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 48.64778, 128.02835, 218.34856, 346.4732, 486.90723, 637.16486, 792.85645, 947.14417, 1100.0029, 1247.7168, 1391.1373, 1539.402, 1694.2136, 1840.8918, 1972.7554, 2101.6472, 2233.7146, 2363.3884, 2489.229, 2608.4607, 2707.724, 2661.326, 2659.9243, 2745.0857, 2857.6711, 3084.9436, 3229.258, 3282.37, 3327.7722, 3370.5798, 3379.5886, 3228.3118, 3002.6453, 3083.2422, 3216.7673, 3280.463, 3344.239, 3147.7424, 2805.0317, 2728.6196, 2643.2253, 2091.063, 1433.0349, 575.6042, 104.4518, 747.7852, 1725.9738, 2213.897, 1889.4774, 1717.8846, 1720.9277, 1481.0529, 1451.6934, 1496.9896, 1533.9843, 1455.903, 1273.7245, 1076.4614, 944.17145, 820.09155, 653.7127, 461.18857, 286.797, 147.20808, 38.52187, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], "gt": [153.18826, 149.41406, 142.8327, 138.95538, 132.86967, 121.87293, 117.52625, 116.83201, 113.04073, 109.92602, 113.93937, 116.622314, 119.54532, 115.73791, 117.96523, 102.81362, 421.47934, 108.156906, 102.61196, 104.04698, 111.60087, 112.91344, 114.607704, 118.300964, 116.222694, 117.595116, 120.69791, 147.29416, 172.26561, 175.16829, 194.3625, 167.28473, 156.28201, 185.46138, 209.93407, 216.49504, 245.47766, 234.24677, 278.80374, 282.54236, 286.2364, 359.69653, 1590737.6, 237.11137, 283.54062, 282.0038, 884.594, 316.24377, 283.6969, 311.01895, 300.82245, 320.50656, 283.0136, 341.40622, 331.39835, 332.0016, 261.28595, 263.35202, 300.71408, 301.8121, 305.6299, 314.90225, 341.29306, 343.01514, 445.36728, 321.22867, 284.32724, 261.96912, 318.90146, 319.28342, 284.8267, 235.36584, 222.45235, 270.65665, 252.45891, 294.18994, 279.60944, 285.48102, 267.6564, 338.03616, 380.73367, 345.55405, 346.73608, 287.8207, 216.94576, 195.0212, 225.27817, 263.46017, 222.51614, 188.66417, 181.93085, 200.81438, 204.21544, 197.03679, 186.71611, 162.98248, 164.91559, 149.41406, 142.8327, 138.95538, 132.86967, 121.87293, 117.52625, 116.83201, 113.04073, 109.92602, 113.93937, 116.622314, 119.54532, 115.73791, 117.96523, 102.81362, 421.47934, 108.156906, 102.61196, 104.04698, 111.60087, 112.91344, 114.607704, 118.300964, 116.222694, 117.595116, 120.69791, 147.29416, 172.26561, 175.16829, 194.3625, 167.28473, 156.28201, 185.46138, 209.93407, 216.49504, 245.47766, 234.24677, 278.80374, 282.54236, 286.2364, 359.69653, 1590737.6, 237.11137, 283.54062, 282.0038, 884.594, 316.24377, 283.6969, 311.01895, 300.82245, 320.50656, 283.0136, 341.40622, 331.39835, 332.0016, 261.28595, 263.35202, 300.71408, 301.8121, 305.6299, 314.90225, 341.29306, 343.01514, 445.36728, 321.22867, 284.32724, 261.96912, 318.90146, 319.28342, 284.8267, 235.36584, 222.45235, 270.65665, 252.45891, 294.18994, 279.60944, 285.48102, 267.6564, 338.03616, 380.73367, 345.55405, 346.73608, 287.8207, 216.94576, 195.0212, 225.27817, 263.46017, 222.51614, 188.66417, 181.93085, 200.81438, 204.21544, 197.03679, 186.71611, 162.98248], "p_E": [8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05, 8.016667e-05], "p_N": [0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257, 0.000257]}}

Response (HTTP 200)

{"status": "Infeasible", "objective_value": null, "limit_violations": {"grid_import_limit_exceeded": false, "grid_export_limit_hit": false}, "batteries": [], "grid_import": [], "grid_export": [], "flow_direction": [], "grid_import_overshoot": [], "grid_export_overshoot": []}

Side note on the response: grid_import_limit_exceeded: false is a bit misleading. The M cap acts as an undeclared grid-import limit, but unlike p_max_imp it has no reporting. So the very condition the flag exists to surface (an import limit being hit) is happening invisibly, while the flag says false. If the main issue is fixed, this case would return Optimal with grid_import_limit_exceeded: true instead of Infeasible with false.

Options to fix

1. Do not bound grid import by the optimizer's M. Leave n[t] effectively unbounded, or size M from the actual data.
2. Validate and clamp inputs before solving, and reject clearly impossible values with a descriptive error (offending field + slot index) instead of a bare Infeasible.
3. In evcc, homeProfile could clamp outlier slots so one bad meter sample doesn't poison the profile for weeks.
4. Manually delete bad samples in the evcc database and hope for the EMS manufacturer to fix the bad readings with a firmware update.

I'm not sure which fix would fit the project's goals / architecture best. I can try to create a PR based on the feedback here.

Disclaimer: The fault analysis was done with Claude Code, the issue description was written by me (a human being). From my understanding, this is compatible with the AI guidelines of the project. If you think otherwise, please close this issue. The intention is to avoid such an outage in the future for me and other users.

[andig]

Wondering if we just shouldn‘t average the household demand and instead tage something like median instead of fixing this on the optimizer side? This should imho make it more stable wrt putlier values?