Skip to content

Add MQTT device firmware version #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DevOldSchool merged 3 commits into from
Jan 18, 2026
Merged

Add MQTT device firmware version #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
454a5eb
Added firmware version to MQTT device and ensured discovery is cleare…
DevOldSchool Jan 18, 2026
9ad3fbc
Switched legacy nginx build to manual as this doesn't change often.
DevOldSchool Jan 18, 2026
557a6d4
Removed imports that are not a part of this PR.
DevOldSchool Jan 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/build-legacy-nginx.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
name: Build and Push Legacy Nginx

# Switch to a manual step as this does not change often.
on:
workflow_dispatch:
release:
types: [ published ]

jobs:
build-and-push:
Expand Down
2 changes: 1 addition & 1 deletion .home-assistant/sensors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
WHEN labels.label LIKE '%h1%' OR labels.label LIKE '%h2%'
THEN (0.6 * 230 * readings.value) / 1000.0
WHEN labels.label LIKE '%h3%'
THEN (readings.value / 10.0)
THEN 1.0 * 230 * ((readings.value / 10.0) / 200)
ELSE readings.value
END AS W
FROM readings
Expand Down
77 changes: 55 additions & 22 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,20 +36,61 @@ chmod +x ./generate-certs.sh
This will create `server.key` and `server.crt` inside the `legacy-nginx` directory, where the `docker-compose.yml` file
expects to find them.

### 2. Run the Services
### 2. Configuration

The `hub-server` can be configured using environment variables in the `docker-compose.yml` file.

#### Identifying your Hub Version

Efergy Hubs (H1, H2, and H3) send data in different formats and to different endpoints. To ensure the `POWER_FACTOR` and data parsing are correct, you need to identify your hub version.

The `hub-server` will automatically log the detected version at the `INFO` level when it receives the first packet from your device.

1. **Check the logs**: View the logs of the `hub-server` container as the device sends data (usually every 6-30 seconds):
```shell
docker logs -f hub-server
```
2. **Look for the detection message**:
- `Detected Efergy Hub version: H1`
- `Detected Efergy Hub version: H2`
- `Detected Efergy Hub version: H3`

3. **Update Configuration**: Once identified, update your `docker-compose.yml` with the appropriate `POWER_FACTOR`:
- **H1 / H2**: `0.6`
- **H3**: `1.0`

*Note: If you don't see the detection message, you can temporarily set `LOG_LEVEL: DEBUG` in `docker-compose.yml` to see all incoming requests and identify the endpoint (`/recjson` = H1, `/h2` = H2, `/h3` = H3).*

#### Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `TZ` | Timezone (e.g., `Australia/Brisbane`) | `Australia/Brisbane` |
| `LOG_LEVEL` | Logging verbosity (`DEBUG`, `INFO`, `WARN`, `ERROR`) | `INFO` |
| `MAINS_VOLTAGE` | Local mains voltage (e.g., `230` for AU/UK, `120` for US) | `230` |
| `POWER_FACTOR` | Power factor (H1/H2 use `0.6`, H3 uses `1.0`) | `0.6` |
| `HISTORY_RETENTION_MONTHS` | How many months of data to keep (`0` = keep everything) | `0` |
| `MQTT_ENABLED` | Enable or disable MQTT publishing (`true`/`false`) | `false` |
| `MQTT_BROKER` | IP address or hostname of your MQTT broker | `10.0.0.220` |
| `MQTT_PORT` | Port for your MQTT broker | `1883` |
| `MQTT_USER` | Username for MQTT broker | `None` |
| `MQTT_PASS` | Password for MQTT broker | `None` |
| `HA_DISCOVERY` | Enable Home Assistant MQTT Discovery (`true`/`false`) | `false` |
| `ENERGY_MONTHLY_RESET` | Reset cumulative energy in HA each month (`true`/`false`) | `false` |
| `DEVICE_URL` | Link to device management (e.g., Portainer or NAS UI) | `.../powermeter_hub_server` |

### 3. Run the Services

With the certificates in place, you can start both services using Docker Compose.

```shell
# Build and start the containers in detached mode
docker-compose up --build -d
# Start the containers in detached mode
docker-compose up -d
```

This will:
* Build the `hub-server` image from its Dockerfile.
* Build the `legacy-nginx` image from its Dockerfile.
* Start both containers. The `legacy-nginx` service is exposed on port `443`.
* Automatically create the SQLite database on first run and mount the data directory for persistence.
* Automatically create the SQLite database on the first run and mount the data directory for persistence.

### 3. Redirect the Efergy Hub

Expand Down Expand Up @@ -126,22 +167,14 @@ If your Efergy Hub server is running on HA OS, you can integrate the readings in

1. **Configure Environment Variables** for MQTT

Update your environment variables in the `docker-compose.yml` file:
```
# Optional: logging level (DEBUG, INFO, WARN, ERROR)
LOG_LEVEL=INFO

# Enable MQTT (true/false)
MQTT_ENABLED=true

# MQTT broker details
MQTT_BROKER=homeassistant.local
MQTT_PORT=1883
MQTT_USER=mqtt-broker-username-here
MQTT_PASS=your-password-here

# Home Assistant MQTT Discovery
HA_DISCOVERY=true
Configure your environment variables in the `docker-compose.yml` file as described in the [Configuration](#configuration) section.
For Home Assistant with MQTT, you should at least set:
```yaml
MQTT_ENABLED: true
MQTT_BROKER: homeassistant.local
MQTT_USER: mqtt-broker-username-here
MQTT_PASS: your-password-here
HA_DISCOVERY: true
```

2. **Home Assistant Auto-Discovery**
Expand Down
12 changes: 8 additions & 4 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
services:
hub-server:
# build: ./hub-server
image: ghcr.io/devoldschool/powermeter_hub_server/hub-server:latest
container_name: hub-server
expose:
Expand All @@ -11,21 +10,26 @@ services:
TZ: Australia/Brisbane
# Logging
LOG_LEVEL: INFO
# Common voltages AU,UK = 230 | US = 120 https://en.wikipedia.org/wiki/Mains_electricity_by_country
MAINS_VOLTAGE: 230
# H1 and H2 devices used a power factor of 0.6, H3 uses 1.0
POWER_FACTOR: 0.6
# How many months of readings and aggregated values to keep, 0 = keep everything
HISTORY_RETENTION_MONTHS: 0
# MQTT
# MQTT https://www.home-assistant.io/integrations/mqtt/
MQTT_ENABLED: true
MQTT_BROKER: homeassistant.local
MQTT_PORT: 1883
MQTT_USER: mqtt-broker-username-here
MQTT_PASS: your-password-here
# Home Assistant
HA_DISCOVERY: true
# Setting to true will reset the cumulative energy value each month
# Setting to true will reset the cumulative energy value each month in Home Assistant
ENERGY_MONTHLY_RESET: false
# Device URL, set this to your management page, example Portainer or your NAS
DEVICE_URL: "https://github.com/DevOldSchool/powermeter_hub_server"

legacy-nginx:
# build: ./legacy-nginx
image: ghcr.io/devoldschool/powermeter_hub_server/legacy-nginx:latest
container_name: legacy-nginx
ports:
Expand Down
9 changes: 8 additions & 1 deletion hub-server/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# Hub server config
SERVER_PORT = int(os.getenv("SERVER_PORT", "5000"))
MAINS_VOLTAGE = int(os.getenv("MAINS_VOLTAGE", "230"))
# H1 and H2 devices use a power factor of 0.6, H3 uses 1.0
POWER_FACTOR = float(os.getenv("POWER_FACTOR", "0.6"))

# Logging level, values are DEBUG, INFO, WARN, ERROR, CRITICAL
Expand Down Expand Up @@ -34,6 +35,10 @@

# Formulas
# AC single phase milliamps to watts calculation P(W) = (PF × V(V) × I(mA)) / 1000
# {{ (__POWER_FACTOR__ * __MAINS_VOLTAGE__ * (value_json.value | float)) / 1000 }}

# AC three phase milliamps to watts calculation P(W) = (√3 x PF × VL-L(V) × I(mA)) / 1000
# {{ (1.732 * __POWER_FACTOR__ * __MAINS_VOLTAGE__ * (value_json.value | float)) / 1000 }}

POWER_VALUE_TEMPLATE_H1_RAW = os.getenv("POWER_VALUE_TEMPLATE_H1_RAW", "{{ (__POWER_FACTOR__ * __MAINS_VOLTAGE__ * (value_json.value | float)) / 1000 }}")
POWER_VALUE_TEMPLATE_H1 = POWER_VALUE_TEMPLATE_H1_RAW.replace("__MAINS_VOLTAGE__", str(MAINS_VOLTAGE)).replace("__POWER_FACTOR__", str(POWER_FACTOR))
Expand All @@ -43,9 +48,11 @@
POWER_VALUE_TEMPLATE_H2 = POWER_VALUE_TEMPLATE_H2_RAW.replace("__MAINS_VOLTAGE__", str(MAINS_VOLTAGE)).replace("__POWER_FACTOR__", str(POWER_FACTOR))
POWER_UNIT_OF_MEASUREMENT_H2 = os.getenv("POWER_UNIT_OF_MEASUREMENT_H2", "W")

POWER_VALUE_TEMPLATE_H3 = os.getenv("POWER_VALUE_TEMPLATE_H3", "{{ (value_json.value | float) / 10 }}")
POWER_VALUE_TEMPLATE_H3_RAW = os.getenv("POWER_VALUE_TEMPLATE_H3_RAW", "{{ __POWER_FACTOR__ * __MAINS_VOLTAGE__ * (((value_json.value | float) / 10) / 200) }}")
POWER_VALUE_TEMPLATE_H3 = POWER_VALUE_TEMPLATE_H3_RAW.replace("__MAINS_VOLTAGE__", str(MAINS_VOLTAGE)).replace("__POWER_FACTOR__", str(POWER_FACTOR))
POWER_UNIT_OF_MEASUREMENT_H3 = os.getenv("POWER_UNIT_OF_MEASUREMENT_H3", "W")

ENERGY_SENSOR_LABEL = "energy_consumption"
ENERGY_NAME = os.getenv("ENERGY_NAME", "Energy consumption")
ENERGY_ICON = os.getenv("ENERGY_ICON", "mdi:lightning-bolt")
ENERGY_DEVICE_CLASS = os.getenv("ENERGY_DEVICE_CLASS", "energy")
Expand Down
39 changes: 30 additions & 9 deletions hub-server/database.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,8 @@ def setup(self) -> None:
cursor.execute("""
CREATE TABLE IF NOT EXISTS labels (
label_id INTEGER PRIMARY KEY AUTOINCREMENT,
label STRING UNIQUE
label STRING UNIQUE,
firmware_version STRING
)
""")
cursor.execute("""
Expand Down Expand Up @@ -147,10 +148,17 @@ def setup(self) -> None:
ON energy_hourly (hour_start)
""")

# Add a firmware_version column if it doesn't exist
cursor.execute("PRAGMA table_info(labels)")
columns = [row[1] for row in cursor.fetchall()]
if "firmware_version" not in columns:
logging.info("Adding 'firmware_version' column to labels table")
cursor.execute("ALTER TABLE labels ADD COLUMN firmware_version STRING")

logging.debug("Database setup complete.")


def _get_or_create_label_id(self, cursor: sqlite3.Cursor, label: str) -> int:
def _get_or_create_label_id(self, cursor: sqlite3.Cursor, label: str, firmware_version: str) -> int:
"""
Gets a label_id from the cache or database.
If the label doesn't exist, it's created.
Expand All @@ -171,22 +179,31 @@ def _get_or_create_label_id(self, cursor: sqlite3.Cursor, label: str) -> int:
return self._label_cache[label]

# If not in cache, check database
cursor.execute("SELECT label_id FROM labels WHERE label=?", (label,))
cursor.execute("SELECT label_id, firmware_version FROM labels WHERE label=?", (label,))
row = cursor.fetchone()

if row:
label_id = row[0]
label_id, existing_firmware = row
# Update firmware only if it's provided and different
if firmware_version is not None and firmware_version != existing_firmware:
cursor.execute(
"UPDATE labels SET firmware_version=? WHERE label_id=?",
(firmware_version, label_id)
)
else:
# Not in DB, so create it
cursor.execute("INSERT INTO labels(label) VALUES (?)", (label,))
cursor.execute(
"INSERT INTO labels(label, firmware_version) VALUES (?, ?)",
(label, firmware_version)
)
label_id = cursor.lastrowid
logging.debug(f"Created new label '{label}' with id {label_id}")

self._label_cache[label] = label_id
return label_id


def log_data(self, label: str, value: float, timestamp: Optional[int] = None) -> None:
def log_data(self, label: str, value: float, firmware_version: str, timestamp: Optional[int] = None) -> None:
"""
Logs a new data point to the database.

Expand All @@ -196,6 +213,7 @@ def log_data(self, label: str, value: float, timestamp: Optional[int] = None) ->
Args:
label: The string identifier for the data (e.g., 'efergy_h2_123456').
value: The floating-point value of the reading.
firmware_version: The firmware version of the hub.
timestamp: The Unix timestamp. If None, current time is used.
"""
if timestamp is None:
Expand All @@ -204,7 +222,7 @@ def log_data(self, label: str, value: float, timestamp: Optional[int] = None) ->
try:
with self._get_connection() as conn:
cursor = conn.cursor()
label_id = self._get_or_create_label_id(cursor, label)
label_id = self._get_or_create_label_id(cursor, label, firmware_version)

# Insert the actual reading
cursor.execute(
Expand All @@ -225,8 +243,11 @@ def get_all_labels(self):
try:
with self._get_connection() as conn:
cursor = conn.cursor()
cursor.execute("SELECT label FROM labels ORDER BY label ASC")
return [row[0] for row in cursor.fetchall()]
cursor.execute("SELECT label, firmware_version FROM labels ORDER BY label ASC")
return [
{"label": row[0], "firmware_version": row[1]}
for row in cursor.fetchall()
]
except Exception as e:
logging.error(f"Failed to fetch labels: {e}")
return []
Expand Down
20 changes: 14 additions & 6 deletions hub-server/hub_server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ def __init__(self,
self.database = database
self.mqtt_manager = mqtt_manager
self.published_discovery = set()
self.detected_versions = set()
super().__init__(server_address, request_handler_class, bind_and_activate)


Expand Down Expand Up @@ -178,6 +179,8 @@ def do_POST(self):
post_data_bytes = self.rfile.read(content_length)
logging.debug(f">>> POST body: {post_data_bytes.decode('utf-8', 'ignore')}")

firmware_version = self.headers.get("X-Version", "")

db = getattr(self.server, "database", None)
if not db:
logging.error("Database not initialized on server instance.")
Expand All @@ -190,15 +193,15 @@ def do_POST(self):
logging.debug(f"Received ping from sensors: {sensor_ids}")
elif parsed_url.path in ["/h2", "/h3"]:
hub_version = parsed_url.path.strip("/")
self.process_sensor_data(post_data_bytes, hub_version, db)
self.process_sensor_data(post_data_bytes, hub_version, firmware_version, db)
elif parsed_url.path == '/recjson':
# v1 hub sends URL-encoded form data: json=<pipe-delimited-data>
hub_version = 'h1'
decoded_body = post_data_bytes.decode('utf-8', 'ignore')
if decoded_body.startswith('json='):
# Extract the actual sensor data
sensor_data = decoded_body[5:] # Skip 'json='
self.process_sensor_data(sensor_data.encode('utf-8'), hub_version, db)
self.process_sensor_data(sensor_data.encode('utf-8'), hub_version, "", db)
else:
logging.warning(f"Unexpected /recjson body format: {decoded_body[:100]}")
else:
Expand Down Expand Up @@ -247,9 +250,13 @@ def do_CONNECT(self):
self._send_response(200, b"success")


def process_sensor_data(self, post_data_bytes: bytes, hub_version: str, database: Database):
def process_sensor_data(self, post_data_bytes: bytes, hub_version: str, firmware_version: str, database: Database):
"""Parses and logs sensor data from the POST body."""
parsed_results = parse_sensor_payload(post_data_bytes, hub_version)
parsed_results = parse_sensor_payload(post_data_bytes, hub_version, firmware_version)

if hub_version.upper() not in self.server.detected_versions:
logging.info(f"Detected Efergy Hub version: {hub_version.upper()}")
self.server.detected_versions.add(hub_version.upper())

for data in parsed_results:
try:
Expand All @@ -264,12 +271,13 @@ def process_sensor_data(self, post_data_bytes: bytes, hub_version: str, database
sid = data["sid"]
label = data["label"]
value = data["value"]
firmware_version = data.get("firmware_version", firmware_version)

logging.debug(f"Logging sensor: {label}, raw: {value}")
database.log_data(label, value)
database.log_data(label, value, firmware_version)

# Publish power reading
self.server.mqtt_manager.publish_power(label, sid, hub_version, value)
self.server.mqtt_manager.publish_power(label, sid, hub_version, firmware_version, value)

except Exception as e:
logging.error(f"Unexpected error processing parsed data {data}: {e}")
Expand Down
Loading