Skip to content

Building a Custom Input Module

Kyle Gabriel edited this page Jul 25, 2020 · 20 revisions

An Input module import system has been implemented in Mycodo to allow new Inputs to be created within a single file and imported into Mycodo for use within the system. An Input module can be imported into Mycodo on the Configure -> Input page and will appear in the Add Input dropdown selection list on the Setup -> Data page.

Bult-in Modules: Mycodo/mycodo/inputs

Example module that generates random data: minimal_humidity_temperature.py

Example module with all options: example_all_options_temperature.py

This will be a brief tutorial covering the components of a simple Input module with the purpose of helping you develop your own. I'll be using the MCP9808 temperature sensor Input module (located at Mycodo/mycodo/inputs/mcp9808.py) for this example, as it contains the minimum components necessary for an Input module to operate. Following this tutorial will be additional options and functions available to Input modules for advanced use.

The first part of an input module is the measurements_dict dictionary. measurements_dict contains all measurements that you would wish to store in the measurements database. This could include measurements that the sensor is capable of measuring in addition to measurements that could be calculated. For the MCP9808, since only temperature is measured, measurements_dict looks like this:

measurements_dict = {
    0: {
        'measurement': 'temperature',
        'unit': 'C'
    }
}

Had this sensor been capable of measuring more than one condition or if calculated measurements were desired, additional dictionary entries could be added with an increasing dictionary key (0, 1, 2, etc.). This number is presented to the user as the channel of the Input. For instance, a temperature/humidity sensor may be able to measure only temperature and humidity, but dew point could be calculated from these two measurements. If dew point was desired to be stored in the measurement database, this would need to be included in measurements_dict, like so:

measurements_dict = {
    0: {
        'measurement': 'temperature',
        'unit': 'C'
    },
    1: {
        'measurement': 'humidity',
        'unit': 'percent'
    },
    2: {
        'measurement': 'dewpoint',
        'unit': 'C'
    }
}

Additionally, measurements_dict also requires both the units and measurements to exist in the measurement/unit database. You are required to add measurements and units from the Configure -> Measurements page if they don't already exist, prior to importing the Input module.

Next is the INPUT_INFORMATION dictionary, where all information about an input is contained. Let's take a look at this dictionary to see what's required for it to operate.

INPUT_INFORMATION = {
    'input_name_unique': 'MCP9808',
    'input_manufacturer': 'Microchip',
    'input_name': 'MCP9808',
    'input_library': 'Adafruit_MCP9808',
    'measurements_name': 'Temperature',
    'measurements_dict': measurements_dict,

    'dependencies_module': [
        ('pip-pypi', 'Adafruit_GPIO', 'Adafruit_GPIO'),
        ('pip-git', 'Adafruit_MCP9808', 'git://github.com/adafruit/Adafruit_Python_MCP9808.git#egg=adafruit-mcp9808'),
    ],

    'interfaces': ['I2C'],
    'i2c_location': ['0x18', '0x19', '0x1a', '0x1b', '0x1c', '0x1d', '0x1e', '0x1f'],
    'i2c_address_editable': False,

    'options_enabled': [
        'i2c_location',
        'period',
        'pre_output'
    ],
    'options_disabled': ['interface']
}

input_name_unique is an ID for the input that must be unique from all other inputs, as this is what's used to differentiate it from others. If you're copying an Input module and using it as a template, you must change at least this name to something different before it can be imported into Mycodo.

input_manufacturer is the manufacturer name, input_name is the input name, input_library is the library or libraries used to read the sensor, and measurements_name is the measurement or measurements that can be obtained from the input. These are all used to generate the Input name that's presented in the list of Inputs that can be added on the Data page. These names can be anything, but it's suggested to follow the format of the other Inputs.

measurements_dict merely points to our previously-created dictionary of measurements, measurements_dict, and is required for the Input module to operate properly.

Next is dependencies_module, which contains all the software dependencies for the Input. This is structured as a list of tuples, with each tuple comprised of three strings. The first tuple string is used to indicate how the dependency should be installed. This can be "pip-pypi" to install a package from pypi.org with pip, "pip-git" to install a package from github.org with pip, or "apt" to install a package via apt. The second item is the Python library name that will be attempted to be imported to determine if the dependency has been met. The last item is the name used to install the library, either via pip or apt. In the above example, Adafruit_GPIO is a library available via pypi.org, and will be installed via pip with the command pip install Adafruit_GPIO. Adafruit_MCP9808 is installed via pip as well, but uses a git repository instead of pypi, with the command pip install -e git://github.com/adafruit/Adafruit_Python_MCP9808.git#egg=adafruit-mcp9808. When a user attempts to add this Input, Mycodo will check if these libraries are installed. If they are not installed, Mycodo will then automatically install them. See the other input modules as examples for how to properly format the dependencies list.

Next is interface, and is used to determine how to communicate with the Input. Since this input communicates with the sensor via the I2C bus, we specify I2C. It's possible for some sensors to be able to communicate in multiple ways, such as UART, 1WIRE, SPI, FTDI, among others. Add all applicable communication methods to the interfaces list. In the above example, the sensor only has an I2C interface. i2c_location defines a list of all possible I2C addresses that the sensor can be found at. If these are the only addresses that the sensor allows, i2c_address_editable should be set to False. However, if the sensor allows the user to reconfigure the address to something unique, this should be set to True to display an editable box for the user to define the I2C address rather than a pre-defined dropdown list.

Next is options_enabled and options_disabled, which determine which Input options appear on the web interface with other Input options. Since we want to be able to set the I2C location (address and bus), we enable this to allow the option fields I2C Address and I2C Bus to appear as options the user can set. Additionally, the Period (seconds) and Pre Output options are enabled to allow them to be used with the Input as well. interface is disabled, meaning it will be visible with the other options, but it will be disabled and unable to be edited, allowing it to only indicate what interface has been selected for the Input.

The following table shows for each interface enabled, what needs to be included in options_enabled, how to set default options, as well as the variables that are available within the module.

Interfaces options_enabled INPUT_INFORMATION default options variables
I2C i2c_location 'i2c_location': ['0x64', '0x65'], 'i2c_address_editable': True self.input_dev.i2c_address, self.input_dev.i2c_bus
UART uart_location 'uart_location': '/dev/ttyAMA0', 'uart_baud_rate': 9600 self.input_dev.uart_location, self.input_dev.baud_rate
FTDI ftdi_location 'ftdi_location': '/dev/ttyUSB0' self.input_dev.ftdi_location
1WIRE location   self.input_dev.location

There are other options that can be used within INPUT_INFORMATION. Refer to the other Input and example modules to see how they can be used.

The InputModule class contains all the functions to query measurements from the sensor itself as well as any other functions, such as those to perform calibration. This class inherits from the AbstractInput base class in base_input.py.

class InputModule(AbstractInput):
    """ A sensor support class that monitors the MCP9808's temperature """

    def __init__(self, input_dev, testing=False):
        super(InputModule, self).__init__(input_dev, testing=testing, name=__name__)

        self.sensor = None

        if not testing:
            self.initialize_input()

    def initialize_input(self):
        """ Initialize the MCP9808 sensor class """
        from Adafruit_MCP9808 import MCP9808

        self.sensor = MCP9808.MCP9808(
            address=int(str(self.input_dev.i2c_location), 16),
            busnum=self.input_dev.i2c_bus)
        self.sensor.begin()

    def get_measurement(self):
        """ Gets the MCP9808's temperature in Celsius """
        self.return_dict = copy.deepcopy(measurements_dict)

        if not self.sensor:
            self.logger.error("Sensor not set up")
            return

        try:
            self.value_set(0, self.sensor.readTempC())
            return self.return_dict
        except Exception as msg:
            self.logger.exception("Input read failure: {}".format(msg))

__init__() is executed upon the class instantiation, and is generally reserved for defining variable default values. Here, we set self.sensor to None to indicate it has not yet been set up.

initialize_input() is executed following initialization. This is where our dependency library or libraries (if there are any) are typically imported and any classes initialized. In the above example code, we create an instance of the class MCP9808 and assign this object to the variable self.sensor. We initialize the class with the I2C address (input_dev.i2c_location) and I2C bus (input_dev.i2c_bus) parameters from input_dev, which is our database entry for this particular Input. The MCP9808 library also requires begin() to be called to properly initialize the library/sensor prior to querying the sensor for measurements.

Next we have the get_measurement() function. This is called every Period to acquire a measurement from the sensor and store it in the measurement database. This function contains the code to query the sensor itself and the function to store the measurement that was obtained.

This sets the value in the template measurements dictionary that will be used to store the measurements in the measurements database. Only the measurements set in this way are stored in the measurements database following the return of the get_measurements() function. An optional timestamp may be provided as a datetime object in UTC time.

In the above code, we begin with self.return_dict = copy.deepcopy(measurements_dict) that creates a blank template of our measurement dictionary. Next, we check if self.sensor is still None or if it was successfully set up in initialize_input(). Next, self.sensor.readTempC() is used to return a temperature measurement, and is passed as a parameter within self.value_set(0, self.sensor.readTempC()) in order to store this measurement to our newly-created measurement template dictionary under the key 0 (which is our temperature measurement in measurements_dict). If we desired to store additional measurements, such as humidity, under key 1 of our template, we would call self.value_set(1, humidity_value), where humidity_value is a float value representing the humidity. Last, we return self.return_dict to indicate all measurements were successfully obtained and the Input module should save the measurements that were stored in the template to the measurements database.

Beyond the ability to configure how to communicate with the Input device (e.g. I2C address, I2C Bus, UART device, UART baud rate, FTDI device, 1-Wire address), you can create your own options that will be presented for the user to configure and be available to use within the Input module as variables. Below are examples of the various types.

def constraints_pass_positive_value(mod_input, value):
    """Check if the user input is acceptable"""
    errors = []
    all_passed = True
    if value <= 0:  # Ensure value is positive
        all_passed = False
        errors.append("Must be a positive value")
    return all_passed, errors, mod_input

INPUT_INFORMATION = {
    # Custom options that can be set by the user in the web interface.
    'custom_options_message': 'This is a message displayed for custom options.',
    'custom_options': [
        {
            'id': 'variable_boolean',
            'type': 'bool',
            'default_value': True,
            'name': 'Checkbox',
            'phrase': 'Description of the checkbox option'
        },
        {
            'id': 'variable_integer',
            'type': 'integer',
            'default_value': 10,
            'constraints_pass': constraints_pass_positive_value,
            'name': 'Integer Value',
            'phrase': 'Description of the integer option'
        },
        {
            'id': 'variable_float',
            'type': 'float',
            'default_value': 5.2,
            'constraints_pass': constraints_pass_positive_value,
            'name': 'Float Value',
            'phrase': 'Description of the float option'
        },
        {  # This starts a new line for the next options
            'type': 'new_line'
        },
        {  # This message will be displayed after the new line
            'type': 'message',
            'default_value': 'Another message between options',
        },
        {
            'id': 'variable_select_dropdown',
            'type': 'select',
            'default_value': '2',
            'options_select': [
                ('1', 'Selection One'),
                ('2', 'Selection Two'),
                ('3', 'Selection Three'),
                ('5', 'Selection Four'),
            ],
            'constraints_pass': constraints_pass_measure_range,
            'name': 'Select Options',
            'phrase': 'Description of the selection option'
        },
        {
            'id': 'variable_select_device',
            'type': 'select_device',
            'default_value': '',
            'options_select': [
                'Output',
            ],
            'name': 'Select Device',
            'phrase': 'Description of the device selection'
        },
        {
            'id': 'variable_select_measurement',
            'type': 'select_measurement',
            'default_value': '',
            'options_select': [
                'Input',
                'Math'
            ],
            'name': 'Select Measurement',
            'phrase': 'Description of the measurement selection'
        }
    ]
}

class InputModule(AbstractInput):
    def __init__(self, input_dev, testing=False):
        super(InputModule, self).__init__(input_dev, testing=testing, name=__name__)

        # Initialize custom option variables to None
        self.variable_boolean = None
        self.variable_integer = None
        self.variable_float = None
        self.variable_select_dropdown = None
        self.variable_select_device_id = None
        self.variable_select_measurement_device_id = None
        self.variable_select_measurement_measurement_id = None

        # Set custom option variables to defaults or user-set values
        self.setup_custom_options(INPUT_INFORMATION['custom_options'], input_dev)

        self.logger.info("Variable values: {}, {}, {}, {}, {}, {}, {}".format(
            self.variable_boolean,
            self.variable_integer,
            self.variable_float,
            self.variable_select_dropdown,
            self.variable_select_device_id,
            self.variable_select_measurement_device_id,
            self.variable_select_measurement_measurement_id))

        self.logger.debug("Turning Output with ID {} ON".format(
            self.variable_select_device_id))
        from mycodo.mycodo_client import DaemonControl
        self.control = DaemonControl()
        self.control.output_on(self.variable_select_device_id)

        last_measurement = self.get_last_measurement(
            self.variable_select_measurement_device_id,
            self.variable_select_measurement_measurement_id,
            max_age=3600)
        self.logger.debug("Last measurement (past 3600 seconds): {}".format(
            last_measurement))

In the above code, there are options set in custom_options for the user to be able to set a boolean value, set an integer value, set a float value, select text from a dropdown, select an output device from a dropdown, and select a specific measurement of an Input or Math controller from a dropdown. Note that the option id must contain no spaces, as it will be used later to determine the variable names (which cannot have spaces). There's also the ability to add a message at the top of these options or in between options, as well as create a new line. Optionally, constraints_pass may be set to a function to verify the user input. In this example, constraints_pass_positive_value() tests whether the value entered is positive. If the user tries to save a non-positive value, the specified error is presented and the value is not saved.

When we get to __init__(), we see we must first name the custom option variables to the same name as the option id and set them to None. The only exception is the device and measurements selection options. The device option variable name must have "_id" appended to the end and the measurement option becomes two variables, one with "_device_id" appended to the end and the other with "_measurement_id" appended to the end. These correspond to either the device ID itself (Input, Outputs, PID, etc.) or the measurement ID (such as the ID of channel 0 of an Input, representing temperature). Following setting these variables to None, we execute self.setup_custom_options(), which sets them to the values the user configured in the web interface. Following this, we have a log line that will print the values of all the variables. Next we demonstrate how to use the output device variable to turn the output on. And last we demonstrate how to use the measurement variables to retrieve and print the last stored value of the selected measurement.

These are functions contained in the abstract base class AbstractInput of base_input.py. These may be used within any Input module that inherits these functions.

  • Parameter channel: integer, the numeric key of the measurements_dict dictionary of measurements.
  • Returns: True if measurement channel enabled, False if measurement channel disabled.

This function allows users to determine which measurements the user has configured to be stored in the measurement database. This function is called in value_set() to determine if the measurement should be added to the database.

  • Parameter channel: integer, the numeric key of the measurements_dict dictionary of measurements.
  • Returns: The value stored with value_set() since get_measurement() was first called.

This function is useful for referencing previously-acquired measurements without having to explicitly declare variables to hold the measurements. For example, a temperature and humidity sensor will acquire the measurements temperature and humidity, but dew point can also be calculated from these two measurements. In the below code, we check whether each measurement is enabled, then use value_set() to store the measurement. Then, we check if measurements 0, 1, and 2 are all enabled, we calculate the dew point with calculate_dewpoint() and set measurement 3.

if self.is_enabled(0):
    self.value_set(0, self.read_temperature())

if self.is_enabled(1):
    self.value_set(1, self.read_humidity())

 if self.is_enabled(2) and self.is_enabled(0) and self.is_enabled(1):
    self.value_set(2, calculate_dewpoint(self.value_get(0), self.value_get(1)))
  • Parameter name: string, a dictionary key to store measurements
  • Parameter init_max: integer, function must first be called with this, sets how many measurements to store/average.
  • Parameter measurement: integer/float, the measurement to store in a rolling list to average.
  • Returns: An average of past-stored measurements.

This function is used to average measurements. This is useful for smoothing out noisy signals such as from analog-to-digital converters. For instance, you can first call filter_average("light", init_max=20) in initialize_input() prior to storing several values.

light_average = None
for _ in range(20):
    light_average = self.filter_average("light", measurement=self.sensor.get_light())

self.value_set(0, light_average)

You could also merely perform a rolling average of several past measurements rather than performing multiple measurements per Input Period.

self.value_set(0, self.filter_average("light", measurement=self.sensor.get_light()))
  • Parameter lockfile: string, the full path to the lock file to be created.
  • Parameter timeout: integer, the timeout period for acquiring a lock.
  • Returns: True if successfully locked, False if not successfully locked

Returns: None

This is a non-blocking locking method for gaining access to shared resources, such as devices, files, sensors, or anything else. An example if it's use if below. Notice self.locked[lockfile] is used to verify whether a lock has been obtained or not.

if self.lock_acquire("/var/lock/lockfile", timeout=10):
    try:
        # Code to run while we have a lock
    finally:
        self.lock_release("/var/lock/lockfile")
  • Parameter lockfile: string, the full path to the lock file.
  • Returns: True if locked, False if not locked

Determines if a lock is already in place for a specific lockfile.

  • Parameter lockfile: string, the full path to the lock file.
  • Returns: None

Releases a lock and forces the deletion of the lockfile.

This function can be used to determine if an Input is currently acquiring a measurement by returning the value of self.acquiring_measurement. This entails self.acquiring_measurement is set to True prior to measurement acquisition and set to False following it.