ESP32 ADC input tuning via Berry script

[nostroff]

I've successfully connected a Total Dissolved Solids sensor to an ESP32 (configured as an ADC Analog input). The TDS sensor outputs an analog voltage that can be converted to a parts per million (PPM) value. The sensor details are here:
http://www.cqrobot.wiki/index.php/TDS_(Total_Dissolved_Solids)_Meter_Sensor_SKU:_CQRSENTDS01

Currently, Tasmota natively supports converting raw ADC values to temperature, light, pH (among others) using the AdcParam command and hard-coded logic.

The following Berry script converts the raw TDS sensor voltage to a PPM value:

import json
import string
sensors=json.load(tasmota.read_sensors())
def rule_adc_2(value)
  var compTemp = 1.0 + 0.02 * (((sensors['AM2301']['Temperature'] - 32.0) * 0.5555) - 25.0)
  var voltage=value / 1000.0
  var compVolt = voltage / compTemp
  var tdsValue = (133.42 * compVolt * compVolt * compVolt - 255.86 * compVolt * compVolt + 857.39 * compVolt) * 0.5
  #print("compTemp:", compTemp, ", compVolt:", compVolt, ", tdsValue:", tdsValue)
  tasmota.publish_result(string.format('{"TDS":%f}', tdsValue), 'TDS_VALUE')
end

tasmota.add_rule("Tele#ANALOG#A2", rule_adc_2)

The PPM conversion relies on a separate temperature reading and a non-linear algorithm (and a conversion from Fahrenheit to Celsius).

Could you please consider adding support for a new AdcParam value where param1 would be the name of a Berry rule/function, and where the Berry rule/function returns a value? The goal is to allow arbitrary conversion of raw ADC values into meaningful values.

If there's an easier and/or better approach, can you please point me in the right direction?

Thanks.

[apbarratt]

I've just received this same sensor, seems this is what's coming up when searching for "Tasmota TDS" now. I've not encountered Berry SCript before but would agree that some standard method of entering conversion scripts for raw analog inputs would be very helpful.

[sfromis]

While this does not address the specific suggestion, my strategy for handling such adjustments would be to instead create a driver in Berry, providing the data both on the Tasmota front page and in the regular JSON payloads for sensor data. Such a driver could subscribe to the sensor readings with a tasmota.add_rule('Analog#A2, ...) to get updates (note not only at teleperiod), and include functions web_sensor and json_append to provide the data. It can add itself as a driver using tasmota.add_driver, and the driver would not itself be doing the MQTT publish, as it would instead be part of the broader sensor payload.

As it would also pick up analog data via the regular JSON payloads (internally generated about every second), one small annoyance is that the "upgraded" value put into the sensor payload would be based on the prior analog value, as Berry code cannot directly read analog, maybe a couple of seconds behind only. In your case, I'd want to generalize such a driver to recognize Tasmota being configured for temperature in C/F, to make it work with both.

[sfromis]

This driver (which I improved a bit via a PR) is a good illustration of how a driver can work. While this is for an I2C sensor, and thus has specific coding for that, it still has the mentioned things included, without being very long.
https://github.com/TomsTek/tasmota-berry-TMP117-driver/blob/main/tmp117.be

[nostroff]

@sfromis A big thanks for providing an alternative approach.

The following Berry script is based on your TMP117 driver. It adds a new driver that performs the TDS sensor conversion and handles C/F temperatures (per your suggestion). As this approach looks perfectly reasonable, the value of adding a new AdcParam value that calls a Berry function seems questionable, where the benefits would be to make Tasmota more intuitive and align Tasmota with ESPEasy where sensor values can have a user-supplied conversion algorithm.

#-
 - Analog input driver written in Berry
 -
 - Support for TDS (Total Dissolved Solids) Meter Sensor_CQRSENTDS01
 - http://www.cqrobot.wiki/index.php/TDS_(Total_Dissolved_Solids)_Meter_Sensor_SKU:_CQRSENTDS01
 -#

class CQRSENTDS01: Driver
  static Sensor_name = "CQRSENTDS01"  # sensor name
  static Tds_label  = "TDS"           # label for temperature in language of preference

  var Tds            # global TDS value in PPM

  #- read raw voltage (and temperature) and return TDS value -#
  def read_tds()
    import json
    import string
    # DEBUG print("sensors:",tasmota.read_sensors())
    var Sensors=json.load(tasmota.read_sensors())
    # Configure to read temp value
    var Temp = Sensors['AM2301']['Temperature']
    # Configure to read TDS voltage value
    var Voltage=Sensors['ANALOG']['A2'] / 1000.0
    if ! Temp
        Temp = 25
    end
    if ! Voltage
        Voltage = 0
    end
    var Fahrenheit = tasmota.get_option(8) == 1
    var CompTemp
    if Fahrenheit
        CompTemp = 1.0 + 0.02 * (((Temp - 32.0) * 0.5555) - 25.0)
    else
        CompTemp = 1.0 + 0.02 * (Temp - 25.0)
    end

    var CompVolt = Voltage / CompTemp
    self.Tds = int((133.42 * CompVolt * CompVolt * CompVolt - 255.86 * CompVolt * CompVolt + 857.39 * CompVolt) * 0.5)
    if ! self.Tds
        self.Tds = 0
    end
    # DEBUG print("CompTemp:", CompTemp, ", CompVolt:", CompVolt, ", tds:", self.Tds)
    return self.Tds
  end

  #- trigger a read every second -#
  def every_second()
    self.read_tds()
  end

  #- display sensor value in the web UI -#
  def web_sensor()
    import string
    if ! self.Tds
        self.Tds = 0
    end
    var Msg = string.format("{s}%s %s{m}%d PPM{e}", self.Sensor_name, self.Tds_label, self.Tds)
    tasmota.web_send_decimal(Msg)
  end

  #- add sensor value to teleperiod -#
  def json_append()
    import string
    if ! self.Tds
        self.Tds = 0
    end
    var Msg = string.format(',"%s":{"%s":%d}', self.Sensor_name, self.Tds_label, self.Tds)
    tasmota.response_append(Msg)
  end

end
cqrsentds01 = CQRSENTDS01()
tasmota.add_driver(cqrsentds01)

[sfromis]

Not a solution. Using Status 10 is similar to using read_sensors in calling all drivers to produce sensor payloads. It does not address the issue mentioned. OTOH, as long as you are doing things in that separate every_second cycle, it can work without problems, with read_sensors being much preferable, unless you want the extra console/MQTT traffic from using Status 10 very often.

And in case of having a analog driver callable Berry conversion function, that should most probably neither use read_sensors nor Status 10.

What both barbudor and I suggested instead of read_sensors (and certainly instead of Status 10) was using tasmota.add_rule to have a function to pick up the analog value, and save it in a instance variable for other functions to use. I'd say that this is a bit better than calling read_sensors in every_second but both strategies can work reasonably. However, this would probably still not be a viable strategy if the Tasmota analog driver became able to call a Berry function to adjust the value.

Obvious exception would be in case of extra architecture being created to support this situation.

I'd say that creating a driver using a tasmota.add_rule to pick up values is probably a fairly decent solution for most situations, with the small nick of the converted value being based on a slightly older analog input.

[nostroff]

@sfromis Can you please comment on the Berry driver shown below? Per your suggestion, the driver uses rules for picking up the TDS sensor voltage and the temperature. If this script is "correct" and you think it would be useful to other Tasmota users, please consider adding it to the cookbook,

#-
 - Analog input driver written in Berry
 -
 - Support for TDS (Total Dissolved Solids) Meter Sensor_CQRSENTDS01
 - http://www.cqrobot.wiki/index.php/TDS_(Total_Dissolved_Solids)_Meter_Sensor_SKU:_CQRSENTDS01
 - Based on TMP117-driver driver by sfromis and TomsTek (https://github.com/TomsTek/tasmota-berry-TMP117-driver)
 -#

class CQRSENTDS01: Driver
  static sensor_name = "CQRSENTDS01"  # sensor name
  static tds_label  = "TDS"           # label for temperature in language of preference

  var tds            # global TDS value in PPM
  var comp_temp      # global temperature compensation
  var voltage        # global TDS sensor voltage
  var prev_voltage

  def init()
    # initialize globals
    self.tds = 0
    self.comp_temp = 25.0
    self.voltage = 1.0
    self.prev_voltage = self.voltage
    # create rules to trigger when TDS sensor voltage updates and temp sensor updates
    tasmota.add_rule("Tele#ANALOG#A2", /value -> self.trigger_voltage(value))
    tasmota.add_rule("Tele#AM2301#Temperature", /value -> self.trigger_temp(value))
  end

  def trigger_voltage(voltage)
    if voltage != nil
      self.voltage = voltage
    end
    # DEBUG print("voltage:", voltage)
  end

  def trigger_temp(temp)
    if temp != nil
      if tasmota.get_option(8) == 1
        self.comp_temp = 1.0 + 0.02 * (((temp - 32.0) * 0.5555) - 25.0)
      else
        self.comp_temp = 1.0 + 0.02 * (temp - 25.0)
      end
    end
    # DEBUG print("temp:", temp, "comp_temp:", self.comp_temp)
  end

  #- calculate TDS value from voltage and temp -#
  def calc_tds()
    # to reduce cpu usage, only perform calculation if voltage changes
    if self.prev_voltage != self.voltage
      self.prev_voltage = self.voltage
      var comp_voltage = (self.voltage / 1000.0) / self.comp_temp
      self.tds = int((133.42 * comp_voltage * comp_voltage * comp_voltage - 255.86 * comp_voltage * comp_voltage + 857.39 * comp_voltage) * 0.5)
      # DEBUG print("comp_temp:", self.comp_temp, ", comp_voltage:", comp_voltage, ", tds:", self.tds)
    end
  end

  #- trigger a read every second -#
  def every_second()
    self.calc_tds()
  end

  #- display sensor value in the web UI -#
  def web_sensor()
    import string
    if ! self.tds
      self.tds = 0
    end
    var msg = string.format("{s}%s %s{m}%d PPM{e}", self.sensor_name, self.tds_label, self.tds)
    tasmota.web_send_decimal(msg)
  end

  #- add sensor value to teleperiod -#
  def json_append()
    import string
    if ! self.tds
      self.tds = 0
    end
    var msg = string.format(',"%s":{"%s":%d}', self.sensor_name, self.tds_label, self.tds)
    tasmota.response_append(msg)
  end

end
cqrsentds01 = CQRSENTDS01()
tasmota.add_driver(cqrsentds01)

[sfromis]

Using triggers with tele# defeats the regular way of sensor drivers working, providing updated values on the Tasmota front page, and available to rules (Berry or not) outside of the telemetry period. It also means that the "small nick" I mentioned about slightly older values changes to much older (up to TelePeriod), and that the every_second calculation you have no longer makes sense. Your previous version did not "suffer" from this.

Obviously, coding styles are different, and I'm not going into nitpicking about things I'd do differently, most important is that the code works without clear "holes". The TMP117 driver I linked was not by me, I merely provided a "minimal-change" PR to make it more general about handling temperature offsets and units in harmony with how user selections of these works in Tasmota. The driver is fine as is, but "unsurprisingly" I'd have done some things differently.

Assuming that this Berry examples repository project gets off the ground, this would IMO be a better place to have such user-contributed code than the cookbook page. #17490

[nostroff]

@sfromis Thanks for holding my hand through the Berry learning curve (and not trashing my coding style). Your guidance has been very helpful. The 2 rules have been updated to trigger whenever their values change (instead of at the telemetry period):

tasmota.add_rule("ANALOG#A2", /value -> self.trigger_voltage(value))
tasmota.add_rule("AM2301#Temperature", /value -> self.trigger_temp(value))

Perhaps @s-hadinger will add support for calling a Berry script to handle ADC conversions or add support for gpio.analog_read (to eliminate the need for a rule for capturing the analog value as suggested by @barbudor).

[nostroff]

Please uncomment the 2 #DEBUG print lines to see the temperature, sensor voltage and calculated TDS values. Note that the TDS calculation requires a temperature compensation value which might be incorrect or is in degrees C vs. degrees F. If you measure the TDS sensor voltage using a voltmeter and know the temperature, you can calculate the TDS value manually.

[ozett]

Hello folks,
i am a coding-beginner and now playing a bit with berry and have read this and other threads about berry-solutions.
I hope it's okay to ask something here, because the preliminary work with the driver revolves around my problem.

This driver here, as far as I understand it, reads values ​​and does conversions, but the triggers for actions are still rules.
And the update of Rules (as for a trigger) is not faster than 1 second, right?

My experiment to play with berry is: I'm trying to read the hall effect sensor from an ESP32 and switch a relay on values ​​below zero.
If I do this all only with rules, Problem is: Changes are already displayed in the webgui before the rule triggers somehow later.

My question: how can react faster than 1 second to value changes on rules?
Can or must it only be done with a berry driver? A driver, wich not feeds rules and status-updates, but itself switches a relay?

Or am I totally on the wrong track here?
My apologies if I hijacked this thread, but I found the above discussion to be more than halfway towards my questions on berry

[nostroff]

Assuming you're using the hall effect sensor that's built into the ESP32 and you've configured GPIO36 as HallEffect 1 and GPIO39 as HallEffect 2, have you tried to use the rules engine and define a rule like the following:

ON ESP32#HallEffect<0 DO Power1 On ENDON

How much slower is the rule firing than the webgui update? From my limited understanding, I would have expected the rule to fire at least as fast as the webgui update. It may help to write a message to the console when the rule is fired, as shown below (so that you can trigger the sensor and see when the console message appears):

ON ESP32#HallEffect<0 DO BACKLOG logentry,On; Power1 On ENDON

As for using a Berry script, in theory you could create a driver that calls every_250ms() or every_100ms() to read the hall effect sensor and then do some action based on the value, except Berry does not currently support reading the hall effect sensor value. This is similar to what @barbudor suggested for a Tasmota Berry improvement:

The best would be to have a direct gpio_adc in Berry that can be read in Berry (like the digital ones)

The difference is that instead of adding something like: gpio.analog_read, it may be something like gpio.hall_read.

Hopefully, the Berry coding master @s-hadinger will comment on adding support for gpio.analog_read and gpio.hall_read.

[sfromis]

In general, it is better to open a new thread for a new topic, instead of taking an existing thread off-topic.

Briefly, if you want to try to get faster reactions, you could poll the sensors using cyclic tasmota.read_sensors() calls to read values. Not in a loop, but using a relevant timer callback.

[ozett]

Hopefully, the Berry coding master @s-hadinger will comment on adding support for gpio.analog_read and gpio.hall_read.

thats what i thought while hijacking this thread here.
and thanks to your comments im am not totally wrong with my assumption.

just in short here @nostroff : big thanks for commenting.
i have rules like you showed, and i see 2 to 4 values changing in the gui before the rule triggers.

@sfromis : also big thanks for commenting.
i will try to experiment with timer callbacks for the hall-sensor, because i had sufficient success with short loops around 50ms and switching a LED in berry. maybe i can expand my simple code-skeletons with tasmota.read_sensors()

for more opportunity to ask berry questions on experimenting with a berry driver
will open a new discussion, (and later link it here)

follow-up ->

i have build working skeleton which has a quick response on the sensor,#18116
appreciate any comment on this.. thanks

[matmay]

Okay, I am also looking into writing my own driver for sensors outputting analog values.
Is there now a common ground on how such a berry driver should look like?
I'll provide a formula as well as units and name for the final output.

[sfromis]

You can read back in this thread about how such a Berry driver can be done.