Asyncio IO version of uMqtt.Simple

[garynthompson]

Hi All,

I've recently refactored the Simple MQTTClient to be asynchronous, is this something worth sharing?

I started off with the synchronous version on a thread, discovered thread limitations, then used Peter Hinch's version which worked great for a while but as I was optimising my project to work more offline I realised that micropython-mqtt is better suited where MQTT is central to the project as opposed to where MQTT is opportunistic. As I am working cross platform I want to also trial using the same library on Linux devices as my MQTT Client requirements are very simple.

If this sounds like it is worth sharing, I am happy to prepare a PR back to micropython-lib over the next few months once I have studied contributor guidelines, confirmed standard error workflows and added some documentation. My own projects are aimed to be Micropython and CPython compatible, which means I could unit test in CPython but I haven't seen any implementation patterns in the existing micropython-lib repo so am happy to get advice on this.

Edit: Draft Gist: https://gist.github.com/garynthompson/b62ddf958fec860c547587bfdcca4157#file-a_mqtt_simple-py

Kind regards,

Gary.

[Carglglz]

I think yours is a better version of this micropython/micropython-lib#722.
I'm not sure if there is interest since I think most people would use micropython-mqtt, however it would be nice to see something like this officially supported in micropython-lib, so thanks for sharing anyway 👍🏼

My own projects are aimed to be Micropython and CPython compatible, which means I could unit test in CPython but I haven't seen any implementation patterns in the existing micropython-lib repo so am happy to get advice on this.

You can run unit tests in MicroPython too, see e.g. python-stdlib/datetime/test_datetime.py and to integrate them in CI check function ci_package_tests_run in tools/ci.sh

[garynthompson]

Thanks for the feedback, I really appreciate the tip to the unit tests. I searched the discussion forums and issue lists checking if someone else had worked on this and forgot to check the PRs.

As a summary for anyone else reading this, one of the biggest change that I implemented from the original uMQTT was a receive loop instead of a wait function. Since asyncio is being used, cooperative multitasking is implied and a receive loop ensures that all receives are handled in a coordinated manner while also minimising blocking if the client is used from multiple tasks. There is still a possibility of memory leaks for the buffers used by the receive loop that I will be looking into.

[peterhinch]

On past experience I think it is unlikely that the maintainers will accept a new MQTT version in micropython-lib - a revised MQTT library has been discussed a number of times.

If you see a role for a reduced version of mqtt_as I suggest you publish it independently. I would recommend documenting the features it offers which the official library does not, and also listing features of mqtt_as which have been removed.

The biggest of the latter is the removal of WiFi control. While this enables the code to be ported to Unix, it does remove the original purpose of mqtt_as which is resilience: an ability to cope with WiFi and broker outages in a transparent manner. I think a new MQTT client should state what level of resilience it supports. The purpose of MQTT is to provide reliable communication over a link which can be subject to outages - for example the reason for QOS > 0 is to cater for the case where a PUBACK is not received. If a link were perfect, there would be no need for QOS > 0 (a point rather missed in the official library).

While I appreciate it is a work in progress I can see a number of issues with the code. Ones that jump out at me are:

• It seems to lack control over concurrency. For example _receive_loop() can send data but an application may have launched a publish task leading to a risk of corrupted transmission.
• PID's increase without limit, contrary to the MQTT spec.
• If a QOS 1 publication fails to receive its PUBACK message, the publication task hangs.
• Subscribe-only applications are not supported (the broker connection will time out).
• Behaviour should be defined where an application launches a QOS 1 publication while another publication task instance is busy waiting for a PUBACK.
• Creating a client_id from a random number means that, if a client reconnects, the broker will see the new connection as a new client. This is problematic when clean_session==False. You might want to find another source of local ID (MAC address?).

[garynthompson]

Thank you @peterhinch for the detailed response and review. I appreciate your experience in this. I've always been hesitant to roll my own package where there are existing ones such as mqtt_as, I would rather invest my effort in offering something back to existing packages. I do also suspect my needs are niche and I'll consider them a bit more deeply before proceeding.

Separation of Wifi control was one of the main reasons I thought to hack at uMQTT simple rather than dig deeper into whatever blocking issue I was having previously. In my project I wanted Wifi control managed independently of the MQTT client as I'm building a little framework and MQTT is optional, Wifi is mostly used for SNTP.

[garynthompson]

I am happy to close this off with a summary in case it helps anyone else who comes along. If anyone comes across this I will assume you are interested in cooperative multitasking via asyncio and interacting with an MQTT broker.

• uMQTT is the "batteries included" solution which is okay and may be good enough for many projects that don't need concurrency, or to get you off the ground. The code certainly worked in 2025 with little effort against a Mosquitto broker.
• micropython-mqtt should be your default option once you outgrow uMQTT or need asyncio. If you intent MQTT to be central to your project, consider this library.
• If you want to experiment with cross-platform, take a look at this version of mqtt_as that I missed earlier
• If these don't meet your requirements, you may want to perform some deep soul searching.

For that last point, I probably will still tinker with different options as I gain a better understanding of my use case.

[peterhinch]

@garynthompson You might be interested in this short doc.

tl;dr It outlines a way forward with mqtt_as, which would address the Unix compatibility issue amongst others.