SSL read fails with big chunks of data.

[nemidiy]

Hi all,

I am actually coming from a a different project that leverages on this lib to implement an MQTT client.
One of the features I am trying to get working is OTA updates through MQTT over TLS.
The problem I see is that when the payload is sent over the TCP channel the ESPAsyncTCP lib fails reading the SSL buffer and kills the tcp connection.
This is what I get from the debug messages :

$ platformio device monitor -b 115200
--- Miniterm on /dev/ttyUSB0  115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
Fake value: 6382404
006382.458 [ASYNC_TCP] _sent[1]:   69, unacked=14444, acked=1149800, space=2484
006382.465 [ASYNC_TCP] _sent[1]:  308, unacked=14136, acked=1150108, space=2792
006382.467 [ASYNC_TCP] _recv[1]: 69
006382.468 [TCP_SSL] tcp_ssl_ssl_read: 0
006382.482 [TCP_SSL] tcp_ssl_ssl_read: 4
〽 Sending statistics...
  • Interval: 15s (10s including 5s grace time)
  • Wi-Fi signal quality: 100%
  • Uptime: 6385s
006385.388 [ASYNC_TCP] _sent[1]:   69, unacked=15198, acked=1150177, space=1730
006385.411 [ASYNC_TCP] _sent[1]: 1062, unacked=14136, acked=1151239, space=2792
006385.412 [ASYNC_TCP] _recv[1]: 69
006385.413 [TCP_SSL] tcp_ssl_ssl_read: 0
006385.427 [TCP_SSL] tcp_ssl_ssl_read: 4
006385.553 [ASYNC_TCP] _recv[1]: 138
006385.554 [TCP_SSL] tcp_ssl_ssl_read: 0
006385.562 [TCP_SSL] tcp_ssl_ssl_read: 4
006385.566 [TCP_SSL] tcp_ssl_ssl_read: 0
006385.570 [TCP_SSL] tcp_ssl_ssl_read: 4
Fake value: 6392411
006392.469 [ASYNC_TCP] _sent[1]:   69, unacked=14444, acked=1151308, space=2484
006392.477 [ASYNC_TCP] _sent[1]:  308, unacked=14136, acked=1151616, space=2792
006392.479 [ASYNC_TCP] _recv[1]: 69
006392.480 [TCP_SSL] tcp_ssl_ssl_read: 0
006392.494 [TCP_SSL] tcp_ssl_ssl_read: 4
〽 Sending statistics...
  • Interval: 15s (10s including 5s grace time)
  • Wi-Fi signal quality: 100%
  • Uptime: 6395s
006395.400 [ASYNC_TCP] _sent[1]:   69, unacked=15198, acked=1151685, space=1730
006395.421 [ASYNC_TCP] _sent[1]: 1062, unacked=14136, acked=1152747, space=2792
006395.423 [ASYNC_TCP] _recv[1]: 69
006395.424 [TCP_SSL] tcp_ssl_ssl_read: 0
006395.438 [TCP_SSL] tcp_ssl_ssl_read: 4
006395.619 [ASYNC_TCP] _recv[1]: 138
006395.620 [TCP_SSL] tcp_ssl_ssl_read: 0
006395.629 [TCP_SSL] tcp_ssl_ssl_read: 4
006395.633 [TCP_SSL] tcp_ssl_ssl_read: 0
006395.637 [TCP_SSL] tcp_ssl_ssl_read: 4
006396.130 [ASYNC_TCP] _recv[1]: 1460
006396.132 [TCP_SSL] tcp_ssl_ssl_read: -261
006396.138 [TCP_SSL] tcp_ssl_read: read error: -261
006396.142 [ASYNC_TCP] _recv[1] err: -261
006396.143 [TCP_SSL] tcp_ssl_free: 19
006396.150 [ASYNC_TCP] _close[1]: AsyncClient 0x3FFF0B70
✖ MQTT disconnected, reason: 0
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...

The log shows that the SSL connection is healthy until .232 (mqtt broker) starts sending bigger chunks of data the size of the MTU (1460) passes them on to SSL layer and the read error happens. I also did a trace with tcpdump to see what was going on. here you can see that after the forth ~1516 size package the connection gets closed by .201 (the nodemcu)

https://imgur.com/a/q1I5g6J

I am an outsider when it comes to SSL to be honest so I am not sure this is a problem of the lib
or something different. I would appreciate any help.
Thanks!

[nemidiy]

Doing further research I found this : espressif/ESP8266_RTOS_SDK#142
If that is accurate then it means there is no easy way out.

[stale]

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale]

[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions.

[uddhavdave]

Hi @nemidiy , I am facing a similar issue when sending out large (>1Kb) MQTT message . I would like to know how you solved the issue if you have, if not did you consider changing to other ssl Library?

I am using Arduino/esp8266 SDK along with async-mqtt-client library and desperately need a solution to this problem. Pls help.

[uddhavdave]

Also , can you tell me how do you have this much space 006382.465 [ASYNC_TCP] _sent[1]: 308, unacked=14136, acked=1150108, space=2792 ? are you using ESP8266, if yes can you tell me a way to increase the size of it. I have tried multiple ways but nothing helps.

[nemiliani]

hey @alphabeta1212 , I ended up moving to ESP32, but essentially my problem was that since I was using SSL, and decrypting is a rather heavy task for the MCU, my producer (a script pushing a chunk of OTA firmware into the MQTT broker) was too fast for my MCU (the consumer).
I tried a lot of things like increasing the size of the TCP window, but the MCU would eventually choke.

The only way I found to have it working is while taking this bug chunks of data stop doing whatever
else so that the MCU is as relaxed as possible to handle the load.

Here is the entire story :
homieiot/homie-esp8266#648

[uddhavdave]

Thats right. thanks for the reply and suggestion. By the way my problem was solved on changing the LWIP variant.