MLX90640 Thermography Camera for ESP32

Description

Program for thermal imaging camera with MLX90640 far-infrared (IR) thermal sensor srray for XIAO ESP32S3 and ESP32-2432S028R

Features

It comes with a graphical user interface to properly configure the MLX90640 for different scenarios.

• Display frame rate: 1 FPS (min) / 16 FPS (max)
• Resolution: 32 x 24 (min) / 256 x 192 (max)
• Heat map mode: Rainbow / Inferno
• Measurement range: Auto scaling / Manual adjustment.
• Measurement features: Automatic min/max / a specified point.
• Screen capture: Save 320 x 240 bitmap (24-bit) to SD card.
• Video recording: Record 32 x 24 raw data continuously onto SD card.
• File explorer: Display thumbnails / Play videos on SD card.
• Offline video viewer: Offline video playback on PC
• Touch screen: Calibration on the first launch / in the configuration menu.

Demo

File Explorer

MLX90640Viewer

Hardware Setup

For GPIO pin assignments, see pin_assign.h.

XIAO ESP32S3

Block Diagram

Wiring

Prototype

ESP32-2432S028R (AKA Cheap Yellow Display)

Connecting MLX90640 to I2C

Li-Po Battery

Prototype

I2C pull-up registers

Software Setup

Requirements

• Arduino IDE : 2.3.4
• Board package : esp32 by Espressif Systems : 3.1.1
  Board type:
  • XIAO_ESP32S3
  • ESP32-2432S028R CYD
• Adafruit_MLX90640 1.1.1
• GFX library
  • LovyanGFX : 1.2.0 or
  • TFT_eSPI : 2.5.43
• Touchscreen library (only for TFT_eSPI on CYD)
  • XPT2046_Touchscreen : Version 1.4 or
  • XPT2046_Bitbang_Slim : v2.0.1

File Structure and Dependencies

MLX90640.ino
├─ pin_assign.h
├─ heatmap.h
│
├─ mlx.hpp
├─ gfx.hpp
│   │ <LovyanGFX.h>
│   ├─ boards
│   │   ├─ LGFX_CYD_2432S028R.hpp
│   │   └─ LGFX_XIAO_ESP32S3_ST7789.hpp
│   │ <TFT_eSPI.h>
│   ├─ User_Setup.h
│   └─ boards
│       ├─ Setup_CYD_2432S028R.h
│       ├─ Setup_XIAO_ESP32S3_ST7789.h
│       └─ XPT2046_ScreenPoint.h
├─ task.hpp
├─ touch.hpp
├─ sdcard.hpp
├─ filter.hpp
├─ interpolation.hpp
└─ widget.hpp
    ├─ draw.hpp
    │   └─ marker.h
    └─ widgets.hpp
        └─ widgets.h

Configuration

1. Depending on the board type, define pinouts of the I2C for MLX90640 and SPI for LCD / touch screen / SD card in pin_assign.h.

2. Configure each step in MLX90640.ino. Generally, there is no need to change the default values.

• Step1: Debugging mode (default: false)
• Step2: Operational specifications
  • ENA_INTERPOLATION (default: true)
  • ENA_MULTITASKING (default: true)
  • ENA_OUTWARD_CAMERA (true: Outward, false: Selfie)
• Step3: GFX library
  • USE_LOVYAN_GFX (default: true, highly recommended)
    • USE_AUTODETECT (default: true, or manual settings)
  • USE_TFT_ESPI
    • User_Setup.h (definition required)
• Step4: Flash memory
  • USE_PREFERENCES (default: false, true is recommended.)
• Step5: Resolution
  For low performance MCUs. No modifications required.

Dual-Core / Multitask

It achieves parallel processing by utilizing multi-cores with FreeRTOS and double-buffering for thermal images.

Timing diagram

• Timing diagram (Drawio)
  

Touch Screen Calibration

The “Calibration Screen” will appear when you run the sketch for the first time after compile & uploading, or when you select to run calibration from the menu.

Calibration Screen

If USE_PREFERENCES is set to true, you can save the calibration results to flash, but when it's set to false, you need to open the serial monitor with 115200 bps before running the calibration and embed the displayed data into touch.hpp after calibration is completed.

Known Issue

• When TFT_eSPI is selected,
  • rendering will be slower than LovyanGFX on ESP32-2432S028R.
  • screenshots can not be saved correctly on XIAO_ESP32S3.

Related Project

• MLX90640Viewer
  The MLX90640Viewer is a tool that applies heatmaps and visualizes the output of the Melexis MLX90640 32x24 IR array, which is stored as continuous raw data in a file.

• MLX90640 for Arduino UNO R4

Resouces

• Melexis MLX90640

  • Datasheet
    • I2C Frequency: 1MHz (Fast mode+)
  • Breakout board: GY-MCU90640
    • Alliexpress

• Seeed Studio XIAO ESP32S3

  • ESP32-S3 Series Datasheet
  • ESP32 Technical Reference Manual

• witnessmenow/ESP32-Cheap-Yellow-Display