TinyML Occupancy Classification Example with Consentium IoT

This example demonstrates how to use Consentium IoT's TinyML and EdgeNeuron AI platforms to perform real-time occupancy classification using the AMG8833 Grid-EYE thermal sensor and TensorFlow Lite Micro. The project is designed for edge devices like the ESP32 and Raspberry Pi Pico W.

---

Features

• Edge Compatibility: Works seamlessly with ESP32 and Raspberry Pi Pico W.

• Real-time Inference: Employs TensorFlow Lite Micro for low-latency predictions.

• Occupancy Classification: Identifies room occupancy in the following categories:

  • Empty

  • One Person

  • Two Persons

  • Three Persons

---

Prerequisites

Hardware Requirements

1. Edge Board: ESP32 or Raspberry Pi Pico W

2. Sensor: AMG8833 Grid-EYE Thermal Sensor

3. Connections: I2C communication setup (SCL and SDA)

Software Requirements

• Arduino IDE with relevant libraries:

  • EdgeNeuron.h

  • EdgeMath.h

  • SparkFun_GridEYE_Arduino_Library.h

---

Code Overview

1. Header Section

The header provides essential information about the example, including its purpose, supported features, and licensing.

2. Libraries and Configuration

#include <Wire.h>
#include <EdgeNeuron.h>
#include <EdgeMath.h>
#include <SparkFun_GridEYE_Arduino_Library.h>
#include "edge_neuron_model.h"

• Wire.h: For I2C communication.

• EdgeNeuron.h: Facilitates model initialization and inference.

• EdgeMath.h: Handles preprocessing like scaling and finding maximum probabilities.

• SparkFun_GridEYE_Arduino_Library.h: Provides an interface for the AMG8833 sensor.

---

3. Memory Allocation for TensorFlow Lite Model

constexpr int kTensorArenaSize = 10000;
alignas(16) uint8_t tensor_arena[kTensorArenaSize];

• Tensor Arena: Allocates memory for model inference.

• Input and Output Tensors:

  • kInputSize: 64 (8x8 thermal sensor pixels).

  • kOutputSize: 4 (class probabilities).

---

4. Class Labels

String class_array[] = {"Empty", "One_person", "Two_persons", "Three_persons"};

Defines human-readable labels for the predicted classes.

---

5. Setup Function

void setup() {
  // Initialize serial communication
  Serial.begin(115200);
  
  // Initialize the Grid-EYE sensor
  grideye.begin();
  
  Serial.println("[Consentium INFO] GridEYE sensor initialized successfully!");

  // Initialize the EdgeNeuron model
  if (!initializeModel(edge_neuron_model, tensor_arena, kTensorArenaSize)) {
    Serial.println("[Consentium ERROR] Model initialization failed!");
    while (true); // Halt execution
  }
  Serial.println("[Consentium INFO] Model initialized successfully!");
}

Tasks Performed:

1. Initializes the serial monitor for debugging.

2. Configures the Grid-EYE sensor for data capture.

3. Initializes the TensorFlow Lite Micro model using the EdgeNeuron platform.

---

6. Loop Function

Main Steps:

1. Capture Data from Grid-EYE Sensor

   for (int i = 0; i < kInputSize; i++) {
     input_data[i] = grideye.getPixelTemperature(i);
     if (isnan(input_data[i])) {
       input_data[i] = 0.0; // Replace invalid sensor data
     }
   }

   • Captures the 8x8 thermal grid data from the sensor.

   • Replaces invalid data points (NaN) with 0.0.

---

1. Preprocess Input Data

   edgemath.scaler_transform(input_data, scaler_mean, scaler_scale, kInputSize);

   • Scales the input data using mean and scale values from training.

---

1. Set Model Input Tensor

   for (int i = 0; i < kInputSize; i++) {
     setModelInput(input_data[i], i);
   }

   • Loads the preprocessed data into the model's input tensor.

---

1. Run Model Inference

   if (!runModelInference()) {
     Serial.println("[Consentium ERROR] Inference failed!");
     return;
   }

   • Executes the TensorFlow Lite model for real-time inference.

---

1. Retrieve Output and Predict Class

   for (int i = 0; i < kOutputSize; i++) {
     output_data[i] = getModelOutput(i);
   }
   int class_pos = edgemath.argmax(output_data, kOutputSize);
   Serial.println("[Consentium INFO] Predicted Class: " + class_array[class_pos]);

   • Retrieves the predicted probabilities for each class.

   • Uses argmax to identify the class with the highest probability.

---

1. Log Class Probabilities

   for (int i = 0; i < kOutputSize; i++) {
     Serial.print("  ");
     Serial.print(class_array[i]);
     Serial.print(": ");
     Serial.println(output_data[i]);
   }

   • Outputs the predicted class probabilities to the serial monitor.

---

7. Delay for the Next Cycle

delay(1000);

Introduces a 1-second delay between consecutive inferences.

---

Example Output

Serial Monitor Logs

***************************************************
*    Consentium IoT - Occupancy Classification   *
*        Powered by EdgeNeuron AI Platform       *
***************************************************

[Consentium INFO] GridEYE sensor initialized successfully!
[Consentium INFO] Model initialized successfully!
[Consentium INFO] Capturing temperature data from Grid-EYE...
[Consentium INFO] Data capture complete.
[Consentium INFO] Preprocessing input data...
[Consentium INFO] Data preprocessing complete.
[Consentium INFO] Running inference...
[Consentium INFO] Inference complete. Time taken: 23 ms
[Consentium INFO] Predicted Class: One_person
[Consentium INFO] Class probabilities:
  Empty: 0.12
  One_person: 0.85
  Two_persons: 0.03
  Three_persons: 0.00

---

References

• Consentium IoT Documentation

• For additional support, email [email protected]

---

License

This project is licensed under the MIT License. Redistribution must include the header file.