Building an ESP32 Data Logger with SD Card and Timestamps (Low-Power Guide)

Why the data logger is the unglamorous core of everything #

Every analytics dashboard, every automation rule, every model that claims to “understand” a process is only as good as the data feeding it. And that data has to come from somewhere – usually a sensor sitting on a machine, in a field, or down a borehole, far from a wall socket. The job of a data logger is to acquire that data validly, cheaply, and without disturbing the thing it’s measuring.

In a lab that’s trivial. In the real world – remote sites, no mains power, months between visits — it comes down to three numbers you have to respect from the first line of code: how low your sleep current goes, how high your peak draw spikes when you acquire and transmit, and how often you actually need to move data. Get those right and a coin-cell-class logger runs for years. Get them wrong and you’re changing batteries every few months, and the maintenance bill quietly eats the value of the data.

This guide builds a real ESP32 data logger — sensor → timestamp → microSD — and then shows the sampling and power decisions that separate a bench demo from a field-ready node.

What you’ll build #

• An ESP32 (or ESP32-S3) that wakes on a timer, reads a sensor, and writes a unix-timestamped CSV row to a microSD card.
• Deep sleep between samples, with the clock and a small data buffer preserved in RTC memory so nothing is lost across sleep cycles.
• A batched-write strategy: hold samples in RTC RAM and only touch the SD card occasionally, because the SD write itself costs energy.

Prerequisites: Arduino IDE with the ESP32 board package, a microSD module (SPI), and any sensor (this example uses a simple analog input; an I²C or 4–20 mA sensor drops straight in).

Wiring #

The two *_EN lines are the single most important part of the wiring. A microSD module and most sensor front-ends leak current continuously if you leave them powered. Switching them through a small P- or N-channel MOSFET — powered up only for the milliseconds you actually use them — is what lets the whole node fall to microamps between samples.

The core sketch #

This is the complete logger: it wakes, reads, timestamps, buffers, and writes to SD in batches, then deep-sleeps. It’s adapted from a working NORVI field deployment and trimmed to the essentials.

/*
 * ESP32 Data Logger — microSD + unix timestamps + deep sleep
 * Wakes on a timer, samples a sensor, buffers in RTC memory,
 * and writes timestamped rows to microSD in batches.
 */
#include <Arduino.h>
#include <SPI.h>
#include "SD.h"
#include "esp_sleep.h"
#include <time.h>
#include <WiFi.h>              // only used for the occasional NTP time-sync

// ---------- Pins (adjust to your board) ----------
#define SD_CS        42        // microSD chip-select (SPI)
#define SCLK         35
#define MISO_PIN     36
#define MOSI_PIN     37
#define SD_EN        12        // MOSFET gate: powers the SD module only when needed
#define SENSOR_EN     5        // MOSFET gate: powers the sensor only during a read
#define SENSOR_ADC   14        // example analog sensor pin

// ---------- Timing ----------
#define uS_TO_S          1000000ULL
#define SAMPLE_EVERY_S   900          // take a reading every 15 minutes
#define FLUSH_EVERY_N    4            // write buffered samples to SD every 4 wakes (~1 h)
#define SYNC_TIME_EVERY  96           // re-sync the clock roughly once a day

// ---------- Persisted across deep sleep (RTC memory) ----------
RTC_DATA_ATTR time_t   rtcEpoch     = 0;   // last known unix time
RTC_DATA_ATTR uint32_t wakeCounter  = 0;   // wakes since last time-sync
RTC_DATA_ATTR uint16_t sampleCount  = 0;   // samples currently held in RTC buffer

struct Sample { time_t ts; float value; };
#define RTC_BUFFER 32
RTC_DATA_ATTR Sample rtcBuffer[RTC_BUFFER];   // hold data in RTC RAM, not on SD

// WiFi only needed for the periodic NTP resync (swap for an RTC/modem if offline)
const char* WIFI_SSID = "your-ssid";
const char* WIFI_PASS = "your-pass";

// Dead-reckon the time from the last sync; the deep-sleep timer keeps counting.
time_t currentTimestamp() {
  return rtcEpoch + (time_t)(wakeCounter * SAMPLE_EVERY_S);
}

bool syncTimeNTP() {
  WiFi.begin(WIFI_SSID, WIFI_PASS);
  uint32_t t0 = millis();
  while (WiFi.status() != WL_CONNECTED && millis() - t0 < 15000) delay(200);
  if (WiFi.status() != WL_CONNECTED) return false;

  configTime(0, 0, "pool.ntp.org");         // UTC
  struct tm tm;
  if (!getLocalTime(&tm, 10000)) { WiFi.disconnect(true); return false; }

  rtcEpoch = mktime(&tm);
  wakeCounter = 0;
  WiFi.disconnect(true);
  WiFi.mode(WIFI_OFF);
  return true;
}

float readSensor() {
  digitalWrite(SENSOR_EN, HIGH);            // power the sensor
  delay(50);                                // let it settle
  int raw = analogRead(SENSOR_ADC);
  digitalWrite(SENSOR_EN, LOW);             // cut sensor power immediately
  return raw * (3.3f / 4095.0f);            // convert to your engineering units
}

bool flushToSD() {
  digitalWrite(SD_EN, HIGH);                // power the SD module
  delay(50);
  SPI.begin(SCLK, MISO_PIN, MOSI_PIN, SD_CS);
  if (!SD.begin(SD_CS)) { digitalWrite(SD_EN, LOW); return false; }

  if (!SD.exists("/log.csv")) {             // write the header once
    File h = SD.open("/log.csv", FILE_WRITE);
    if (h) { h.println("timestamp,value"); h.close(); }
  }

  File f = SD.open("/log.csv", FILE_APPEND);
  if (!f) { SD.end(); digitalWrite(SD_EN, LOW); return false; }
  for (uint16_t i = 0; i < sampleCount; i++)
    f.printf("%ld,%.3f\n", (long)rtcBuffer[i].ts, rtcBuffer[i].value);
  f.close();

  SD.end();
  SPI.end();
  digitalWrite(SD_EN, LOW);                 // cut SD power
  sampleCount = 0;                          // buffer written — reset it
  return true;
}

void setup() {
  Serial.begin(115200);
  pinMode(SD_EN, OUTPUT);     digitalWrite(SD_EN, LOW);
  pinMode(SENSOR_EN, OUTPUT); digitalWrite(SENSOR_EN, LOW);

  // Sync the clock on cold boot, then only about once a day.
  if (rtcEpoch == 0 || wakeCounter >= SYNC_TIME_EVERY) syncTimeNTP();

  // 1) One timestamped sample, into the RTC-memory buffer.
  Sample s;
  s.ts = currentTimestamp();
  s.value = readSensor();
  if (sampleCount < RTC_BUFFER) rtcBuffer[sampleCount++] = s;
  wakeCounter++;

  // 2) Only touch the SD card when the buffer is full or the flush window hits.
  if (sampleCount >= RTC_BUFFER || (wakeCounter % FLUSH_EVERY_N) == 0)
    flushToSD();

  // 3) Back to sleep. On wake, the ESP32 restarts execution from setup().
  esp_sleep_enable_timer_wakeup((uint64_t)SAMPLE_EVERY_S * uS_TO_S);
  esp_deep_sleep_start();
}

void loop() { /* never runs — deep sleep restarts from setup() */ }

Three things in that sketch are doing the real work, and they’re worth understanding rather than copying.

Timestamps that survive deep sleep #

A logger is worthless if you can’t trust when each reading happened. The trick is that variables marked RTC_DATA_ATTR live in RTC memory, which stays powered through deep sleep — so rtcEpoch and wakeCounter survive. On cold boot we fetch real time once over NTP, then dead-reckon: each wake is SAMPLE_EVERY_S later than the last, so currentTimestamp() just adds them up. We only pay for a full time-sync about once a day to correct the deep-sleep timer’s drift.

Store the timestamp as a unix epoch (time_t) — one integer, sorts naturally, converts to any timezone downstream, and takes far less space than a formatted date string on both the card and the air. If you’re fully offline, replace the NTP call with a cheap external RTC (DS3231) or the time reported by a cellular modem.

Sampling strategy: sample often, write and transmit rarely #

Here’s the counter-intuitive part. Acquiring a reading is cheap. Moving it is expensive. Two operations dominate a logger’s energy budget, and neither is the measurement:

1. Writing to the SD card. Spinning up the card, mounting the filesystem, and committing a write is a surprisingly heavy, multi-millisecond burst — far more than reading a sensor.
2. Transmitting (if the node is connected). A modem registering on the network and negotiating a TLS session to a server can cost tens of times more energy than the payload itself.

The strategy that falls out of this is the same for both: batch. Keep readings in RTC memory as they’re taken, and only pay the expensive operation once per batch. In the sketch above, samples accumulate in rtcBuffer[] and hit the SD card only every FLUSH_EVERY_N wakes — so a reading every 15 minutes becomes one SD write per hour instead of four. If you add a cellular uplink later, apply the identical idea: buffer many records, connect once, publish them all in a single session, then clear the buffer. You amortise the modem’s registration-and-connect cost — the part that actually drains the battery — across dozens of readings.

Only spill to the SD card when you have to: when the RTC buffer would overflow, or when the next transmission is hours away and you can’t risk holding unsent data in volatile RAM. RTC memory is small but effectively free to write; the SD card is roomy but costs energy every time you open it. Use each for what it’s good at.

The reporting interval, not the battery size, is what sets field life. As a real reference point, the same job on a productised cellular logger stretches from ~1.8 years at a 15-minute interval to ~7.5 years at hourly — purely by transmitting less often.

Power notes: the numbers to watch #

• Deep-sleep current. This is the number that decides multi-year life, because the node spends 99%+ of its time here. Power-gate the SD module, the sensor, and any level shifters (the *_EN MOSFETs above). A well-gated ESP32 in hibernation lands around ~10 µA; a poorly-gated one with a always-on SD module can sit at several milliamps and flatten a battery in weeks.
• Peak draw. Know your spikes — sensor excitation, the SD write, and above all a radio session. Size the battery and any capacitors for the peak, not the average.
• Don’t write what you don’t have to. Every avoided SD open and every avoided transmission is energy kept in the battery. The best write, like the best transmission, is the one you never make.
• Know your MCU’s floor. The ESP32 is superb for prototyping a logger and for WiFi-connected nodes, but its deep-sleep floor (~10 µA, plus RTC) is a real limit. When a deployment needs to run for years on a primary cell, that’s the point where teams move to an MCU built for it — an ultra-low-power ARM Cortex-M0+ like the STM32L0, which idles in the low-microamp range.

From breadboard to field #

This build gets you a working, battery-conscious logger. Turning it into something you can bolt to a pole and forget for years is a different amount of work: a certified cellular modem and antenna, a proper 4–20 mA / RS-485 front end, an IP67 enclosure, a battery pack sized for the mission, and firmware on an MCU that sleeps at microamps.

Need battery power, IP67, and cellular out of the box? That’s what the NORVI EC-M12 is for. It productises exactly this logger — sample, timestamp, buffer, transmit — but on an ultra-low-power STM32L072CZ (not an ESP32), with a SIMCOM/Quectel Cat-M1/NB-IoT modem, a 16-bit ADC front end for 4–20 mA and 0–10 V, RS-485 Modbus, microSD, and dual 19,000 mAh lithium cells behind an IP67 shell. Real deployments — reservoir level monitoring to ThingsBoard, RS-485 Modbus over MQTT — run for years on a single pack using the exact batching strategy above.

→ Browse the range on the battery-powered cellular data logger hub and pull the EC-M12 datasheet.

Further reading #

• EC-M12 user guide — full pinout, ADC and RS-485 front end, programming setup.
• Low-power / sleep-mode application — modem sleep sequence and current figures.