I personally develop all my software using embedXcode. It is an extension to the Xcode development environment to develop software for embedded platforms. After development I port the software back as sketch for the Arduino IDE.
If you check the Arduino sketch, you probably note the almost empty file “DataLoggerMinimal”. This file just creates an instance of Application and is calling the setup() and loop() method on this instance.
The Application Class
The Application class assembles all components into the final application. It generates a little overhead, but adds a “namespace” to the component instances which avoids naming conflicts. It requires the initialisation of the former global instances in the constructor of the Application class.
Application::Application()
: dht(3, DHT22), rtc(), modeSelector(), storage(), logSystem(0, &storage)
{
}
The Setup Method
The setup() method starts with the initialisation of all component instances:
void Application::setup()
{
// Initialize the serial interface.
Serial.begin(57600);
// Initialize all libraries
Wire.begin();
dht.begin();
rtc.begin();
modeSelector.begin();
storage.begin();
After the initialisation, a header is sent to the serial interface:
// Write some initial greeting.
Serial.println(F("Lucky Resistor's Data Logger Version 1"));
Serial.println(F("--------------------------------------"));
Serial.flush();
Next I check if the real time clock is running. If the clock is not running, the application is stopped and an error is signalled using the red LED.
if (!rtc.isrunning()) {
Serial.println(F("Warning! RTC is not running."));
signalError(3);
}
The Read Mode
// Check the mode.
if (modeSelector.getMode() == ModeSelector::Read) {
Serial.print(F("Read selected. Sending "));
const uint32_t numberOfRecords = logSystem.currentNumberOfRecords();
Serial.print(numberOfRecords);
Serial.println(F(" records."));
for (uint32_t i = 0; i < numberOfRecords; ++i) {
LogRecord record = logSystem.getLogRecord(i);
record.writeToSerial();
}
Serial.println(F("Finished successfully. Enter sleep mode."));
Serial.flush();
If read mode is detected using the ModeSelector component, the LogSystem is used to read and output all stored entries to the serial interface.
set_sleep_mode(B010); // Enter power-down mode.
cli(); // no interrupts to wake the cpu again.
sleep_mode(); // enter sleep mode.
After the output, I put the microcontroller into power-down mode. Because I disable all interrupts using cli(), there is no way the controller ever wakes up again (except from an external interrupt like a press of the reset button of course).
The Format Mode
} else if (modeSelector.getMode() == ModeSelector::Format) {
Serial.println(F("Format (!) selected. Format is starting in ~10 seconds."));
// using LED on pin 13 to blink aggresively.
pinMode(SIGNAL_LED, OUTPUT);
digitalWrite(SIGNAL_LED, LOW);
for (int8_t i = 1; i <= 10; ++i) {
Serial.print(i);
Serial.println("...");
Serial.flush();
for (int8_t j = 0; j < i; ++j) {
digitalWrite(SIGNAL_LED, HIGH);
delay(50);
digitalWrite(SIGNAL_LED, LOW);
delay(100);
}
delay(1000);
}
Serial.println(F("Erasing all logged records..."));
logSystem.format();
Serial.println(F("Format finished successfully. Enter sleep mode."));
Serial.flush();
set_sleep_mode(B010); // Enter power-down mode.
cli(); // no interrupts to wake the cpu again.
sleep_mode(); // enter sleep mode.
If the format mode is detected, the first part of this code produces a countdown flashing the red LED on the microcontroller. After this warning period, the format() method on the LogSystem is called and the microcontroller is put into power-down mode as explained before.
The Read Mode
If the log mode is detected, there is first a bunch of output which is sent to the serial interface. Basically some important numbers are calculated and presented to the user (if he has the serial interface attached).
} else {
// Write about the logging mode.
Serial.print(F("Logging selected. Interval = "));
Serial.println(modeSelector.getIntervalText());
Serial.print(F("Maximum records: "));
Serial.println(logSystem.maximumNumberOfRecords());
Serial.print(F("Current records: "));
Serial.println(logSystem.currentNumberOfRecords());
// calculate how long we can record data.
const uint32_t availableRecords = logSystem.maximumNumberOfRecords()-logSystem.currentNumberOfRecords();
Serial.print(F("Avaliable records: "));
Serial.println(availableRecords);
const uint32_t recordingTime = (availableRecords*modeSelector.getInterval());
Serial.print(F("Recording time: "));
sendDurationToSerial(recordingTime);
Serial.println();
Serial.print(F("Current time: "));
_currentTime = rtc.now();
sendDateTimeToSerial(_currentTime);
Serial.println();
const DateTime recordingEndTime = DateTime(_currentTime.unixtime() + recordingTime);
Serial.print(F("Recording end time: "));
sendDateTimeToSerial(recordingEndTime);
Serial.println();
Next the pin for the red LED is set as output and the LED is turned off.
// Enable the red led as output.
pinMode(SIGNAL_LED, OUTPUT);
digitalWrite(SIGNAL_LED, LOW);
Now I setup timer two of the microcontroller to the slowest possible speed. This timer is used to wake the microcontroller after putting it into power-save mode. See more details in the section “Putting the Microcontroller to Sleep”.
Next I calculate a good interval to check the RTC if we reached the time to record the next measurement. I use a 1/10 of the selected interval, but at maximum 60 seconds.
// Keep the sleep interval between 1s and 1m
_sleepDelay = min(modeSelector.getInterval() / 10, 60);
Now I prepare the next record time. This is the time when the next measurement is taken and logged.
// Set the next record time.
_nextRecordTime = DateTime(_currentTime.unixtime() + modeSelector.getInterval());
Putting the Microcontroller to Sleep
To save as much power as possible, I put the microcontroller to power-save mode while waiting for the next measurement. The microcontroller will only wake from this mode if an interrupt is triggered. I can use timer two of the microcontroller to trigger this required interrupt.
Sadly the Pro Trinket has no pin for the external timer clock, I could use it with the external 1s clock of the RTC to use timer two as a perfect power-save timer. I have to use the internal clock of the processor.
The Pro Trinket is running with 16MHz. Setting the prescaler to 1024 will increase the timer counter at ~15kHz. Timer two is a 8-bit counter and we use the overflow interrupt it will count 256 times until the interrupt is signalled. This is approximate 61Hz or 0.0164 seconds. It seems very short, but putting the microcontroller repeatedly into power-save mode for this time period will save a huge amount of power.
I first create an empty interrupt handler:
// Create an empty interrupt for timer2 overflow. // The interrupt is only used to wake from sleep. EMPTY_INTERRUPT(TIMER2_OVF_vect)
In the log mode, in the setup() method, I prepare timer two to generate an interrupt on overflow. This happens after 0.0164 seconds after setting the timer counter to zero. At the end I enable interrupts.
// Prepare the timer2 to wake from sleep.
ASSR = 0; // Synchronous internal clock.
TCCR2A = _BV(WGM21)|_BV(WGM20); // Normal operation. Fast PWM.
TCCR2B |= _BV(CS22)|_BV(CS21)|_BV(CS20); // Prescaler to 1024.
OCR2A = 0; // Ignore the compare
OCR2B = 0; // Ignore the compare
TIMSK2 = _BV(TOIE2); // Interrupt on overflow.
sei(); // Allow interrupts.
The rest is done in the powerSave() method:
void Application::powerSave(uint16_t seconds)
{
// Go to sleep (for 1/60s).
SMCR = _BV(SM1)|_BV(SM0); // Power-save mode.
const uint32_t waitIntervals = (seconds*61); // This is almost a second.
for (uint32_t i = 0; i < waitIntervals; ++i) {
TCNT2 = 0; // reset the timer.
SMCR |= _BV(SE); // Enable sleep mode.
sleep_cpu();
SMCR &amp;= ~_BV(SE); // Disable sleep mode.
}
}
This method will approximate keep the microcontroller in power-save mode for the given number of seconds.
First I select the correct sleep mode (power-save), calculate the required number of wait intervals and start a loop counting them.
In the loop I reset timer counter two to zero and put the microcontroller to sleep. It will wake after 0.0164 seconds. The loop will repeat this until the number of seconds passed.
This is not optimal as mentioned above. The optimal solution would be an external signal from the RTC. The RTC has a 1s clock output which would be perfect for this task. It could be used to trigger the timer two counter. So I could program how many seconds I would like to wait in the timer and the interrupt would only signalled after this number of seconds.
The Loop Method
The loop starts with the measurement of the values from the sensor. This is a very slow operation.
void Application::loop()
{
// Read the values from the sensor
const uint8_t humidity = dht.readHumidity();
const uint8_t temperature = dht.readTemperature();
The measured values are stored with the current date and time to the log system.
// Write the record
LogRecord logRecord(_currentTime, temperature, humidity);
if (!logSystem.appendRecord(logRecord)) {
// storage is full
signalError(5);
}
The next block waits until it is time for the next measurement.
// Wait until we reached the right time.
while (true) {
powerSave(_sleepDelay);
_currentTime = rtc.now();
const int32_t secondsToNextRecord = _nextRecordTime.unixtime()-_currentTime.unixtime();
if (secondsToNextRecord<_sleepDelay) { if (secondsToNextRecord > 0) {
powerSave(secondsToNextRecord);
}
break;
}
}
First I wait the calculated number of seconds (1/10 of the selected interval), then check the real time clock for the current time. I calculate the difference to the desired time. This will give me the number of seconds to the next record. If this number is smaller than the calculated sleep delay, I wait the remaining seconds and leave the loop. Otherwise the loop is repeated.
// Read the current time for the log entry.
_currentTime = rtc.now();
// Increase the next record time. This will keep the timing stable, even
// if we do not wake up precise at the right time.
_nextRecordTime = DateTime(_nextRecordTime.unixtime() + modeSelector.getInterval());
At the end of the loop() method, I store the current time for the next record and calculate the next record time based on the last record time. This will guarantee some stability because the used system to wait for the next record is not very precise.
The Storage Class
The storage class provides a simple abstraction to the used memory to store the records.
class Storage
{
public:
Storage();
~Storage();
public:
void begin();
uint32_t size();
uint8_t readByte(uint32_t index);
void writeByte(uint32_t index, uint8_t data);
};
For a desktop application I would write an abstract base class as interface and different classes with concrete implementations. In this limited environment, a simple class using different implementations is enough.
The implementation simple uses the EEPROM code from the Arduino library to implement the methods.
uint32_t Storage::size()
{
return EEPROM.length();
}
void Storage::writeByte(uint32_t index, uint8_t data)
{
EEPROM.update(index, data);
}
uint8_t Storage::readByte(uint32_t index)
{
return EEPROM.read(index);
}
I am using update() instead of write() to minimise the writes to the EEPROM.
The ModeSelector Class
The mode selector class provides a wrapper around the BCD DIL switch on the board. It returns the selected mode and time interval.
This is a good example, how to use classes like this to abstract hardware in your devices. It makes your code more modular and easier to understand. At the application level you do not have to know the pins where the DIL switch is connected neither the meanings of the numbers.
The class starts with a number of definitions which allow a simple change of the used pins.
#define MODE_SELECTOR_PIN_D1 4 #define MODE_SELECTOR_PIN_D2 5 #define MODE_SELECTOR_PIN_D4 6 #define MODE_SELECTOR_PIN_D8 8
The class itself provides only the required methods.
class ModeSelector
{
public:
enum Mode {
Log,
Read,
Format
};
public:
ModeSelector();
~ModeSelector();
public:
void begin();
Mode getMode();
uint32_t getInterval();
String getIntervalText();
private:
uint8_t _selectedValue;
};
The value is only read once in the begin() method using getSelectedValue().
void ModeSelector::begin()
{
pinMode(MODE_SELECTOR_PIN_D1, INPUT_PULLUP);
pinMode(MODE_SELECTOR_PIN_D2, INPUT_PULLUP);
pinMode(MODE_SELECTOR_PIN_D4, INPUT_PULLUP);
pinMode(MODE_SELECTOR_PIN_D8, INPUT_PULLUP);
delay(100);
_selectedValue = getSelectedValue();
}
The getSelectedValue() is very simple. Because I use the pull up resistors from the ATmega, LOW means 1 and HIGH means 0. The namespace around this method avoids any naming conflicts.
namespace {
uint8_t getSelectedValue()
{
uint8_t result = 0;
if (digitalRead(MODE_SELECTOR_PIN_D1) == LOW) {
result |= B0001;
}
if (digitalRead(MODE_SELECTOR_PIN_D2) == LOW) {
result |= B0010;
}
if (digitalRead(MODE_SELECTOR_PIN_D4) == LOW) {
result |= B0100;
}
if (digitalRead(MODE_SELECTOR_PIN_D8) == LOW) {
result |= B1000;
}
return result;
}
}
The getMode(), getInterval() and getIntervalText() are very simple and need no explanation.
The LogSystem Class
This class provides an abstraction to the process of write and read of log entries. The header file starts with an interesting definition:
#define LOG_SYSTEM_YEAR_BASE 2015
To save space in memory, the year is only stored with two digits. Because a data logger will never work with dates in the past, this value sets the base year which can always be set to the current year.
If the log system finds a year lower than “15”, it will automatically assume it is a year from the next century. So the value “14” will be converted into “2114”.
Next is the definition of a log record. This is just to pass a log record to the system or read it from the system. It is not related to the format how the records are stored in memory.
class LogRecord
{
public:
LogRecord(const DateTime &amp;dateTime, int8_t temperature, uint8_t humidity);
LogRecord();
~LogRecord();
public:
bool isNull() const;
DateTime getDateTime() const;
int8_t getTemperature() const;
int8_t getHumidity() const;
void writeToSerial() const;
private:
DateTime _dateTime;
int8_t _temperature;
uint8_t _humidity;
};
The log system itself provides a minimal set of methods for the data logger.
class LogSystem
{
public:
LogSystem(uint32_t reservedForConfig, Storage *storage);
~LogSystem();
public:
uint32_t maximumNumberOfRecords() const;
uint32_t currentNumberOfRecords() const;
LogRecord getLogRecord(uint32_t index) const;
bool appendRecord(const LogRecord &amp;logRecord);
void format();
private:
uint32_t _reservedForConfig;
Storage *_storage;
uint32_t _currentNumberOfRecords;
uint32_t _maximumNumberOfRecords;
};
The Implementation
After the simple implementation of the LogRecord class, the struct with the internal representation of a log entry follows.
struct InternalLogRecord
{
uint8_t year : 7; // 00-99
uint8_t month : 4; // 0-11
uint8_t day : 5; // 0-30
uint16_t time : 14; // 0-8639 (multiply with 10 to get seconds per day)
uint8_t humidity : 7; // 0-100
int8_t temperature : 7; // -64 - +63
uint8_t crc : 4; //
};
This is a special form of struct, called “bitfield”. It is a way to produce extremely compact structures and let the compiler do all the bit shifting for us.
The numbers after the “:” are bit counts. Usually you specify there less bits than the datatype can hold. Please read all the details here.
The time is stored in 10s interval. Using this method I can store the time of the day in 14 bits.
Each record has some form of minimalistic CRC to check the integrity of the record. It prevents the software from reading corrupt data, but to be honest, for a proper protection an eight bit CRC would be the minimum.
The next method calculates the start of a record in the storage. It is used in the following methods to provide a single point where the algorithm can be changed.
inline uint32_t getRecordStart(uint32_t offset, uint32_t index)
{
return offset + (sizeof(InternalLogRecord) * index);
}
The next two methods provide some kind of raw methods to read, write and zero the internal log records without any interpretation done.
inline InternalLogRecord getInternalRecord(Storage *storage, uint32_t offset, uint32_t index)
{
InternalLogRecord record;
uint8_t *recordPtr = reinterpret_cast<uint8_t*>(&amp;record);
uint32_t storageIndex = getRecordStart(offset, index);
for (uint8_t i = 0; i < sizeof(InternalLogRecord); ++i) { *recordPtr = storage->readByte(storageIndex);
++storageIndex;
++recordPtr;
}
return record;
}
inline void setInternalRecord(Storage *storage, uint32_t offset, InternalLogRecord *record, uint32_t index)
{
uint8_t *recordPtr = reinterpret_cast<uint8_t*>(record);
uint32_t storageIndex = getRecordStart(offset, index);
for (uint8_t i = 0; i < sizeof(InternalLogRecord); ++i) { storage->writeByte(storageIndex, *recordPtr);
++storageIndex;
++recordPtr;
}
}
void zeroInternalRecord(Storage *storage, uint32_t offset, uint32_t index)
{
uint32_t storageIndex = getRecordStart(offset, index);
for (uint8_t i = 0; i < sizeof(InternalLogRecord); ++i) { storage->writeByte(storageIndex, 0);
++storageIndex;
}
}
I use reinterpret_cast to look at the internal structure as a pointer to an array of bytes. This simplifies the read and write process. The reinterpret_cast expression is the preferred method to express a cast which requires an reinterpretation. Writing (uint8_t*) would work too, but do not express my intention.
The CRC algorithm I use calculates a CRC-16 and XORs all nibbles of this result together. This is far from optimal, but provides a minimal protection against data corruption.
uint8_t getCRCForInternalRecord(InternalLogRecord *record)
{
uint16_t crc = 0xFFFF;
InternalLogRecord recordForCRC = *record;
recordForCRC.crc = 0;
uint8_t *recordPtr = reinterpret_cast<uint8_t*>(&amp;recordForCRC);
for (uint8_t i = 0; i < sizeof(InternalLogRecord); ++i) { crc = _crc16_update(crc, *recordPtr); ++recordPtr; } return (crc>>12) ^ ((crc&amp;0x0f00)>>8) ^ ((crc&amp;0x00f0)>>4) ^ (crc&amp;0x000f);
}
To prevent data corruption I also check all values if they are in valid ranges. The method isInternalRecordValid() checks all values and the CRC of the record.
bool isInternalRecordValid(InternalLogRecord *record)
{
if (record->year > 99 ||
record->month > 11 ||
record->day > 30 ||
record->time > 8639 ||
record->humidity > 100) {
return false; // out of range.
}
const uint8_t crc = getCRCForInternalRecord(record);
return crc == record->crc;
}
The Constructor
The constructor of the log system simple scans the storage for valid records until a invalid or null record is detected. It stores the number of valid records in the normal RAM. This can be a slow process, but speed is luckily of no concern for a data logger.
I could store the number of records in the storage itself, but this would have a number of disadvantages.
- Because no write operation can be atomic (there is the risk of power loss at any time, from the user, from the battery), the stored number can be incorrect.
- There can always be corrupt records, better check all records at the start.
- The built-in EEPROM has a limited number of writes, storing the number of records would require to write to some bytes after each new record. This will probably fail after recording 10000 records.
LogSystem::LogSystem(uint32_t reservedForConfig, Storage *storage)
: _reservedForConfig(reservedForConfig), _storage(storage), _currentNumberOfRecords(0), _maximumNumberOfRecords(0)
{
// Calculate the maximum number of records.
_maximumNumberOfRecords = (storage->size() - reservedForConfig) / sizeof(InternalLogRecord);
// Scan the storage for valid records.
uint32_t index = 0;
InternalLogRecord record = getInternalRecord(_storage, _reservedForConfig, index);
while (!isInternalRecordNull(&amp;record)) {
if (!isInternalRecordValid(&amp;record)) {
break;
}
++index;
record = getInternalRecord(_storage, _reservedForConfig, index);
}
_currentNumberOfRecords = index;
}
The Get Method
Next is the method to read log entries. It will simple read the internal record and convert it into an instance of LogRecord.
LogRecord LogSystem::getLogRecord(uint32_t index) const
{
if (index >= _currentNumberOfRecords) {
return LogRecord();
}
InternalLogRecord record = getInternalRecord(_storage, _reservedForConfig, index);
const uint8_t hours = (record.time / 360);
const uint8_t minutes = (record.time / 6) % 60;
const uint8_t seconds = (record.time % 6) * 10;
uint16_t year = record.year + (LOG_SYSTEM_YEAR_BASE/100*100);
if (record.year < (LOG_SYSTEM_YEAR_BASE%100)) {
year += 100;
}
DateTime dateTime(year, record.month+1, record.day+1, hours, minutes, seconds);
return LogRecord(dateTime, record.temperature, record.humidity);
}
The Append Method
The append method first zeros the entry at index+1 (if possible), then writes the new record to index. This is the safest way to write new records. The power may fail at any point without causing much corruption.
bool LogSystem::appendRecord(const LogRecord &amp;logRecord)
{
if (_currentNumberOfRecords >= _maximumNumberOfRecords) {
return false;
}
// zero the following record if possible
if (_currentNumberOfRecords+1 < _maximumNumberOfRecords) {
zeroInternalRecord(_storage, _reservedForConfig, _currentNumberOfRecords+1);
}
// convert the record into the internal structure.
InternalLogRecord internalRecord;
memset(&amp;internalRecord, 0, sizeof(InternalLogRecord));
const DateTime dateTime = logRecord.getDateTime();
internalRecord.year = dateTime.year() % 100;
internalRecord.month = dateTime.month() - 1;
internalRecord.day = dateTime.day() - 1;
uint16_t timeValue = static_cast<uint16_t>(dateTime.hour()) * 360;
timeValue += static_cast<uint16_t>(dateTime.minute()) * 6;
timeValue += dateTime.second() / 10;
internalRecord.time = timeValue;
internalRecord.humidity = logRecord.getHumidity();
internalRecord.temperature = logRecord.getTemperature();
internalRecord.crc = getCRCForInternalRecord(&amp;internalRecord);
setInternalRecord(_storage, _reservedForConfig, &amp;internalRecord, _currentNumberOfRecords);
_currentNumberOfRecords++;
return true;
}
The Format Method
The format method just zeros the first two records in the storage. This avoids unnecessary writes and is enough for this system to reset the record count.
void LogSystem::format()
{
zeroInternalRecord(_storage, _reservedForConfig, 0);
zeroInternalRecord(_storage, _reservedForConfig, 1);
}
Conclusion
As you can see, there is not much magic in this code. It follows many rules of good software.
- Use clear and speaking names for classes, methods and variables (In the implementation too!).
- Use interfaces to encapsulate complex code.
- Divide an application into manageable components.
- Add documentation to each API method.
- Add comments in the code if it is not obvious.
- Let the compiler do the optimisation, unless you run into timing problems.
And there are a number of additional rules I would suggest.
- Use a consistent and clear coding style (e.g. use mine.)
- Avoid preprocessor macros if possible.
- Use
#pragma onceinstead of old-style code guards. - Keep methods short and simple.
And a number of rules which apply to Arduino development.
- Use
_BV(X)instead of(1 << X). - Put all strings/constants into flash memory
- Always use explicit types:
uint8_t,int16_t, etc.