Tag Archives: data-logger

Data Logger Deluxe – Finished + Video

I finally finished the deluxe version of the data logger. The software took longer than expected, because of the tight flash memory limit. I created a short video which should give you an impression how you can use the menus on the data logger.

The video is showing the menu driven user interface. It is controlled using a capacitive keypad. Everything is sealed in this case to protect the electronic components from the high humidity.

RUNNING OUT OF FLASH MEMORY

I usually start software in the best way possible and postpone any optimisations to the end of the process. So I created everything in an abstract object oriented way, but quickly had to go back to a procedural approach for all objects which were singletons anyway.

To keep everything as logical and abstract as possible, I used namespaces to encapsulate the separate modules. This lead to a very clean approach while saving all the memory which was used for object pointers in the code.

WORKING ON THE DOCUMENTATION

Currently I am working on a detailed documentation, which will show step by step how to build this deluxe version of the data logger. There will also be a page with some notes and explanations about the software. You can already have a look at the software here.

Data Logger Deluxe Version Almost Done

Just to let you know the deluxe version of the data logger is almost done. The hardware is ready and tested – currently I finish the menu based software for the data logger.

  • Menu based control with 5 capacitive inputs
  • Record, view records, send records to serial, erase all, settings.
  • Wireless charging.

Data Logger – Sensor Limits

After fixing the timing problem with the sensor library, I started the data logger and recorded data in 10 seconds intervals until the memory was full. This time the AM2302 (DHT22) sensor was read every time, no cached values were used.

It created a really interesting data set you can see in the diagram below (click on the diagram to see all details):

Data Logger Chart Fixed

The actual measured data values are the small dots in the background, and the lines are polynomial interpolated over these values.

For the temperature (blue) you can see the sensor precision limit of 0.1 and an interesting jitter when the value makes the transition from one value to another. In contrast, the humidity values (orange) do not have the same jitter like the temperature values, but you can see something strange at the crossing point of the two series.

In Detail

I created a new diagram where you can see this point in detail:

Data Logger Sensor Problems

The jitter of the humidity values is minimal, but suddenly there are huge jumps of the values. Compare the jumps with the corresponding temperature values. Because the humidity value depends on the temperature, the jitter from the temperature causes the humidity values to jump.

Conclusions

The test was successful: There is now real sensor data for each measurement.

If this sensor is used for a real time display of humidity and temperature values, there has to be some integrated interpolation to flatten the jitter of the read values. This is no big deal for the data logger, because you can easily do all the corrections and interpolations afterwards.

Related Files

You can download the CSV file with the measurements and the Excel sheet I used with the links below:

Data Logger Simple – Software Bugfix

After analysing recorded data from the simple version of the data logger, I found a problem I had to fix with a new release v1.1 of the software.

If you look at this chart, which is showing data recorded in 30 second intervals, you can clearly see the values do not really change every 30 seconds.

Data Logger Problem Chart

The values change every 40 minutes and this can not be correct, even the sensor is slow. I found the problem in the DHT library from Adafruit. It is not actually a problem, but the library relies on the function millis() to measure the time between two reads.

From the DHT sensor library:

boolean DHT::read(void) {
  // Check if sensor was read less than two seconds ago and return early
  // to use last reading.
  uint32_t currenttime = millis();
  if (currenttime < _lastreadtime) {
    // ie there was a rollover
    _lastreadtime = 0;
  }
  if (!_firstreading && ((currenttime - _lastreadtime) < 2000)) {
    return _lastresult; // return last correct measurement
  }
  _firstreading = false;
  _lastreadtime = millis();

The function millis() relies on timer1 which is counting the time for this function. To save power, I put the microcontroller into sleep mode, which stops all timers except timer2. Therefore the time is not counted anymore, and millis() returns a wrong value.

Continue reading Data Logger Simple – Software Bugfix

Long-Term Test Finished

I finished my long-term test of the minimal version of the data logger project after a few days. To simulate the situation for the actual usage, I setup the data logger to record data in 1 hour intervals.

The test answered these questions:

  • Does the data logger function properly over several days?
  • How much battery does the data logger use?

I could not answer the first question until I stopped the data logger and read all values over the TTL cable. I found this is a little bit tricky, because the microcontroller did reset as soon I attached the TTL cable. This automatically restarted the logging and logged some additional value, visible in the chart.

The battery life is hard to measure. I used a multimeter and measured the voltage on the battery. At the end, the voltage was still over 4.0V, which means the battery pack is almost full charged.

I would need to connect the battery to an analog input of the microcontroller, using a MOSFET as a switch. So I could measure the battery voltage over time.

Importing the Data

Working on Mac OS X, I am using the screen command as serial console. If you add the parameter -L to the command, the whole output is automatically logged in a file.

LR: ~lr$ screen -L /dev/cu.usbserial-FTHXXXXX 57600

I opened the file using BBEdit and removed any lines before and after the actual data. After the edit I saved this edited file as “data.csv”.

Temperature Humidity ChartTo get a little bit more options for the diagram, I used Microsoft Excel. After the import of the data, I formatted the columns, added a title and created a X/Y chart from it. After fine tuning all the titles, formats and lines I created a nice chart and could export it as PNG file.

You can see the chart as feature image of this article, or click on the thumbnail on the right side to see the full resolution version.

Conclusions

If you look at the chart you can clearly see the problem of the minimal version. It logs just integer values without fractions which is clearly not enough for the temperature. The sensor is capable of a higher precision, but not accuracy. But the higher precision will make trends better visible in the data.

What is Coming Next?

Next I will implement the simple version of the data logger. This will add additional 32KB of storage to the device, using the Non-Volatile FRAM Breakout from Adafruit. With this “vast” amount of storage, I can use a 14 bytes record which stores the date and time as unix time-stamp, using a float for humidity and temperature and a real CRC-16.

Date/Time    4b
Humidity     4b
Temperature  4b
crc          2b
---------------
total       14b

This will allow 2340 records in the FRAM, which sums up to 97 days recording time at one hour intervals.

Data Logger Documentation Online

I just published the minimal version of the data logger project. The minimal version uses the built-in EEPROM of the microcontroller to log sensor data. I decided to create this additional minimal version because it works for very simple cases and does not need any external memory.

Data Logger Wires

The data logging mode is controlled using a BCD DIL switch. Using this switch you can select logging mode, read mode and format mode.

Data Logger Mode Select

The documentation is explaining how to create this data logger in detail. All about the parts, how to build the hardware, how to build the software and a detailed page which explains the software in detail. There is also a short overview of the project and how to use the final device.

Start reading the detailed documentation here:

The Data Logger Project

If you have any questions or feedback about this project, feel free to leave comments on these pages.

Are My Documents Safe in the Root Cellar at ~80% Humidity?

Update 2015-08-21: Added the wiring diagram.
Update 2015-08-24: The documentation for the minimal version is online here.

There is a large cellar where I could store unused items and documents, but the catch is the humidity there. It is a root cellar near a small brook and the humidity varies between 75% up to 90%.

Archived material should never be exposed to humidity greater than 65%, therefore I have to isolate all documents in boxes from the air of the cellar. But are this boxes safe? Do they keep the humidity away from the documents – even for years?

To have a look into the box environment, I need a data logger. It would be simple to buy one, but much more fun to build one. So a new project is born: I call it the “Data Logger” project.

There are a few phases to realise this project:

  • Prototype (done)
  • Simple Version (in work)
  • Deluxe Version with Display and Case

The Prototype

For the prototype I used a large solderless breadboard from 3M. There I actually just tested the components for the simple version, especially if the protocols are working and if I can use them in the way intended. I will keep this section short and explain everything in detail in the next section. Just to remind you, I tested everything on a breadboard before I soldered the components to a stripboard.

Parts of The Simple Version

The simple version should be a data logger which automatically starts logging temperature and humidity values it it is powered on. Using a small switch, I can select between logging, reading and erase. In the reading mode, the logger will send all logged values to the serial interface.

Because I need quick results, I will build the whole device using components from Adafruit. Here a list of components I will use for the simple version of the data logger:

Continue reading Are My Documents Safe in the Root Cellar at ~80% Humidity?