Streaming and graphing real time data with the Raspberry Pi

So you have a fancy little sensor/contraption and its working great, the system is portable maybe even in a waterproof case or is mobile with no room for a display. However, how do you know the quality of your data or if the system is working without carrying a display or connecting up an SSH session? A black and white terminal impresses no one and is a terrible way to view trends in data.

There is an answer and it is fantastically flexible. You turn your Raspberry PI into a wireless access point and web server. Then with a little HTML, JavaScript, JSON and the chart.js library, you have a clean and dynamic interface for your project.


The particular project in this example uses the following hardware:


When assembled and strapped to a buoy anchored in the swan river it is capable of logging and analysing the motion and frequency of waves. This data is collected as part of the research for Woodside’s River Lab run in partnership with UWA’s Oceans Institute. Here you can see the real-time data stream from the buoy in a lab environment.


The Sensor

The BNO055 sensor performs all of the internal calculation and compensation to output clean acceleration and heading information, however as a demonstration to students and visitors the data is hard to visualise when it must be copied and charted from a CSV file. To improve the accessibility of the data and increase its value this tutorial will utilise several scripting & markup languages and several key libraries:

  • Python
  • HTML

  • CSS
  • JavaScript


Configure the Raspberry Pi

To configure the Raspberry Pi and the website there are several steps. Keep in mind any existing configuration or modifications to a vanilla Raspbian Jessie install may have unforeseen ramifications to this tutorial. Thankfully for my fingers many of these steps are already well documented, the links below will take you to the tutorials I used for some of these steps.

  1. Install Apache Web Server
  2. Configure Raspberry Pi as an Access Point
  3. Modifying your python script
  4. Configuring your script to run on boot
  5. Writing a webpage with chart.js

Having configured your Raspberry Pi as a hot spot web server as per tutorials 1 & 2 above, you should be able to load your web page on a laptop or mobile phone connected to the access point according to the IP address you configured in the second tutorial. If you followed the tutorial verbatim this is 172.24.1.1 although in my application I configured my /etc/network/interfaces AND /etc/dnsmasq.conf files with the address 192.168.0.1 as that is common across other hardware we develop.


Modifying your python script

To improve the portability of our data and leverage existing data handling tools built into python and JavaScript we need to output our sensor data as a JSON file. It is a simple and human readable file format for structuring data reminiscent of a dictionary/list within many languages. Python has some built-in functions that make it easy to output a dictionary as a JSON file. In this example, we want to create one file that details the current sensor reading as well as the calibration state of the sensor.

Most importantly, our JSON data is written to the directory for our web server /var/www/html/whatever.JSON  This means our web server can share the file when it is requested by a client. Any JSON file can be placed and requested natively via http://192.168.0.1/whateverTheFileNameIs.JSON this is great for viewing configuration information and system states without the additional effort of writing and formatting a full web page.

JSON can be viewed directly in the browser with no html required. Great for checking system states

Running the script on boot allows for truly headless operation. Although some form of visual feedback that the code is running, such as flashing an LED, will save any head scratching should the Pi fail to boot or script fail. To run the script on boot we will be modifying the /etc/rc.local file again with another line.

 (sleep:10;python /whatever/the/file/path/is/script.py)& 

As all operations within the rc.local file are already executed with privileges you will not need to include sudo in the command. Full copies of the source code used in this example can be found here.

A word of caution. As you are writing & reading the same file concurrently your code must be tolerant of not being able to access the file on occasion.

Now we have everything configured on the raspberry pi and our data is streaming we can start to build our chart.js data interface.


Andrew Van Dam

An electronics expert based in Australia dedicated to creating great tools for the maker community.


Before and After with Adobe Lightroom

On our recent trip/honeymoon to Koh Samui, I thought I would conjure my inner photographer. It’s fair to say that I still have a lot to learn. The camera I had was a Canon 80D. (Yes not a full frame camera, but I primarily purchased it for the video functionality.) However, saying that it was pretty cool getting down to the beach and having a go at mashing the buttons and seeing what came out at the other end. The results were actually pretty average….as expected. Practice makes perfect hey! Anyhoo nothing a bit of adobe lightroom can’t fix!

I’m definitely no expert on Adobe Lightroom, however, a couple hours messing around and the following pics were ‘Enhanced’.

Drag the slider to reveal the edited image.


This is my very first attempt at a long exposure shot. Trying to draw out all the light I could. To the naked eye, it was pitch black down at the beach, except for the lights of the restaurant. It was difficult to check settings/change settings on the fly as I was doing it due to low light.


To my eye, the green grass did look as good as the Adobe Lightroom edit. I guess I still have a lot of work to do on the camera settings. It was very bright and I was a little bit hungover…


I tried to do something a bit different here. The edit was supposed to highlight the face of the statue and make it stand out more. How do you think I went? any tips? Maybe I could have blurred our the background.


This is what happens when you are really hungover, laying on a very comfortable cushion chair and have absolutely zero motivation to do anything. Yep, take photos of trees and the sky for practice! In the Lightroom version, I tried to bring the green and blue out, however, I think the white in the balloon has become a bit green.


Last one promise! The idea here was to try and dray some of the brown back out of the rock face and the greens into the shrubs. It worked ok I think. Still some work to do.


Well, after all of that it is clear I still have a lot of work to do on the whole “Camera Settings” and “Technique side of photography. Would it be correct in saying that a good photographer should not need to edit his/her photos at all to make them better?


Inspecto Roboto: A dirty robot, just how we like it.

Our resident electronics expert, Andrew has been busy in the man cave this weekend. After reporting some dampness inside his house he decided to take matters into his own hands. The outcome: ‘Inspecto Roboto’! Andrew has pulled apart his racing drone and a few other bits and pieces to hack together a pretty sweet inspection robot for under the house. Check out his build below. And don’t forget to check out some of his other cool projects on his website.

 

Check out the Video in HD here.


Some of the Key components

  • Lynx motion Tri-Tracks chassis
  • 2 x 12V geared DC motors
  • LM298 motor controller drivers

  • 4s LiPo battery pack
  • LED light bar
  • Servos

  • Taranis hobby RC transmitter
  • FRsky receiver
  • Teensy 3.2



The 'Inspecto Roboto' - A confined space inspection robot.

My house has a bit of a moisture problem and I wanted to inspect the floor crawl space to see if there was an obvious fault. It is a horrible cramped environment that whilst I can just wiggle inside the rat carcass at the door confirmed my doubts about venturing any further. So I decided to tackle the problem with robotics!

This was a weekend build and whilst I just used parts lying around I had a lot of nice spares and old projects to work with. Essentially this was a hybrid system using parts from an old racing drone & a lynx motion robotics kit.

Locomotion was a lynx motion Tri-Tracks chassis. It has a nice set of rubberized Tracks as well as an acrylic frame, two 12V geared DC motors and other associated hardware. A basic LM298 motor controller drives each set of tracks from a 4s LiPo battery pack tucked inside. The camera system was from an old racing drone with an FPV display to make driving it around easier. The servo allows for tilting the camera up, panning is just turning the robot on the spot. Finally, a large LED light bar from a vehicle was installed up against the front with some additional acrylic pieces to act as bushes and help retain it in place. The light is controlled via a relay as it is convenient to switch it off when unrequired as it draws about 12W.

The controller was a Taranis hobby RC transmitter paired with an FRsky receiver and a teensy 3.2. The teensy decodes the signals & operates the motors, lights & servo. I have written about using these transmitters with microcontrollers before (Here) and they are a great option for long-range and reliable communication which xBees & other serial radios often struggle with.

The whole system was a bit of a hack but as someone who often spends forever ‘gilding the lily’ on a project until they grow bored of it and don’t finish it, this was a fun and pleasant change.


Andrew Van Dam

An electronics expert based in Australia dedicated to creating great tools for the maker community.