Project updates

Things have been a little quiet as of late due to me being busy.

The various projects have been progressing nicely in the background and I’ll be getting their progress written up into blog posts as soon as time permits.

In the interim here is a quick update on where things are at:

Wireless thermostat

My Arduino YUN has arrived. This is going to be the perfect piece of equipment for completing my WiFi thermostat project. It takes away the need to worry about hooking up a separate wireless shield. I’ll just need to attach one of my touch screen shields to it and hook up a relay and thermistor and it will be ready to go.

My next wireless thermostat post will go into more detail on this.

Ethernet Thermostat

The Ethernet thermostat is almost complete. The next post on this will detail each component I used, how to set it up and will also discuss connecting it to baseboard heaters and the types of relay needed.

Thermostat controller

The Java based thermostat controller running on the Pi is coming along nicely. The next posts on this will tie in with my earlier post on parallel computing.

The controller is going to leverage Hadoop which is running on other devices in my home to process temperature data. Expect a guide to setting up Hadoop on the Raspberry Pi and also a further look at MPI for C based applications that can also leverage spare processing capacity.

There will also be code uploaded to BitBucket which you can download and use as a base for your projects.


Upcoming reviews

I have some upcoming reviews on the Arduino YUN, the Pebble smart watch and Mint floor sweeper.



MPI (Message Passing Interface) – Part 1


In a previous post I touched on the fact we would be examining HPC (High Performance Computing), parallel processing and Hadoop in relation to Home Automation.

As our home network grows with multiple devices, tablets, microprocessors and similar running on it we reach a point where we have many CPU’s which are often sitting dormant. Being able to leverage this spare computational power for working on other tasks in our home poses an interesting challenge.

One method we can use in the case of multiple Raspberry Pi units is to use applications that run using MPI (Message Passing Interface). The MPI provides us with a set of base routines and functionality for communicating between multiple CPU’s.

Originally written with C and Fortran in mind, there are many libraries for languages including Python and Java that allow us to implement MPI based applications.

MPI has eight basic concepts, these are:

Communicator – An object that connects groups of processes in the MPI session.

Point-to-point basics – Allows process to process communication

Collective basics – Performs communication amongst all processes in a process group

Derived datatypes – Used for dealing with datatypes in MPI applications

One-sided communication – A set of operations used for dealing with scenarios where synchronization would be an inconvenience for example the manipulation of matrices and calculating matrix products.

Collective extensions –  For example non-blocking collective operations (A combination of non-blocking point-to-point functionality with collective basics) which can help reduce the over-head on communications.

Dynamic process management –  The ability for an MPI process to help with the creation of new processes or establish communication between MPI processes.

MPI-IO – The abstraction of the I/O management on parallel systems to the MPI

These eight concepts can help us to decide what functionality we wish to use in our home automation applications that implement MPI.

To give an example of how this may be useful in the home let’s look at the following example.

Redundant media unit and thermostat controller

In this scenario we have a thermostat controller running on a Raspberry Pi that calculates various temperature data and adjusts the thermostats, blinds and similar as needed. This thermostat controller for example may also use weather data to decide whether a device that acts as a damp/water sensor should be switched on or off based upon predicted rain fall levels.

Over time as we store more historical data and get weather streams via the web we have an ever increasing amount of data to work with. More data should hopefully provide more precision. However working with more data also means we are crunching more numbers and thus placing an ever increasing burden on the CPU of the Raspberry Pi.

We can optimize our code and database in order to help tackle performance issues, however this will only get us so far.

Now let’s imagine we have a second Raspberry Pi running XMBC in our home, which we use on occasions to watch streamed TV from the web. This Raspberry Pi may be sitting idle for much of the day, so it would be great if we could leverage it’s CPU to crunch our weather data while it is sitting dormant.

This is where using MPI can help. A program written using MPI can check if the media unit is being used and if not set its CPU to work on processing temperature data.

Using the eight basic concepts of MPI we can write an application that will handle the communication between the thermostat controller and XMBC RPi and pass calculations back and forth.

This model also allows us to add further devices to our network that can be used by the thermostat controller for parallel processing.


Our thermostat controller is being written in Java and will run on Tomcat, therefore in the next post on MPI we will look at its Java implementation and how we can use this in our code.

We will explore the techniques used for writing distributed applications and look at a scalable model that allows for the addition of extra devices to the network.

Building the controller system – Part 1


We are now going to look at the first step of building the controller system for our home automation system using the Raspberry Pi as the “brains”.

In order to host web applications, write data to our temperature database and host a method for serving the data stored in the DB via http we are going to need to install a number of tools.

For calculating the temperature and communicating with the thermostats we are going to use Java and setup a Java based web server. Following from this we will once again look at HTSQL and setup an HTSQL server for reading the temperature data from our database. We will also in later posts consider using Hadoop for a scalable parallel based controller capable of handling lots of data.

Having built the controller, we will then need to re-visit the post on Postgres and build out our database.

In part one of this post we will look at installing Java Apache Tomcat.

Setting up Apache Tomcat

Apache Tomcat provides us with a web server that can run Java based applications, distributed through WAR file.

Before we install the WAR file we should setup a new user on our Raspberry Pi. Tomcat will then run under this user, and we can lock down the user so it can only perform certain tasks.

To add a user run the following command from the RPi shell:

useradd controller
passwd <password>

This will create a new user on the system. Under the home directory you will see a list of the user on your RPi. New directories can be added here using the mkdir command. You should create a home directory for your new user controller

mkdir controller

Ownership of the directory can be changed using the chown command, for example:

chown controller:users /home/controller

Now we have a user who will be used for running our controller code and servers under.

Our next step is to install Java on the Raspberry Pi.

Installing Java

We will start by installing the latest version of Java onto our Raspberry Pi. The first command we should run is to remove the existing version installed. Keeping your version of Java up-to-date is important as security patches are released that address underlying security flaws.

To remove your existing Java version run the following from the command line:

sudo apt-get purge openjdk*

Once this is complete we can run the following commands to install Java

 sudo apt-get update
 sudo apt-get install openjdk-7-jdk

In the above command we have installed the Java Development Kit version 7. You should install the latest version, a list can be found at the following URL:

Once you have installed Java run the clean up command:

sudo apt-get clean

You now have the latest version of Java running on your Raspberry Pi. If you need a basic guide to the language you can read IBM’s introduction guide:

Installing Tomcat

We now need to install the Tomcat web server onto the Raspberry Pi. You can read more on Tomcat at the Apache Tomcat website at the link below:

Let’s start by downloading the tar.gz for the web server.

From the command line on your RPi run the following command, make sure you are located in the controller directory under /home:


As with installing Java you should replace the version number above with the latest version.

Once wget has finished running you should be find a .tar.gz file located in the controller directory.

We can untar/unzip the file as follows:

tar xzf apache-tomcat-7.0.39.tar.gz

Now change directory into the conf folder located in the new apache tomcat folder created when untar/unzipping

cd apache-tomcat-7.0.39/conf

We can now add our controller user to the configuration so they can start and stop the server.

Open the tomcat-users.xml and add the following:

<user username="controller" password="raspberry" roles="manager-gui"/>

Save the file. We can now test that Tomcat runs.

Navigate to the bin directory e.g.

cd ../bin

then run

sudo sh

This will start your Tomcat server.

Tomcat by default runs on port 8080 of the localhost. In the next post we will take a look at the management console and explain how it works and look at further configuration of our Tomcat server.

Quick update

Just a quick update.

1.) Exams are done, so plenty of time to work on various projects.

2.) In the coming weeks I’ll be doing an interview with Marco of Open Home Automation.

3.) Wrapping up the controller module on the Raspberry Pi

4.) Completing the Ethernet and wireless thermostats

5.) A review of the Mint robot sweeper

6.) A closer look at Nest

7.) Baseboard heaters and wireless thermostats

8.) HPC with the Raspberry Pi


Temperature Database Prototyping – Hardware, Postgresql and HTSQL


In this post I am going to detail prototyping a test Postgresql database server, installing Postgres support on the Raspberry PI (for HTSQL) and then serving up the contents of the DB server via HTSQL over http.

The purpose of this task is to familiarise ourselves with each of the components in the system before we design how the actual database server will be setup and built out.

Throw away test machine

I like to keep a couple of old Windows machine on hand for building out quick test servers that I can use to prototype a system and then wipe out ready for my next project. I find this saves a lot of cost of going out and buying hardware, only to discover your original plan has changed for some technical/design reason and then having to purchase more equipment.

For this task I am using an old Pentium 4 machine running XP with about 60 Gigs of HDD space. I’ve hooked it up with a monitor and keyboard and plugged it into my router via a network cable. At this point it is worth grabbing the IP address of the machine you have just plugged in. This should be available via your routers DHCP table.

Consult your router documentation on how to access this information if you are not familiar with the process. Note: I checked my DHCP table after installing Ubuntu – since this is the only Ubuntu machine on my network currently, it appeared as “ubuntu” in the table

Now my first step is going to be to set it up as a dual-boot machine and load Ubuntu desktop edition onto it.

Ubuntu setup

Depending on the type of machine you use for prototyping the instructions below will obviously be different, for those of you with an old XP machine at hand, the steps below should work for you.

  1. Downloaded Ubuntu’s latest version windows installed here.
  2. Run the installer and follow the on-screen instructions – Note: select your OS type as Ubuntu.
  3. Once your machine has rebooted into Windows follow the next steps:
  4.  Right click My computer and select Properties
  5. Select the Advanced tab from the pop-up box
  6. Under Start-up and Recovery select Settings
  7. Select Time to display list of operating systems checkbox – change the drop-down to 10 seconds
  8. Select the Default operating system drop-down and change this to “Ubuntu”
  9. Save your changes and re-boot your machine into Ubuntu.

Setting up Postgresql

Our Ubuntu machine is going to act as a database server, a Postgresql one in fact. For those of you not familiar with Postresql, it is a powerful, free, open source relational database and has been around for 15+ years. You can read more about it here.

Your first step will be to install the necessary software on Ubuntu.

Follow the instructions below to set up Postgresql

1. Open Terminal in Ubuntu and type the following command:

sudo apt-get install postgresql postgresql-client postgresql-contrib

2. The above command once it has completed running will have installed all the necessary components to start a Postgresql server on your Ubuntu machine.

3. Navigate via the command line to the following directory:


Note: depending on the version of Postgres you have installed the directory 9.1 may be different. As of August 26, 2012 9.1 is the latest version installed via the apt-get command above.

4. You will now need to edit the postgresql.conf in order to tell Postgresql to listen on the IP address assigned to your new dev machine via DHCP. Using vi type:

sudo vi postgresql.conf

Once you have the file open, edit the following line (you may have to un-comment it, and it will probably be set to localhost):

listen_addresses = 'ip address from dhcp table'

Save the file and exit.

5. We can now login to the “template1” DB that comes setup by default with pgsql

sudo -u postgres psql template1

From here for example we can change the password of the postgres account:

ALTER USER postgres with encrypted password 'my_new_password';

6. We now want to create a test database to serve via HTSQL, to do this we type the following:


7. With our new database created we can add a dummy table and a dummy row of data:

CREATE TABLE temperature (code char(5) );
INSERT INTO temperature (code) VALUES ('12345');

Feel free to add some more test rows.

8. Now we can quit postgresql and edit the configuration in order to allow remote access to the DB. In order to do this, you will need to edit the pg_hba.conf file

Change the configuration to include the following (you can use Vi):

local   all         postgres                          md5
host   all         postgres              samenet            md5

Then save and exit

9. We now need to restart our postgresql instance to pick up the changes, this can be done by:

sudo /etc/init.d/postgresql restart

Once the instance has restarted, we are now ready to make some configuration changes on the Raspberry PI.

Raspberry PI configuration

In order to connect our HTSQL server to our newly built Postgresql DB we need to install some extra packages on the Raspberry PI. First of all you will need ssh into the RPI.

Once logged in, follow the steps below:

1. Install python developer packages and the HTSQL postgres package as follows:

sudo apt-get install libpq-dev python-dev
pip install HTSQL-PGSQL

Once these have successfully been installed you should now see:

Successfully installed HTSQL-PGSQL psycopg2

Now we can connect to the postgresql instance.

2. We can test our connection to the testtemp database we created as follows:

htsql-ctl shell -p pgsql://postgres@host:port/database

The host above should be replaced with the IP address of the Ubuntu server you setup with postgresql installed on it. Also remember to include the port number after the IP address, by default postgresql runs on 5432. The final option is the database we created – temptest.

Once you can log in via the HTSQL shell to the DB instances then you are good to go with actually running a server.

3. Exit out of the HTSQL shell and then create a HTSQL server as follows:

htsql-ctl server 'engine://username:password@host:port/database'

engine should be replaced with pgsql, username and password are those that you set on the DB server, host and port are the DB servers IP address and postgresql port (as mentioned above this will probably be 5432) and finally database is our temptest database.

Once you have started the server you will see:

Starting an HTSQL server on raspberrypi:8080 over temptest

4. We can now check that everything is running as expected. Load up your web browser and in the URL bar type:  http://<ip of raspberry pi>:8080

If everything is working as expected you will see

Welcome to HTSQL!

Please enter a query in the address bar.

5. You can then hit the test table we created as follows:

http://<ip of raspberry pi>:8080/temperature

And see some test data (note I have added a few extra rows to my temperature table of random test data)

1 12345
2 1245
3 56789

6. And that’s it, you now have HTSQL running on the Raspberry PI, pointing to a database server.


The above demonstrates the process of setting up a database server and using HTSQL on the Raspberry PI to communicate with it.  However this is only the beginning of what is required in order to build a system that can record our thermostat temperature data.

Using a prototype is useful for the purposes of ensuring we are building a modular system, where each component can be swapped out as necessary. Now we have demonstrated that we can plug a database into the system, we can concentrate on sourcing the right hardware that will replace the prototype.

Of course whilst our Raspberry PI now runs HTSQL, there is still a lot of work to be done on that end of things. We need to configure the RPI so that it is secure and also build out our controller software – which will use the data stored in the Postgres DB to tune our heating system.

The next steps then are going to be to complete the setup of the Raspberry PI and use our prototype DB server to test this configuration against. Once this is complete we will then concentrate on designing a relational database for the heating data, and building out the server hardware.