Remote SQLite access

The problem:

I want to use SQLite as a file system format for my configuration setting and log management on my IoT devices I tend to use small embedded Linux units that run headless. Accessing SQLite you have to be local to the db file so this makes it hard to do db maintenance on my devices. The choices I have for access are:

  • Use a network file share to make the db available.
  • Install PHP and use use phpLiteAdmin
  • Remote login with ssh and use the SQLite shell
  • Roll my own web service to do the job.

What would be handy is an API to the database that any app could consume. That way I can automate management apps to tend to my devices SQLite storage. This can allow remote configuration, log caching , etc that can center on the SQLite file as a database. Since I am using Python for device side hardware interfacing chores it seemed natural to do something with it. There are several Python web frame works that could help here and I chose to use Flask. Client side will be a HTML 5 app served by the API.

The repo for the app is at https://github.com/m2ag-labs/sqlite-remote – MIT license. This tool is intended for development use and light duty maintenance. The most common thing I do is trouble shoot messaging issues and pruning tables. For more intensive usage I’m thinking phpLiteAdmin would be a better choice, but you would have to install PHP and a webserver.

Using Ohmite Force sensitive potentiometers.

Force sensing potentiometers offer an alternative to mechanical potentiometers and rotary encoders. These devices do away the mechanical action of the switch and instead offer a touch sensitive alternative. After looking around the web I was not able to find a suitable library for implementing these devices so I ended up creating my own for both Arduino and Circuitpython.

The libraries implement functionality for Ohmite’s FSP series of devices. The devices include one round sensor (FPS03CE), and two linear devices (FSP01CE and FSP02CE). The linear devices come in two lengths.

All devices are single touch. This means that you can only detect on touch location on the device at a time. If we tried to detect a finger on each end of a linear device it would register as being in the middle.

The libraries have a similar api, each implement the functions necessary to get the force applied to the device as well as the position of the touch.

For all devices the force is reported as voltage. Generally 3.2 volts is the highest we will see on a 3.3 volt system. The position is reported as an integer. For the FPS03CE the integer represents 0 to 360 degrees from the tail (connector) of the device. For the linear devices it is 0 to 100 mm for the FSP01CE and 0 to 55 mm for the FSP02CE. Default for the two linear devices is for zero at the tail, but the call to position call can take a parameter to return with 0 at the head end of the device.

Integration Guide:

I was only able to find one source for the integration guide — this pdf at Mouser. It is pretty comprehensive, but contains a couple of errors that an integrator should be aware of.

First: In the section for FSP0(1/2)CE on page 4:

This section from the integration guide specifies V2 as an analog input. It does not need to be.

The document specifies that V2 should be an ADC pin. In measuring the force and position I have not read an Analog value from there. Not a show stopper, but there is no need to tie up an analog input unnecessarily. I changed this to a digital input and these sensors work correctly.

Second: In the section for FSP03CE on page 7:

This table for the FSP03CE pin out is not correct.

Using Figure 10: Pin 4 is actually the the wiper pin, while pin 1 is the third drive electrode. Plus the Pin name for pin 2 is incorrect. Luckily this diagram makes it clear where the connections go.

From the integration guide.

Hook up:

Refer to this diagram when hooking up one the FSP devices.

Use this schematic when wiring up one of the FSP devices. VRef will use a digital I/O

Each device requires one analog input and multiple digital I/O lines. I used 22K for the FSP03CE voltage divider and 18K for both the liner variants. All are 1/4 watt 5%. VRef will use a digital I/O line. These resistor values worked for my application. The sensitivity of response to touch is effected by these values. This is detailed in the integration guide.

The libraries for Arduino and CircuitPython are on on git hub. I tried to make them work similarly but there are a couple differences.

Arduino:
#include "M2aglabs_Ohmite.h"

//Set this to the one the Arduino uses
#define ANALOG_RESOLUTION 12
#define LINEAR_THRESHOLD  0.3
#define ROUND_THRESHOLD  0.3


/*
	Round sensor
	WIPER, VREF, D0, D120, D240
	Linear Sensor
	WIPER, VREF, V1, V2, true/false (Short or Long)
	 
    WIPER and VREF are on two sides of a resistor. VREF floats for position measurements,
	PULLS low for force. 
*/

M2aglabs_Ohmite roundSensor(A5, 0, 2, 1, 3);
M2aglabs_Ohmite lLinear(A2, 7, 6, 10, true);  //true means short
M2aglabs_Ohmite sLinear(A4, 5, 4, 9, false);  //false is long sensor 



void setup() {

	Serial.begin(115200);
	/*
		The lib is set for a default of 10 for analog resolution and 3.3. for voltage. If the voltage is 5.0,
		set it here. 
	*/
	analogReadResolution(ANALOG_RESOLUTION);
	roundSensor.begin(ANALOG_RESOLUTION);
	sLinear.begin(ANALOG_RESOLUTION);
	lLinear.begin(ANALOG_RESOLUTION); 

	//Set options --
	/*
	Serial.println(roundSensor.readRange());
	Serial.println(roundSensor.readRange(1500)); 
	Serial.println(roundSensor.zeroOffset());
	Serial.println(roundSensor.zeroOffset(500));
	*/

}

void loop() {
	roundSensorActions();
	linearSensorActions();
}

void linearSensorActions() {

	int spos, lpos; //Position is an integer 
	float fsp, flp; //Force is a float

	fsp = sLinear.getForce(); 
	
	if (fsp > LINEAR_THRESHOLD) { 
		//False reads from tail to tip. 
		spos = sLinear.getPosition(false);
		Serial.print("s: ");
		Serial.print(fsp);
		Serial.print(" : ");
		Serial.println(spos);

	}

	flp = lLinear.getForce(); 
	if (flp > LINEAR_THRESHOLD) {
		lpos = lLinear.getPosition(false);	
		Serial.print("l: ");
		Serial.print(flp);
		Serial.print(" : ");
		Serial.println(lpos);
	}
}



void roundSensorActions() {

	//Get the force from the round sensor 
	float force = roundSensor.getForce();
	//IF it looks like we are touching it, calculate the position. 
	if (force > ROUND_THRESHOLD) {
		
		Serial.print("force: ");
		Serial.print(force);

		int angle = roundSensor.getPosition(); 
		Serial.print(" raw angle: ");
		Serial.print(angle);

		angle = constrain(angle, 0, 360);

		Serial.print(" adjusted: ");
		Serial.println(angle);
	}
	return;
} 

The function calls are documented in the header for the library.

CircuitPython
import board
from digitalio import DigitalInOut, Direction, Pull
from analogio import AnalogIn
import time
from m2aglabs_fsp import Ohmite

# Round sensor (FSP03CE) -- it needs a lot of inputs
wiper = board.A5
v_ref = board.D0
D_0 = board.D2
D_120 = board.D1
D_240 = board.D3

# Long linear sensor (FSP01CE)
l_wiper = board.A4
l_ref = board.D5
l_v1 = board.D4
l_v2 = board.D9

# Long linear sensor (FSP02CE)
s_wiper = board.A2
s_ref = board.D7
s_v1 = board.D6
s_v2 = board.D10

s_lin = Ohmite(s_wiper, s_ref, s_v1, s_v2, type=2) # FSP02
l_lin = Ohmite(l_wiper, l_ref, l_v1, l_v2, type=1) # FSP01
s_rnd = Ohmite(wiper, v_ref, D_0, D_120, D_240) #FSP03 can add type=0, but default is 0

######################### MAIN LOOP ##############################

s_rnd.begin()
l_lin.begin()
s_lin.begin()

while True:

    s_force = s_lin.get_force()
    force = s_rnd.get_force()
    l_force = l_lin.get_force()

    if s_force > 0.4:
        position = s_lin.get_position(False)
        print(s_force, position)
      
    # for long linear
    if l_force > 0.4:
        position = l_lin.get_position()
        print(l_force, position)
       
    # for round sensor
    if force > 0.09:
        angle = s_rnd.get_position()
        print(force, angle)

The circuit python code is a lot simpler. The differences from Arduino are:

  • the ‘type=’ key word argument sets the type of sensor. 0, or round, is the default
  • the analog resolution for CircuitPython is always 65536 there is no need to set it
  • I didn’t add setters for zero offset and read range. These can be adjusted by editing the library for now. This will be added shortly.
Using the library:

Usage is fairly straightforward. The general steps are:

  • Instantiate the object
  • Call begin
  • Poll for force
  • If there is force applied read the position

These libraries have only been tested on a Metro M4 and Itsybitsy M4 to date. There is nothing that is SAMD51 specific so the Arduino library should work on other devices. I’ll be using some of these sensors on a pro-micro soon, we’ll see how it goes.

Two settings to be aware of is the _ZERO_OFFSET and _READ_RANGE. These effect each sensors overall range. The _ZERO_OFFSET specifies the normal zero reading of the ADC. With a finger at the 0 position of the sensor there is still a voltage present. Depending on the sensor, the voltage will be in the 200 to 800 millivolt range. If the sensor will not go to zero try adjusting this. For round sensors the 0’s are at 0 degrees (at the tail) then clockwise to 120 degrees, then 240. This is detailed in the integration guide.

Read range sets the maximum value of the voltage at the max end of the sensor. So if the lengths come out short, or max is hit before the end reached try adjusting this setting.

Both libraries are available on git hub.

https://github.com/m2ag-labs/m2aglabs_ohmite

https://github.com/m2ag-labs/m2aglabs_ohmite_python

Nodejs, Socket.io and Intel Edison

One of the things I really like about the Intel Edison (and the Galileo) is that they can be run as little Linux computers. We can use a lot of tools that are available for Linux machines. One of those tools in particular is nodejs.

Nodejs is a very powerful server side platform. It comes preinstalled on the Edison and Galileo Yocto images so it is very easy to get started with. We can implement code on the Edison and Galileo with the Intel XDK IoT Edition. With this combination of tools it is very easy to create applications that can monitor and control a remote Edison via a web based interface.

To get started I have created a repo on Git Hub that contains a project for the XDK. This project utilizes several technologies — node js, express , socket.io and  mraa library  on the server side. Jquery and an open source library to implement a gauge are used on the client side. I will be using the MCP9808 setup that I blogged about previously but any sensor (or group of sensors) could be utilized.

Before we jump into the code lets make sure we our Edison in up to date. I am using version 146 of the stock Yocto image. To check the version of your Edison execute the following:

configure_edison --version

If you need to update your Edison you can use the following:

configure_edison --upgrade

If you upgrade your Edison you will have to set up the wifi again.

To install the code on your Edison use the Intel XDK to open the project downloaded from Git hub. If you are using the Edison Arduino board open app.js and change line 23 to use i2c bus 6. Upload the project and use the build action to install all the Node dependencies. If you had an app running on the Edison previously you will need to stop it and start this one with the start and stop buttons.

The server code is located in the root of the project in the app.js file.  Here are lines 6 thru 10:

6  var express = require('express');
7  var app = express();
8  app.use(express.static(__dirname));
9  var server = app.listen(8085);
10 var io = require('socket.io').listen(server);

The first  four lines set up Express. Express is a full featured web framework for Node, but here I am just going to use it to serve up my static files from the file system.  The static server is set up on line 8. Line 9 instructs our server to listen on port 8085. Socket.io is initialized on line 10 and instructed to listen for connections on line 10.

Express may seem like overkill for this application, but I like to use if because it is easy to configure and is well supported. There are other, lighter foot print modules that can be used. You can find these other http servers by looking in the NPM Registry .

The real work in the server is done on lines 28 thru 44 :

28 io.sockets.on('connection', function (socket) {
 
30  setInterval(function () { 
31 socket.emit( 'temp' , JSON.stringify(getTemp())); //send temp every interval
32 }, 2000);

35 socket.on('toggle_led', function(data){
36   if(data === 'on'){
37     myOnboardLed.write(0);
38   } else {
39     myOnboardLed.write(1); 
40   }
41 });
 

44 });

Line 28 sets up socket.io to listen for connections. Each client connecting to this device will start the callback that sends the current temperature every 2 seconds (lines 30 – 32 ) and listen for the ‘toggle_led’ event (lines 35 – 41) to allow an led to be turned on and off.

The client side code is just as simple. The index.html page contains divs that hold the guage and a button.  The javascript code that does the work is in /js/app.js . This is all pretty standard stuff, but the first time I worked with socket.io I was stumped for awhile trying to find the source file for the socket.io client (Line 11 in index.html). This file is served automatically from our node server when we run it and is included in the server side socket.io code. All we need to do is add the line to load it in index.html.

If we look at the client side javascript in /js/app.js :

1  var socket = io.connect();
2
3  socket.on('temp', function (data) {
4    var status = JSON.parse(data);
5    Gauge.Collection.get('temp').setValue(status.farenheit); 
6   });
7
8  function toggle_led(state){
9  
10   socket.emit('toggle_led' , state ); 
11   var button = $('#led_button'); 
12   if(state === 'on'){
13      button.attr("onclick", "toggle_led('off')"); 
14      button.html('Turn off led'); 
15    } else {
16      button.attr("onclick", "toggle_led('on')"); 
17     button.html("Turn on led");1
19    }
20 }

We see that the code is fairly simple as well. The socket.io client is initialized on line 1 with a connection to the server we loaded our script from. We then setup a listener for the “temp” event on line 3. If we get a “temp” event the callback function will parse the json string from the server and update the guage in our page to the current value of our MCP9808. (Check the guage git hub page for comprehensive details on its use.)

To toggle the onboard led we have implemented the function at line 10 to emit the “toggle_led” event to the server. We also swap out the events on the button so we can just use one button to toggle the led.

The thing to remember is that the socket.emit sends an event with the name that we provide (in this case “toggle_led”). We need to ensure that the listener for this event uses the exact string. I’ve messed this up a couple of times.

It should look something like this when we have it running:

When the page is running it will look something like this.

When the page is running it will look something like this.

So there you have it. We have implemented a simple web page on the Edison so that we can get the temperature from out MCP9808 and display it. We have implemented the ability to toggle an led on the Edison from the web page. These examples are pretty basic, but the demonstrate a good starting point for more enhanced functionality that could be implemented. Feel free to use the template as a starting point in your projects.

I2C Interfacing on Intel Edison

In a previous post I discussed how to read the temperature from a MCP9808 temperature sensor. In this post I want to back up a bit to demonstrate some techniques for figuring out how to interface with an i2c chip and some basic verification techniques. I will be using the Intel Edison on the mini-breakout board, but these techniques will apply to most Linux based embedded systems.

The MCP9808 is an i2c bus device. I2C is short for Inter-Integrated Circuit, a two wire bus used for connecting components to each other on electronic devices. I2C is just one option used for interconnecting devices. Other options  include. SPI, 1-wire, and Serial. When using devices for an embedded project we generally find SPI or I2C on devices. I tend to go for I2C devices when I can because they allow me to get away with using less wires to hook up.

When we do integrate a device it is handy to ensure the device is recognized by the system before we try to write code for it. There are tools that are already installed on the Edison called I2C tools. A part of the LM-Sensors project, I2C tools allow low level access to I2C busses and the devices connected to them.

Before we get started lets make sure our Edison is up to date. We can do this by logging in via ssh (or use the terminal) and executing the command:

configure-edison --upgrade

You need to have your Edison attached to a network.

Let’s get started. There are four commands we are interested in that are summarized in the table here:

Bus scanning i2cdetect
Device register dumping i2cdump
Device register reading i2cget
Device register setting i2cset

Bus scanning:  

We use the i2cdetect command to both find the busses available as well as the devices on the bus. If we execute

i2cdetect -l

we can get a list of the currently installed I2C busses. The output is as as follows:

The result of i2cdetect -l on Intel Edison.

The result of i2cdetect -l on Intel Edison.

We can see we have 8 I2C buses on the Edison. But according to the docs we are only able to use bus 1 and 6 on the mini-break out board. I have only used bus 1 in my projects so far. If we wanted to see the devices on bus 1, we would enter:

i2cdetect -r 1

and get something like the following output:

The result of i2cdetect -r 1. This gives us a dump listing the devices on i2c bus 1.

The result of i2cdetect -r 1. This gives us a dump listing the devices on i2c bus 1.

When executing this command we get the standard warning that executing this command may jack up our device. It’s true, we can cause errors if we run this command on some of the other busses, but it has always cleared up for me with a power cycle of the Edison. Usually the device just hangs.

Looking at the output we can see we have four devices on bus 1. At 0x18 we have a MCP9808 temperature sensor, 0x3C is a SSD1306 display, 0x48 is an ADS1819 ADC , and 0x77 is a BMP085 barometric pressure and temperature sensor.

Device register dumping:

Now that we know what devices are on the board (or more accurately all the devices we wired to the Edison are recognized by our system) we can start working with the devices. Each device has internal registers that control the device and provide output from the device. We use the i2cdump command to take a look at these registers. Let’s take a look at the MCP9808

i2cdump 1 0x18 w

The form of the command is 12cdump (bus | device address | option). The ‘w’ option give us word output which makes it easier (for me at least) to read the registers. Executing the command gives us output that looks like this:

i2cdump of our MCP9808 on i2c bus 1 in word format. There is a lot of register space for devices, but we see we only have 13 registers in our device.

i2cdump of our MCP9808 on i2c bus 1 in word format. There is a lot of register space for devices, but we see we only have 13 registers in our device.

To understand what we are seeing in the dump we need to study the data sheet for our MCP9808.

If we study the data sheet for the MCP9808 we can decipher the registers we dumped.

If we study the data sheet for the MCP9808 we can decipher the registers we dumped.

Keeping in mind that the MCP9808 is a little endian device, we see the 16 bit data registers storing their values with the least significant  byte (LSB) first and most significant byte (MSB) second. We just need to do a quick byte swap to look at register 0 to see the value is 0x001d. According to the spec sheet the MSB should always be 0x00 as well as the LSB bits 7-4. Bits 3-0 in the LSB are the pointer used to select which register we read and write to when communicating to the device. The MCP9808 is a surprisingly complex device, so we won’t look at all the registers. But a thorough explanation of the registers and their function begins on page 16 of the data sheet.

Read a specific register:

We could certainly read the registers from the output of  i2cdump but there is a specific command for reading a single register. If we wanted to read the the current temperature the data sheet tells us the register containing this value is 0x05. To read this register we would issue this command:

i2cget 1 0x18 0x05 w

The command is in the form i2cget (bus | device address | register | option) . This gives is the following output:

Use i2cget to read the current temperature from our MCP9808.

Use i2cget to read the current temperature from our MCP9808.

We get the requisite warning and then the little endian contents of the register. Swapping the bytes to 0xC152 and referencing page 25 of the data sheet we can get the current temperature in Celsius. Briefly —   Bits 15-13 are flags that are used when we set the device to monitor a high and low set point. They can tell us if a threshold was passed, and which one. We don’t have those set so the values are not significant to us in this case. Bit 12 of the word is the sign of the two’s compliment value, since it is 0 our value is positive. Stripping those four bits out we are left with 0x0152 which is the temperature value. The rest is a base conversion and a little math — the MSB is 1 decimal, the LSB is 82. So using the data sheet formula  –> 1 * 16 + 82 / 16  = 21 degrees Celsius. Thats a cool 69.8 F in the office today.

Set a register:

If we want to manually control the MCP9808 we can set its registers with the i2cset command. Before we use the command a little back ground.

The MCP9808 has two modes of operation — continuous conversion and shutdown mode. Shutdown mode puts the device into a low power state and , as you might have guessed, stops updating the temperature measurement. If we read the temperature register while the device is in shutdown we will retrieve the last measurement made. In a battery powered device it might be a good idea to shutdown the device and only power it up when we need to update the temperature. This is controlled by bit 8 of the config register located at 0x01.

We can use the following command to enter into shutdown mode:

 i2cset 1 0x18 0x01 0x0001 w

Remember that we need to swap the bytes in the word due to endianness. This can be tested by reading the temperature while heating up the sensor (I just put my finger on it). You can use i2cget or i2cdump to read register 0x05 of the device. To set the sensor to continuous conversion mode use this command:

i2cset 1 0x18 0x01 0x0000 w

This post covered the basics of verifying a device is detected on an I2C bus and determining if the device is functioning properly. We used the four I2C tools commands to detect the busses and devices installed, dump a device registers, read a specific register, and to write a register. With these four commands we can certainly see if our devices are working correctly as we implement an embedded system.

The i2c tools do have other modes of operation that I did not touch on here. These modes can be read about on the I2C tools docs page. Armed with these tools and the data sheet for a device we are ready to get coding.