The m2ag-thing-framework (formerly m2ag-iot-framework) has reached alpha 3. The functionality of this release represents MVP for the project. The framework generates (RFC?) compliant webthings and is compatible with the Mozilla IoT gateway.
Designed with security and privacy in mind the m2ag-thing-framework comes out of the box pre-configured for ssl for every thing and http basic auth for the api and management app. Coming in beta one will be a jwt implementation for the webthing security to provide peace of mind that devices can not be easily hijacked.
The device side implementation can happily run inside your firewall and never connect to the internet after the initial install. All data is kept locally. Data logging and rules engine support can be provided by the Mozilla IoT gateway. All files needed to run the thing are local after configuration.
The html5 management app is used to configure and monitor the device. Custom devices pages are automatically generated for installed things allowing debug and monitoring. The pages can be individually addressed to they can be used as ui’s for simple cases. The management app needs to be connected to the cloud while modules are being added to a thing, but afterwards does not.
The m2ag.labs iot framework is intended as a way to remove tedium from the iot development experience as well as make low cost diy IoT projects feasible for people who may not otherwise pursue them.
Given that a majority of projects are generally geared towards similar ends, a lot of boilerplate can be automatically generated to take care of the basics. These basics include security, communication, installation and remote control.
Security first architecture. With the prevalence of network connected devices making it into the wild there is a good chance one will be a trojan. Any connected device could monitor network traffic and poke at ports on discovered devices. With that in mind this framework defaults to ssl and at least minimal authentication. For alpha 1 there is ssl on all external connections, and HTTP basic auth on the framework api. JWT auth for the webthing will be coming in alpha2.
Self signed certificates must be used. I understand this can be difficult for some users to tackle but it will have to be diy for alpha 1. This will entail generating the certificate authority and distributing a root certificate to all systems/browsers/node apps/ that need to verify thing connections. I have a post here about that — and a good link to follow is here. There will be an option for m2ag.labs to manage this in the future (probably beta 1 or 2 — a couple of months).
Configure your device without coding (well kinda). This is done by specifying configuration and then dynamically constructing the needed classes at runtime. Each things element will get a thing description. This description will include all things related entries to generate a w3c compliant webthing. Each thing will take a component class that will provide an interface to the actual device drivers. The component class is really just a wrapper around the actual device library to provide a standard interface for exchanging data. Adafruit circuit python libraries and gpio zero will be favored in my implementations, but most python driver libraries and i/o access can be used.
The above configuration builds these things:
Common things/components will be available in a repo (coming soon) but it is pretty simple to create the required files. The needed circuit python files need to be installed separately for now.
The m2ag.labs IoT framework will be MIT licensed.
Preliminary installer available. After placing your self singed certificates in .m2ag-labs/ssl run:
The installer is a rough version, and will run for quite some time. Check the console for errors. If all goes well you can browse to https://(your thing) and load the config page. The installer assumes a fresh install of Raspbian Buster. It will do an update before starting.
The framework can be installed on the Mozilla Gateway image, but will conflict with the iptables the image implements. The config page expects to be at 443, so the gateway has to move to use it’s default port. In the m2ag-labs/installer/extras/nginx folder there is a config to run the gateway on 8443 using your self signed certificates. The external Mozilla connection (via page kite) is not affected by this. Alpha 2 installer will look for the gateway image and adjust things accordingly. Alpha 2 will also implement stricter control over the systems ports to only allow those needed. No insecure connections will be allowed.
Does not load down the device. Resource usage is relatively light. The framework and things do not over tax even the Raspberry PI zero w.
A B3+ barely notices the framework:
There is plenty of head room to add other services. Any thing that runs on a PI can be used as a thing (a little coaxing maybe required).
I have to use a slightly modified version of server.py for the webthing framework to get things to work behind a proxy. I’ll be following up with the webthing folks about this for alpha 2. You can see those tweaks here:
You are welcome to try the framework out. It’s totally at your risk, and expect breaking changes until we hit beta. But I find the framework useful in its current state. Support will be limited to the comments section here for now.
Currently the use of properties is only supported by the dynamic thing builder. Events and actions will have to be added by hand. Dynamically generated events are slated for Alpha 2, actions will be addressed in beta 1.
Code quality: It’s a little of the hack and burn variety. Future releases will see the ui become more componentized with webcomponents and more abstraction of the generated python classes.
The m2ag.labs IoT framework currently targets the Raspberry PI and Raspbian. The framework is in pre-alpha but us useable with custom modules installed. Custom modules will be the topic of next weeks post, along with some example code.
The usual warnings apply — this framework may not be compatible with the alpha or later versions of the framework. Some the of the internals need some work, but the framework can be made to work for you now.
Install Raspbian Lite
Perform initial start up
Configure WIFI, SSH, i2c, change password and hostname
I use Raspbian Lite unless there is a need for desktop UI on the device. Generally the stuff I do runs headless and is controlled via a web app.
After the image is written do the initial start up for the PI. The easiest way is to add a ssh file to the boot directory of the image we just created and connect the device to an ethernet hub. Just ssh to firstname.lastname@example.org and go. Another way is to plug in a monitor and keyboard but I am too lazy to go get one. The RaspberryPI site has instructions on setting up headless with wifi but I haven’t had much luck getting this to work lately (the wifi won’t start) .
After we get logged in (pi/raspberry) we need to configure wifi and enable ssh (if we haven’t already) It’s a good idea to go a head a change:
the default password
the time zone
Next – install the X509 certificates. This has been discussed previously on m2aglabs.com. I haven’t decided if I will add a configuration option to run without ssh. I’m not thinking it is a good idea. But — ssh can be turned off with a little manual file editing. For the device, it is in the file device/comm/comm.py. The logger is just one file. For the client look in device/api/static/js/comm.js. I don’t think it is a good idea to not use ssh, even in a controlled network.
To create and deploy the certificates follow the instructions in this link or use the tool at https://github.com/dakshshah96/local-cert-generator to generate the certs. Remember — the root CA has to be imported to each OS that wants to access the IoT device securely. The blog post at m2aglabs.com contains links to instructions for the most common systems. I use Mac, IOS and Linux around here but Windows and Android are all good.
After the certificates are generated place them in the pi users home directory in a .certs subdirectory. Both the Flask app and Mosquitto MQTT server will access them from here. We are looking for the filenames server.crt and server.key.
Next – cd to home/pi and get the install.sh with wget :
This script does apt update and upgrade first, then the install. It can take quite a while to complete. When the script is complete the system should be installed and running. Check the script results for errors. Then navigate to your device on port 5000 (raspib.local:5000) and get the client page opened (be sure to have imported the root CA so ssl will work.
You will get a warning about no user name password. Click on ‘credentials’ in the upper right and enter the default username/password (pi/raspberry).
The device will use the hostname as a login for mqtt, default password is raspberry. When we get the page and the login we can see that the api app is running and ssl is correctly recognized by the browser.
Click on ‘Get Device’ list to populate the list. You should see a listing for the device that you can click on:
This selects the device setup us up for an mqtt connection to the IoT core of or device. Commands can be sent via the Communications tab:
If you have a piezo buzzer attached to pin 13 the preceding command will make it play a tone. In the command — the prm object can carry your custom command to your IoT device. Returned data will be available in the output area. I have a post here that details how I set my buzzer up.
Limited DB access via Database tab,
The code is pre-alpha so there are plenty of warts. There is a problem with the HTML5 app that make the initial mqtt connection a little contrary but it can be made to work. Just try sending a command a couple of times if you can’t get the initial command to work. Much work needs to be done to HTML5 app ( would you like to help? ).
The UI and accompanying API are going to get some usability improvements next, followed by some work on the IoT device.
One of the things I want for my IoT devices is a smart power button. When off or in low power I would like to be able to press the button and have the device power on. If the device is on I want to press the power button and have the device shutdown safely and either power off or indicate that power can be shut off.
The PI has built in support for the seed of my solution in the form of device tree overlays (dto) that can activate kernel support for a shut down signal and a powered down signal. I will be using these two features to implement a simple shutdown button and running indicator led. Not quite the feature set I am looking for, but this may be useful to pull out for some circumstances. Besides, if I can control a led at shut down I could send a signal to put the PI in low power or have it’s UPS shutdown.
There are many sources on the web that detail these two dto’s . The two I found most helpful were:
With this configuration I am monitoring pin 17 for a low signal to initiate system shut down. When the system has halted, pin 19 will go low. Pin 19 will go from low to high on startup.
When pin 19 signals shut down there are two low to high transitions that look like this:
So — using this line to control a running led would have the led on while running. When the system shuts down the led will blink once and then go out indicating it is safe to power off. Keeping the led lit adds 10 ma to the measured current draw of the PI. If this is a concern the logic could be reversed and the led could be made to illuminate at system shutdown. I’m going to be using the led as a running indicator, on while running.
These simple circuits are all that we need to implement our running indicator and shutdown switch.
In case it is not clear — those upside diamonds are grounds at the bottom of the schematic. I like the other style ground symbols better.
D1 is a generic led I had on my desk. R1 is 500 ohms because I have so many of them. R3 and C1 are for switch debounce. You can leave these out if you want. I added them because my switch was glitchy when activated:
And there you have it. With a simple file edit and circuit we now have a shutdown switch and a running indicator for our PI. We no longer have to log in and issue a shutdown command and have to assume it is safe to power off our PI. This is a good first step, but it would be handy if we could get this to be a smart power switch. We will be looking at this in an up coming post.
A while back I released some simple code to allow access to a SQLite database on my embedded devices. A python/html5 app that provided an HTML5 interface to a python api. My goal is a simple to use method that allows quick and easy access to databases on raspberry pi (or similar) used for configuration and logging on my IoT devices. The requirements are minimal :
Some assembly is required — mainly in the form of generating self signed certificates for the devices and browsers that will be used to access them. Check out my previous post about securing local IoT devices. Generate the certs that you need before installing the sqlite.remote tool.
Since I use this on the Raspberry PI it is installed in the pi user’s home directory. Any system that supports python should work, as should any user.
Then cd to the install directory and chmod the installer script:
chmod +x install.sh
Then ./install.sh to install dependencies. This app requires flask and flask_httpauth. Depending on your current update status the dependencies may already be installed.
After the install is completed the install can be tested by changing to the project root and running:
python3 api.py sqlite-remote.sqlite
This should start the api server and show a message like this in the console:
As you can see we are using the barebones development server to run our api. Despite the message to not use this in production, I am using this in production. My usage only calls for one or two users at a time infrequently hitting the api. I decided to trade a simple implementation for a more robust web server and more complicated install. Plus, the web server would generally sit idle for day to day use.
It should also be noted that the api is being served over https. This is requires a self signed certificate to be installed on the device. This is configured at the bottom of api.py:
Adjust the paths to the correct certificate files. The api can be run without ssl if desired. For me, I want to provide at least a minimal level of security to my IoT devices. I intend this to be a config and control interface, but the devices will generally be inside a controlled firewall. Self signed certs and a basic login meets my needs.
As part of the install a service a file is and copied to the systemd system directory. If the default location (/home/pi/m2ag-sqlite-remote) is changed, change it in this file. To enable the service:
sudo systemctl m2ag-sqlite-remote enable
sudo systemctl m2ag-sqlite-remote start
After the api is running navigate to your device at:
The default user is “pi” and the password is “raspberry”. These can be set in the credentials popup:
Select close to get back to query screen. If the query button is pressed with an empty query an “Ok” message will be returned. This tells us the configuration is correct:
To change the password use e the credentials popup:
‘Ok’ will appear in the status on a successful change.
To add new users first select * from users and get the hash for pi’s password and then insert a user using the same password hash. You can then use the new user’s credentials and change the password with the set password dialog.
This app should handle most updates and edits to tables. Since the services setup specifies the database at startup additional tables to be added to the sqlite-remote.sqlite database. The only requirement for the app is the user table with a text username and password fields. This table could be added to any database. Just insert the pi user (or any other user) into the user tables via the command line app or some other tool.
It is hoped that you find this app useful, please feel free to open issues on github or comment here if there are problems with the app. Keep in mind it is barebones on purpose, all I need it to do is update tables remotely for me. If more complex usage is envisioned it may be modifications will need to be made.
If you find this work helpful perhaps you would consider supporting m2ag.labs open source efforts by buying us coffee. Any amount would be appreciated. Please use this link to do so:
In this post I want to demonstrate some techniques for figuring out how to interface with an i2c chip and some basic verification techniques. I will be using a Raspberry PI 3 B, but these techniques will apply to most Linux based embedded systems with an i2c bus.
I2C is short for Inter-Integrated Circuit, a two wire bus used for connecting components to each other on electronic devices. I2C is just one option used for interconnecting devices. Other options include. SPI, 1-wire, and Serial. When using devices for an embedded project we generally find SPI or I2C on devices. I tend to go for I2C devices when I can because they allow me to get away with using less wires to hook up.
When we integrate a device it is handy to ensure the device is recognized by the system before we try to write code for it. There are tools available for this called I2C tools. Formerly a part of the LM-Sensors project, I2C tools allow low level access to I2C busses and the devices connected to them. These tools are not installed on the PI by default but are available via apt:
sudo apt install i2c-tools
Let’s get started. The four commands we are interested in are summarized in this table:
We use the i2cdetect command to both find the busses available as well as the devices on the bus. If we execute
we can get a list of the currently installed I2C busses. The output is as as follows:
We can see we have 1 I2C bus on the PI. If we wanted to see the devices on bus 1, we would enter:
i2cdetect -r 1
and get something like the following output:
When executing this command we get the standard warning that executing this command may jack up our device. It’s true, we can cause errors if we run this command on some busses on other devices (like the obsolete Intel Edison), but it has always cleared up for me with a power cycle of the Device. Usually the device just hangs.
Looking at the output we can see we have four devices on bus 1. At 0x18 we have a MCP9808 temperature sensor, 0x3C is a SSD1306 display, 0x48 is an ADS1819 ADC , and 0x77 is a BMP085 barometric pressure and temperature sensor.
Device register dumping:
Now that we know what devices are on the board (or more accurately all the devices we wired to the PI are recognized by our system) we can start working with the devices. Each device has internal registers that control the device and provide output from the device. We use the i2cdump command to take a look at these registers. Let’s take a look at the MCP9808
i2cdump 1 0x18 w
The form of the command is 12cdump (bus | device address | option). The ‘w’ option give us word output which makes it easier (for me at least) to read the registers. Executing the command gives us output that looks like this:
To understand what we are seeing in the dump we need to study the data sheet for our MCP9808.
Keeping in mind that the MCP9808 is a little endian device, we see the 16 bit data registers storing their values with the least significant byte (LSB) first and most significant byte (MSB) second. We just need to do a quick byte swap to look at register 0 to see the value is 0x001d. According to the spec sheet the MSB should always be 0x00 as well as the LSB bits 7-4. Bits 3-0 in the LSB are the pointer used to select which register we read and write to when communicating to the device. The MCP9808 is a surprisingly complex device, so we won’t look at all the registers. But a thorough explanation of the registers and their function begins on page 16 of the data sheet.
Read a specific register:
We could certainly read the registers from the output of i2cdump but there is a specific command for reading a single register. If we wanted to read the the current temperature the data sheet tells us the register containing this value is 0x05. To read this register we would issue this command:
i2cget 1 0x18 0x05 w
The command is in the form i2cget (bus | device address | register | option) . This gives is the following output:
We get the requisite warning and then the little endian contents of the register. Swapping the bytes to 0xC152 and referencing page 25 of the data sheet we can get the current temperature in Celsius. Briefly — Bits 15-13 are flags that are used when we set the device to monitor a high and low set point. They can tell us if a threshold was passed, and which one. We don’t have those set so the values are not significant to us in this case. Bit 12 of the word is the sign of the two’s compliment value, since it is 0 our value is positive. Stripping those four bits out we are left with 0x0152 which is the temperature value. The rest is a base conversion and a little math — the MSB is 1 decimal, the LSB is 82. So using the data sheet formula –> 1 * 16 + 82 / 16 = 21 degrees Celsius. Thats a cool 69.8 F in the office today.
Set a register:
If we want to manually control the MCP9808 we can set its registers with the i2cset command. Before we use the command a little back ground.
The MCP9808 has two modes of operation — continuous conversion and shutdown mode. Shutdown mode puts the device into a low power state and , as you might have guessed, stops updating the temperature measurement. If we read the temperature register while the device is in shutdown we will retrieve the last measurement made. In a battery powered device it might be a good idea to shutdown the device and only power it up when we need to update the temperature. This is controlled by bit 8 of the config register located at 0x01.
We can use the following command to enter into shutdown mode:
i2cset 1 0x18 0x01 0x0001 w
Remember that we need to swap the bytes in the word due to endianness. This can be tested by reading the temperature while heating up the sensor (I just put my finger on it). You can use i2cget or i2cdump to read register 0x05 of the device. To set the sensor to continuous conversion mode use this command:
i2cset 1 0x18 0x01 0x0000 w
This post covered the basics of verifying a device is detected on an I2C bus and determining if the device is functioning properly. We used the four I2C tools commands to detect the busses and devices installed, dump a device registers, read a specific register, and to write a register. With these four commands we can certainly see if our devices are working correctly as we implement an embedded system.
The i2c tools do have other modes of operation that I did not touch on here. These modes can be read about on the I2C tools docs page. Armed with these tools and the data sheet for a device we are ready to get coding.