Category Archives: raspberrypi

m2ag.labs iot framework alpha 1

July 7, 2020

The m2ag.labs iot framework reaches alpha 1.

The m2ag.labs iot framework is intended as a way to remove tedium from the iot development experience as well as make low cost diy IoT projects feasible for people who may not otherwise pursue them.

Given that a majority of projects are generally geared towards similar ends, a lot of boilerplate can be automatically generated to take care of the basics. These basics include security, communication, installation and remote control.

Security first architecture. With the prevalence of network connected devices making it into the wild there is a good chance one will be a trojan. Any connected device could monitor network traffic and poke at ports on discovered devices. With that in mind this framework defaults to ssl and at least minimal authentication. For alpha 1 there is ssl on all external connections, and HTTP basic auth on the framework api. JWT auth for the webthing will be coming in alpha2.

Self signed certificates must be used. I understand this can be difficult for some users to tackle but it will have to be diy for alpha 1. This will entail generating the certificate authority and distributing a root certificate to all systems/browsers/node apps/ that need to verify thing connections. I have a post here about that — and a good link to follow is here. There will be an option for m2ag.labs to manage this in the future (probably beta 1 or 2 — a couple of months).

Configure your device without coding (well kinda). This is done by specifying configuration and then dynamically constructing the needed classes at runtime. Each things element will get a thing description. This description will include all things related entries to generate a w3c compliant webthing. Each thing will take a component class that will provide an interface to the actual device drivers. The component class is really just a wrapper around the actual device library to provide a standard interface for exchanging data. Adafruit circuit python libraries and gpio zero will be favored in my implementations, but most python driver libraries and i/o access can be used.

The above configuration builds these things:

Common things/components will be available in a repo (coming soon) but it is pretty simple to create the required files. The needed circuit python files need to be installed separately for now.

Opensource implementation:

The m2ag.labs IoT framework will be MIT licensed.

Preliminary installer available. After placing your self singed certificates in .m2ag-labs/ssl run:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/m2ag-labs/m2ag-iot-installer/master/install.sh)"

The installer is a rough version, and will run for quite some time. Check the console for errors. If all goes well you can browse to https://(your thing) and load the config page. The installer assumes a fresh install of Raspbian Buster. It will do an update before starting.

The framework can be installed on the Mozilla Gateway image, but will conflict with the iptables the image implements. The config page expects to be at 443, so the gateway has to move to use it’s default port. In the m2ag-labs/installer/extras/nginx folder there is a config to run the gateway on 8443 using your self signed certificates. The external Mozilla connection (via page kite) is not affected by this. Alpha 2 installer will look for the gateway image and adjust things accordingly. Alpha 2 will also implement stricter control over the systems ports to only allow those needed. No insecure connections will be allowed.

Does not load down the device. Resource usage is relatively light. The framework and things do not over tax even the Raspberry PI zero w.

A B3+ barely notices the framework:

There is plenty of head room to add other services. Any thing that runs on a PI can be used as a thing (a little coaxing maybe required).

It’s all on github:

https://github.com/m2ag-labs/m2ag-iot-client

https://github.com/m2ag-labs/m2ag-iot-thing

https://github.com/m2ag-labs/m2ag-iot-installer

I have to use a slightly modified version of server.py for the webthing framework to get things to work behind a proxy. I’ll be following up with the webthing folks about this for alpha 2. You can see those tweaks here:

https://github.com/m2ag-labs/webthing-python

You are welcome to try the framework out. It’s totally at your risk, and expect breaking changes until we hit beta. But I find the framework useful in its current state. Support will be limited to the comments section here for now.

Currently the use of properties is only supported by the dynamic thing builder. Events and actions will have to be added by hand. Dynamically generated events are slated for Alpha 2, actions will be addressed in beta 1.

Code quality: It’s a little of the hack and burn variety. Future releases will see the ui become more componentized with webcomponents and more abstraction of the generated python classes.

m2ag.labs iot framework block diagram

m2ag.labs IoT framework

The m2ag.labs IoT framework is a collection of techniques for running very capable IoT devices completely disconnected from any specific cloud provider. The framework, currently in pre-alpha, is an moderately opinionated implementation that seeks to cover the basics of communication, security and configuration of most IoT devices needs. The implementor will have a consistent framework with which to start projects with and only need to focus on the IoT functionality to be implemented.

The framework itself is intended to be used inside a controlled firewall with no access to the internet. Because of this the self signed X509 certificates are used to provide SSL and MQTTS connectivity for the device.

The framework consists of several distinct parts:

  • API
  • UI
  • Action Engine
  • Logger
  • Device
  • COMM module
  • Device manager
  • Hardware interface
  • Multiple component modules to implement functionality

Installer:

A basic install script will ensure required elements are installed and put in a basic configuration. Initially there will be some configuration that is manually completed but as the framework matures a more complex installation and configuration system will be implemented.

API:

A REST API to allow access to logs database and configuration pages for the device. In general configuration is set through this API.

The API includes the device database which serves as the persistent memory for the device as well as configuration registers.

UI:

HTML5 based app to provide default access to both the config API and the Device. If device is a HUB, will coordinate a fleet of devices.

Initially just basic connectivity will be implemented. As the framework matures preconfigured modules will be used to allow basic UI of common components to displayed.

Device:

A device is the implementation of the IoT functionality. As the python app service is concerned – it manages MQTT comm, configuration changes and general live activities the specific application requires. Monitor a temperature or pressure and send an alarm. Did a position sensor change — send a message. Basic IoT stuff. Other aspects of the device implementation – the support services – are basic Linux/Raspberry PI configurations. The non-python functions of the device will require separate setup steps. These steps will be documented in the installer.

The device coordinates:

COMM module:

Is a generic interface to allow MQTT access. MQTTs is the mechanism for interacting with the IoT functionality.

Each device may be its own MQTT server, or it may connect to a central MQTT server so that events can be aggregated.  The architecture supports connecting to cloud services in a on your own terms manner. The implementor has complete control of what data gets sent to the cloud, if any.

Device manager:

The device manager coordinates activity between the COMM, API and Hardware. Events are monitored and generated here (TBD a better location — probably on the logger service)

Hardware manager:

Handles command processing amongst the python monitored modules. This module contains a list of submodules that comprise the actual implementation of functionality. An object representing the exposed functionality of a module can be addressed in a simple yet expressive manner that can cause any state change that may be required for the implementation.

Component Modules:

 Generally the object will wrap an hardware item – like a bmp280. The framework will generally wrap an Adafruit Circuit Python library for the module with helpers to facilitate standardized data transfer over the mqtt interface.

Component modules may aggregate components to abstract an more complicated component. The comm framework strives to be agnostic to message traffic.

In general — gpizero will be wrapped to implement desired functionality. One of the framework goals is to be as simple as possible to allow easy adoption of the framework with a minimum of technical ability. You won’t need to necessarily be a software engineer to be able to make use of the framework.

Action Engine:

A separate process that monitors MQTT topics for programmed events. Can collect data and trigger actions. This process also logs the MQTT traffic. Each device in a fleet can run it’s own action engine, or they can defer to a central point.

Logger:

A  logger task monitors MQTT traffic to gather telemetry about the IoT system. Will aggregate traffic and log to a SQLite database of the hub.  This is a separate python app that is running as a service on the hub device (a solo device is its own hub).

The idea is to provide a standardized implementation of the common aspects of getting a system running. Control and monitoring, combining components and events can be coordinated in a easy to implement pattern. 

An installer process will do the preliminary install. This will get the api, logger and device services installed as and enabled as well as getting mosquitto mqtt, circuit python, etc.

Some critical tasks — like generating and staging X509 certificates need to take place before install. The frame work strives to be reasonably secure without too much effort.

Alpha versions of the framework will be available in the upcoming weeks.

If you find this work helpful perhaps you would consider supporting m2ag.labs open source efforts by buying us coffee. Any amount would be appreciated. Please use this link to do so:

Small Donate Button

Thanks for stopping by.

Embedded audio alerts with gpizero and simple circuit

For the devices I tend to make the Raspberry PI is intended to be a controller or sensor of some sort that does not usually have a visual interface attached. It is nice to have the device setup so that all critical functions are easily accessible, and the device be able to give indications of type. Certainly LEDS are easy to implement on the PI. A simple transistor allows us to interface a wide assortment of devices. In this case I want to interface a piezo buzzer to provide audible alerts for device activity.

Now granted, I could attach a plain speaker to the appropriate port and play audio files. There is a lot of overhead to doing that and it would be way overkill for what I am trying to accomplish. Generally I am looking for a beep at power up, maybe a tone when the network is down, an alarm for a sensor – you know.

A simple circuit is all you need –

A few pennies worth of discrete components implements the buzzer

6.8 K worked for me, but you should match R1 to the transistor you are using. This makes for an audible tone that is not too loud. If you need a reference to calculate R1 this is a good source: https://www.electronics-tutorials.ws/transistor/tran_4.html

Python code to implement tones could not be simpler. Using gpiozero‘s buzzer can give basic beep or tonal buzzer can be used to create compelling tones. Pwmled can also be used to give an attention getting alert.

from gpiozero import PWMLED
import time

buzzer = PWMLED(13)
buzzer.pulse(fade_in_time=0.5, fade_out_time=0.2, n=10, background=True)
time.sleep(10)

Vary the fade_in_time and fade_out_time to experiment with tones. I’ll be using some variation of this implementation in the upcoming m2ag.labs IoT framework for general device side alerting. As I develop tones I will add them the the m2ag.labs example repo. I’d be happy to include your tones of you create a pull request.

If you find this work helpful perhaps you would consider supporting m2ag.labs open source efforts by buying us coffee. Any amount would be appreciated. Please use this link to do so:

Small Donate Button

Thanks for stopping by.

a simple circuit to implement a run indicator and shutdown button

Raspberry PI running indicator and shutdown button

One of the things I want for my IoT devices is a smart power button. When off or in low power I would like to be able to press the button and have the device power on. If the device is on I want to press the power button and have the device shutdown safely and either power off or indicate that power can be shut off.

The PI has built in support for the seed of my solution in the form of device tree overlays  (dto) that can activate kernel support for a shut down signal and a powered down signal. I will be using these two features to implement a simple shutdown button and running indicator led. Not quite the feature set I am looking for, but this may be useful to pull out for some circumstances. Besides, if I can control a led at shut down I could send a signal to put the PI in low power or have it’s UPS shutdown.

There are many sources on the web that detail these two dto’s . The two I found most helpful were:

shutdown signal:
https://www.raspberrypi.org/forums/viewtopic.php?t=217442
 
toggling gpio at shutdown:
https://www.raspberrypi.org/forums/viewtopic.php?t=223157

It should be noted that specifying the gpio pin to use is different in these two overlays. My implementation looks like this (in /boot/config.txt):

dtoverlay=gpio-shutdown,gpio_pin=17,active_low=1,gpio_pull=up
dtoverlay=gpio-poweroff,gpiopin=19,active_low=1

With this configuration I am monitoring pin 17 for a low signal to initiate system shut down. When the system has halted, pin 19 will go low. Pin 19 will go from low to high on startup.

When pin 19 signals shut down there are two low to high transitions that look like this:

the has shut down signal toggles low to high once before staying low
The has shut down signal toggles low to high once before staying low

So — using this line to control a running led would have the led on while running. When the system shuts down the led will blink once and then go out indicating it is safe to power off. Keeping the led lit adds 10 ma to the measured current draw of the PI. If this is a concern the logic could be reversed and the led could be made to illuminate at system shutdown. I’m going to be using the led as a running indicator, on while running.

These simple circuits are all that we need to implement our running indicator and shutdown switch.

a simple circuit to implement a run indicator and shutdown button
For a dollars worth of parts we can add some helpful features.

In case it is not clear — those upside diamonds are grounds at the bottom of the schematic. I like the other style ground symbols better.

D1 is a generic led I had on my desk. R1 is 500 ohms because I have so many of them. R3 and C1 are for switch debounce. You can leave these out if you want. I added them because my switch was glitchy when activated:

Without r3/c1:

glitchy shutdown signal
With out R3/C1 my shutdown signals usually look like this

With r3/c1

glitch free shutdown signal
With r3/c1 installed the shutdown signal is glitch free

And there you have it. With a simple file edit and circuit we now have a shutdown switch and a running indicator for our PI. We no longer have to log in and issue a shutdown command and have to assume it is safe to power off our PI. This is a good first step, but it would be handy if we could get this to be a smart power switch. We will be looking at this in an up coming post.

These links are relevant for what was done here:

These links are relevant for what was done here:
 
https://www.onsemi.com/pub/Collateral/2N3903-D.PDF
 
https://www.thegeekpub.com/246471/debouncing-a-switch-in-hardware-or-software/
 
https://hackaday.com/2015/12/09/embed-with-elliot-debounce-your-noisy-buttons-part-i/
 
https://www.electronics-tutorials.ws/transistor/tran_4.html
 
Logic captures by the awesome Saleae logic analyzer

If you find this work helpful perhaps you would consider supporting m2ag.labs open source efforts by buying us coffee. Any amount would be appreciated. Please use this link to do so:

Small Donate Button

Thanks for stopping by.

Control an Arduino from Python with PyCmdMessenger

March 6, 2020

There are many methods that can be used to provide communication with a host computer and an Arduino device. ArduinoCmdMessenger is a stable and relatively easy to use choice. It enables simple messages to be sent to the Arduino to control actions and return data.

Command Messenger itself can be installed via the Arduino Library Manager. The latest is version 4.0. It comes with examples in both C# and Visual Basic. My requirement is Python, so the PyCmdMessenger will be used. PyCmdMessenger can be installed via pip3. 

Both the projects have not had many updates in the recent years, but I find in combination they address the needs of my projects.

The problem:

I want to use Raspberry PI for IoT work. I want to use an Arduino to run some sensors that the PI will poll and publish.

True, the PI can handle most sensors and do the work for me. But in this particular case I have one of Adafruit’s CCS811  air quality sensors. The only way to get the sensor to work directly on the Raspberry PI is to slow the i2c bus speed down as described here -> https://learn.adafruit.com/adafruit-ccs811-air-quality-sensor/raspberry-pi-wiring-test .

I don’t really want to slow the i2c bus down on the PI as I have it busy with some other sensors. Plus — the implementation I have in mind is the CCS811 and a BME280 comprise an add on module for other projects. Using an Arduino Leonardo clone, the DF Robot Beetle, I can treat the implementation as a plug and play add on for other projects such as our up coming Raspberry Alarm Clock.

There are other options like Firmata. But I was not looking to couple the Arduino code so closely. I could roll my own implementation, but I’d rather use something that is already available. I  just want to be able to reliably pass commands and data back and forth asyhronously.  PyCmdMessenger provides a solution for this.

An example project can be found in the m2ag.labs examples repo. The code is documented and can be modified to use alternate sensors.

Prototype Sensor module.
Prototype Beetle sensor module. There is plenty of unused capacity on the Beetle that could be put to use.

Links:

https://github.com/thijse/Arduino-CmdMessenger

https://github.com/harmsm/PyCmdMessenger

I2C Interfacing on the Raspberry PI

In this post I want to demonstrate some techniques for figuring out how to interface with an i2c chip and some basic verification techniques. I will be using a Raspberry PI 3 B, but these techniques will apply to most Linux based embedded systems with an i2c bus.

I2C is short for Inter-Integrated Circuit, a two wire bus used for connecting components to each other on electronic devices. I2C is just one option used for interconnecting devices. Other options  include. SPI, 1-wire, and Serial. When using devices for an embedded project we generally find SPI or I2C on devices. I tend to go for I2C devices when I can because they allow me to get away with using less wires to hook up.

When we integrate a device it is handy to ensure the device is recognized by the system before we try to write code for it. There are tools available for this called  I2C tools. Formerly a part of the LM-Sensors project, I2C tools allow low level access to I2C busses and the devices connected to them. These tools are not installed on the PI by default but are available via apt:

sudo apt install i2c-tools 

Let’s get started. The four commands we are interested in are summarized in this table:

Bus scanningi2cdetect
Device register dumpingi2cdump
Device register readingi2cget
Device register settingi2cset

Bus scanning:  

We use the i2cdetect command to both find the busses available as well as the devices on the bus. If we execute

i2cdetect -l

we can get a list of the currently installed I2C busses. The output is as as follows:

We can see we have 1 I2C bus on the PI.  If we wanted to see the devices on bus 1, we would enter:

i2cdetect -r 1

and get something like the following output:

The result of i2cdetect -r 1. This gives us a dump listing the devices on i2c bus 1.
The result of i2cdetect -r 1. This gives us a dump listing the devices on i2c bus 1.

When executing this command we get the standard warning that executing this command may jack up our device. It’s true, we can cause errors if we run this command on some busses on other devices (like the obsolete Intel Edison), but it has always cleared up for me with a power cycle of the Device. Usually the device just hangs.

Looking at the output we can see we have four devices on bus 1. At 0x18 we have a MCP9808 temperature sensor, 0x3C is a SSD1306 display, 0x48 is an ADS1819 ADC , and 0x77 is a BMP085 barometric pressure and temperature sensor.

Device register dumping:

Now that we know what devices are on the board (or more accurately all the devices we wired to the PI are recognized by our system) we can start working with the devices. Each device has internal registers that control the device and provide output from the device. We use the i2cdump command to take a look at these registers. Let’s take a look at the MCP9808

i2cdump 1 0x18 w

The form of the command is 12cdump (bus | device address | option). The ‘w’ option give us word output which makes it easier (for me at least) to read the registers. Executing the command gives us output that looks like this:

i2cdump of our MCP9808 on i2c bus 1 in word format. There is a lot of register space for devices, but we see we only have 13 registers in our device.
i2cdump of our MCP9808 on i2c bus 1 in word format. There is a lot of register space for devices, but we see we only have 13 registers in our device.

To understand what we are seeing in the dump we need to study the data sheet for our MCP9808.

If we study the data sheet for the MCP9808 we can decipher the registers we dumped.
If we study the data sheet for the MCP9808 we can decipher the registers we dumped.

Keeping in mind that the MCP9808 is a little endian device, we see the 16 bit data registers storing their values with the least significant  byte (LSB) first and most significant byte (MSB) second. We just need to do a quick byte swap to look at register 0 to see the value is 0x001d. According to the spec sheet the MSB should always be 0x00 as well as the LSB bits 7-4. Bits 3-0 in the LSB are the pointer used to select which register we read and write to when communicating to the device. The MCP9808 is a surprisingly complex device, so we won’t look at all the registers. But a thorough explanation of the registers and their function begins on page 16 of the data sheet.

Read a specific register:

We could certainly read the registers from the output of  i2cdump but there is a specific command for reading a single register. If we wanted to read the the current temperature the data sheet tells us the register containing this value is 0x05. To read this register we would issue this command:

i2cget 1 0x18 0x05 w

The command is in the form i2cget (bus | device address | register | option) . This gives is the following output:

Use i2cget to read the current temperature from our MCP9808.
Use i2cget to read the current temperature from our MCP9808.

We get the requisite warning and then the little endian contents of the register. Swapping the bytes to 0xC152 and referencing page 25 of the data sheet we can get the current temperature in Celsius. Briefly —   Bits 15-13 are flags that are used when we set the device to monitor a high and low set point. They can tell us if a threshold was passed, and which one. We don’t have those set so the values are not significant to us in this case. Bit 12 of the word is the sign of the two’s compliment value, since it is 0 our value is positive. Stripping those four bits out we are left with 0x0152 which is the temperature value. The rest is a base conversion and a little math — the MSB is 1 decimal, the LSB is 82. So using the data sheet formula  –> 1 * 16 + 82 / 16  = 21 degrees Celsius. Thats a cool 69.8 F in the office today.

Set a register:

If we want to manually control the MCP9808 we can set its registers with the i2cset command. Before we use the command a little back ground.

The MCP9808 has two modes of operation — continuous conversion and shutdown mode. Shutdown mode puts the device into a low power state and , as you might have guessed, stops updating the temperature measurement. If we read the temperature register while the device is in shutdown we will retrieve the last measurement made. In a battery powered device it might be a good idea to shutdown the device and only power it up when we need to update the temperature. This is controlled by bit 8 of the config register located at 0x01.

We can use the following command to enter into shutdown mode:

 i2cset 1 0x18 0x01 0x0001 w

Remember that we need to swap the bytes in the word due to endianness. This can be tested by reading the temperature while heating up the sensor (I just put my finger on it). You can use i2cget or i2cdump to read register 0x05 of the device. To set the sensor to continuous conversion mode use this command:

i2cset 1 0x18 0x01 0x0000 w

This post covered the basics of verifying a device is detected on an I2C bus and determining if the device is functioning properly. We used the four I2C tools commands to detect the busses and devices installed, dump a device registers, read a specific register, and to write a register. With these four commands we can certainly see if our devices are working correctly as we implement an embedded system.

The i2c tools do have other modes of operation that I did not touch on here. These modes can be read about on the I2C tools docs page. Armed with these tools and the data sheet for a device we are ready to get coding.