Getting started with Azure Sphere: Part 2 (integration with Azure services)

Last week, I wrote about my experiences getting some sample code running on an Avnet Azure Sphere Starter Kit. That first post walked through installing the SDK, setting up my development environment (I chose to use Visual Studio Code), configuring the device (including creating a tenant, claiming the device, connecting the device to Wi-Fi, and updating the OS), and downloading and deploying a sample app.

Since then, I’ve managed to make some steps forward with the Element 14 out of the box demo by Brian Willess (part 1, part 2 and part 3). Rather than repeat Brian’s posts, I’ll focus on what I did to work around a few challenges along the way.

Working around compiler errors in Visual Studio Code using the command line

My first issue was that the Element 14 blogs are based on Visual Studio – not Visual Studio Code and I was experiencing issues where Code would complain it couldn’t find a compiler.

Thanks my colleague Andrew Hawker who was also experimenting with his Starter Kit, but using a Linux VM, I had a workaround. That workaround was to run CMake and Ninja from the command line, then to sideload the resulting app package onto the device from the Azure Sphere Developer Command Prompt:

cmake ^
-G "Ninja" ^
-DCMAKE_TOOLCHAIN_FILE="C:\Program Files (x86)\Microsoft Azure Sphere SDK\CMakeFiles\AzureSphereToolchain.cmake" ^
-DAZURE_SPHERE_TARGET_API_SET="4" ^
-DAZURE_SPHERE_TARGET_HARDWARE_DEFINITION_DIRECTORY="C:\Users\%username%\AzureSphereHacksterTTC\Hardware\avnet_mt3620_sk" ^
-DAZURE_SPHERE_TARGET_HARDWARE_DEFINITION="avnet_mt3620_sk.json" ^
--no-warn-unused-cli ^
-DCMAKE_BUILD_TYPE="Debug" ^
-DCMAKE_MAKE_PROGRAM="ninja.exe" ^
"C:\Users\%username%\AzureSphereHacksterTTC\AvnetStarterKitReferenceDesign"
ninja
azsphere device sideload deploy --imagepackage AvnetStarterKitReferenceDesign.imagepackage

I wasn’t able to view the debug output (despite my efforts to use PuTTY to read 192.168.35.2:2342) but I was confident that the app was working on the device so moved on to integrating with cloud services.

Brian Willess has since updated the repo so it should now work with Visual Studio Code (at least for the high level application) and I have successfully tested the non-connected scenario (part 1) with the changes.

Integration with Azure IoT Hub, device twins and Azure Time Series Insights

Part 2 of the series of posts I was working though is where the integration starts. The basic steps (refer to Brian Willess’ post for full details) were:

  1. Create an Azure IoT hub, which is a cloud-hosted back-end for secure communication with Internet of Things (IoT) devices, of which the Azure Sphere is just one of many options.
  2. Create and configure the IoT Hub Device Provisioning Service (DPS), including:
    • Downloading a certificate from the Azure Sphere tenant (using azsphere tenant download-CA-certificate --output CAcertificate.cer at the Azure Sphere Developer Command Prompt) and using this to authenticate with the DPS, including validation with the verification code generated by the Azure portal (azsphere tenant download-validation-certificate --output validation.cer --verificationcode verificationcode) and uploading the resulting certificate to the portal.
    • Creating an Enrollment Group, to enrol any newly-claimed device whose certificate is signed by my tenant. This stage also includes the creation of an initial device twin state, editing the JSON to include some extra lines:
      "userLedRed": false,
      "userLedGreen": false,
      "userLedBlue": true
    • The initial blue illumination of the LED means that we can see when the Azure Sphere has successfully connected to the IoT Hub.
  3. Edit the application source code (I used Visual Studio Code but any editor will do) to:
    • Uncomment #define IOT_HUB_APPLICATION in build_options.h.
    • Update the CmdArgs line in app_manifest.json with the ID Scope from the DPS Overview in the Azure portal.
    • Update the AllowedConnections line in app_manifest.json with the FQDNs from the DPS Overview (Global Device Endpoint) and the IoT Hub (Hostname) in the Azure portal.
    • Update the DeviceAuthentication line in app_manifest.json with the Azure Sphere tenant ID (which may be obtained using azsphere tenant show-selected at the Azure Sphere Developer Command Prompt).
  4. Build and run the app. I used the CLI as detailed above, but this should now be possible within Visual Studio Code.
  5. Use the device twin capabilities to manipulate the device, for example turning LEDs on/off (though clearly there are more complex scenarios that could be used in real deployments!).
  6. Create a Time Series Insights resource in Azure, which is an analytics solution to turn IoT data into actionable insights.
    • Create the Time Series Insights environment using the existing IoT Hub with an access policy of iothubowner and consumer group of $Default.
  7. Add events inside the Time Series Insights to view the sensor readings from the Azure Sphere device.
Time Series Insights showing sensor data from an Azure Sphere device.

Time Series Insights can get expensive for a simple test project without any real value. I could quickly have used my entire month’s Azure credits, so I deleted the resource group used to contain my Azure Sphere resources before moving on to the next section…

Integration with Azure IoT Central

Azure IoT Central is a hosted IoT platform. It is intended to take away much of the underlying complexity and let organisations quickly build IoT solutions using just a web interface.

Following part 3 in Brian Willess’ Azure Sphere series, I was able to get my device working with IoT Central, both using the web interface to control the LEDs on the board and also pushing sensor data to a dashboard. As before, these are the basic steps and refer to Brian Willess’ post for full details:

  1. Create a new IoT Central application.
  2. Select or create a template:
    • Use the IoT device custom template.
    • Either import an existing capability model (this was mine) or create one, adding interfaces (sensors, buttons, information, etc.) and capabilities.
    • Create custom views – e.g. for LED device control or for device metrics.
  3. Publish the template.
  4. Configure DPS:
    • Download a certificate from the Azure Sphere tenant using azsphere tenant download-CA-certificate --output CAcertificate.cer at the Azure Sphere Developer Command Prompt. (This is the same certificate already generated for the IoT Hub example.)
    • Upload the certificate to IoT Central and generate a validation code, then use azsphere tenant download-validation-certificate --output validation.cer --verificationcode verificationcode to apply this.
    • Upload the new validation certificate.
  5. Create a non-simulated device in IoT Central:
  6. Run ShowIoTCentralConfig.exe, providing the ID Scope and a shared access signature key for the device (both obtained from the Device Connection details in IoT Central) and the Device ID (from the device created in the previous step). Make a note of details provided by the tool
  7. Configure the application source code to connect to IoT Central:
    • Uncomment #define IOT_CENTRAL_APPLICATION in build_options.h.
    • Update the CmdArgs line in app_manifest.json with the ID Scope obtained from the Device Connection details in IoT Central.
    • Update the AllowedConnections line in app_manifest.json with the FQDNs obtained by running ShowIoTCentralConfig.exe.
    • Update the DeviceAuthentication line in app_manifest.json with the Azure Sphere tenant ID (which may be obtained using azsphere tenant show-selected at the Azure Sphere Developer Command Prompt).
  8. Build and run the application.
  9. Associate the Azure Sphere device with IoT Central (the device created previously was just a “dummy” to get some configuration details). IoT Central should have found the real device but it will need to be “migrated” to the appropriate device group to pick up the template created earlier.
  10. Open the device and enjoy the data!

I hadn’t expected IoT Central to cost much (if anything, because the first two devices are free) but I think the app I’m using is pretty chatty so I’m being charged for extra messages (30,000 a month sounds like a lot until you realise it’s only around 40 an hour on a device that’s sending frequent updates to/from the service). It seems to be costing just under £1/day (from a pool of credits) so I won’t be worrying too much!

What’s next for my Azure Sphere device?

Having used Brian Willess’ posts at Element 14 to get an idea of how this should work, I think my next step is to buy some external sensors and write some real code to monitor something real… unfortunately the sensors I want are on back order until the summer but watch this space!

Getting started with Azure Sphere: Part 1 (setup and running a sample app)

Late in 2019, I got my hands on an Azure Sphere Starter Kit, which I’ve been intending to use for an IoT project, using some of the on-board sensors for temperature and potentially an external one for humidity…

For those who aren’t familiar with Azure Sphere, it’s Microsoft’s Secure Internet of Things (IoT) solution using certified chips, a custom operating system and a security service. My device is an Avnet Azure Sphere MT3620 Starter Kit and this blog post focuses on getting it up and running with one of the sample applications that Microsoft provides, using Windows 10 (other options include Linux).

Installing Visual Studio Code and the Azure Sphere SDK

Having obtained the kit, the next stop was Microsoft’s Getting Started with Azure Sphere page. I downloaded and installed Visual Studio Code (I don’t really need the whole Visual Studio 2019 application – though I later found that a lot of the advice on the Internet assumes that’s what you’re using…) and then immediately found that there are two versions of the Azure Sphere Software Development Kit (SDK). According to the Microsoft docs, either can be used with Visual Studio Code but I found the setup for the Azure Sphere SDK for Visual Studio failed when it can’t find Visual Studio (not really surprising) and so I used the Azure Sphere SDK for Windows.

Connecting the hardware

I plugged in the Avnet Azure Sphere Starter Kit, using the supplied USB cable, and watched as Windows installed drivers after which a virtual network interface was present and three COM ports appeared in Device Manager.

Setting up my dev environment

Installing Visual Studio Code and the Azure Sphere SDK was only the first part of getting ready to create code for the device. I needed to install the Azure Sphere extension (easily found in the Extensions Marketplace):

The Azure Sphere extension also installs two dependencies:

  • C/C++
  • CMake Tools

I also need to install CMake (in my case it was version 3.17.1). Not really knowing what I was doing, I followed the defaults but on reflection, I probably should have let CMake add its directory to the system %PATH% variable (I later uninstalled and reinstalled CMake to do this, but could just have added C:\Program Files\CMake\bin to the Path in the user environment variables).

The final installation was Ninja. Windows Defender SmartScreen blocked this app, but I was later able to work around that, by unblocking in the properties for ninja.exe:

I missed the point in the Microsoft documentation that said I needed to manually add Ninja to the %PATH% environment variable but I later went back and added the folder that I copied ninja.exe to (which, for me, was C:\Users\%username%\Tools).

(The above steps were my second attempt – the first time I installed MinGW-W64 to work around issues when Visual Studio Code couldn’t find a compiler, together with several changes in settings.json. I later removed all of that and managed to compile and deploy a sample application using just the settings above…)

Configuring the Azure Sphere device for use

There are a few steps required to configure the device for use. These are all completed using the Azure Sphere Developer Command Prompt, which was installed earlier, with the SDK.

Creating an Azure Sphere tenant and claiming the device

Each Azure Sphere device must be “claimed” and associated with a “tenant”. I followed the Microsoft documentation to do this…

azsphere login --newuser user@domain.tld

After completing Multi-Factor Authentication (MFA) and confirming I wanted to allow Azure Sphere to use my account, I was logged in but with a warning that I don’t have access to any Azure Sphere tenants, so I created one:

azsphere tenant create --name "Mark Wilson"

Warning – more research required: I used a Microsoft Account, as per the Microsoft instructions, but am now concerned I should have used an Azure Active Directory (Organisational/Work or School) account (especially as Role Based Access Control is supported from Azure Sphere 19.10 onwards). As a device can only be claimed once and, once claimed, the device is permanently associated with the Azure Sphere tenant, I’m stuck with these settings now…

I then went ahead and claimed the device:

azsphere device claim

Connecting to Wi-Fi and updating the device operating system

I checked the current OS version on the device:

azsphere device show-deployment-status

As can be seen, not only is the OS out of date, but the device is not connected to a network, so I connected to Wi-Fi:

azsphere device wifi show-status
azsphere device wifi add --ssid "SSID" --psk password
azsphere device wifi show-status

Now, with network connectivity in place, the device had a fighting chance of an OS update and according to the Microsoft documentation:

The Azure Sphere device checks for Azure Sphere OS and application updates each time it boots, when it initially connects to the internet, and at 24-hour intervals thereafter. If updates are available, download and installation could take as much as 15-20 minutes and might cause the device to restart.

Configure networking and update the device OS

I tried several restarts using azsphere device restart with no success. In the end, I left the device connected overnight and, by the morning, it had updated to 20.03.

Finally, I enabled application development on the device, ready to download some code and deploy an application:

azure sphere device enable-development

Downloading a sample app

My initial attempts to use the app that I wanted to didn’t work so I decided to test my setup with one of the Microsoft Quick Starts.

I needed to use git to clone the Azure Sphere Samples Repo, so that meant installing git. Then, from the Terminal in Visual Studio Code, I ran git clone https://github.com/Azure/azure-sphere-samples.git.

I then opened the Samples\HelloWorld\HelloWorld_HighLevelApp folder in Visual Studio Code, ready to build and deploy the app.

Building and deploying the app

Having set up my dev environment, set up the device and downloaded some sample code, I followed the instructions in the Visual Studio Code Azure Sphere Extension to run the following in the Command Palette: Azure Sphere: Configure Settings (selecting High-Level Application) and CMake: Build.

I was then able to build and deploy the sample app to my Azure Sphere device, by starting a debug session (F5) .

and was rewarded with a blinking LED on the board!

Azure Sphere Starter Kit with blinking LED

I can also view the application status with azsphere device app show-status.

Next steps

The next step is to get the app I really wanted to use working on the device, making use of some of the on-board sensors and then integrating this with some of the Azure services. I’m having trouble compiling that code at the moment, so that blog post may be a while longer…

Further reading

Dreaming of a better commute


Travelling in and out of London this week for the course I’ve been attending has reminded me why working from home (mostly) is a huge blessing. At least 4 hours’ travel a day for a relatively simple 60-mile commute? No thank you!

I did, however, use two different routes with contrasting experiences and that made me think – why does it have to be this way? And what might it be like one day?

Commute route 1: Olney to London via Bedford (Thameslink/East Midlands Trains)

After driving to Bedford and finding a space in the car park (not always easy), the next question was where are the ticket machines? The option to pay and display with optional mobile phone/SMS/app payment seems to have been replaced by a system to pay as you leave the car park on foot (albeit with an optional mobile app). It uses ANPR to recognise my car but the user interface is confusing and there’s no option for contactless payment (surely a perfect use case for fast commodity transactions like this?). At £7.90 for a day’s parking (when the only reason you would ever park there is to catch a train!), it’s expensive too.

Then, at the station I bought a ticket – again falling foul of a confusing user interface (not helped by Thameslink’s corporate colours not really highlighting what I need to see). I switched to another machine and followed a different (but more familiar) purchase journey on the touch screen whilst another customer switched queues because of a broken card reader in the machine she was using.

Catching the train is simple, with frequent services but lots of stops and the (07:34) train is packed well before reaching London.

The good thing about this route (on Thameslink – not on East Midlands Trains) is that it goes right to the heart of the city (not the West End) although I change at Farringdon to get on the underground towards Tower Hill. Sadly, with no barriers to pass through and crowds of commuters I didn’t see an Oyster touch in/out machine, which I realise after boarding the train – wouldn’t it be good if there were more of these machines or if you could swipe on the train!? I touch out at the end of the journey but am charged the full fare and it takes me a lot of time on the phone waiting to sort out the charging…

Commute route 2: Olney to London via Milton Keynes (London Midland/Virgin Trains)

After a faster drive to Milton Keynes (MK is famous for its roundabouts but there’s a real benefit in the national speed limit grid road network), I park close to the station. The actual station parking is extortionate (so much so that I know some people who don’t pay, preferring to take the risk of an occasional fine) but off-street parking is available and half price if I pay by phone (£4.18).

I buy a ticket at the station but know to always allow time for queuing: there are 6 machines and 4 booths but that’s never enough! It’s 06:54 so I dash for the 06:55 London Midland service, but see that the (faster) 06:53 Virgin train has only just arrived (even though it’s showing as “on time”).

We set off towards London, only to be delayed by a vehicle striking a bridge at Watford and are overtaken by the slower London Midland service that I nearly caught earlier! Eventually, we get moving and arrive in London 20 minutes late…

A dream of a better commute

These real world stories are just single journeys and it could all be so different on another day. So let’s compare with what it could be like:

  • My calendar shows that I’m planning to be in London for the day.
  • My alarm wakes me with enough time to get ready, and the lights in the house gently warm up to wake me from my slumber.
  • I drive to the station and, as I park my phone recognises my location and that I’m stationary, asks me if I need to pay for parking and then takes care of the details.
  • Arriving on the station concourse, my digital personal assistant has pre-bookèd my train ticket and there’s a boarding pass on my phone. No paper tickets are required as the barriers can simply scan a QR code on my screen (or even use NFC?)
  • There’s a steady flow of trains (on time of course!) and as I switch to the Underground, payment is dealt with as I pass through turnstiles using a contactless payment card – and, even if I end up on the platform via a different route I can pick my boarding point (verified using location services) and ensure I’m correctly billed, using a smartphone app…
  • Realising there are delays on the line, my phone reschedules appointments as required, or otherwise ensures that contacts are aware I will be delayed.

It’s not difficult – all of this technology is available today but it just doesn’t quite work together… all of this talk of an Internet of Things brings it tantalisingly close but train companies, car park operators and other organisations still cling on to outdated methods. So it seems I’ll be dreaming for a little while longer…

 

Short takes: tl;dr; online influence (#digitalsurrey); and the Internet of things (#cloudcamp)


It’s been another crazy week without any time for blogging so here are some quick highlights from the stuff I would like to have written about (and still might, time permitting!)

tl;dr

I was reading one of Matthew Baxter-Reynold’s articles on the Guardian website a few days ago and he gave a summary of the key points under the heading tl;dr.  I hadn’t seen that before but it turns out it’s an Internet meme – tl;dr is an abbreviation for “too long; didn’t read” – something that I suspect many of my blog posts suffer from. Maybe I’ll start including a tl;dr section in future…

Return on Influence

On Tuesday evening, Mark W Schaefer (@MarkWSchaefer) spoke at Digital Surrey about the use of influence marketing on the web. It was an enlightening talk and certainly something to consider as organisations increasingly judge our online influence in deciding how to (or whether to) react to and interact with us. My personal view is that Klout and its ilk are over-rated (Klout in particular is very much led by volume of online activity – if I go on holiday for a few days, my Klout takes a hit) but, if I were to give a “tl;dr” view on Mark’s talk it would probably include this diagram:

  1. Surround yourself with people who care about you (and your views) and have a pre-disposition to “move” (i.e. like, retweet, advertise, etc.) your content.
  2. Create unique and interesting content – have something to say (in order to make it “move”) – make it relevant, interesting, timely and entertaining.
  3. Be consistent in engagement – not just broadcasting but being authentically helpful and looking for opportunities to interact.

Common sense? Perhaps – but it’s how Mark suggests we build influence.  Read more in Mark’s book – Return On Influence: The Revolutionary Power of Klout, Social Scoring, and Influence Marketing.

(Jas Dhariwal has made a recording of Mark’s talk available.)

The Internet of things

The Internet of what? Well, depending on your source of technology reading material, you might have head that we’re increasingly connecting lots of “things” to the Internet – sensors, for example – and Wednesday saw a CloudCamp Special in London on The Internet of things. As usual, the evening was introduced by Simon Wardley (@swardley) with his well-practiced (but still interesting) talk on the cycle of innovation leading up to his vision of “augmented intelligence” supported by utility computing (cloud), big data, and intelligent mobile applications.

Then, onto the lightning talks with: Andy Bennet (@databasescaling)’s introduction to the Internet of things (it’s not new!); Raphael Cohn’s fascinating recital of how Smith Electric Vehicles overcame a major business issue in that “electric trucks rule, but batteries suck, and mobiles die”; Kuan Hon (@kuan0)’s rundown on cookie laws (which have a much broader impact than just websites); Paul Downey intruducing us to the wonderful world of open source hardware (which is far more extensive than I ever imagined); and Chris Swan (@cpswan)’s review of the Internet of Things in some of his favourite science fiction novels. Oh yes, a a couple of guys from Betfair stood up and tried to plug their new application cloud, which I’m sure is very good but seemed a little too like a vendor pitch to me…

Wrapping up with a panel discussion, before beer and pizza, it was a thoroughly agreeable way to spend the evening and I learned loads about the Internet of things… hopefully I’ll write some more on the topic over the coming weeks.

Starting to play with the Internet of things


Unlike some people, who find it invasive, I love the concept of the Internet of things. I’m truly excited by some of the possibilities that a world driven by data opens up. Sure, there are issues to overcome (primarily around privacy and connectivity) – but anyone who believes their data isn’t already being captured by service providers (even if those providers don’t yet know how to handle the massive volumes of data) is in for a shock. So why not embrace the possibilities and use our increasingly smart world to our collective advantage?

In my recent presentation to the BCS Internet Special Interest group, I referred to the Technology Strategy Board‘s Future Internet Report, which talks about [emphasis added by me]:

“An evolving convergent Internet of things and services that is available anywhere, anytime as part of an all-pervasive omnipresent socio–economic fabric, made up of converged services, shared data and an advanced wireless and fixed infrastructure linking people and machines to provide advanced services to business and citizens.”

The report also acknowledges the need for more than just “bigger pipes” to handle the explosion in data volumes. We do need a capable access mechanism but we also need infrastructure for the personalisation of cloud services and for machine to machine (M2M) transactions; and we also need convergence to enable a transformational change in both public and private service delivery.

That’s the big picture but scaling back down to a personal level, one of my colleagues, David Gentle (@davegentle – who happens to be the main author of Fujitsu’s Technology Perspectives microsite) highlighted a site called Pachube to me last week. I first came across Pachube a few months back but [partly because it used to be a chargeable service (it became free at the start of this month)] it got added to my “list-of-things-to-have-a-better-look-at-one-day” (that day rarely comes, by the way!). This time I had a better look and I found it to be pretty cool.

Pachube is basically a cloud-based broker for connected devices with a web service to manage real-time data and a growing ecosystem of applications to feed and consume data. That sounded like it might need some programming (i.e. could be difficult for me these days) but then I found a method to hook an energy monitor up to the web, with no coding required!

I’ve written before about the EnergyFit (Current Cost) power meter that E-ON sent me. I wasn’t a fan of E-ON’s software so I hooked it up to Google PowerMeter for a while, but that service has closed down (along with Microsoft’s Hohm service – which I don’t think even made it to the UK). Using a USB to serial driver and a companion application I now have one of my computers feeding data from my Current Cost meter to the Pachube website, where it gets transformed into JSON, XML or CSV format and “magic” can be performed. I used the Mac OS X software versions of the driver and the application but there are also Windows (driver/application) and Linux (driver/application) variants that I have not tested. The process of setting up a Pachube feed has also changed slightly since the original guidance was written but the basic steps are:

  1. Install the USB-serial drivers.
  2. Install the application
  3. Run the application and select the appropriate serial port (for me, on my Mac, that is /dev/tty.usb-serial).
  4. Create a feed (a push feed – and however many times I turn it private it seems to switch back to public…).
  5. Paste the XML version of the feed into the application.
  6. Set up a secure sharing (API) key (you probably don’t want to use the master key) and paste it into the application.
  7. Save preferences and wait for the application to start feeding data, at which point the feed should show as live

The application I used and the Pachube website seem to work together to configure the datastreams within the feed (one for temperature and one for power) and it’s all set to go.

Once the feed is live, there are a load of apps listed on the Pachube website with everything from graphs and visualisations to mapping tools and augmented reality. I decided to create a page to display some of these, starting out with a customisable PNG-based graph from my feed. That worked, so I added another, together with a PachuDial and a couple of PachuBlog gadgets (sadly, these are Flash-based, so don’t work on the iPad…). Next I created a second feed to consume the power usage from the first one and measure the associated carbon footprint.

Having played around with energy usage, I found that I could also use Pachube to monitor my Twitter account (a pull feed this time) – which might be useful too.

Now I’ve mastered the basics with my Current Cost meter, I might try some home automation using Arduino devices – although that looks to have quite a steep learning curve on the electronics front… In the meantime, you can see the Home electricity usage and Twitter statistics pages that I created using just the Pachube platform and some basic HTML.

[Update 30 November 2011: added comment about Pachube becoming free to use]