Removing password protection from PDF files

Important note: this post wont help you if you have a PDF file and don’t know the password. This is for removing passwords on PDFs that you have legal access to, but don’t want to be password-protected any more.

A while ago, one of my employers started emailing payslips in PDF format. Now, I know there are many issues around accessibility with PDFs, but it works for me – I get a digital version of a document that looks exactly as the printed one would have. Except that someone decided email (even to a company-secured account) was not secure enough, and they password-protected the files. In theory, this stops another employee from opening my payslip. In practice, they used a known piece of personally identifiable information (PII).

Anyway, I wanted to keep a copy of the files on my own file storage. I can do this because, technically, they are not company data and they are (or at least should be) private to me. Indeed the company in question has since moved to a system that emails a link to a personal email account, inviting the employee to download their payslip from a portal.

I didn’t want the copies of the payslips that I held to be password protected. That meant I needed to remove those passwords.

QPDF

QPDF is a computer program, and associated library, for structural, content-preserving transformations on PDF files. It’s not for creating, viewing or converting PDF files.

One of the things it can do, is remove the password protection on a file. Remember, this is a file that I have legal access to, so removing the password protection is not a crime. I’m not hacking the file – in fact I need to know the password in order to remove it.

QPDF can do much more than remove passwords (for example I think I could use it to create new versions of a PDF file with just a subset of the pages), but this was what I needed to do.

A little side-note

This was the second time I performed this exercise. I first did it a few years ago, but only on the payslips I’d received up until that date. Later ones were still password-protected. I didn’t document my method the first time around though… so I had to work it all out again. This time I decided to write it down…

A little PowerShell Script

It looks like, the first time I ran this, I downloaded a Windows executable version of QPDF and either wrote, or more likely found, a PowerShell script to adapt. The script is called payslips.ps1 and looks like this:

$children = Get-ChildItem # Save files in a variable. Piping the rest of the script from Get-ChildItem in a single line was a bad idea
$children | ForEach-Object {
Write-Debug "Working on $_.Name"; #Doesn't actually display a lot
$fileName =[System.IO.Path]::GetFileNameWithoutExtension($_.Name); #Strip name, we will append "tmp"
$ext =[System.IO.Path]::GetExtension($_.Name);
$tempFile = $fileName + "tmp" + $ext; # Append "_tmp" Move-Item -Path $.Name -Destination $tempFile; #Move the file to a temporary location
..\qpdf.exe --password=AB123456C --decrypt $tempFile $_.Name; #Use qpdf to decrypt it, save in original location
#Remove-Item $tempFile #Remove temporary file
}

ABC123456C should be replaced with the actual password. Actually, it shouldn’t, because including credentials in code is sloppy security practice. There are better ways to pass the password, but I’m just converting 50 files as a one-off exercise, not building a repeatable business process. If you go on to use this in a business environment, please don’t do it this way!

Release notes

The script makes a temporary copy of each file, suffixed with _tmp but preserving the file extension.

If you run the script against the current folder, it will run against all files, not just PDFs. That means it will rename itself and all the QPDF files with _tmp. This will cause it to fail.

It looks like, when I ran this a few years ago, I used a files.txt file to control this behaviour. files.txt was just a list of filenames and is easily generated using the following command:

dir /b /a-d > files.txt

But, this time, I couldn’t see how to provide that as a parameter to QPDF, so I had to:

  1. Place all the files to be converted in a subfolder of the folder containing QPDF and my PowerShell script.
  2. Edit the payslips.ps1 script to refer to ..\qpdf.exe (i.e. qpdf.exe in the folder above the current one).
  3. Change directory into the subfolder.
  4. Run payslips.ps1 from the subfolder – i.e.:
..\payslips.ps1

This means it will only run against the files in the subfolder, and not against QPDF, the script, or anything else.

It doesn’t seem to remove the temporary files. I didn’t try to work out why. It had already created what I needed by then.

Featured image: author’s own

Weeknote 2024/04: Coffees, and staying curious

Another week, and lots of positive feedback from colleagues on these weeknotes, so they keep going. This time I’ve written it over the course of the week, rather than in one huge writing session at the weekend. I’m not sure it really helped… it’s still way too long. Anyway, here it is.

(I’m also slightly concerned that some people think I have too much time on my hands. I really don’t. I just stay up too late and don’t get enough sleep!)

This week at work

I struggle to write about work at the moment. I’m doing lots of cool stuff, but I don’t really want to tell competitors what Node4 is developing. Even so, it’s no secret that we’re driving forwards with our Digital delivery (that’s why Node4 bought TNP, risual, Tisski, and ThreeTwoFour) – and public cloud is a big part of that, particularly in the Microsoft space.

My presentation to the Node4 Go To Market community on our public cloud transformation capabilities seemed to go well. And it would be remiss of me not to say that, if you want to know more about how we can potentially help your organisation on its Microsoft Azure journey then I, or my colleagues, would be pleased to have a conversation. Feel free to get in touch on email, or book some time with me.

Beyond that, I joined an interesting call with IDC, looking at the European cloud market in 2024. And I’m just getting involved in a project with some cool tech to help address the ransomware challenge.

Most exciting though is that I’ve submitted a request to join Node4’s Innovate Leadership Development Pathway for 2024. This looks to be a great programme, run over several months, that results in an ILM qualification. The reason I’m excited is that, for the first time in a while, I feel that I’m in a role where I can exploit my leadership potential. I had a career diversion into management, because I thought I needed that experience. Then I got out of it, only to fall back into it (and was very unhappy for quite a long time). Management and leadership are very different things, and over the years I’ve learned that I want to be a leader, not a manager.

Coffees (virtual and IRL)

Much is made of “watercooler moments” as a reason to return to the office (RTO). Well, is there any reason that such moments can’t happen outside the office too?

In 2023, Matt Ballantine ran a “100 coffees” experiment to chat without any particular agenda. It was a big success so it’s rolled on into 2024, currently at around 138. (I was number 49.) Incidentally, you don’t have to drink coffee. It’s about taking the time to chat with people and other beverages are equally acceptable. Or, as Matt describes it in a post he wrote for his employer, Equal Experts, about the process and its benefits:

“Coffee here is a metaphor. A metaphor for being intentional about making space in our working days to create serendipity, build relationships, reflect, have new ideas, share old ideas and a wealth of other benefits that come from conversations without agenda.”

Matt Ballantine: “How to have coffee”

Earlier in the month I had some “coffees” with some colleagues I no longer work with on an daily basis. It was brilliant just to check in and see what they are up to, to keep myself in touch with what’s going on in a different part of the organisation. This week, in addition to some “quick chats” with a couple of my peers, I met several people outside the company for “coffee”. Their roles included: a Chief Evangelist; a Managing Director; and a Digital Transformation Consultant.

One I hadn’t seen since we worked together over a decade ago. Another is part of a “coffee club” that Matt set up to encourage us have a monthly conversation with someone we don’t normally talk to. And one has become a friend over the years that we’ve been catching up for coffee and occasional lunches. My own lack of confidence makes me think “what do I have to add to this conversation”, but invariably I learn things. And I assume that the value of meeting up with no agenda to “just have a chat” goes both ways.

Some of the things we talked about

Our conversation topics were wide and varied. From family life to:

  • Recognising when to buy services vs. learning to do something yourself.
  • “Thought leadership” and qualitative vs. quantitative metrics – looking at the “who” not the size of the reach.
  • Next-generation content management systems.
  • How localisation is more than just translation – sometimes you might rearrange the contents on the page to suit the local culture.
  • How UK town centres seem to encourage chains to flourish over independent retailers.
  • The frustrations of being an end user in a world of corporate IT security (managed devices, classifying information, etc.)
  • Being proud of your kids.
  • What travel was like when we were young, when our location wasn’t being tracked, and when our parents must have been super-worried about where we were. (Is the world more dangerous, or just more reported?)
  • Finding your tribe by showing things in the background on virtual meetings.
  • Bad service and food vs. great coffee but no space. And on what makes a good English breakfast.
  • Parenting young adults and supporting their life decisions.
  • Publishing newsletters, weeknotes, blogs. Owning your own content, and why RSS is still wonderful.
  • Fountain pens, a place for everything (and everything in its place) – and why I’d like to be more like that… but have to accept I’m just not.
  • Four day weeks, balancing work, health and exercise (or lack of).

That’s the whole point. No agenda. See where the conversation leads. Get to know each other better. Learn new things. Build relationships.

And all three “coffees” ran out of time!

This week in tech

  • Here’s something I wrote a blog post about. I had intended there to be more posts, but I overestimated the amount of time I have for these things:
  • A couple of weeks ago, I mentioned I’d been looking at Calendly. It turned out to be a trial (I missed that) and I need to subscribe for some of the features that I set up. So, I guess that experiment didn’t work out…
  • I don’t understand why Google opening a new data centre in the UK this is news. All of the hyperscalers already have data centres in the UK. This is just another one. I’m not sure that they contribute much to the economy though, except maybe in construction and through services consumed (electricity, water, etc.). As for the PM’s statement that “Google’s $1 billion investment is testament to the fact that the UK is a centre of excellence in technology and has huge potential for growth”. Poppycock. It shows there is a demand for cloud computing services in the UK. It’s got nothing to do with excellence.
  • I found a new setting in Microsoft Teams that makes my video feed look like I’m using a decent camera! It’s so much better than the old background blur.

Some posts I liked elsewhere

  • On digital inclusion…
  • Of course, not everyone finds online easy. And we have to recognise that sometimes, for any age group, there’s a need for a human connection…

Life

Some readers may know that I have been using the Zoe personalised nutrition programme to see what insights I can get into my diet. I’ve tweeted a bit, and it deserves a longer blog post, but I found this article in the Times very interesting. Jay Rayner has a slightly less reverent view in The Guardian. (Kate Bevan shared both of these articles.)

And I have a holiday to look forward to… or at least a mini-break. Mrs W and I have just booked a long weekend in Tallinn for a few weeks’ time…

This week’s watching

After finishing our recent dramas, it was time to start something new. Several people had recommended Lessons in Chemistry (on Apple TV) and we’re really enjoying it. As an aside, we still have a long way to go on diversity, inclusion and equality but, oh my, we’ve come a long way since the 1950s.

This week’s listening

I listen to a lot of podcasts when I’m walking the dog, or when I’m driving alone. The Archers is the first on my list but please don’t judge me.

I also like to listen to The Bottom Line, though sometimes find Evan Davis’ views on modern work to be a little “traditional”. This week’s episode on e-commerce returns was fascinating, though I do wonder why no major UK retailers (e.g. Next, John Lewis) or online-only retailers like Amazon or even Wiggle wanted to take part…

I used to listen to The Rest Is Politics – it’s a great podcast but there is just too much of it – I found the volume of content overwhelming. But I did listen the Rest Is Politics Leading interview with Bill Gates. I was looking for a link to the podcast episode to share, but I found it’s available on YouTube too, so you can watch or listen:

Some of the things I took away from the interview were:

  • It’s well-known that Bill Gates dropped out of Harvard, but it’s clear he was a very smart kid… he quietly mentions finishing his classes a year early.
  • I was interested in his responses to tough questions – like asking if his approach at Microsoft was “flattening competition not creating excellence”. And on monopolistic views of the world and how they needed to lower prices to gain market share. Remember the mission was to get a computer onto every desk and into every home.
  • On his position as a rich and powerful person, and why he follows the philanthropic path that he does of trying to kill malaria rather than direct giving to those in poverty.
  • On family, the impact he can have on his granddaughter’s future world, and the advantages/disadvantages of growing up with wealthy/famous parents
  • On the future of AI.
  • On politicians he admires (and giving very guarded responses!)
  • His rather odd (IMHO) views on climate change.
  • On learning from Warren Buffet, and on a lifetime of staying curious.

Maybe that’s what I should call this blog… “staying curious”.

This week in the press

On the PR front, I had a brief quote in Digitalisation World’s Tech Transformations for 2024 article.

…and not in the press

After initially being flattered to be contacted by a major UK newspaper for comment on the importance of public sector work to Fujitsu, I declined to comment. Not sure if it was my media training or common sense, but it feels right. I had already written a brief post on LinkedIn, but a lot will have changed in the time since I left and anything I can remember would already been in the public domain.

More thoughts on the Post Office Scandal

I was going to write about this last week, but I was still reeling from some of the comments I’d received on social media, so thought on for a bit more.

Understandably, this is a very emotive subject. Lives were ruined. Some who were affected took their own lives. It’s nothing short of a tragedy.

Even so, it was upsetting to be told last week on Twitter/X that anyone who has Fujitsu on their CV should never work again (or words to that effect). I was at ICL or Fujitsu for around 16 years over one internship and two periods of employment. In common with most people there, I had nothing to do with (or knowledge of) Horizon, other than knowing of its existence, in a separate business unit. And, in common with most people who saw the recent ITV Drama, I was shocked and appalled.

I can’t defend Fujitsu – but I am going to use someone else’s words, because they sum up the situation about their future in the UK public sector market perfectly for me:

“A lot of innocent people [may] lose work at Fujitsu. All of us who have worked for outsourcing partners will know the nature of contracts means many will know nothing of other ongoing projects. Today many workers at Fujitsu [may] be ‘at risk’ for something they had no control over.”

From a technical perspective, I found this video from Dave Farley to be an excellent explanation of the types of technical issues in the Horizon system that led to accounting errors. Then add in believing the computer over the humans, together with an unhealthy dose of corporate mismanagement (as is being uncovered by the ongoing inquiry), and you get the full horror of the Post Office Scandal.

This week in photos

Looks like I didn’t take many, but I did wrap up the week with a nice dog walk in the winter sunshine.

Featured image by Engin Akyurt from Pixabay.

Using NFC tags to automate my home

Imagine a home with a smart thermostat to control the heating, smart lights to control the lighting, and smart sockets to control other electrical devices. Well, for some people, that doesn’t require a lot of imagination – it’s just the way things are!

My home isn’t quite like that. We still have an analogue thermostat switch on the hall wall (one day we might upgrade) but there are various smart sockets around and we do have some smart lighting. The smart sockets control things like the heater in my Man Cave, the Christmas tree lights (mid December-January 6th only) or the fairy lights in the garden. And I wrote some posts about the smart lights, in this two part series in 2021:

NFC tags

Many of us are familiar with Near Field Communication (NFC), even if we don’t know it. It’s the technology used in contactless payment cards. To learn more, I thoroughly recommend watching Professor Hannah Fry‘s Secret Genius of Modern Life TV episode about the bank card, on BBC iPlayer.

And those smartphones we carry everywhere with us, well, if they are NFC-enabled, they can read NFC tags to perform other operations. It’s not just for making electronic payments!

All you need is to buy some tags and, as much as I try to avoid the big online marketplace that sells everything from A to Z, that’s where I picked mine up.

What follows is for iOS, as my family are all iPhone users (tested on 17.2.1). Android users can do similar things, but you’ll use a different app.

Shortcuts and automations

The iOS Shortcuts app has a section called Automation.

  1. Click + to create a new automation.
  2. Scroll down to NFC.
  3. Click Scan and scan your tag, then name it.
  4. Pick when to run the automation (immediately, or after confirmation) and whether to be notified.
  5. Select what the automation will do.

That’s it. Just touch the top of the iPhone to the tag and it will run your automation. Stick it to the desk, the wall, or wherever is handy to run the automation.

Things I discovered

In this experiment, I did find out a few things…

  • I should have bought tags with sticky pads. Or maybe not – a glue stick seems to work pretty well for attaching them to things.
  • They don’t seem to work on metal things (like my desk lamp, or the metal switch sockets in the Man Cave). I guess it interferes with the signal, so you’ll need to stick them nearby.
  • The iOS Shortcuts app will integrate with many applications, but not directly with Alexa, it seems. I have tested a couple of workarounds though:
    • Play a recording that issues the Alexa command.
    • More elegantly, use the scripting option to Open the Alexa app, wait a second, speak “Alexa”, speak the Alexa command.
  • You can also create quite advanced no-code scripts to launch menus and ask for input – for example scan the same tag and ask whether to turn on the device or turn it off, then take action accordingly.

There’s more in this thread on Twitter/X…

Conclusion

NFC tags are cheap (especially when bought in bulk) and an effective way to automate tasks around the home (or at work, in the car, or wherever). There’s lots more that you can do with NFC tags and YouTube is full of videos to provide inspiration. Have fun!

Featured image from the Computer Science Wiki, used under a Creative Commons Attribution-NonCommercial-ShareAlike licence.

Weeknote 2024/03: missing cyclocross; digitally transforming my family; installing Ethernet and much, much more

Another week has flown by – this time I kept notes to keep track of it all in the hope it would speed up the weeknote writing. It didn’t, so I need to work on the format. Anyway, this is how it looks this week.

My week at work

Understandably, I can’t write much about what I do in the day job. Suffice to say, it’s been busy, busy, busy. I’m preparing for a presentation to the Node4 Go To Market (GTM) team next week. This will be me, along with my colleague Bjoern, presenting to the entire salesforce and trying to convince them why they should be selling more of the services we’re responsible for. And, in parallel, I’m refreshing the collateral to support the sales of those same services.

I also spent some time on a call with one of our business partner this week, learning more about how they are developing their offers and how we can potentially do more work together.

My week in cycling

I know, this blog is supposed to be about tech, but I also have two very sporty teenagers that I’m very proud of. Their sports activities are a big part of my week (and my life in general).

Last weekend, I should have been in Falkirk, supporting my eldest son, Matt, at the 2024 British National Cyclocross Championships. As things transpired, that was not to be…

At 2023’s National Champs (Matt’s first senior year), Cameron Mason was so dominant in the Elite/U23 Men’s race that only 7 riders were permitted to finish the race (under the 80% rule). It’s a big investment of time and money to travel the length of the country for a short race but we would have been there, if Matt felt he was ready for it. Unfortunately, after a challenging few weeks with a return to racing after spending the autumn leading cycle tours in Greece, he decided to end his cyclocross season early. Apart from podiums in the local Central Cyclocross League (CCXL), third place in the Central Regional CX Champs was to be his only significant result this season. He’s preparing to build two new bikes for the 2024/25 cyclocross season – and he has plans for the 2024 road season too. I’m sure those plans will end up in these weeknotes in due course.

Just as a side note, after the demise of GCN Plus, it’s great to see BBC Scotland providing mainstream TV coverage of the national champs!

My week in technology

Adding AirPlay to an old Hi-Fi amplifier

With a bonus weekend at home, I got to finish up a tech project that’s been on the list for a while – adding AirPlay to my old 1990s Technics amplifier. When I moved in with my wife, my mid-range Hi-Fi stack was labelled “black loud crap”. As a result, it was banished from the house, but still lives on in the Man Cave. Adding an old Raspberry Pi 2B running as an audio gateway has provided the necessary tech to cast audio, without needing to invest in more Sonos (or IKEA Symfonisk) as I have in the rest of the house. This is the guide I followed, at PiMyLifeUp.

There’s the odd stutter, which I think may be due to a weak 2.4MHz Wi-Fi signal. It could also be down to running on a relatively old Pi 2. It certainly beats connecting Spotify via Alexa which used my account and so only worked for me and not the whole family. Plus it also works with other apps, like Pocket Casts and Audible.

Wilson family digital transformation

Late last year, I convinced Mrs W that we could use the family calendar on our iPhones to manage our busy family life. Previously, the paper calendar on the kitchen wall was the single source of the truth. That’s not too helpful when we’re not at home. This digital transformation of the Wilson family has been a huge success but it’s also shown me that people use calendars in different ways!

For example, our eldest son is currently away skiing. Is that one long appointment for 2 weeks? Or do we just need to know the dates he leaves and returns? And how do we record our youngest son’s Hockey training sessions? Is it the actual session times, or the times we leave the house and return? I’m trying not to be too “Mark” about this, but it’s an interesting insight into how other people think!

On a related note, I also learned this week that not everyone sees pictures in their mind, like I do. I don’t know what they do “see”, but it explains why not everyone can visualise what something will look like when it’s finished!

AirTag all the the things

After a trial with an Apple AirTag in my luggage (very useful when it wasn’t put on the plane at Stansted one holiday), I’ve been expanding our use of these devices. One use case that’s been particularly helpful is my youngest son’s keys… as he’s already had to replace at least one set that he lost before I tagged them. Now I regularly hear the “FindMy sound” as he searches for them before leaving the house.

On a similar note, for Christmas, my eldest son bought me an Apple FindMy-compatible tracker for my glasses. It doesn’t have the Precision Finding feature of the AirTag, but it does tell me where they were last seen, and lets me play a sound. Now, when I leave them somewhere, I can listen for the chirp of the Orbit sensor. It’s a bit strange charging my glasses though, but this is relatively infrequent.

Other bits and pieces of tech

  • After seeing a thread about date formats, ISO standards and RFCs, I thought about my frustrations with people who write dates “the wrong way”. By the wrong way, I mean not putting the most significant portion first. The US convention of mm/dd/yyyy is nonsensical. UK dd/mm/yyyy is better, but I generally name files using yyyymmdd etc. because they appear in order. On that basis, I realised that my naming for these weeknotes should be year/week number (inspired by Sharon O’Dea). Previously I had erroneously named them week number/year. From this week forwards, that is corrected.
  • After watching a YouTube video, I successfully resuscitated an apparently-dead Li-ion battery pack (the on-board circuitry needed its capacity recharging before it would accept a charge). This is potentially dangerous – I’m not responsible for anything that happens if you try it, but it worked for me, and saved me quite a few quid. Some say to use a resistor for safety. Others stress to only “jump start” momentarily (as I did).
  • I was looking at some communications from Vodafone about the 3G switch off… and wondered if that is the same part of the spectrum as 4G… i.e. more channels freed up for 4G/5G or will 4G/5G have access to extra spectrum now? Twitter helped me out with that…
  • Hopefully that section between the hall (OpenReach ONT) and the garage “datacentre” (ISP router) is all the Ethernet I need to run, but I have plenty of spare cable if I need to pull any more for a potential CCTV project… (I’ve been watching lots of videos about Reolink cameras).
  • Oh yes, one more thing. I finally changed my LinkedIn profile picture… my previous professional headshot was taken when I was in my late 30s, I think. I’m nearly 52 and afraid it’s time to look my age. This may not sound like news but it took me ages to find something suitably professional that I liked!

My week in TV

I’ll spare you all my YouTube highlights this week but, over in streaming TV land, Mrs W and I wrapped up three excellent series:

  • Mr Bates vs the Post Office (ITV);
  • Slow Horses (Apple TV); and
  • The Crown (Netflix).

This last season of The Crown has been criticised for being too dramatic but I thought it was well done. There will be no season 7 and it feels like it was left at just the right point, at the marriage of Charles and Camilla (then Prince and Princess of Wales) and the early days of William and Katherine’s relationship (the current Prince and Princess of Wales). It even contained a nod to Queen Elizabeth II’s funeral, with her involvement in the plans but also some scenes that linked to the actual events last year.

And in case we hadn’t had enough Toby Jones, we’ve started watching season 2 of Detectorists, for some light hearted relief from the more serious stuff.

Other things that should probably be a blog post on their own

I was going to write some more, but I’m getting bored of writing this now – goodness knows how you feel, dear reader. So there may need to be an overflow post or two about these topics, or maybe the tweets will say enough:

  • Well-paid IT folks moaning about the inconvenience that strikes have on their lives… playing to the “them and us” narrative.
  • Rebooting the car to get Apple CarPlay to work again!
  • CTOs with 30 years of industry experience being approached about a job that claims to need a technical degree.
  • Storytelling. And how pictures can convey messages that words alone cannot. Or bring meaning to words when they are in another language that you only have a passing knowledge of.
  • Rail fare “simplification” and the very different approaches taken by LNER (UK Government-owned) and ScotRail (Scottish Government-owned).
  • Public sector IT contracts, and the need to be a good client – it’s not all about the supplier.
  • The increasingly anti-social nature of social media.

My week in pictures

Featured image: author’s own

Weeknote 2/2024: time flies

Writing this weeknote may explain why this week has felt so busy. I clearly try to squeeze far too many personal projects in around my work and family life…

This week:

  • I spent quite a bit of time looking at ways to reduce the cost of moving to Azure for our clients. I’m not going to post the details here, but I think my colleagues and I made some good progress. I’ll have more to share soon, but in the meantime, you can always get in touch. Here’s my Microsoft Bookings link if you want to have a chat about how Node4 could help your business.
  • Talking of which, I set up Calendly to try and pull my various calendars together. It’s kind of like Microsoft Bookings on steroids. Sadly it didn’t do the thing I’m really struggling with and show me my work, home and family calendars in one place. Outlook does that for me, but the family calendar in iCloud doesn’t seem to update…
  • I have been staying up far too late watching too much YouTube. Last night the algorithm decided that I needed to know how to install a French Drain. Actually, it was right… it might help with some of the drainage issues in our back garden. Other gems have included:
  • I did try to create a static archive of my tweets though. It’s not quite as I would like, so let’s just say that’s “work in progress”. Maybe more next week, when I have a working solution.
  • And I questioned how “normal users” must feel when presented with nonsensical or jargon-filled computer messages:
  • The photos are still uploading from my NAS to Azure. Several people have recommended other solutions (e.g. Backblaze or Synology C2) but the granular charging on Azure means that I think my current solution may well cost a little less if I tick over the 1TB mark…

I’ve been watching…

As well as the YouTube content I mentioned above, and the dramas I wrote about last week… 3/4 of my family watched the Barbie movie together last weekend. I really quite enjoyed it. Actually, Mark Kermode’s Guardian review nails it: “It’s a riotously entertaining candy-coloured feminist fable that manages simultaneously to celebrate, satirise and deconstruct its happy-plastic subject. Audiences will be delighted. Mattel should be ecstatic.”

And, away from the screen, I got to watch my youngest son play Hockey at Bedford last night.

I’ve been reading…

Not enough. But I am slowly reading the materials for my amateur radio foundation licence exam… and I’ve made the Man Cave a better place to kick back and relax (including a place to read, or listen to an audiobook):

Some thoughts that won’t make it to a full blog post…

The ITV Drama about the Post Office Scandal is a brilliant illustration of the power of storytelling. Graham Chastney wrote about this before I got around to it and his post about how we are wired for stories is pretty much what I wanted to say. Dan Barker’s tweet looks a bit deeper at why years of quality journalism wasn’t enough and it took an ITV Drama to bring the story to the masses.

Rachel Coldicutt examines why we seem inclined to believe the machine, not the person:

I’ve certainly experienced “management by dashboard”, when a report, which was believed by so many, was flawed and presented bad insights on data.

And, whilst I’m still embarrassed that my former employer is so deeply embroiled in a scandal that led to so many miscarriages of justice, I’m more and more inclined to think that software is imperfect, and that the failure of leadership (and consequential mismanagement of the issues) was the main problem here, as outlined by Professor Damien Page:

What else?

Not enough writing. No press coverage this week. Still working out what my new content strategy is as I try to use less “social” media and make blogging less of a time hoover. Next week’s weeknote might not be such a rush…

Featured image: Author’s own
(screenshot from Microsoft Visio)

Weeknote 1/2024: A new beginning

Wow, that was a bump. New Year celebrations over, a day off for the public holiday, and straight back to work.

After a lot of uncertainty in December, I’ve been keen to get stuck in to something valuable, and I’m not breaking any confidentiality by saying that my focus right now is on refreshing the collateral behind Node4’s Public Cloud offerings. I need to work across the business – my Office of the CTO (OCTO) role is about strategy, innovation and offering development – but the work also needs to include specialist sales colleagues, our marketing teams, and of course the experts that actually deliver the engagements.

So that’s the day job. Alongside that, I’ve been:

  • Avoiding stating any grand new year resolutions. I’ll only break them. It was literally hours before I broke my goal of not posting on Twitter/X this year. Though I did step away from a 453-day streak on Duolingo to focus my spare time on other, hopefully less gamified, pursuits:
  • Doing far too little exercise. A recurring health condition is impacting my ability to walk, run, cycle and to get back to Caveman Conditioning. It’s getting a bit better but it may be another week before I can have my new year fitness kick-start.
  • Eating badly. Logging everything in the Zoe app is helping me to see what I should avoid (spoiler: I need to eat more plants and less sweet stuff) but my willpower is still shockingly bad. I was also alarmed to see Prof. Tim Spector launching what appeared to be an ultra-processed food (UPF) product. More on that after I’ve got to M&S and actually seen the ingredients list for the Zoe Gut Shot, but others are telling me it’s not a UPF.
  • Redesigning the disaster recovery strategy for my photos. I learned the hard way several years ago that RAID is not a backup, and nothing exists unless it’s in three places. For me that’s the original, a copy on my Synology NAS, and copy in the cloud. My cloud (Azure) backups were in a proprietary format from the Synology Hyper Backup program, so I’ve started to synchronise the native files by following a very useful article from Charbel Nemnom, MVP. Unfortunately the timestamps get re-written on synchronisation, but the metadata is still inside the files and these are the disaster copies – hopefully I’ll never need to rely on them.
  • Watching the third season of Slow Horses. No spoilers please. I still have 4 episodes to watch… but it’s great TV.
  • Watching Mr Bates vs. The Post Office. The more I learn about the Post Office Scandal, the more I’m genuinely shocked. I worked for Fujitsu (and, previously, ICL) for just over 15 years. I was nothing to do with Horizon, and knew nothing of the scandal, but it’s really made me think about the values of the company where I spent around half my career to date.
  • Spreading some of my late Father-in-law’s ashes by his tree in the Olney Community Orchard.
  • Meeting up with old friends from my “youth”, as one returns to England from his home in California, for a Christmas visit.

Other things

Other things I found noteworthy this week:

  • Which came first, the chicken or the egg scissors or the blister-pack?

Press coverage

This week, I was quoted in this article:

Coming up

This weekend will see:

  • A return to Team MK Youth Cycle Coaching. Our local cyclo-cross league is finished for the 2023/4 season so we’re switching back to road cycling as we move into the new year.
  • Some home IT projects (more on them next week).
  • General adulting and administration.

Next week, I’ll be continuing the work I mentioned at the head of this post, but also joining an online Group Coaching session from Professor John Amaechi OBE. I have no idea what to expect but I’m a huge fan of his wise commentary. I’m also listening to The Promises of Giants on Audible. (I was reading on Kindle, but switched to the audiobook.)

This week in photos

Featured image: Author’s own
(this week’s flooding of the River Great Ouse at Olney)

The problem with a subscription culture (“but it’s just the price of a coffee”)

“If you enjoyed this content, please support the channel by liking and subscribing – and hit the bell icon. If you feel able, then consider a regular donation via Patreon.”

Seems reasonable enough, doesn’t it? Except that we live in a world of subscriptions.

In my house we currently have Microsoft 365 (Family and E1) and Azure, Google Photos, Apple iCloud, Spotify Premium, BBC (TV Licence), Netflix, Amazon Prime (free trial for one month only), Apple TV (free trial for 3 months only), Disney (nope, just culled that), GCN (until it disappears next week) and that’s before we get to apps (Calm, Duolingo, Zoe), newsletters, podcasts, and YouTube channels.

A voluntary donation

I recently found that I was watching a lot of content from one YouTube channel in particular. And it was really helpful with one of my hobbies. So were many others, but I particularly wanted to support this one. Creating video content is time-consuming and it had helped me a lot.

So I signed up to Patreon and donated what seemed reasonable as a contribution. £1 a month for ad-free early access to new content, generally 2 videos a month.

Not much, but a contribution. After all, this is one chap videoing what he learns from his own hobby and sharing with others. It felt like a decent thing to do.

All was good and then the early access emails stopped arriving.

Not a big enough donation…

I queried with the channel producer and he, very politely, pointed out that I was not meeting the minimum contribution. At some point there was a membership structure introduced: $2 for the welcome tier; $5 for a premium level. So I upped my contribution to $2. Except it was actually £2. And Patreon added a fee on top. “$2” was charged at £2.40 (currently around $3).

I know how much effort goes into creating content. Twenty years of writing/maintaining this blog, 14 on Twitter/X, some years on Instagram and elsewhere… and creating video content is next-level…

“But it’s just the price of a coffee”

“But it’s just the price of a coffee”, you may say. Well, I drink one coffee a day… my subscriptions would add up to a lot of coffees.

£2.40 a month for a YouTube channel isn’t much money. But that’s just for two or three ad-free videos (if I watch them before they go public) – the whole of Netflix is £10.99. I don’t know how many Patreon supporters there are – but with just shy of 70,000 subscribers, 1% would be 700. Let’s say 20% pay a fiver (to the channel – so ignoring the Patreon fee) and the rest are on the welcome tier. That’s not a bad income (£1820 a month if my maths is correct). And that’s in addition to the YouTube ad revenue, which I imagine would be significant with that number of subscribers.

What’s the answer?

I don’t know what the answer is. But it feels like there are three “camps” for those producing and publishing content on the Internet in 2023:

  1. Altrustic individuals giving away our knowledge for free.
  2. People trying to turn their hobby into a living, and monetising their efforts.
  3. Corporates providing a service for a monthly subscription.

None of those is a bad thing. Arguably the first is unsustainable. But so is the second, if pitched at the price of a coffee a month and multiplied by many individual producers. Maybe we need micro-transactions. Or maybe we don’t. Maybe I’m just ranting and rambling but what should be the price for good, quality, ad-free content?

Featured image: generated via Microsoft Copilot, powered by DALL-E 3.

Weeknote 49/2023: Back to work

So I’m having another go at writing weeknotes. I think it might even be a mindful exercise and good for me…

I was going to make this “Weeknote 2696” – because that’s the number of weeks I’ve been on this earth (plus 3 days), so it’s not a bad number to use. Then I looked back and realised I did manage a reasonable number of weeknumber/year posts a while back, so that’s the format. I’ll write these on a Friday though, so weekend thoughts will spill over to the next week…

This week’s discoveries/events included:

  • Returning to work after 10 days off, during which I seemed to have forgotten everything!
  • Reinforcing the view that a “strategic discussion about business challenges” with the wrong audience will still end in a conversation about technology. That was even after I’d been clear in the pre-meeting communications, calendar invite, and agenda. Now, I’ll engage the technical team that should have been involved the first time around…
  • Catching up for an overdue virtual coffee with Matt Ballantine (he of the #100coffees experiment), a long time acquaintance whom I now count as a friend.
  • Chatting with Mark Reynolds from Hable about organisational change. That seemed particularly appropriate after British Cycling had emailed me about changes to their coaching framework. It was clearly important to their Learning and Development team but just noise to me, with no clear call to action.
  • Experience of failed digital transformation at Costco, where it appears you can renew your membership online, but it might take 24 hours for the processes in the warehouse to catch up. I made some progress by deliberately crashing and reloading the app. But even then it needed a human to enable my digital membership card. Repeat after me: it’s no good implementing new tech, unless you sort out the business process too!
  • Milton Keynes Geek Night number 46 – at a fabulous new venue (South Central Institute of Technology).
  • Starting to learn about amateur radio, after Christian Payne (Documentally) gifted me a Quansheng UV-K5(8) at Milton Keynes Geek Night. I promised that I would take my foundation exam to get a licence.
  • Taking up the floor in my loft, to expose the heating pipes, to prove to the heating engineers that the pipes are fine and there’s something else in the system that needs to be fixed…
  • Wrapping up the week with a visit to my new favourite local pub (The Bell and Bear in Emberton), with my friend James, for a pint of Marc Antony. This beer appears to have been renamed. It was previously the correct spelling for me – Mark Antony!

Some press coverage

Featured image: author’s own

Celebrating ChatGPT’s first birthday…

I was asked to comment on a few GPT-related questions by Node4’s PR agency. I haven’t seen where those comments are featured yet, but I decided to string them together into my own post…

How has ChatGPT prompted innovation over the past year, transforming industries for the better?

ChatGPT’s main impact in 2023 has been on driving a conversation around generative AI, and demonstrating the potential that this technology has to make truly impactful change. It’s certainly stoked a lot of interest in a lot of sectors and is clearly right at the top of the hype curve right now.

It’s sparked conversations around driving innovation and transforming a variety of industries in remarkable ways, including:

  • Revolutionising Customer Experiences: AI-powered chatbots, bolstered by sophisticated language models like ChatGPT, can engage with customers in natural language, offering real-time assistance, answering queries, and even providing personalized recommendations. This level of interaction not only improves customer satisfaction but also opens new avenues for businesses to understand their customers on a deeper level.
  • Enhancing Decision-Making and Strategic Planning: By harnessing the power of AI, leaders can make informed decisions that are driven by data rather than intuition alone. This has impacted decision-making processes and strategic planning across industries.
  • Transforming the Economy: ChatGPT, with its human-like writing abilities, holds the promise of automating all sorts of tasks that were previously thought to be solely in the realm of human creativity and reasoning, from writing to creating graphics to summarizing and analysing data. This has left economists unsure how jobs and overall productivity might be affected [as we will discuss in a moment].
  • Developer Tools: OpenAI and Microsoft have unveiled a series of artificial intelligence tool updates, including the ability for developers to create custom versions of ChatGPT. These innovations are designed to make it easier for developers to incorporate AI into their projects, whether they’re building chatbots, integrating AI into existing systems, or creating entirely new applications.

These advancements signal a new direction in tackling complex challenges. The impact of ChatGPT on workers and the economy as a whole is far-reaching and continues to evolve. 2023 was just the start – and the announcements from companies like Microsoft on how it will use ChatGPT and other AI technologies at the heart of its strategy show that we are still only getting started on an exciting journey.

Elon Musk has recently claimed that AI will end all jobs, is this actually a reality or is it scaremongering?

Mr Musk was one of original backers of OpenAI, the company that created ChatGPT; however he resigned from their board in 2018. At the time that was over conflicts of interest with Tesla’s AI work and Musk now has another AI startup called xAI. He’s well-known for his controversial opinions, and his comment about AI ending all jobs is aimed to fuel controversy.

Computing has been a disruptor throughout the last few generations. Generative AI is the latest iteration of that cycle.

Will we see jobs replaced by AI? Almost certainly! But new jobs will be created in their place.

To give an example, when I entered the workplace in the 1990s, we didn’t have social media and all the job roles that are built around it today; PR involved phoning journalists and taking clippings from newspapers to show where coverage had been gained for clients; and advertising was limited to print, TV, radio, and billboards. That world has changed immensely in the last 30 years and that’s just one sector.

For another example, look at retail where a huge number of transactions are now online and even those in-store may be self-service. Or the rise in logistics opportunities (warehousing, transportation) as a result of all our online commerce and the fuel for ever more variety in our purchases.

The jobs that my sons are taking on as they enter the workplace are vastly different to the ones I had. And the jobs their children take a generation later will be different again.

Just as robots have become commonplace on production lines and impacted “blue collar” roles, AI is the productivity enhancement that will impact “white collar” jobs.

So will AI end work? Maybe one day, but not for a while yet!

When and if it does, way out in the future, we will need new social constructs – perhaps a Universal Basic Income – to replace wages and salaries from employment, but right now that’s a distant dream.

How far is there still to go to overcome the ethical nightmares surrounding the technology e.g. data privacy, algorithm bias?

There’s a lot of work that’s been done to overcome ethical issues with artificial intelligence but there’s still a lot to do.

The major source of bias is the training data used by models. We’re looking at AI in the context of ChatGPT and my understanding is that ChatGPT was trained, in part, on information from the world-wide web. That data varies tremendously in quality and scope.

As it becomes easier for organisations to generate their own generative AI models, tuned with their own data, we can expect to see improved quality in the outcomes. Whether that improved quality translates into ethical responses depends on the decisions made by the humans that decide how to use the AI results. OpenAI and its partners will be keen to demonstrate how they are improving the models that they create and reducing bias, but this is a broader social issue, and we can’t simply rely on technology.

ChatGPT has developed so much in just a year, what does the next year look like for its capabilities? How will the workplace look different this time next year, thanks to ChatGPT?

We’ve been using forms of AI in our work for years – for example predictive text, design suggestions, grammar correction – but the generative AI that ChatGPT has made us all familiar with over the last year is a huge step forwards.

Microsoft is a significant investor in OpenAI (the creators of ChatGPT) and the licensing of the OpenAI models is fundamental to Microsoft’s strategy. So much so that, when Sam Altman and Greg Brockman (OpenAI’s CEO and CTO respectively) abruptly left the company, Microsoft moved quickly to offer them the opportunity to set up an advanced AI Research function inside Microsoft. Even though Altman and Brockman were soon reinstated at OpenAI, it shows how important this investment is to Microsoft.

In mid-November 2023, the main theme of the Microsoft Ignite conference was AI, including new computing capabilities, the inclusion of a set of Copilots in Microsoft products – and the ability for Microsoft’s customers to create their own Copilots. Indeed, in his keynote, Microsoft CEO Satya Nadella repeatedly referred to the “age of the Copilot”.

These Copilots are assistive technologies – agents to help us all be more productive in our work. The Copilots use ChatGPT and other models to generate text and visual content.

Even today, I regularly use ChatGPT to help me with a first draft of a document, or to help me write an Excel formula. The results are far from perfect, but the output is “good enough” for a first draft that I can then edit. That’s the inspiration to get me going, or the push to see something on the page, which I can then take to the next level.

This is what’s going to influence our workplaces over the next year or so. Whereas now we’re talking about the potential of Copilots, next year we’ll be using them for real.

What about 5 or 10 years time?

5 or 10 years is a long time in technology. And the pace of change seems to be increasing (or maybe that’s just a sign of my age).

I asked Microsoft Copilot (which uses ChatGPT) what the big innovations of 2013 were and it said: Google Glass (now confined to the history books); Oculus Rift (now part of Meta’s plans for augmented reality) and Bitcoin (another controversial technology whose fans say it will be world-changing and critics say has no place in society). For 2018 it was duelling neural networks (AI); babel-fish earbuds (AI) and zero-carbon natural gas.

The fact that none of these are commonplace today tells me that predicting the future is hard!

If pushed, and talking specifically about innovations where AI plays a part, I’d say that we’ll be looking at a world where:

  • Our interactions with technology will have changed – we will have more (and better) spoken interaction with our devices (think Apple Siri or Amazon Alexa, but much improved). Intelligent assistants will be everywhere. And the results they produce will be integrated with Augmented Reality (AR) to create new computing experiences that improve productivity in the workplace – be that an industrial setting or an office.
  • The computing services that are provided by companies like Node4 but also by the hyperscalers (Microsoft, Amazon, et al.) will have evolved. AI will change the demands of our clients and that will be reflected in the ways that we provide compute, storage and connectivity services. We’ll need new processors (moving beyond CPUs, GPUs, TPUs, to the next AI-driven approach), new approaches to power and cooling, and new approaches to data connectivity (hollow-core fibre, satellite communications, etc.).
  • People will still struggle to come to grips with new computing paradigms. We’ll need to invest more and more effort into helping people make good use of the technologies that are available to them. Crucially, we’ll also need to help current and future generations develop critical thinking skills to consider whether the content they see is real or computer-generated.

Anything else to add?

Well, a lot has been made of ChatGPT and its use in an educational context. One question is “should it be banned?”

In the past, calculators (and even slide rules – though they were before my time!) were banned before they became accepted tools, even in the exam hall. No-one would say today “you can’t use Google to search for information”. That’s the viewpoint we need to get to with generative AI. ChatGPT is just a tool and, at some point, instead of banning these tools, educational establishments (and broader society) will issue guidelines for their acceptable use.

Postscript

Bonus points for working out which part of this was original content by yours truly, and which had some AI assistance…

Featured image: generated via Microsoft Copilot, powered by DALL-E 3.

What did we learn at Microsoft Ignite 2023?

Right now, there’s a whole load of journalists and influencers writing about what Microsoft announced at Ignite. I’m not a journalist, and Microsoft has long since stopped considering me as an influencer. Even so, I’m going to take a look at the key messages. Not the technology announcements – for those there’s the Microsoft Ignite 2023 Book of News – but the real things IT Leaders need to take away from this event.

Microsoft’s investment in OpenAI

It’s all about AI. I know, you’re tired of the hype, suffering from AI fatigue, but for Microsoft, it really is about AI. And if you were unconvinced just how important AI is to Microsoft’s strategy, their action to snap up key members of staff from an imploding OpenAI a week later should be all you need to see:

Tortoise Media‘s Barney Macintyre (@barneymac) summed it up brilliantly when he said that

“Satya Nadella, chief executive of Microsoft, has played a blinder. Altman’s firing raised the risk that he would lose a key ally at a company into which Microsoft has invested $13 billion. After it became clear the board wouldn’t accept his reinstatement, Nadella offered jobs to Altman, Brockman and other loyalist researchers thinking about leaving.

The upshot: a new AI lab, filled with talent and wholly owned by Microsoft – without the bossy board. An $86 billion subsidiary for a $13 billion investment.”

But the soap opera continued and, by the middle of the week, Altman was back at OpenAI, apparently with the blessing of Microsoft!

If nothing else, this whole saga should reinforce just how important OpenAI is to Microsoft.

The age of the Copilot

Copilot is Microsoft’s brand for a set of assistive technologies that will sit alongside applications and provide an agent experience, built on ChatGPT, Dall-E and other models. Copilots are going to be everywhere. So much so that there is a “stack” for Copilot and Satya described Microsoft as “a Copilot company”.

That stack consists of:

  • The AI infrastructure in Azure – all Copilots are built on AzureAI.
  • Foundation models from OpenAI, including the Azure OpenAI Service to provide access in a protected manner but also new OpenAI models, fine-tuning, hosted APIs, and an open source model catalogue – including Models as a Service in Azure.
  • Your data – and Microsoft is pushing Fabric as all the data management tools in one SaaS experience, with onwards flow to Microsoft 365 for improved decision-making, Purview for data governance, and Copilots to assist. One place to unify, prepare and model data (for AI to act upon).
  • Applications, with tools like Microsoft Teams becoming more than just communication and collaboration but a “multi-player canvas for business processes”.
  • A new Copilot Studio to extend and customise Microsoft Copilot, with 1100 prebuilt plugins and connectors for every Azure data service and many common enterprise data platforms.
  • All wrapped with a set of AI safety and security measures – both in the platform (model and safety system) and in application (metaprompts, grounding and user experience).

In addition to this, Bing Chat is now re-branded as Copilot – with an enterprise version at no additional cost to eligible Entra ID users. On LinkedIn this week, one Microsoft exec posted that “Copilot is going to be the new UI for work”.

In short, Copilots will be everywhere.

Azure as the world’s computer

Of course, other cloud platforms exist, but I’m writing about Microsoft here. So what did they announce that makes Azure even more powerful and suited to running these new AI workloads?

  • Re-affirming the commitment to zero carbon power sources and then becoming carbon negative.
  • Manufacturing their own hollow-core fibre to drive up speeds.
  • Azure Boost (offloading server virtualisation processes from the hypervisor to hardware).
  • Taking the innovation from Intel and AMD but also introducing new Microsoft silicon: Azure Cobalt (ARM-based CPU series) and Azure Maia (AI accelerator in the form of an LLM training and inference chip).
  • More AI models and APIs. New tooling (Azure AI Studio).
  • Improvements in the data layer with enhancements to Microsoft Fabric. The “Microsoft Intelligent Data Platform” now has 4 tenets: databases; analytics; AI; and governance.
  • Extending Copilot across every role and function (as I briefly discussed in the previous section).

In summary, and looking forward

Microsoft is powering ahead on the back of its AI investments. And, as tired of the hype as we may all be, it would be foolish to ignore it. Copilots look to be the next generation of assistive technology that will help drive productivity. Just as robots have become commonplace on production lines and impacted “blue collar” roles, AI is the productivity enhancement that will impact “white collar” jobs.

In time we’ll see AI and mixed reality coming together to make sense of our intent and the world around us. Voice, gestures, and where we look become new inputs – the world becomes our prompt and interface.

Featured images: screenshots from the Microsoft Ignite keynote stream, under fair use for copyright purposes.