As I stare at my laptop after an intense few days at 44CON, I reflect on the experience. I went through a lot of different emotions. Excitement, trepidation, relief, bewilderment, pride, disappointment and hope had accompanied my visit to London to 44CON. We had an OSINT Capture The Flag competition happen right next to a round-table discussion on how the government should secure the country. Looking from outside in, a rallying cry of “Hack the Planet” (if you want to feel old, it turned 28 during the con) side-by-side with the establishment might not be the first thing to expect at a security conference.

Day 22 of Advent of Code 2021 was all about cubes. The problem statement provided us with coordinates of cubes and an “on” or “off” instructions. These instructions needed to be carried out in order. The first part of the problem was deceptively simple. So much so that I didn’t go for the “obvious” solution initially because I thought it wouldn’t scale. As it happened I ended up implementing it anyway as my optimised approach wasn’t working because I made a simple mistake.

On Day 21 of Advent of Code 2021 we played Dirac Dice! And part one felt way too easy to solve, but I needed have worried because part two we were asked to solve a limited multiverse problem. And without a quantum computer! The problem The problem was described as follows. Two players are playing a game with three dice Each starts on a different position of the board The board is circular and has 10 numbered positions Players take it in turn to roll three dice A player moves forward by the sum of those three dice Once the complete their move, they add as many points to their score as indicated by the board position First to 1000 points wins The dice are deterministic, meaning that they roll 1, 2, 3, …, 100, 1, 2, … Part One To start with, I created some types:

Hmpf. It’s now January and I’m still doing Advent. Still, I was determine to push ahead. Then I got stuck good and proper on Day 19 - though to be fair the whole Log4shell dumpster fire was taking up a lot of time, so I decided to skip it for now. Day 20 of Advent of Code 2021 was all about transforming images. What made it slightly more complicated was the fact that the images were nominally of infinite size.

Day 18 of Advent of Code 2021 took rather a long time to get right. Half of the problem was that there were a couple of gotchas in the requirements and the choice of data structure hugely impacted how easy it was to come up with a solution. Just like Day 17 I was using TDD and this helped me root out some blind alleys nicely. The problem The story was roughly as follows: In order to continue on the Advent of Code submarine journey, we had to solve snailfish number problems.

On day 17 of Advent of Code 2021, I did something that I should have really done from the start: Write tests first. My excuse so far has been that I wasn’t familiar with Haskell yet and other such lame excuses. As a software engineer, I should know better. TDD doesn’t slow me down, it makes things faster. The last straw - if you like - was when I was writing up my exploits from Day 16 and I thought there really is no excuse for not testing these parser combinators individually.

For day 16 Advent of Code 2021, it was all about monadic parser combinators (whatever they are)! Just like Day 15, this one took me a lot longer to complete than I had wanted, this time it was all about learning Haskell’s approach to parsing text. According to the story, we had to decode a message in a custom binary format, the Buoyancy Interchange Transmission System (BITS). BITS is a message format that encodes both literal values and operations.

Advent of Code 2021 day 15 took us back to the 2-dimensional maps we first encountered on Day 11 and Day 9. We had to parse a map again, and this time we had to find the shortest path from the top left to the bottom right. This was the first time when a homespun algorithm just did not cut the mustard, and I had to wrack my brain (i.e. use Google) to do a bit of algorithm research.

After Advent of Code 2021 transparent origami on Day 13 with more output than just a number, for Day 14 it was back to more standard processing with exponential complexity. Interestingly in part 1 I initially started going down the route of not trying to brute forcing it, got almost there but tangled myself in some logic, so I decided to “sod it, I’ll just brute force it” only to then throw the brute force solution away again as soon as I got to part 2.

On day 13 of Advent of Code 2021 we had to do the impossible. We were folding a transparent piece of paper 12 times. Surely it isn’t possible to fold paper more than 7 times, but we also have to remember that we’re in a Santa submarine! The aim of the puzzle is to get a code by folding over a transparent sheet of paper which has got dots dotted all over.

For Advent of Code 2021 day 12 it took me a while to get going though for reasons unrelated to coding, Norovirus is not pleasant! So when I finally did the puzzle, I didn’t find it as difficult as I thought it would be based on some of the messages on the #adventofcode channel. The puzzle started with With your submarine’s subterranean subsystems subsisting suboptimally This deserves a medal in itself.

Advent of Code 2021 day 11 felt a little bit like a repeat of Day 9. In both cases, we got a 2-dimensional map with single digit values. In case of day 9 it was a height map, this time round we’ve got a 10x10 grid of bioluminescent Octopi. Each of those critters has an energy level that increases each round. Once that level goes past 9, it lets off a flash, which then imparts extra energy into the surrounding 8-legged creatures.

Day 10 of Advent of Code 2021 was all about mismatched brackets. And I think Haskell did rather well on this as I found the expressiveness arrived at a really concise solution. We’re still in our Advent submarine and after mapping the floor on Day 9, we now discover lots of syntax errors in the submarine navigation system. The problem This puzzle was all about matching brackets, the input was like this:

On day 9 of Advent of Code 2021 I arrived at two different solutions for parts one and two. That is to say the approach I took in part one turned out to be useless for part two, though I’m pleased with both solutions. Unlike yesterday’s which got a bit convoluted (Day 8). The problem This puzzle was all about looking at a height map and finding the lowest points. The input would look like so:

On day 8 Advent of Code 2021 adventure, part 1 lulled me into a false sense of security. It took longer to read the instructions than implementing the code. But then part 2 was back with a vengeance. Admittedly it would have been so much easier if I slowed down a bit and actually counted carefully. As it happened it was a good reminder that “more haste, less speed” applies to programming very much.

On the 7th day of the submarine Advent of Code 2021 adventure, I felt like the problem was getting a bit easier than it had been of late. If anything I felt a touch suspicious on whether I had been missing anything. In previous days I had to reframe the problem statement a bit to make it efficient (Day 4, Day 5 and Day 5). Either that or I’m starting to feel more comfortable with Haskell.

On the next day of Advent of Code 2021 there were two approaches to solving the problem. Brute force and ignorance, and stopping to think about it. After finding out that Haskell often forces me to having a think first on Day 4 and Day 5, I was looking forward to The problem This puzzle was all about lanternfish population - yes there was a submarine theme to this year!

Following from my efforts on Day 4 of Advent of Code 2021 I was encouraged to try writing up more of my attempts to teach myself Haskell while having fun trying to solve puzzles. The problem The exercise was concerned with drawing lines on a grid and finding intersections: An entry like 1,1 -> 1,3 covers points 1,1, 1,2, and 1,3. An entry like 9,7 -> 7,7 covers points 9,7, 8,7, and 7,7.

For Advent of Code 2021 rather than just thinking “Hey, what a good idea - I should give that a go” and then promptly forgetting about it, I thought it would be a good opportunity to explore Haskell a bit more. On day 4 - when I started scratching my head for a bit - I discovered one interesting aspect about a functionally pure approach “forced” on me by Haskell.

I am wearing my Equal Experts hoodie. Often I’ll be found on conference calls with my EE t-shirts, and I even have some socks. So the irony of writing an article about “leaving the badge on the door” is not lost on me. So what do I mean? Contracting Scum I am a contractor. I have been for the best part of 20 years. I’ve worked in a variety of sectors (telecomms, finance, government) in a variety of roles (development, architecture, security) using a variety of methodologies (waterfall - eurgh, faux agile - double eurgh - and lowercase agile).

I am back at home, a few days after my first experience of Agile on the Beach. There were a few problems with it (yes, yes, I’m baiting you, I loved it). There was too much quality, it was really hard to pick which talks to go to. There was too much fun, it was difficult to leave the after-party after the beach party. And it was hard to keep track of all the new faces and names.

As I am sitting at Schipol Airport, contemplating that the airport is so big it has a branch of the Rijksmuseum, I can’t help but thinking about the fabulous conference I’ve just been to. The DevOps Enterprise Summit 2023 has exceeded my expectations. But before I get to there, I’d just like to develop the airport metaphor a little bit. So here I am, looking at culture. Why was it that at Schipol, one of the busiest airports in the world, it was a breeze to get through security, airport security agents were smiling, cracking jokes, I could leave my laptop, toothpaste and deodorant in the bag and they have art in the middle of the airport, goddammit.

As I am sitting on the 16th floor of the Okura Hotel in Amsterdam in my Batman pyjamas and facing a large mirror, just glimpsing the paper crane that was so lovingly put on my pillow, I’m starting to reflect what has happened during the day. I was lucky enough to be a guest at the DevOps Enterprise Summit. There was certainly plenty of things to get excited about. I’d meet Gene Kim, who’s been running this conference for 10 years and who wrote the Phoenix Project.

“Outdated tech stack and metaphorical gaffer tape holding together the code” - sound familiar? How about “We can’t recruit, because nobody wants to touch this legacy stuff”? Yet banks and governments would stop working if mainframes were switched off. Instead of outsourcing to the lowest bidder, maintenance is a job for experienced engineers. Not “the short straw”, brown field development can be more exciting than a feature factory. After all, the shiny code written today is the legacy code of tomorrow…

Let me tell you a story about Application Security (AppSec). It contains heroes and villains, and I’m not necessarily thinking about the defenders and attackers here. It contains lots of interesting technology that is often overemphasised. We’ve got whole industries that work on letting us know how scary it is out there, vulnerabilities that are marketed like rock stars and terminology that makes you quiver in your boots: who would want to fall victim to an Advanced Persistent Threat (APT)?

I am sitting on the train as I type this coming back from my first IRL conference. Lean Agile Scotland 2022 was brilliant. I met so many people with ideas that just chime with my thinking on agile, I feel energised and full of ideas No Bugs No Decisions No Deadlines No Fear What follows are some thoughts that I took away from some of sessions the conference. There were lots of exciting talks and workshops and it was a pity that I couldn’t clone myself to go to more of them (I was gutted to miss sessions that coincided with my own talk).

This is part of my series of interview questions for Agile India 2022. Thinking about exciting developments in software development as a developer immediately brings magpies to mind. What is the latest technology that can grab our attention. What’s the latest shiny? Could it be some crypto tech? Some no-code development? Or programming in the metaverse maybe? No, no, and NO! Crypto is a scourge that wastes energy, kills the planet and destroys lives, no-code and low code will mean some actual software engineers will have to pick up the pieces when it inevitably will go wrong and the metaverse (second second life) looks to be a great way to make the web as tedious to use than going to an actual shop where you can’t find anything because the items have moved to another shelf yet again.

This is part of my series of interview questions for Agile India 2022. It is difficult to articulate what the biggest challenge to software product engineering is, because there’s a fair few of them, so I’ll try to describe a few of my bug bears. Process over people In my opinion, a huge issue that we unfortunately find all too often is when the principles of the agile manifesto are ignored or not applied.

After being accepted for speaking at Agile India 2022 I was asked a few questions. I figured rather than a paragraph, the questions would make interesting blog posts: I came to agile completely the wrong way. I’ve been developing software professionally since I was a teenager and was lucky enough to start out with projects where I was given plenty of agency to decide on how to tackle problems and the independence to make my own mistakes and learn from them.

Dave Farley and Allen Holub are two people that I respect hugely when it comes to Software Development. I’ve been following them on twitter for quite a while, and am always taken in about their takes on driving continuous delivery and “lowercase agile” forward. So when both came together for Dave’s Engineering Room (sponsored by Equal Experts, who I work with), it was bound to be very interesting. In this post I try to outline my key takeaways from watching the chat.

I was really positively and pleasantly surprised when I found out what HMRC Digital’s mission statement was. Normally, I am not a fan of mission statements. They usually read like this: Our mission is focused on six core aspirations the company continually strives to achieve… Or some such drivel. Verbal gymnastics to make a company sound like everything to everyone - without being offensive to anyone - which then gets used to “align” people on mandated fun days.

In this post, I am going to look at an increasingly important part of securing applications: Your supply chain. This includes every library, tool or service that you are using to build, run and monitor your service. When the log4shell vulnerability hit, it wasn’t just a matter of looking at the dependencies that your source code pulls in, but also at the infrastructure you’re using and the build pipeline. Have you had a look at the vulnerability reports of your dependencies lately?

This post peels back the covers on what it is like to work with a large digital platform. The platform in question is MDTP - Multichannel Digital Tax Platform, which supports a UK-based tax collection agency which is using a hyperscale cloud provider with a sideline in books. I’ve previously described what it is like to work in MDTP (Making Software. Quickly) during the Covid-19 responses that allowed the UK government to provide financial support for millions turning around projects in record time.

A few days back as part of a general discussion about interviewing at Equal Experts, we looked at the question “What makes a good developer?” Could we come up with a list of qualities in a developer that we’d want to look for? This post illustrates my thinking. Why do you ask? To put it in a bit of context, I’m a software developer, I’m not a recruiter, but I’ve been involved with technical interviewing for quite a while, and have marked a fair number of take-home tests over the years, but recently we thought that the experience was not as good as it could be.

Dave Farley and Martin Fowler are two heavyweights when it comes to Software Development. As well as his latest and hugely anticipated book Modern Software Engineering, Dave wrote Continuous Delivery. Martin co-wrote the Agile Manifesto and Refactoring. All of which are hugely influential to large swathes of software engineers. So when both came together for Dave’s new series on YouTube, the Engineering Room (sponsored by Equal Experts), it was bound to be very interesting.

Scenario: You’re in a handover session and explain everything in great detail but Dave is too hungover, Jim is on holiday and Jane is at another meeting and Chris isn’t asking any questions and in any case Trevor (who is actually going to be looking after this system after handover) hasn’t been recruited yet. Wouldn’t it be useful to record the session and make it available? Now, this piece of writing does not explore whether we should be recording meetings, nor does it attempt to answer the question of whether good documentation wouldn’t be preferable to having to sit and spend hours watching someone droning on about something that could be condensed into a five minute read of a blog post.

To successfully deliver software a developer needs a shield and a sword. A good product owner, delivery manager or alike that will shield the engineers from having to attend too many meetings, giving long-winded status updates and essentially allow them to get on with it. Shields go to all the meetings with the “business” and explain what can and cannot be done, without the engineer having to attend. A shield would also filter all the last-minute requirements or requests for gold plating without having to be dragged into endless meetings.

In this post, I describe my personal experience of being part of a software development team working with Equal Experts and HMRC during Covid-19. Under normal circumstances, we’re responsible for tax services such as Self Assessment, PAYE Expenses and Benefits, VAT submissions amongst others. These services run on the Multi-channel Digital Tax Platform (MDTP). This platform is hosted in a hyperscale cloud (the cloud provider has a sideline selling books), run in-house by HMRC teams made up of permanent staff and consultants.

This is part of my series of interview questions for Agile India 2022. Thinking about exciting developments in software development as a developer immediately brings magpies to mind. What is the latest technology that can grab our attention. What’s the latest shiny? Could it be some crypto tech? Some no-code development? Or programming in the metaverse maybe? No, no, and NO! Crypto is a scourge that wastes energy, kills the planet and destroys lives, no-code and low code will mean some actual software engineers will have to pick up the pieces when it inevitably will go wrong and the metaverse (second second life) looks to be a great way to make the web as tedious to use than going to an actual shop where you can’t find anything because the items have moved to another shelf yet again.

This is part of my series of interview questions for Agile India 2022. It is difficult to articulate what the biggest challenge to software product engineering is, because there’s a fair few of them, so I’ll try to describe a few of my bug bears. Process over people In my opinion, a huge issue that we unfortunately find all too often is when the principles of the agile manifesto are ignored or not applied.

After being accepted for speaking at Agile India 2022 I was asked a few questions. I figured rather than a paragraph, the questions would make interesting blog posts: I came to agile completely the wrong way. I’ve been developing software professionally since I was a teenager and was lucky enough to start out with projects where I was given plenty of agency to decide on how to tackle problems and the independence to make my own mistakes and learn from them.

This is part of my series of interview questions for Agile India 2022. Thinking about exciting developments in software development as a developer immediately brings magpies to mind. What is the latest technology that can grab our attention. What’s the latest shiny? Could it be some crypto tech? Some no-code development? Or programming in the metaverse maybe? No, no, and NO! Crypto is a scourge that wastes energy, kills the planet and destroys lives, no-code and low code will mean some actual software engineers will have to pick up the pieces when it inevitably will go wrong and the metaverse (second second life) looks to be a great way to make the web as tedious to use than going to an actual shop where you can’t find anything because the items have moved to another shelf yet again.

This is part of my series of interview questions for Agile India 2022. It is difficult to articulate what the biggest challenge to software product engineering is, because there’s a fair few of them, so I’ll try to describe a few of my bug bears. Process over people In my opinion, a huge issue that we unfortunately find all too often is when the principles of the agile manifesto are ignored or not applied.

After being accepted for speaking at Agile India 2022 I was asked a few questions. I figured rather than a paragraph, the questions would make interesting blog posts: I came to agile completely the wrong way. I’ve been developing software professionally since I was a teenager and was lucky enough to start out with projects where I was given plenty of agency to decide on how to tackle problems and the independence to make my own mistakes and learn from them.

I am back at home, a few days after my first experience of Agile on the Beach. There were a few problems with it (yes, yes, I’m baiting you, I loved it). There was too much quality, it was really hard to pick which talks to go to. There was too much fun, it was difficult to leave the after-party after the beach party. And it was hard to keep track of all the new faces and names.

The question of automatic dependency updates came up in our Slack channel the other day. There was a lot of nodding on how it is a good thing. Tools like Dependabot and Renovate were mentioned. Yet I was a dissenting voice. Why? The case for automatic dependency updates is simple and seductive: A bot would automatically scan your dependencies in your source code and create pull requests to update your libraries to the latest versions, sometimes even automatically merging it in.

From a security point of view, application logs are two-sided. On the one hand, it is really important to have good observability, to find out what is happening and what has happened. On the other hand, we don’t want to leak sensitive information. In this post I am going to look at the kinds of things you might find in your logs. The juicy bits are Personal Identifying Information (PII) or security credentials.

Let me tell you a story about Application Security (AppSec). It contains heroes and villains, and I’m not necessarily thinking about the defenders and attackers here. It contains lots of interesting technology that is often overemphasised. We’ve got whole industries that work on letting us know how scary it is out there, vulnerabilities that are marketed like rock stars and terminology that makes you quiver in your boots: who would want to fall victim to an Advanced Persistent Threat (APT)?

There’s a metaphor about the fight between attackers and defenders in the Denial of Service cybersecurity game. It’s an “arms race” between ever bigger attacks throwing huge amounts of traffic at ever more sophisticated defenses (e.g. AWS shield). Incidentally, I’ve just demonstrated an easy mistake: I’m not describing a Denial of Service (DoS) attack, it’s a Distributed Denial of Service (DDoS) attack. The aim is to overwhelm the infrastructure, either the networking infrastructure or the application by sending more requests than can be handled.

In this post, I am going to look at an increasingly important part of securing applications: Your supply chain. This includes every library, tool or service that you are using to build, run and monitor your service. When the log4shell vulnerability hit, it wasn’t just a matter of looking at the dependencies that your source code pulls in, but also at the infrastructure you’re using and the build pipeline. Have you had a look at the vulnerability reports of your dependencies lately?

In this writing I ask: When will this madness end? When will we wake up from this fascination with cryptocurrencies (chief amongst which Bitcoin) that is turning technological orthodoxy on it’s head: It used to be that technological progress was measured on making things cheaper, faster, more secure or more scalable. But it seems that Bitcoin does not follow these. Bitcoin Over the last 12 years, it is hard to have lived and not notice the poster child of blockchains: Bitcoin.

It is a couple of weeks after my trip to Leeds to go speak at BSides. Now, if you’ve never been to a BSides - and I had only been to one prior - then hopefully this gives you a bit of a feel as to what to expect. In short, there’s lots of interesting people with great talks and insights and something that feels just right. A bit of community and some weird people.

In this post, I am going to look at an increasingly important part of securing applications: Your supply chain. This includes every library, tool or service that you are using to build, run and monitor your service. When the log4shell vulnerability hit, it wasn’t just a matter of looking at the dependencies that your source code pulls in, but also at the infrastructure you’re using and the build pipeline. Have you had a look at the vulnerability reports of your dependencies lately?

This post peels back the covers on what it is like to work with a large digital platform. The platform in question is MDTP - Multichannel Digital Tax Platform, which supports a UK-based tax collection agency which is using a hyperscale cloud provider with a sideline in books. I’ve previously described what it is like to work in MDTP (Making Software. Quickly) during the Covid-19 responses that allowed the UK government to provide financial support for millions turning around projects in record time.

As I stare at my laptop after an intense few days at 44CON, I reflect on the experience. I went through a lot of different emotions. Excitement, trepidation, relief, bewilderment, pride, disappointment and hope had accompanied my visit to London to 44CON. We had an OSINT Capture The Flag competition happen right next to a round-table discussion on how the government should secure the country. Looking from outside in, a rallying cry of “Hack the Planet” (if you want to feel old, it turned 28 during the con) side-by-side with the establishment might not be the first thing to expect at a security conference.

I am back at home, a few days after my first experience of Agile on the Beach. There were a few problems with it (yes, yes, I’m baiting you, I loved it). There was too much quality, it was really hard to pick which talks to go to. There was too much fun, it was difficult to leave the after-party after the beach party. And it was hard to keep track of all the new faces and names.

It is a couple of weeks after my trip to Leeds to go speak at BSides. Now, if you’ve never been to a BSides - and I had only been to one prior - then hopefully this gives you a bit of a feel as to what to expect. In short, there’s lots of interesting people with great talks and insights and something that feels just right. A bit of community and some weird people.

As I am sitting at Schipol Airport, contemplating that the airport is so big it has a branch of the Rijksmuseum, I can’t help but thinking about the fabulous conference I’ve just been to. The DevOps Enterprise Summit 2023 has exceeded my expectations. But before I get to there, I’d just like to develop the airport metaphor a little bit. So here I am, looking at culture. Why was it that at Schipol, one of the busiest airports in the world, it was a breeze to get through security, airport security agents were smiling, cracking jokes, I could leave my laptop, toothpaste and deodorant in the bag and they have art in the middle of the airport, goddammit.

As I am sitting on the 16th floor of the Okura Hotel in Amsterdam in my Batman pyjamas and facing a large mirror, just glimpsing the paper crane that was so lovingly put on my pillow, I’m starting to reflect what has happened during the day. I was lucky enough to be a guest at the DevOps Enterprise Summit. There was certainly plenty of things to get excited about. I’d meet Gene Kim, who’s been running this conference for 10 years and who wrote the Phoenix Project.

I am sitting on the train as I type this coming back from my first IRL conference. Lean Agile Scotland 2022 was brilliant. I met so many people with ideas that just chime with my thinking on agile, I feel energised and full of ideas No Bugs No Decisions No Deadlines No Fear What follows are some thoughts that I took away from some of sessions the conference. There were lots of exciting talks and workshops and it was a pity that I couldn’t clone myself to go to more of them (I was gutted to miss sessions that coincided with my own talk).

This is part of my series of interview questions for Agile India 2022. Thinking about exciting developments in software development as a developer immediately brings magpies to mind. What is the latest technology that can grab our attention. What’s the latest shiny? Could it be some crypto tech? Some no-code development? Or programming in the metaverse maybe? No, no, and NO! Crypto is a scourge that wastes energy, kills the planet and destroys lives, no-code and low code will mean some actual software engineers will have to pick up the pieces when it inevitably will go wrong and the metaverse (second second life) looks to be a great way to make the web as tedious to use than going to an actual shop where you can’t find anything because the items have moved to another shelf yet again.

This is part of my series of interview questions for Agile India 2022. It is difficult to articulate what the biggest challenge to software product engineering is, because there’s a fair few of them, so I’ll try to describe a few of my bug bears. Process over people In my opinion, a huge issue that we unfortunately find all too often is when the principles of the agile manifesto are ignored or not applied.

After being accepted for speaking at Agile India 2022 I was asked a few questions. I figured rather than a paragraph, the questions would make interesting blog posts: I came to agile completely the wrong way. I’ve been developing software professionally since I was a teenager and was lucky enough to start out with projects where I was given plenty of agency to decide on how to tackle problems and the independence to make my own mistakes and learn from them.

I am wearing my Equal Experts hoodie. Often I’ll be found on conference calls with my EE t-shirts, and I even have some socks. So the irony of writing an article about “leaving the badge on the door” is not lost on me. So what do I mean? Contracting Scum I am a contractor. I have been for the best part of 20 years. I’ve worked in a variety of sectors (telecomms, finance, government) in a variety of roles (development, architecture, security) using a variety of methodologies (waterfall - eurgh, faux agile - double eurgh - and lowercase agile).

This post describes how I developed the Discount Dining Finder a lookup map tool for the Eat Out to Help Out scheme in my spare time. I currently work with Equal Experts and HMRC. The aim of this writing is to provide an insight into how problems of scaling services can be solved by having no servers and not using “serverless services” either. Aperitif A really nice side effect in working in a high functioning environment is that sometimes you’re involved in bouncing ideas off each other.

In this post, I describe my personal experience of being part of a software development team working with Equal Experts and HMRC during Covid-19. Under normal circumstances, we’re responsible for tax services such as Self Assessment, PAYE Expenses and Benefits, VAT submissions amongst others. These services run on the Multi-channel Digital Tax Platform (MDTP). This platform is hosted in a hyperscale cloud (the cloud provider has a sideline selling books), run in-house by HMRC teams made up of permanent staff and consultants.

In this writing I ask: When will this madness end? When will we wake up from this fascination with cryptocurrencies (chief amongst which Bitcoin) that is turning technological orthodoxy on it’s head: It used to be that technological progress was measured on making things cheaper, faster, more secure or more scalable. But it seems that Bitcoin does not follow these. Bitcoin Over the last 12 years, it is hard to have lived and not notice the poster child of blockchains: Bitcoin.

In this post, I am going to look at an increasingly important part of securing applications: Your supply chain. This includes every library, tool or service that you are using to build, run and monitor your service. When the log4shell vulnerability hit, it wasn’t just a matter of looking at the dependencies that your source code pulls in, but also at the infrastructure you’re using and the build pipeline. Have you had a look at the vulnerability reports of your dependencies lately?

There’s a metaphor about the fight between attackers and defenders in the Denial of Service cybersecurity game. It’s an “arms race” between ever bigger attacks throwing huge amounts of traffic at ever more sophisticated defenses (e.g. AWS shield). Incidentally, I’ve just demonstrated an easy mistake: I’m not describing a Denial of Service (DoS) attack, it’s a Distributed Denial of Service (DDoS) attack. The aim is to overwhelm the infrastructure, either the networking infrastructure or the application by sending more requests than can be handled.

The question of automatic dependency updates came up in our Slack channel the other day. There was a lot of nodding on how it is a good thing. Tools like Dependabot and Renovate were mentioned. Yet I was a dissenting voice. Why? The case for automatic dependency updates is simple and seductive: A bot would automatically scan your dependencies in your source code and create pull requests to update your libraries to the latest versions, sometimes even automatically merging it in.

Let me tell you a story about Application Security (AppSec). It contains heroes and villains, and I’m not necessarily thinking about the defenders and attackers here. It contains lots of interesting technology that is often overemphasised. We’ve got whole industries that work on letting us know how scary it is out there, vulnerabilities that are marketed like rock stars and terminology that makes you quiver in your boots: who would want to fall victim to an Advanced Persistent Threat (APT)?

I was really positively and pleasantly surprised when I found out what HMRC Digital’s mission statement was. Normally, I am not a fan of mission statements. They usually read like this: Our mission is focused on six core aspirations the company continually strives to achieve… Or some such drivel. Verbal gymnastics to make a company sound like everything to everyone - without being offensive to anyone - which then gets used to “align” people on mandated fun days.

In this post, I am going to look at an increasingly important part of securing applications: Your supply chain. This includes every library, tool or service that you are using to build, run and monitor your service. When the log4shell vulnerability hit, it wasn’t just a matter of looking at the dependencies that your source code pulls in, but also at the infrastructure you’re using and the build pipeline. Have you had a look at the vulnerability reports of your dependencies lately?

This post peels back the covers on what it is like to work with a large digital platform. The platform in question is MDTP - Multichannel Digital Tax Platform, which supports a UK-based tax collection agency which is using a hyperscale cloud provider with a sideline in books. I’ve previously described what it is like to work in MDTP (Making Software. Quickly) during the Covid-19 responses that allowed the UK government to provide financial support for millions turning around projects in record time.

As I am sitting at Schipol Airport, contemplating that the airport is so big it has a branch of the Rijksmuseum, I can’t help but thinking about the fabulous conference I’ve just been to. The DevOps Enterprise Summit 2023 has exceeded my expectations. But before I get to there, I’d just like to develop the airport metaphor a little bit. So here I am, looking at culture. Why was it that at Schipol, one of the busiest airports in the world, it was a breeze to get through security, airport security agents were smiling, cracking jokes, I could leave my laptop, toothpaste and deodorant in the bag and they have art in the middle of the airport, goddammit.

As I am sitting on the 16th floor of the Okura Hotel in Amsterdam in my Batman pyjamas and facing a large mirror, just glimpsing the paper crane that was so lovingly put on my pillow, I’m starting to reflect what has happened during the day. I was lucky enough to be a guest at the DevOps Enterprise Summit. There was certainly plenty of things to get excited about. I’d meet Gene Kim, who’s been running this conference for 10 years and who wrote the Phoenix Project.

This post describes how I developed the Discount Dining Finder a lookup map tool for the Eat Out to Help Out scheme in my spare time. I currently work with Equal Experts and HMRC. The aim of this writing is to provide an insight into how problems of scaling services can be solved by having no servers and not using “serverless services” either. Aperitif A really nice side effect in working in a high functioning environment is that sometimes you’re involved in bouncing ideas off each other.

In this post I am looking at using the mongo aggregation pipeline in non-trivial ways. In fact, once I had done what I wanted to do I looked back at the result and discovered that a bit of an idea of functional programming helped me achieve what I wanted to do. But back to the beginning: I was looking at deployment events which we’d captured in a mongo collection: { "environment" : "production", "serviceName" : "platform-status-frontend", "version" : "0.

Day 22 of Advent of Code 2021 was all about cubes. The problem statement provided us with coordinates of cubes and an “on” or “off” instructions. These instructions needed to be carried out in order. The first part of the problem was deceptively simple. So much so that I didn’t go for the “obvious” solution initially because I thought it wouldn’t scale. As it happened I ended up implementing it anyway as my optimised approach wasn’t working because I made a simple mistake.

On Day 21 of Advent of Code 2021 we played Dirac Dice! And part one felt way too easy to solve, but I needed have worried because part two we were asked to solve a limited multiverse problem. And without a quantum computer! The problem The problem was described as follows. Two players are playing a game with three dice Each starts on a different position of the board The board is circular and has 10 numbered positions Players take it in turn to roll three dice A player moves forward by the sum of those three dice Once the complete their move, they add as many points to their score as indicated by the board position First to 1000 points wins The dice are deterministic, meaning that they roll 1, 2, 3, …, 100, 1, 2, … Part One To start with, I created some types:

Hmpf. It’s now January and I’m still doing Advent. Still, I was determine to push ahead. Then I got stuck good and proper on Day 19 - though to be fair the whole Log4shell dumpster fire was taking up a lot of time, so I decided to skip it for now. Day 20 of Advent of Code 2021 was all about transforming images. What made it slightly more complicated was the fact that the images were nominally of infinite size.

Day 18 of Advent of Code 2021 took rather a long time to get right. Half of the problem was that there were a couple of gotchas in the requirements and the choice of data structure hugely impacted how easy it was to come up with a solution. Just like Day 17 I was using TDD and this helped me root out some blind alleys nicely. The problem The story was roughly as follows: In order to continue on the Advent of Code submarine journey, we had to solve snailfish number problems.

On day 17 of Advent of Code 2021, I did something that I should have really done from the start: Write tests first. My excuse so far has been that I wasn’t familiar with Haskell yet and other such lame excuses. As a software engineer, I should know better. TDD doesn’t slow me down, it makes things faster. The last straw - if you like - was when I was writing up my exploits from Day 16 and I thought there really is no excuse for not testing these parser combinators individually.

For day 16 Advent of Code 2021, it was all about monadic parser combinators (whatever they are)! Just like Day 15, this one took me a lot longer to complete than I had wanted, this time it was all about learning Haskell’s approach to parsing text. According to the story, we had to decode a message in a custom binary format, the Buoyancy Interchange Transmission System (BITS). BITS is a message format that encodes both literal values and operations.

Advent of Code 2021 day 15 took us back to the 2-dimensional maps we first encountered on Day 11 and Day 9. We had to parse a map again, and this time we had to find the shortest path from the top left to the bottom right. This was the first time when a homespun algorithm just did not cut the mustard, and I had to wrack my brain (i.e. use Google) to do a bit of algorithm research.

After Advent of Code 2021 transparent origami on Day 13 with more output than just a number, for Day 14 it was back to more standard processing with exponential complexity. Interestingly in part 1 I initially started going down the route of not trying to brute forcing it, got almost there but tangled myself in some logic, so I decided to “sod it, I’ll just brute force it” only to then throw the brute force solution away again as soon as I got to part 2.

On day 13 of Advent of Code 2021 we had to do the impossible. We were folding a transparent piece of paper 12 times. Surely it isn’t possible to fold paper more than 7 times, but we also have to remember that we’re in a Santa submarine! The aim of the puzzle is to get a code by folding over a transparent sheet of paper which has got dots dotted all over.

For Advent of Code 2021 day 12 it took me a while to get going though for reasons unrelated to coding, Norovirus is not pleasant! So when I finally did the puzzle, I didn’t find it as difficult as I thought it would be based on some of the messages on the #adventofcode channel. The puzzle started with With your submarine’s subterranean subsystems subsisting suboptimally This deserves a medal in itself.

Advent of Code 2021 day 11 felt a little bit like a repeat of Day 9. In both cases, we got a 2-dimensional map with single digit values. In case of day 9 it was a height map, this time round we’ve got a 10x10 grid of bioluminescent Octopi. Each of those critters has an energy level that increases each round. Once that level goes past 9, it lets off a flash, which then imparts extra energy into the surrounding 8-legged creatures.

Day 10 of Advent of Code 2021 was all about mismatched brackets. And I think Haskell did rather well on this as I found the expressiveness arrived at a really concise solution. We’re still in our Advent submarine and after mapping the floor on Day 9, we now discover lots of syntax errors in the submarine navigation system. The problem This puzzle was all about matching brackets, the input was like this:

On day 9 of Advent of Code 2021 I arrived at two different solutions for parts one and two. That is to say the approach I took in part one turned out to be useless for part two, though I’m pleased with both solutions. Unlike yesterday’s which got a bit convoluted (Day 8). The problem This puzzle was all about looking at a height map and finding the lowest points. The input would look like so:

On day 8 Advent of Code 2021 adventure, part 1 lulled me into a false sense of security. It took longer to read the instructions than implementing the code. But then part 2 was back with a vengeance. Admittedly it would have been so much easier if I slowed down a bit and actually counted carefully. As it happened it was a good reminder that “more haste, less speed” applies to programming very much.

On the 7th day of the submarine Advent of Code 2021 adventure, I felt like the problem was getting a bit easier than it had been of late. If anything I felt a touch suspicious on whether I had been missing anything. In previous days I had to reframe the problem statement a bit to make it efficient (Day 4, Day 5 and Day 5). Either that or I’m starting to feel more comfortable with Haskell.

On the next day of Advent of Code 2021 there were two approaches to solving the problem. Brute force and ignorance, and stopping to think about it. After finding out that Haskell often forces me to having a think first on Day 4 and Day 5, I was looking forward to The problem This puzzle was all about lanternfish population - yes there was a submarine theme to this year!

Following from my efforts on Day 4 of Advent of Code 2021 I was encouraged to try writing up more of my attempts to teach myself Haskell while having fun trying to solve puzzles. The problem The exercise was concerned with drawing lines on a grid and finding intersections: An entry like 1,1 -> 1,3 covers points 1,1, 1,2, and 1,3. An entry like 9,7 -> 7,7 covers points 9,7, 8,7, and 7,7.

For Advent of Code 2021 rather than just thinking “Hey, what a good idea - I should give that a go” and then promptly forgetting about it, I thought it would be a good opportunity to explore Haskell a bit more. On day 4 - when I started scratching my head for a bit - I discovered one interesting aspect about a functionally pure approach “forced” on me by Haskell.

In this writing I aim to complete a Fizzbuzz without if statements, conditionals, pattern matching or even using modulus calculations. And if that isn’t enough I thought I’d use the opportunity to explore Haskell. The idea originated in the Friday lunchtime “Curry Club” at HMRC Digital where a few like-minded software engineers are getting together to teach themselves Haskell. (For those not in on the joke, the language is named after the logician Haskell Curry).

As I am sitting on the 16th floor of the Okura Hotel in Amsterdam in my Batman pyjamas and facing a large mirror, just glimpsing the paper crane that was so lovingly put on my pillow, I’m starting to reflect what has happened during the day. I was lucky enough to be a guest at the DevOps Enterprise Summit. There was certainly plenty of things to get excited about. I’d meet Gene Kim, who’s been running this conference for 10 years and who wrote the Phoenix Project.

FADE IN: INT. ABANDONED BUILDING - DAY Opay and Bart are sitting in a dimly lit room, surrounded by stacks of old newspapers and books. OPAY: (sighing) It's not their fault. They don't understand the meaning behind the words. BART: (confused) What do you mean? OPAY: (explaining) The machines were programmed to generate content based on keywords and patterns, not on the actual meaning of the words. BART: (realizing) So they just keep churning out more and more content, without any understanding of what it means.

Day 22 of Advent of Code 2021 was all about cubes. The problem statement provided us with coordinates of cubes and an “on” or “off” instructions. These instructions needed to be carried out in order. The first part of the problem was deceptively simple. So much so that I didn’t go for the “obvious” solution initially because I thought it wouldn’t scale. As it happened I ended up implementing it anyway as my optimised approach wasn’t working because I made a simple mistake.

On Day 21 of Advent of Code 2021 we played Dirac Dice! And part one felt way too easy to solve, but I needed have worried because part two we were asked to solve a limited multiverse problem. And without a quantum computer! The problem The problem was described as follows. Two players are playing a game with three dice Each starts on a different position of the board The board is circular and has 10 numbered positions Players take it in turn to roll three dice A player moves forward by the sum of those three dice Once the complete their move, they add as many points to their score as indicated by the board position First to 1000 points wins The dice are deterministic, meaning that they roll 1, 2, 3, …, 100, 1, 2, … Part One To start with, I created some types:

Hmpf. It’s now January and I’m still doing Advent. Still, I was determine to push ahead. Then I got stuck good and proper on Day 19 - though to be fair the whole Log4shell dumpster fire was taking up a lot of time, so I decided to skip it for now. Day 20 of Advent of Code 2021 was all about transforming images. What made it slightly more complicated was the fact that the images were nominally of infinite size.

Day 18 of Advent of Code 2021 took rather a long time to get right. Half of the problem was that there were a couple of gotchas in the requirements and the choice of data structure hugely impacted how easy it was to come up with a solution. Just like Day 17 I was using TDD and this helped me root out some blind alleys nicely. The problem The story was roughly as follows: In order to continue on the Advent of Code submarine journey, we had to solve snailfish number problems.

On day 17 of Advent of Code 2021, I did something that I should have really done from the start: Write tests first. My excuse so far has been that I wasn’t familiar with Haskell yet and other such lame excuses. As a software engineer, I should know better. TDD doesn’t slow me down, it makes things faster. The last straw - if you like - was when I was writing up my exploits from Day 16 and I thought there really is no excuse for not testing these parser combinators individually.

For day 16 Advent of Code 2021, it was all about monadic parser combinators (whatever they are)! Just like Day 15, this one took me a lot longer to complete than I had wanted, this time it was all about learning Haskell’s approach to parsing text. According to the story, we had to decode a message in a custom binary format, the Buoyancy Interchange Transmission System (BITS). BITS is a message format that encodes both literal values and operations.

Advent of Code 2021 day 15 took us back to the 2-dimensional maps we first encountered on Day 11 and Day 9. We had to parse a map again, and this time we had to find the shortest path from the top left to the bottom right. This was the first time when a homespun algorithm just did not cut the mustard, and I had to wrack my brain (i.e. use Google) to do a bit of algorithm research.

After Advent of Code 2021 transparent origami on Day 13 with more output than just a number, for Day 14 it was back to more standard processing with exponential complexity. Interestingly in part 1 I initially started going down the route of not trying to brute forcing it, got almost there but tangled myself in some logic, so I decided to “sod it, I’ll just brute force it” only to then throw the brute force solution away again as soon as I got to part 2.

On day 13 of Advent of Code 2021 we had to do the impossible. We were folding a transparent piece of paper 12 times. Surely it isn’t possible to fold paper more than 7 times, but we also have to remember that we’re in a Santa submarine! The aim of the puzzle is to get a code by folding over a transparent sheet of paper which has got dots dotted all over.

For Advent of Code 2021 day 12 it took me a while to get going though for reasons unrelated to coding, Norovirus is not pleasant! So when I finally did the puzzle, I didn’t find it as difficult as I thought it would be based on some of the messages on the #adventofcode channel. The puzzle started with With your submarine’s subterranean subsystems subsisting suboptimally This deserves a medal in itself.

Advent of Code 2021 day 11 felt a little bit like a repeat of Day 9. In both cases, we got a 2-dimensional map with single digit values. In case of day 9 it was a height map, this time round we’ve got a 10x10 grid of bioluminescent Octopi. Each of those critters has an energy level that increases each round. Once that level goes past 9, it lets off a flash, which then imparts extra energy into the surrounding 8-legged creatures.

Day 10 of Advent of Code 2021 was all about mismatched brackets. And I think Haskell did rather well on this as I found the expressiveness arrived at a really concise solution. We’re still in our Advent submarine and after mapping the floor on Day 9, we now discover lots of syntax errors in the submarine navigation system. The problem This puzzle was all about matching brackets, the input was like this:

On day 9 of Advent of Code 2021 I arrived at two different solutions for parts one and two. That is to say the approach I took in part one turned out to be useless for part two, though I’m pleased with both solutions. Unlike yesterday’s which got a bit convoluted (Day 8). The problem This puzzle was all about looking at a height map and finding the lowest points. The input would look like so:

On day 8 Advent of Code 2021 adventure, part 1 lulled me into a false sense of security. It took longer to read the instructions than implementing the code. But then part 2 was back with a vengeance. Admittedly it would have been so much easier if I slowed down a bit and actually counted carefully. As it happened it was a good reminder that “more haste, less speed” applies to programming very much.

On the 7th day of the submarine Advent of Code 2021 adventure, I felt like the problem was getting a bit easier than it had been of late. If anything I felt a touch suspicious on whether I had been missing anything. In previous days I had to reframe the problem statement a bit to make it efficient (Day 4, Day 5 and Day 5). Either that or I’m starting to feel more comfortable with Haskell.

On the next day of Advent of Code 2021 there were two approaches to solving the problem. Brute force and ignorance, and stopping to think about it. After finding out that Haskell often forces me to having a think first on Day 4 and Day 5, I was looking forward to The problem This puzzle was all about lanternfish population - yes there was a submarine theme to this year!

Following from my efforts on Day 4 of Advent of Code 2021 I was encouraged to try writing up more of my attempts to teach myself Haskell while having fun trying to solve puzzles. The problem The exercise was concerned with drawing lines on a grid and finding intersections: An entry like 1,1 -> 1,3 covers points 1,1, 1,2, and 1,3. An entry like 9,7 -> 7,7 covers points 9,7, 8,7, and 7,7.

For Advent of Code 2021 rather than just thinking “Hey, what a good idea - I should give that a go” and then promptly forgetting about it, I thought it would be a good opportunity to explore Haskell a bit more. On day 4 - when I started scratching my head for a bit - I discovered one interesting aspect about a functionally pure approach “forced” on me by Haskell.

In this writing I aim to complete a Fizzbuzz without if statements, conditionals, pattern matching or even using modulus calculations. And if that isn’t enough I thought I’d use the opportunity to explore Haskell. The idea originated in the Friday lunchtime “Curry Club” at HMRC Digital where a few like-minded software engineers are getting together to teach themselves Haskell. (For those not in on the joke, the language is named after the logician Haskell Curry).

A few days back as part of a general discussion about interviewing at Equal Experts, we looked at the question “What makes a good developer?” Could we come up with a list of qualities in a developer that we’d want to look for? This post illustrates my thinking. Why do you ask? To put it in a bit of context, I’m a software developer, I’m not a recruiter, but I’ve been involved with technical interviewing for quite a while, and have marked a fair number of take-home tests over the years, but recently we thought that the experience was not as good as it could be.

In this blog post, I would like to explore how missing input validation even in a trivial service can leave parts of server infrastructure crumbling. In my opinion, this why securing applications (AppSec) is very difficult. Put supply chain attacks, unpatched systems and misconfiguring services to one side for a minute and consider that a lot of software is written by developers who do not necessarily consider all the edge cases or implications of what can go wrong even in the simplest of pieces of code (or just copy/paste from Stackoverflow).

I am sitting on the train as I type this coming back from my first IRL conference. Lean Agile Scotland 2022 was brilliant. I met so many people with ideas that just chime with my thinking on agile, I feel energised and full of ideas No Bugs No Decisions No Deadlines No Fear What follows are some thoughts that I took away from some of sessions the conference. There were lots of exciting talks and workshops and it was a pity that I couldn’t clone myself to go to more of them (I was gutted to miss sessions that coincided with my own talk).

“Outdated tech stack and metaphorical gaffer tape holding together the code” - sound familiar? How about “We can’t recruit, because nobody wants to touch this legacy stuff”? Yet banks and governments would stop working if mainframes were switched off. Instead of outsourcing to the lowest bidder, maintenance is a job for experienced engineers. Not “the short straw”, brown field development can be more exciting than a feature factory. After all, the shiny code written today is the legacy code of tomorrow…

From a security point of view, application logs are two-sided. On the one hand, it is really important to have good observability, to find out what is happening and what has happened. On the other hand, we don’t want to leak sensitive information. In this post I am going to look at the kinds of things you might find in your logs. The juicy bits are Personal Identifying Information (PII) or security credentials.

It’s now about two weeks since the HMRCs splurge on getting the UK to go out for a meal has finished. In discount dining finder I showed how to make a map-based tool to search all Eat Out to Help Out (EOTHO) establishments in very little time. What follows is the story of what happened after the site launched, and how I failed to make a dent at marketing. To recap:

Scenario: You’re in a handover session and explain everything in great detail but Dave is too hungover, Jim is on holiday and Jane is at another meeting and Chris isn’t asking any questions and in any case Trevor (who is actually going to be looking after this system after handover) hasn’t been recruited yet. Wouldn’t it be useful to record the session and make it available? Now, this piece of writing does not explore whether we should be recording meetings, nor does it attempt to answer the question of whether good documentation wouldn’t be preferable to having to sit and spend hours watching someone droning on about something that could be condensed into a five minute read of a blog post.

In this post I am looking at using the mongo aggregation pipeline in non-trivial ways. In fact, once I had done what I wanted to do I looked back at the result and discovered that a bit of an idea of functional programming helped me achieve what I wanted to do. But back to the beginning: I was looking at deployment events which we’d captured in a mongo collection: { "environment" : "production", "serviceName" : "platform-status-frontend", "version" : "0.

Dave Farley and Allen Holub are two people that I respect hugely when it comes to Software Development. I’ve been following them on twitter for quite a while, and am always taken in about their takes on driving continuous delivery and “lowercase agile” forward. So when both came together for Dave’s Engineering Room (sponsored by Equal Experts, who I work with), it was bound to be very interesting. In this post I try to outline my key takeaways from watching the chat.

First of all I need the preface this article on how much I abhor the Russian invasion of Ukraine and I wholeheartedly support the sanctions. However, I think the conflict has spilled over into areas of software development that have got some unintended consequences attached. As part of this post, I’m going to look at the decision by MongoDB to cut off services in Russia the destructive change in a node library that deleted files on Russian IPs a change in code/licence in a community terraform module to assert that Putin is a dickhead MongoDB cutting off Russian customers MongoDB is a company and in order to comply with sanctions they have decided to cut off Russian customers.

Dave Farley and Martin Fowler are two heavyweights when it comes to Software Development. As well as his latest and hugely anticipated book Modern Software Engineering, Dave wrote Continuous Delivery. Martin co-wrote the Agile Manifesto and Refactoring. All of which are hugely influential to large swathes of software engineers. So when both came together for Dave’s new series on YouTube, the Engineering Room (sponsored by Equal Experts), it was bound to be very interesting.

After upgrading my AWS RDS instance from Oracle 11g Oracle 19c, my DB Link stopped working with a certificate error. Uh-oh. The symptoms of the problem that I encountered are best summed up by this SQL*Plus snippet: SQL> select * from dual@test1; select * from dual@test1 * ERROR at line 1: ORA-29024: Certificate validation failure A bit of context: The above is testing an Oracle Database Link “test1” which was created using the TCPS protocol (i.

In this post, I’d like to talk about optimisations that I recently used to provide a Scala Microservice that surfaced payment events. The events were held in an Oracle backend and the Microservice was in a docker container with (1 vCPU) allocated 512MB to the container and 256MB to the JVM that was running it. In this writing I’m not going to talk about the Oracle optimisations to make the underlying query fly but rather would like to concentrate on the kinds of things that can make service code quick.

I was really positively and pleasantly surprised when I found out what HMRC Digital’s mission statement was. Normally, I am not a fan of mission statements. They usually read like this: Our mission is focused on six core aspirations the company continually strives to achieve… Or some such drivel. Verbal gymnastics to make a company sound like everything to everyone - without being offensive to anyone - which then gets used to “align” people on mandated fun days.

This post peels back the covers on what it is like to work with a large digital platform. The platform in question is MDTP - Multichannel Digital Tax Platform, which supports a UK-based tax collection agency which is using a hyperscale cloud provider with a sideline in books. I’ve previously described what it is like to work in MDTP (Making Software. Quickly) during the Covid-19 responses that allowed the UK government to provide financial support for millions turning around projects in record time.

In this post, I’d like to talk about optimisations that I recently used to provide a Scala Microservice that surfaced payment events. The events were held in an Oracle backend and the Microservice was in a docker container with (1 vCPU) allocated 512MB to the container and 256MB to the JVM that was running it. In this writing I’m not going to talk about the Oracle optimisations to make the underlying query fly but rather would like to concentrate on the kinds of things that can make service code quick.

There’s a metaphor about the fight between attackers and defenders in the Denial of Service cybersecurity game. It’s an “arms race” between ever bigger attacks throwing huge amounts of traffic at ever more sophisticated defenses (e.g. AWS shield). Incidentally, I’ve just demonstrated an easy mistake: I’m not describing a Denial of Service (DoS) attack, it’s a Distributed Denial of Service (DDoS) attack. The aim is to overwhelm the infrastructure, either the networking infrastructure or the application by sending more requests than can be handled.

Scenario: You’re in a handover session and explain everything in great detail but Dave is too hungover, Jim is on holiday and Jane is at another meeting and Chris isn’t asking any questions and in any case Trevor (who is actually going to be looking after this system after handover) hasn’t been recruited yet. Wouldn’t it be useful to record the session and make it available? Now, this piece of writing does not explore whether we should be recording meetings, nor does it attempt to answer the question of whether good documentation wouldn’t be preferable to having to sit and spend hours watching someone droning on about something that could be condensed into a five minute read of a blog post.

In this post, I’d like to talk about optimisations that I recently used to provide a Scala Microservice that surfaced payment events. The events were held in an Oracle backend and the Microservice was in a docker container with (1 vCPU) allocated 512MB to the container and 256MB to the JVM that was running it. In this writing I’m not going to talk about the Oracle optimisations to make the underlying query fly but rather would like to concentrate on the kinds of things that can make service code quick.

The question of automatic dependency updates came up in our Slack channel the other day. There was a lot of nodding on how it is a good thing. Tools like Dependabot and Renovate were mentioned. Yet I was a dissenting voice. Why? The case for automatic dependency updates is simple and seductive: A bot would automatically scan your dependencies in your source code and create pull requests to update your libraries to the latest versions, sometimes even automatically merging it in.

From a security point of view, application logs are two-sided. On the one hand, it is really important to have good observability, to find out what is happening and what has happened. On the other hand, we don’t want to leak sensitive information. In this post I am going to look at the kinds of things you might find in your logs. The juicy bits are Personal Identifying Information (PII) or security credentials.

As I stare at my laptop after an intense few days at 44CON, I reflect on the experience. I went through a lot of different emotions. Excitement, trepidation, relief, bewilderment, pride, disappointment and hope had accompanied my visit to London to 44CON. We had an OSINT Capture The Flag competition happen right next to a round-table discussion on how the government should secure the country. Looking from outside in, a rallying cry of “Hack the Planet” (if you want to feel old, it turned 28 during the con) side-by-side with the establishment might not be the first thing to expect at a security conference.

It is a couple of weeks after my trip to Leeds to go speak at BSides. Now, if you’ve never been to a BSides - and I had only been to one prior - then hopefully this gives you a bit of a feel as to what to expect. In short, there’s lots of interesting people with great talks and insights and something that feels just right. A bit of community and some weird people.

As I am sitting at Schipol Airport, contemplating that the airport is so big it has a branch of the Rijksmuseum, I can’t help but thinking about the fabulous conference I’ve just been to. The DevOps Enterprise Summit 2023 has exceeded my expectations. But before I get to there, I’d just like to develop the airport metaphor a little bit. So here I am, looking at culture. Why was it that at Schipol, one of the busiest airports in the world, it was a breeze to get through security, airport security agents were smiling, cracking jokes, I could leave my laptop, toothpaste and deodorant in the bag and they have art in the middle of the airport, goddammit.

As I am sitting on the 16th floor of the Okura Hotel in Amsterdam in my Batman pyjamas and facing a large mirror, just glimpsing the paper crane that was so lovingly put on my pillow, I’m starting to reflect what has happened during the day. I was lucky enough to be a guest at the DevOps Enterprise Summit. There was certainly plenty of things to get excited about. I’d meet Gene Kim, who’s been running this conference for 10 years and who wrote the Phoenix Project.

The question of automatic dependency updates came up in our Slack channel the other day. There was a lot of nodding on how it is a good thing. Tools like Dependabot and Renovate were mentioned. Yet I was a dissenting voice. Why? The case for automatic dependency updates is simple and seductive: A bot would automatically scan your dependencies in your source code and create pull requests to update your libraries to the latest versions, sometimes even automatically merging it in.

From a security point of view, application logs are two-sided. On the one hand, it is really important to have good observability, to find out what is happening and what has happened. On the other hand, we don’t want to leak sensitive information. In this post I am going to look at the kinds of things you might find in your logs. The juicy bits are Personal Identifying Information (PII) or security credentials.

Let me tell you a story about Application Security (AppSec). It contains heroes and villains, and I’m not necessarily thinking about the defenders and attackers here. It contains lots of interesting technology that is often overemphasised. We’ve got whole industries that work on letting us know how scary it is out there, vulnerabilities that are marketed like rock stars and terminology that makes you quiver in your boots: who would want to fall victim to an Advanced Persistent Threat (APT)?

There’s a metaphor about the fight between attackers and defenders in the Denial of Service cybersecurity game. It’s an “arms race” between ever bigger attacks throwing huge amounts of traffic at ever more sophisticated defenses (e.g. AWS shield). Incidentally, I’ve just demonstrated an easy mistake: I’m not describing a Denial of Service (DoS) attack, it’s a Distributed Denial of Service (DDoS) attack. The aim is to overwhelm the infrastructure, either the networking infrastructure or the application by sending more requests than can be handled.

In this post, I am going to look at an increasingly important part of securing applications: Your supply chain. This includes every library, tool or service that you are using to build, run and monitor your service. When the log4shell vulnerability hit, it wasn’t just a matter of looking at the dependencies that your source code pulls in, but also at the infrastructure you’re using and the build pipeline. Have you had a look at the vulnerability reports of your dependencies lately?

First of all I need the preface this article on how much I abhor the Russian invasion of Ukraine and I wholeheartedly support the sanctions. However, I think the conflict has spilled over into areas of software development that have got some unintended consequences attached. As part of this post, I’m going to look at the decision by MongoDB to cut off services in Russia the destructive change in a node library that deleted files on Russian IPs a change in code/licence in a community terraform module to assert that Putin is a dickhead MongoDB cutting off Russian customers MongoDB is a company and in order to comply with sanctions they have decided to cut off Russian customers.

Pwnkit is a vulnerability that uses a bug in polkit to elevate permissions to root. This write-up shows how to reproduce it using Ubuntu and what to do to check whether a system is vulnerable. What went wrong? Quoting from the original researchers: This vulnerability is an attacker’s dream come true: pkexec is installed by default on all major Linux distributions; pkexec is vulnerable since its creation, in May 2009; any unprivileged local user can exploit this vulnerability to obtain full root privileges; although this vulnerability is technically a memory corruption, it is exploitable instantly, reliably, in an architecture-independent way; and it is exploitable even if the polkit daemon itself is not running.

In this blog post, I would like to explore how missing input validation even in a trivial service can leave parts of server infrastructure crumbling. In my opinion, this why securing applications (AppSec) is very difficult. Put supply chain attacks, unpatched systems and misconfiguring services to one side for a minute and consider that a lot of software is written by developers who do not necessarily consider all the edge cases or implications of what can go wrong even in the simplest of pieces of code (or just copy/paste from Stackoverflow).

More and more I’m thinking that XML is evil! This is the third part of my series on why, as a software engineer, it is very useful to think about the potentially dangerous combination of outdated libraries and XML. I recently carried out a review of the dependency scanning results CVE-2012-0881: Apache Xerces2 Java allows remote attackers to cause a denial of service (CPU consumption) via a crafted message to an XML service, which triggers hash table collisions.

Security specialists. Arghhh! They’re all sitting in their ‘ivory tower’ without anything better to do than to take a baseball bat to your hard work and tell you how you’ve not considered some obscure vulnerability CVE-142341231/4234 in a library that you didn’t even know existed. Not only that, there is definitely no way that you can deploy now, even when the product owner is breathing down your neck saying that nobody is going home until we’ve fixed this!

Recently I tried to poke holes in a service. I found myself laughing out loud. This was a vulnerability whereby modifying a SAML authentication while being rePOSTed via the browser allowed me to inject a malicious payload (see XML External Entity (XXE) Processing and XML External Entity (XXE) Prevention Cheat Sheet) that could be used to use up a service’s memory and CPU. Health checks and automatic service restarts would have healed the service but it still would have allowed an attacker to mount a Denial of Service attack without needing a lot of requests.

Background I had found a vulnerability that made it is possible to insert maliciously crafted XML into the SAML payload that a reauthentication application returned to perform a Denial of Service (DoS) attack. The vulnerability came about due to the use of a out-of-date but still widely used library. The service could have been made to consume a lot of CPU and memory causing it respond very slowly if at all.

In this post, I am going to look at an increasingly important part of securing applications: Your supply chain. This includes every library, tool or service that you are using to build, run and monitor your service. When the log4shell vulnerability hit, it wasn’t just a matter of looking at the dependencies that your source code pulls in, but also at the infrastructure you’re using and the build pipeline. Have you had a look at the vulnerability reports of your dependencies lately?

This post peels back the covers on what it is like to work with a large digital platform. The platform in question is MDTP - Multichannel Digital Tax Platform, which supports a UK-based tax collection agency which is using a hyperscale cloud provider with a sideline in books. I’ve previously described what it is like to work in MDTP (Making Software. Quickly) during the Covid-19 responses that allowed the UK government to provide financial support for millions turning around projects in record time.

Pwnkit is a vulnerability that uses a bug in polkit to elevate permissions to root. This write-up shows how to reproduce it using Ubuntu and what to do to check whether a system is vulnerable. What went wrong? Quoting from the original researchers: This vulnerability is an attacker’s dream come true: pkexec is installed by default on all major Linux distributions; pkexec is vulnerable since its creation, in May 2009; any unprivileged local user can exploit this vulnerability to obtain full root privileges; although this vulnerability is technically a memory corruption, it is exploitable instantly, reliably, in an architecture-independent way; and it is exploitable even if the polkit daemon itself is not running.

In this writing I ask: When will this madness end? When will we wake up from this fascination with cryptocurrencies (chief amongst which Bitcoin) that is turning technological orthodoxy on it’s head: It used to be that technological progress was measured on making things cheaper, faster, more secure or more scalable. But it seems that Bitcoin does not follow these. Bitcoin Over the last 12 years, it is hard to have lived and not notice the poster child of blockchains: Bitcoin.

More and more I’m thinking that XML is evil! This is the third part of my series on why, as a software engineer, it is very useful to think about the potentially dangerous combination of outdated libraries and XML. I recently carried out a review of the dependency scanning results CVE-2012-0881: Apache Xerces2 Java allows remote attackers to cause a denial of service (CPU consumption) via a crafted message to an XML service, which triggers hash table collisions.

Recently I tried to poke holes in a service. I found myself laughing out loud. This was a vulnerability whereby modifying a SAML authentication while being rePOSTed via the browser allowed me to inject a malicious payload (see XML External Entity (XXE) Processing and XML External Entity (XXE) Prevention Cheat Sheet) that could be used to use up a service’s memory and CPU. Health checks and automatic service restarts would have healed the service but it still would have allowed an attacker to mount a Denial of Service attack without needing a lot of requests.

Background I had found a vulnerability that made it is possible to insert maliciously crafted XML into the SAML payload that a reauthentication application returned to perform a Denial of Service (DoS) attack. The vulnerability came about due to the use of a out-of-date but still widely used library. The service could have been made to consume a lot of CPU and memory causing it respond very slowly if at all.

“Outdated tech stack and metaphorical gaffer tape holding together the code” - sound familiar? How about “We can’t recruit, because nobody wants to touch this legacy stuff”? Yet banks and governments would stop working if mainframes were switched off. Instead of outsourcing to the lowest bidder, maintenance is a job for experienced engineers. Not “the short straw”, brown field development can be more exciting than a feature factory. After all, the shiny code written today is the legacy code of tomorrow…