Kube-Oddities

This blog post comes from a talk me and Márk Sági-Kazár have given a few times now called Kube-Oddities: The Quirks That Keep Kubernetes Interesting. That talk itself was inspired by a previous talk I had given called Pod Deep Dive: The Interesting Bits. In this post I will go through all the fun, weird and quirky things we found out while putting together both these talk.

Over our combined many years of working with Kubernetes, me and Márk have seen some weird things. Many resulting in us scratching our head and going "WTF?". Regardless of if you're brand new to the cloud native world or you're a Kubernetes veteran I hope you'll find something interesting in this post - or at the very least a shared frustration! 😁

Continue reading...

KubeCon EU Amsterdam 2026 - My Guide to Managing the Chaos

With KubeCon just 1 week away I thought it would be good to share some of my tips and suggestions on how to make the most of the week without burning out. I've been to a few KubeCon's now, both EU and US, and I’ve learned a lot about how to make the most of the week and what not to do in order to protect your health!

Continue reading...

Investigating and fixing "StopPodSandbox from runtime service failed" Kubelet errors

For months now (maybe years, who knows 🤷) I've had the following error sprinkled throughout my Kubelet logs across multiple of my worker nodes:

StopPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed to destroy network for sandbox "055b221e44a28ce8d9120f771d5e1ef201f2457ce49c58999a0147104cca2708": failed to get network "cbr0" cached result: decoding version from network config: unexpected end of JSON input

As there has never been any associated impact with this that I could see I just ignored it... for a long time!

Well, fast forward to last Friday night where a sudden hyper-focus hit and I decided I wanted to know what was going on and how to stop it - here's how it went.

Continue reading...

My tips on giving technical talks

I've been giving talks at meetups and conferences for a few years now. I started off after the encouragement of my friends giving their own talk and looking so cool doing it! It's taken a while but I think I'm at a stage now where I'm not only good at it (at least I hope so 😅) but I feel confident and comfortable while doing it. I want everyone to have that same confidence and I want to hear ALL OF YOU giving talks too! You have stories to tell, lessons to share and experience to pass on. So here is my learnings on how I approach giving a talk in front of a crowd of techies, mainly focussed on technical talks but most of this should apply to most public speaking.

Continue reading...

Building Social Media Bots With Node-RED

I run several bots that post to social media and the number seems to keep growing. I've had a few people in the past ask me how I run them so I thought I'd finally get around to writing down how I manage it by leveraging Node-RED.

Continue reading...

Posting to LinkedIn via the API

I have a habit of automating as much of my life as I can. As part of that I have some small automations that handle posting to social media automatically when I publish a new blog post. (More on this in a future post). In the past this was Twitter but that is dead so these days I post to my Mastodon and Bluesky accounts automatically whenever a new blog post is available on my website. Maybe that's how you ended up here today! One platform I had been avoiding for a long time was LinkedIn. I had the impression that they didn't have a free, personal API available for users to use to post status updates. Well, turns out I was mistaken and it's actually not too difficult to setup, there's just a LOT of outdated information out there. So lets fix this...

Continue reading...

Announcing Cloud Native Now

Last week I announced the launch of a new project I've been working on - Cloud Native Now - a new monthly newsletter that will provide a roundup of all the happenings in the cloud native world. This newsletter is my attempt at keeping myself, and others, up-to-date on all the latest news, tools and events happening in the cloud native world. A new issue will be published each month on the last Friday of that month and contain a roundup of articles, announcements, tools, tutorials, events and CFPs relating to cloud native technologies and the community.

Continue reading...

Bootstrapping a Civo cluster with OpenTofu and Flux

As part of a side project I'm currently working on I needed to spin up a new Kubernetes cluster that I could manage via GitOps. I decided to take this opportunity to take a look at OpenTofu and see how it handles as it's been several years now since I last used Terraform. My plan was to use OpenTofu to scaffold a fairly basic Civo Kubernetes cluster and then use Flux to handle installing workloads into the cluster. It took me a little trial-and-error so I thought I'd write up my final setup to help others avoid the issues and to help my future self when I come to do this again in a couple years!

Continue reading...

My Recommended Kubernetes Resources for Newbies

Recently, a friend of mine asked me what resources I'd recommend to start learning about Kubernetes. He was a victim of the layoffs that seem to be so prevalent right now and has experience as a classic SysOps / SysAdmin engineer but no expose to Kubernetes yet and wanted to learn to help improve his job-hunting prospects.

I wasn't sure what to recommend at first, it's been a long time since I was learning Kubernetes for the first time and wasn't sure what was still useful and relevant but what follows is what I ended up sharing with him, and now with all of you.

Continue reading...

My First EMF Camp

Last weekend I finally went to my first ever EMF Camp after wanting to go for 4+ years now. I had a thoroughly good time while I was there and wanted to share my experience to encourage others to go in the future.

Continue reading...

Custom Renovate datasource

I'm quite a fan of Renovate for automating my dependancy updates. I've been using it quite extensively - at Giant Swarm, with my personal infrastructure and with k8s.social - to keep things up-to-date in the various Kubernetes clusters I manage. This has been working great for container images and Helm charts, with both being managed via GitOps and Renovate automating the version updates via PRs, but I have now found myself wanting to automate the updating of the Kubernetes version for my Civo cluster. Renovate doesn't have built in support for this but we can make it work with the use of custom datasources!

Continue reading...

Migrating Redis data within Kubernetes

I’ve recently been seeing some stability issues with Redis that I have running for the k8s.social Mastodon instances. After looking into it I realised that I had it configured in a master/replica architecture but I wasn’t actually making any use of the replicas as Mastodon was configured to do everything via the master. There’s two things wrong with this - firstly I’m wasting resources by having the replicas running but more importantly I created a single point of failure. When the Redis master went down, so did Mastodon.

Not good!

Continue reading...

Managing Kubernetes without losing your cool

This post is based on a webinar i've previously given where I go through some of my favourite tips for working with Kubernetes clusters all day long. The goal of all of these techniques is to make my life easier and (hopefully) less error prone. I start off with the first 5 tips being applicable to anyone working with Kubernetes and can be picked up right away. From there I move on to a couple that would benefit from having some old-skool Linux sys-admin experience. Finally I finish of with some more advanced techniques that require some previous programming experience.

Continue reading...

Restricting cluster-admin Permissions

Generally, and by default, operators of the cluster are assigned to the cluster-admin ClusterRole. This gives the user access and permission to do all operations on all resources in the cluster. There's very good reason for this, an admin generally needs to be able to setup and manage the cluster, including the ability to define and assign roles. But what if we need to block an action performed by cluster admins? We can't do it with RBAC, it only allows for adding of permissions, not taking them away.

Continue reading...

My Recommended Go Resources

I was recently asked by a collegue at work if there are any resource I recommend with getting familiar with Go. It turned into quite a list so I thought i'd share it with everyone (and hopefully keep updating it) in the hopes that others will find it useful.

Continue reading...

Migrating from Docker to Podman

Docker has recently announced that Docker Desktop will soon require a subscription and, based on the size of your company, may require a paid subscription. (It remains free for personal use).
There has been quite a bit of reaction to this news...

Continue reading...

Job hunting / hiring during a pandemic

I’ve recently accepted a new job offer and wanted to share a little about my experience job hunting during a global pandemic. Even after more than a year it was clear that some companies still hadn’t adjusted or taken the time to stop and look at the approach they were taking and the effect it had on the interview candidate. Hopefully some of this will be of use to someone looking to hire during similar situations.
Most of what’s to come is aimed at those doing the hiring rather than the hunting but I think it could be of benefit for both sides to be aware of what might come up.

Continue reading...

Multicloud Kubernetes

I've been using Scaleway's Kapsule managed Kubernetes offering for my personal projects for a while now (this blog is running on it) so I was pretty excited when they announced a new managed Kubernetes offering dubbed Kosmos. What makes Kosmos really interesting is it's sold as a multi-cloud Kubernetes offering.

Continue reading...

T.I.L. CLI flag handling in Bash using getopts

Today I Learnt: CLI flag handling in Bash using getopts

Continue reading...

T.I.L. YAML keys allow for spaces in them

Today I Learnt: YAML keys allow for spaces in them

Continue reading...

T.I.L. Kubernetes label length

Today I Learnt: Kubernetes label length

Continue reading...

T.I.L. How to get the favicon of any site

Today I Learnt: How to get the favicon of any site

Continue reading...

Moving To iOS

Ever since the first batch of Android phones were released in the UK I have been a fan. My first, the HTC Dream (also known as the T-Mobile G1), is still my all time favourite phone. The slide out physical keyboard was fantastic. I’ve had a whole range of different Android-based phones both high-end and low-end, some with quirks, some fairly standard, most rooted and tweaked to their limits.

But, in recent years, my enthusiasm for both Android and Google in general has waned. I have been slowly moving away from various Google services over the past couple years. Google search replace by DuckDuckGo, Gmail replaced with ProtonMail, Drive replaced with a self-hosted Nextcloud, etc. while also getting frustrated with the way most Android devices get abandoned soon after launch.

When the new iPhone 12 was released I decided to give it a try (the blue does look very pretty). Rather than just settling for the new iPhone 12 I decided to also pick up an iPad Air for the family (what I’m currently writing this post on).

It’s been at least a couple weeks now since I made the switch so I thought I’d collect some of my thoughts on how I’m finding the change so that maybe it’d be of use to others.

Continue reading...

T.I.L. Golang's append mutates the provided array

Today I Learnt: Golang's append mutates the provided array

Continue reading...

T.I.L. Named returns in Go functions

Today I Learnt: Named returns in Go functions

Continue reading...