Projects on Nicole van der Hoeven

Spice Runner

Wed, 05 Nov 2025 00:00:00 +0000

Spice Runner is a browser-based endless runner. Think of the “unable to connect” dinosaur game in Chrome, except you are a Fremen running across the sands of Arrakis, jumping sandworms and Harkonnen ornithopters. It’s the demo app for my talk The Spice Must Flow: The Fremen Guide to Sustainable Observability, and it exists to make one point: even a simple, throwaway game generates a surprising amount of telemetry, and that telemetry has real financial and environmental costs.

The Fremen live on a planet that is rich in spice and poor in water, and they thrive by wasting nothing. Our systems are rich in telemetry and poor in budget, and in carbon. Spice Runner is a small, playable version of that problem: a real app, with real players, running on real infrastructure that scales up and down with them while I watch what it costs.

What it demonstrates

Real user monitoring with Grafana Faro. Game sessions, game starts, scores, collisions, restarts, and game overs are all sent as custom events, so the dashboard shows how people actually play.
A leaderboard that’s instrumented end to end. A Go API backed by PostgreSQL and Redis serves a public top-10 leaderboard. It’s instrumented with OpenTelemetry for traces and custom metrics, so a score can be followed from the browser through the API to the database and the cache.
Metrics, logs, and traces on Kubernetes. Prometheus, Grafana Loki, and Grafana Tempo run in the cluster, with Grafana Alloy as the collector and Grafana for dashboards. Telemetry is also sent to Grafana Cloud.
Autoscaling that follows the players. KEDA scales the game’s pods on HTTP request rate, CPU, and memory, and the GKE Cluster Autoscaler adds and removes nodes behind them. Production settings keep at least one pod running and cap it at 20, with a separate demo profile that is far more eager for live demonstrations.
Energy monitoring with Kepler. Power and energy use are tracked per workload, so “what does this cost the planet?” is a query rather than a guess.
Load testing with k6. k6 drives load against the leaderboard API and, with a browser script, plays the game to keep the dashboard populated.

The dashboard

The live dashboard on Grafana Play shows real player data as it happens. The dashboard is also entirely AI-generated: it was built with Grafana Assistant.

Open the live dashboard

Architecture

The app has two layers. The first is the game and its leaderboard: a vanilla JavaScript game served by Nginx, with a Go leaderboard API backed by PostgreSQL and Redis. The second is the observability stack: Faro instruments the browser, and the Nginx exporter, Alloy, Loki, Tempo, and Prometheus cover the backend. Everything runs on Google Kubernetes Engine, with persistent volumes so telemetry survives pod restarts, and autoscalers at both the pod and node level.

The talk

The Spice Must Flow uses Dune as its framing. It covers the hidden environmental and financial costs of an observability stack, how hard it is to measure the cost of ephemeral resources on Kubernetes, how to use KEDA to right-size resources when you don’t know what the right size is, and the most effective thing you can do to cut cost: deleting the telemetry you don’t need.

I first gave it at Øredev 2025 in Malmö, Sweden. You can read the write-up and abstract and see the slides.

Run it yourself

Everything you need to deploy it on your own GKE cluster, including the Kubernetes manifests, the dashboards, and the load tests, is in the GitHub repository. Its README has a quick start.

The Defiant Jazz Incident

Fri, 30 Oct 2026 00:00:00 +0000

The Defiant Jazz Incident is a murder mystery you solve with observability tools. It’s set on the Sleep Floor at Kipsake, a company that rents your sleep: at night, its Dreamers sleep in pods while their sleepwalking selves work the floor. During the Cocoa Hour, the floor’s middle-of-the-night holiday party, the power goes out for five minutes. When it comes back, the night supervisor is dead in the elevator.

Kipsake recorded everything it could: badges, doors, chat, sleep rings, power, microphones, cameras, sleep bands, and personnel records. Your job is to work out who did it, and the only way to do that is through the telemetry.

How it works

The case file is a set of logs, metrics, and traces. A loader sends them to Grafana Loki, Prometheus or Mimir, and Grafana Tempo, either in Grafana Cloud or in a local Docker stack, and you investigate from there with whatever you like: Grafana itself, Grafana Assistant, an MCP server, gcx, or the AI agent of your choice. The data is deliberately messy. Some devices lose clock sync during the outage, some records arrive late, and not everyone on the Sleep Floor remembers what their body did.

The game itself comes in two forms. A terminal version is for demos and playtests: you watch the testimonies and choose what to investigate, while an AI that only sees the telemetry answers your questions. A web version is made to embed in a Grafana dashboard panel right next to the data.

Why a murder mystery?

Investigating an incident is investigating an incident, whether the victim is a production service or a night supervisor. You start with a vague report, a pile of signals that don’t quite agree, and a clock that can’t be fully trusted, and you have to build a timeline you can defend. A case with a known answer is a safe way to practice that, to compare how different tools and AI agents cope with it, and to see how good a question you have to ask before the data answers.

No spoilers

The repository keeps the solution in clearly marked files, and the README tells you which ones to avoid if you want to play. This page deliberately says nothing about who did it.

Try it

The repository has everything: the case brief, the telemetry, the loader, and both versions of the game.

obsidian-okrs

Wed, 12 Jan 2022 00:00:00 +0000

obsidian-okrs is the set of templates I use to set up OKRs (Objectives and Key Results) and goals in Obsidian, as described in the video I made about them (and its blog post).

New Year’s resolutions are infamously left undone. Setting OKRs instead, and reviewing them as you go, gives you a better chance of following through, and keeping them in Markdown notes means the goals live right next to the work they describe.

The templates

The templates are organized into five levels, each one feeding the next:

Core Principles: the values everything else hangs off.
Yearly: a Yearly OKRs note, a template for each individual objective, and a Yearly Review.
Monthly: initiatives, in the form of a monthly review.
Weekly: weekly goals and a weekly review.
Daily: a daily note template.

What you need

The templates use Obsidian’s core Templates plugin plus the community plugins Admonition and Templater. QuickAdd, Periodic Notes, Calendar, and the core Daily Notes plugin are optional extras that make creating the periodic notes smoother.

Try it

Copy the templates from the repository into your vault and follow along with the video.

obsidian-grafana

Mon, 13 Oct 2025 00:00:00 +0000

obsidian-grafana turns an Obsidian vault into time-series data and dashboards. I work at Grafana Labs, helping people monitor their systems with open source tools, and at some point I realized I could use the same tools to monitor my personal knowledge management system. It’s a nerdy and totally unnecessary setup, and I enjoy it a lot. I mention it in my post on how I use visuals in my PKM.

The screenshot above was made by running the repo’s own Grafana setup on a copy of the posts from this website.

How it works

A Python script reads the Markdown files in your vault and extracts metadata: word, line, and character counts, file size, frontmatter fields, tags, wikilinks, and timestamps. It runs every five minutes and has two paths:

A log path for history. Only the notes that changed since the last run are written, as structured JSON, to a log file. Grafana Alloy tails that file and forwards it to Loki, with tags and frontmatter fields as labels, so you can see how your notes change over time.
A metrics path for the current state. A metrics exporter exposes totals such as note count, word count, unique tags, and wikilinks, and Prometheus scrapes them.

Grafana ties the two together with pre-configured dashboards: an overview, a time series dashboard for individual notes, a relationships dashboard for wikilinks, a dashboard of frontmatter types, and a word cloud of your most used words. Everything runs in Docker Compose.

What it’s useful for

Understanding your writing habits and how your vault grows, checking how consistently your notes are tagged, finding the notes that everything links to, and keeping an eye on notes with a given frontmatter status or priority. The README also covers installing the Grafana Assistant plugin into the local Grafana.

Try it

The repository has a quick start. You’ll need Docker and Python 3, and a vault to point it at.

Asimov

Wed, 02 Apr 2025 00:00:00 +0000

Asimov is the companion project to my talk Asimov’s Zeroth Law of Robotics. It’s an AI app built to be observed and tested: The Silent Enterprise, a single-player Star Trek text adventure. You play Data, and Claude is the game master. Every model call, dice roll, and turn shows up as telemetry in a Grafana that you can run on your own machine or in Grafana Cloud, and a k6 test suite, some of it judged by Claude, checks whether the AI follows the rules.

The game master has some known flaws, kept on purpose. It sometimes narrates a rescue that didn’t happen, or “destroys” a drone that’s still standing. The point of the demo is catching those flaws with telemetry, tests, and online evaluators.

The idea

Isaac Asimov’s three Laws of Robotics govern how robots behave. The talk argues for a law that comes before all of them: a robot must be observable, and, in its newest version, testable. AI systems fail differently from traditional software. There’s no single correct output to assert against, a response can be fluent and completely wrong, and models drift. You can’t test what you can’t see, and you can’t trust what you can’t test.

What’s in it

The game. A Go program with a command-line interface and an HTTP API. The game engine owns the rules and the state. Claude interprets the player’s input, makes the game master’s own rolls through a tool, and narrates, so a narration that contradicts the saved state is a defect that can be caught.
Telemetry. Traces, metrics, and logs go over OpenTelemetry to a local Grafana stack (Tempo, Prometheus, and Loki, with the game’s dashboard already loaded) or to Grafana Cloud. Generations can also go to Grafana Cloud’s Agent Observability, where online evaluators judge a sample of them.
Tests. A k6 suite with code checks, LLM judges, dice-trajectory evals, and whole playthroughs. All of it can run for free against a mock game master and judge.
No account needed to start. You can play offline with no API key, and see the local monitoring stack with one command and Docker.

The screenshot above is the game’s local Grafana dashboard.

The talks

I’ve given the talk, in several versions, at KubeCon Europe 2025, Dutch Cloud Native Day 2025, Newcrafts 2025, ExpoQA Madrid 2026, and HUSTEF 2026. The write-ups are linked below. The earlier talks used a Python and Flask version of the app, which has since been rewritten in Go.

Try it

The repository has a quick start for playing offline, a Docker-only path, and release binaries for Windows, macOS, and Linux.