Presentation on Nicole van der Hoeven

Asimov's Zeroth Law of Robotics: Testing for AI (HUSTEF 2026)

Wed, 07 Oct 2026 10:00:00 +0200

This week I gave a keynote at HUSTEF 2026 in Budapest, Hungary. The talk is called Asimov’s Zeroth Law of Robotics: Testing for AI — a testing-focused evolution of a talk I’ve been giving (and rewriting) for a while, previously as Observability for AI at KubeCon EU 2025 in London, Dutch Cloud Native Day 2025 in Utrecht, NewCrafts in Paris, and ExpoQA Madrid 2026. Here’s the abstract:

A robot may not harm humans. A robot must obey humans. A robot must protect its own existence. These are Isaac Asimov’s three Laws of Robotics, created to govern the ethical programming of artificial intelligences. From the Butlerian Jihad to Skynet to cylons, we’ve been immortalizing our collective nightmares about artificial intelligence for years. But there’s an unmentioned law that comes as a prerequisite to all of that: a robot must be testable.

AI systems fail differently than traditional software. There’s no single correct output to assert against. A response can be fluent and completely wrong. Models drift silently over time. And yet we’re shipping AI into production — often without knowing what “passing” even looks like.

Observability is what makes testing possible. When we can see inside our AI systems — trace the chain of reasoning, measure what changed, surface what failed — we can start to write evaluations, define what “good” looks like, and catch regressions we didn’t know to look for.

In this talk, I explore what it means to test AI in the real world, drawing on lessons from building and evaluating an AI assistant for a large-scale observability platform. I’ll cover how evaluations work as the AI equivalent of a test suite, what to measure, how to catch hallucinations, and how to know when your model has regressed.

You cannot test what you cannot see — and you cannot trust what you cannot test.

The big shift in this version is right there in the Zeroth Law itself. In earlier iterations I argued that a robot must be observable. That’s still true — observability is the only way to see the reasoning behind a system and find out what actually happened. But I’ve come to think that pure knowledge isn’t the same as trust. You can know exactly what a system did and still not trust it. The only way to trust something is to put it through its paces: to know how it behaves in the situations you foresee, and even the ones you don’t. So the Zeroth Law has become a robot must be testable.

I framed the whole thing around Asimov’s short story Runaround (from I, Robot) and its robot, Speedy. Speedy’s smuggler owners sent him to collect selenium on Mercury, and he got stuck in a loop — circling the selenium pool, unable to resolve the conflict between obedience (the second law) and self-preservation (the third). His owners couldn’t predict his behaviour and couldn’t reason about it, so they had to do the only thing left: go out into the danger zone and fix him in production. Speedy wasn’t untestable because his owners were incompetent or lacked the tools — he was untestable because he was never built to be tested. That’s the trap I think a lot of AI systems are walking into right now.

The through-line for a testing audience: AI is rewriting how software gets built, but it doesn’t rewrite the discipline of trust. Trust that a system does what we say it does, and that when it breaks we know why and can fix it — that’s always been the job, and it isn’t going anywhere. There’s a whole generation of AI engineers shipping software they haven’t tested and can’t explain, and when someone says “the agent does this,” our job is still to ask how do you know? Testing isn’t just going to survive the age of AI — testing is the reason AI gets to survive, if we make it testable.

I demoed this with my TTRPG side project: an AI Dungeon Master, instrumented and put under test. It turns out that writing evals for something non-deterministic forces you to confront your real requirements. I thought I knew exactly when a dungeon master should call for a dice roll — I GM myself! — but trying to put it into words (“in this situation I’d roll, but in that one I wouldn’t, because I want to keep the story moving”) surfaced decisions I’d never had to make explicit. Those are things we’d normally discover much later, in production. Having to figure them out up front is a win.

The Q&A was genuinely great and ranged well beyond the slides:

“When has an AI been tested enough?” Same answer as for anything else — it’s a mix of risk and budget. If it were a life-critical robot like Speedy, you test every scenario the people relying on it need. For my little demo app, less so.
Testers vs. developers in the age of AI. I said I’d fire developers before testers — provocatively, but the point stands: anyone can write code now. What made a good developer good was never the coding; it was communicating with business and engineers, translating requirements, finding creative-but-safe solutions. Testers already think at that level and translate across teams, which makes them more important in the age of AI, not less.
Defining expected results for non-deterministic systems, multi-turn evals (I give the simulated user a personality — adversarial, cooperative, beginner — rather than a script), and the broader shift from imperative to declarative: we no longer instruct step by step, we declare the end state we want, the way we do with Kubernetes.
Convincing colleagues not to fear AI. I don’t, really. AI is here to stay; the question isn’t whether you approve of it, it’s whether you want to stay in this industry and help make it safe and trustworthy — or sit on the sidelines.

I also recommended the book Slow AI, which lands where I do: not an AI apologist, not a boycott, but a clear-eyed list of what AI is and isn’t good for — and the reminder to keep checking whether a given use of AI is making you more capable or less.

The demo app (the instrumented AI DM) lives on GitHub, and I keep refining it with each iteration of this talk.

I’ll update this post with the recording once HUSTEF publishes it.

Resources

Slides
Demo app on GitHub
Slow AI (book)
Previous versions of this talk: KubeCon EU 2025 · Dutch Cloud Native Day 2025 · NewCrafts 2025 · ExpoQA Madrid 2026

The Great Doubt: What Building an AI Agent Taught Us About Trust (KubeCon Japan 2026)

Thu, 30 Jul 2026 09:00:00 +0900

This week I spoke at KubeCon + CloudNativeCon Japan 2026 in Yokohama, Japan. I gave a talk called The Great Doubt: What Building an AI Agent Taught Us About Trust. Here’s the abstract:

When we started building an AI assistant for observability, we thought the hard part would be making it smart. We were wrong. The hard part was knowing when to trust it.

AI agents hallucinate, forget context, and confidently give wrong answers. Traditional testing doesn’t catch these failures. What’s needed is evaluation grounded in systematic doubt.

This talk shares lessons from shipping an AI agent to production: how to build a golden dataset of use cases you must get right, how to use LLM-as-judge when there’s no ground truth, and how to use OpenTelemetry traces to debug eval failures. You’ll also hear what’s still unsolved: evaluating multi-agent handoffs and closing the feedback loop between what users ask and what your evals cover.

Kyoto School philosopher Nishitani Keiji called this “The Great Doubt” (大疑): questioning every assumption until only what survives is real. For AI agents, that’s not philosophy. It’s the job.

The talk walks through the layers of doubt we worked through while building Grafana Assistant: doubting the agent itself, then doubting our tests, then the LLM-as-judge, then our own scores — and finally doubting whether the doubt was even justified. Each layer needed a different tool, from a golden dataset run in a fully synthetic observability environment, through online evals on live traffic, to per-conversation OpenTelemetry traces that link every eval score back to the exact LLM and tool calls that produced it.

You can find the slides at nicole.to/doubtslides, and you can watch the full talk below:

Resources

Grafana Assistant (generally available for Grafana OSS, Grafana Cloud, and Grafana Play):

Assistant Investigations (Grafana Cloud):

Docs

Agent Observability (Grafana Cloud):

Docs

o11y-bench (OSS):

Site

Investigations Arena:

Video

My notes on Grafana Assistant
My Context Horizon piece Unlearnings from building Grafana Assistant
The Way of Zen by Alan W. Watts
Zen Mind, Beginner’s Mind by Shunryu Suzuki
Religion and Nothingness, Volume 1 by Nishitani Keiji
Philosophers of Nothingness: An Essay on the Kyoto School by James W. Heisig

Asimov's Zeroth Law of Robotics: Observability for AI (ExpoQA Madrid 2026)

Tue, 26 May 2026 10:00:00 +0200

This week I gave the opening keynote at ExpoQA Madrid 2026 in Madrid, Spain. I presented a talk called Asimov’s Zeroth Law of Robotics: Observability for AI, which I had previously given at KubeCon EU 2025 in London, at Dutch Cloud Native Day 2025 in Utrecht, and at Newcrafts in Paris. Here’s the abstract:

A robot may not harm humans. A robot must obey humans. A robot must protect its own existence. These are Isaac Asimov’s three Laws of Robotics, created to govern the ethical programming of artificial intelligences. From the Butlerian Jihad to Skynet to cylons, we’ve been immortalizing our collective nightmares about artificial intelligence for years. But there’s an unmentioned law that comes as a prerequisite to all of that: a robot must be observable.

In this talk, I discuss the different types of AI, the factors that make observing AI different from observing applications, and the telemetry signals specific to AI that we might want to listen to. How do we deal with large data sets? How do we observe for model drift? How do we take into account the costs of LLMs? How can we use distributed tracing to follow event sequences? Part cautionary tale and part technical demo, this talk shows how to instrument and monitor AI apps using OpenTelemetry, Prometheus, OpenLit, and more.

This was a keynote slot at a testing-focused audience, which gave me the chance to lean more into the evals vs. observability framing — what evals can and can’t tell you about an AI system in production, and how distributed tracing fills the gap that benchmarks leave behind. The demo app (a Flask-based “play.py” instrumented with OpenTelemetry, flowing into Tempo for traces, Prometheus for metrics, and Loki for logs) is still on GitHub, and I’ve been refining it with each iteration.

Here’s the recording of the talk:

Resources

Demo app on GitHub

Asimov's Zeroth Law of Robotics: Observability for AI (Newcrafts 2025)

Thu, 06 Nov 2025 12:23:19 +0100

Today I presented at Newcrafts in Paris, France. I gave a talk I’ve given before, called Asimov’s Zeroth Law of Robotics: Observability for AI, which I had previously presented at KubeCon EU in London and at Dutch Cloud Native Day 2025 in Utrecht. I was happy to present it again, as it gave me the chance to further refine my demo application, which you can find on GitHub.

The main change I made was to use the “AI as judge” approach that I had previously only talked about in theory. In this version of the demo app, I use AI in both the game as well as in the testing of the game, showing how AI can be integrated into both the application and its observability/testing stack. It was really interesting to note that testing with AI ended up exposing more defects than my hardcoded test cases did, which goes to show the potential of AI-driven testing.

The Spice Must Flow: The Fremen Guide to Sustainable Observability (Øredev 2025)

Wed, 05 Nov 2025 03:07:38 +0100

Today I gave a talk at Øredev 2025 in Malmö, Sweden, titled +The Spice Must Flow: The Fremen Guide to Sustainable Observability"_. The presentation focused on how to implement sustainable observability practices using open-source tools like Grafana, Prometheus, Loki, Tempo, KEDA, and Kepler. Here’s the abstract:

Nobody wants to be wasteful. But how do you balance that with the need for enough data to fix things when they go wrong? After all, no matter how much it costs, there are SLOs to keep. The spice– the telemetry– must flow.

The Fremen are a people who live on the planet Arrakis, simultaneously rich in spice while poor in water. But the Fremen have managed to thrive, not just survive, on very little while continuing to produce spice. What would it take to apply this “desert planet thinking” to observability?

In this talk, you’ll learn about the hidden environmental and financial costs of your observability stack and how to reduce them. You’ll learn about the complications of measuring cost for ephemeral resources on Kubernetes, how to use KEDA to “right-size” resources when you don’t know what the right size is, and how to convince your teams to drop all but the most meaningful telemetry data. In the end, the spice must flow, but there must still be enough water to sustain all.

Here are some resources mentioned in the talk:

You can play the game Spice Runner here
The spice-runner repo containing everything you need to know about how to deploy it yourself
The slides
The Spice Runner dashboard for observing the application
More about Spice Runner: architecture, tech stack, and what it demonstrates

One Year Later: How I Use Visuals in My PKM (Sketch Your Mind Conference)

Mon, 13 Oct 2025 13:33:18 -0700

Today I gave a talk at my friend Zsolt Viczián’s Sketch Your Mind Conference about how I use visuals in my PKM, a follow-up to my video about Obsidian Excalidraw last year where I shared that I’d changed all my Obsidian templates to hybrid Obsidian Excalidraw notes:

In the talk, I shared how I’ve evolved my use of visuals in my PKM over the last year, including:

Referring to icon and logo libraries that dynamically compile all the atomic visuals I use in my notes
Adopting a visual Zettelkasten-like approach
Creating notes at conferences to track people I’ve met and talks I’ve attended
Using Obsidian Excalidraw to create compelling presentations in lieu of slides
Combining visuals with audio recordings to create literature notes of books I’m reading
Using a full observability pipeline to monitor and analyze the notes in my Obsidian vault

I was particularly excited about the last point. I work at Grafana Labs and help people monitor their computer systems using opensource software. I realized that I could use the same tools to monitor my PKM system, so I set up a pipeline where a Python script scrapes note metadata, and sends information to Loki and Prometheus via Alloy, and then I visualize it all in Grafana. It’s a pretty nerdy and totally unnecessary setup, but here’s the repo for anyone interested in replicating it.

Asimov's Zeroth Law of Robotics (Dutch Cloud Native Day 2025)

Thu, 03 Jul 2025 15:01:24 +0200

Today I presented at the Dutch Cloud Native Day 2025 in Utrecht, the Netherlands. I gave a talk titled Asimov’s Zeroth Law of Robotics: Observability for AI, which I had previously presented at KubeCon EU earlier on in the year, in London. I was very happy to be able to present it again, because it gave me the opportunity to improve the demo app I used, which you can find on GitHub.

Asimov's Zeroth Law of Robotics: Observability for AI (KubeCon EU 2025)

Wed, 02 Apr 2025 13:46:42 +0200

Today I gave a talk at KubeCon EU 2025 in London called “Asmiov’s Zeroth Law of Robotics: Observability for AI”. Here’s the abstract:

A robot may not harm humans. A robot must obey humans. A robot must protect its own existence. These are Isaac Asimov’s three Laws of Robotics, created to govern the ethical programming of artificial intelligences. From the Butlerian Jihad to Skynet to cylons, we’ve been immortalizing our collective nightmares about artificial intelligence for years. But there’s an unmentioned law that comes as a prerequisite to all of that: a robot must be observable.

In this talk, I discuss the different types of AI, the factors that make observing AI different from observing applications, and the telemetry signals specific to AI that we might want to listen to. How do we deal with large data sets? How do we observe for model drift? How do we take into account the costs of LLMs? How can we use distributed tracing to follow event sequences? Part cautionary tale and part technical demo, this talk shows how to instrument and monitor AI apps using OpenTelemetry, Prometheus, OpenLit, and more.

Cartography of the Deep: Distributed Tracing for Testing (Romanian Testing Conference)

Thu, 13 Jun 2024 12:24:02 -0700

I presented a talk today at the Romanian Testing Conference 2024 in Cluj-Napoca, Romania! My talk was called “Cartography of the Deep: Distributed Tracing for Testing” and it covered how to tie in distributed tracing into tests to get a better understanding of your system under test.

Here are my slides!

Abstract

From the Scandinavian Kraken that drags entire ships to their early graves to the Philippine Bakunawa said to rise from the waters to swallow the moon, our existential fears about what lurks at the bottom of our own oceans has inspired us to create hundreds of folkloric monsters across cultures. We’ve charted the surface of our planet, but the vast majority of the ocean floor remains a mystery.

The mysterious depths of modern software architectures makes it difficult for us to test a system for performance and reliability. The easy solution? Unmanned underwater probes: send inputs to the system, and judge its state based on the outputs that return. But this method is often too limited to tell us what’s really going on when something goes wrong. What if there were a way to make our tests themselves more observable?

In this talk, you will learn how to use open-source tools to instrument performance tests, identify bottlenecks at the code level using distributed tracing, and map out a system using a type of software bathymetry. You’ll also learn about the Mariana Trench expeditions, the difficulties of operating in a high-pressure underwater environment, and how to come face to face with monsters of our mythologies.

Adobo & Avocados: Intersectionality in Tech and DevRel (Agile Testing Days 2023)

Thu, 16 Nov 2023 09:45:15 +0100

My friend Marie Cruz and I presented today at Agile Testing Days 2023 in Potsdam, Germany. Our talk was called Adobo & Avocados: Intersectionality in Tech and DevRel. We talked about what intersectionality is, shared our personal experiences regarding diversity and intersecting identities, and gave tips on how to think about intersectionality as we build and use software. You can find the slides here.

Emergent Load Testing - TestCon Europe 2023

Wed, 25 Oct 2023 20:05:36 +0300

I presented at TestCon Europe 2023 in Vilnius, Lithuania about emergent load testing, and how to use it to find the limits of your system. My talk was called Emergent load testing: Rules for Organized Chaos.

You can find the slides here!

Taking Notes for Work With Obsidian

Fri, 16 Jun 2023 19:00:33 +0200

In this talk, I show why I think taking notes for work with Obsidian has not just helped me learn faster but improved my career working in tech as a Developer Advocate and performance engineer. I take notes continuously as a way to keep my skills relevant. This is a rehearsal of a presentation I gave at EuroSTAR 2023.

The Lost Art of Taking Good Notes (EuroSTAR 2023)

Wed, 14 Jun 2023 08:00:21 +0200

Today I gave the talk “The Lost Art of Taking Good Notes” at EuroSTAR 2023, about how I use Obsidian for work and why creating a personal knowledge management system is so valuable in the career of a tester.

Here are the slides.

Load testing and tracing with k6 and Tempo

Thu, 18 May 2023 22:41:42 +1000

At ObservabilityCON on the Road 2023 in Sydney, Australia, I talked about why you should use load testing and distributed tracing, how to do it with k6 and Tempo, and the benefits of being able to see beyond traditional black-box testing.

Resources

How to Avoid the Most Common Kubernetes Monitoring Mistakes

Tue, 16 May 2023 22:41:16 +1000

For ObservabilityCON on the Road Singapore 2023, I talked about some mistakes people make when monitoring applications within Kubernetes clusters. I also demonstrated Grafana’s Kubernetes Monitoring solution and how it can be a shortcut for engineers looking to get started quickly.

Resources

Slides here
Monitoring Kubernetes in general (notes)
The Grafana Kubernetes Monitoring feature (notes)

Emergent Load Testing - rules for organized chaos

Wed, 19 Apr 2023 00:21:06 +0200

Today I spoke at KubeCon EU 2023 about emergent load testing.

Resources

Here are the slides.

Below are some links to extensions and experimental modules that I mentioned:

Grafana
k6 site
k6 repo
k6 docs
k6 browser (experimental module)
Prometheus Remote Write (experimental module)
k6 tracing (experimental module)
xk6-kubernetes (extension)
k6 redis (experimental module)
xk6 disruptor (extension)

Abstract

When we write load testing scripts against our applications, we write them sequentially: A, then B, then C. But this doesn’t accurately reflect the organized chaos of a system in production, nor does it prepare the system for the unexpected.

Emergence is a phenomenon where parts of a whole independently develop properties not originally present in the whole. Emergence is what helps ant workers develop roles without leadership, prompts animals to evolve adaptive traits without forethought, and facilitates non-toxic communities without moderators.

The growing field of emergent software applies this swarm logic to the programs that we write. What would it take to write emergent load testing scripts? It turns out that there are a few ingredients for emergence: a large population size, opportunities to interact, feedback, and an element of control.

In this talk, Nicole van der Hoeven discusses how to bring these elements to load testing by writing a script in Grafana k6 that can independently decide what requests to make next, modify Kubernetes app pods, and disrupt services based on a continual feed of results during runtime– all without manual intervention. She shows how to wield this new breed of load testing to improve confidence in the complex systems we build.

How to create things with your notes

Fri, 17 Jun 2022 17:37:14 +0100

What do you do with all those notes? In this presentation, I talk about how to create things with your notes. I demonstrate my process of going from raw notes to this very presentation and share some of my strategies for creating output with little effort by learning in public and identifying topics to create content about using existing notes.

On May 19th, I did a talk at the Linking Your Thinking Conference about using your notes (or losing them), and about what we can do to make something useful of all our notes. This was my run through!

How to Be a Gish - API, browser, and chaos testing in one script (ExpoQA 2022)

Wed, 01 Jun 2022 14:06:49 +0200

For the ExpoQA 2022 conference in Madrid, Spain, I presented a talk about how to be a gish. In this talk, I covered the following topics:

what a gish is, and how it applies to the field of testing
how testing is a multidisciplinary field, and why generalists triumph over specialists in the face of uncertainty
how the tool we choose to do our testing informs the type of testing and the type of results we get
a hands-on example of how to do API load testing, browser automation, and chaos engineering in one script using a tool like k6

There was no recording of the conference talk, but here is a video of me going through my presentation beforehand:

You can find the slides here.

And here are some photos from the event!

Use It or Lose It - Linking Your Thinking Conference

Thu, 19 May 2022 14:03:13 +0200

Today I spoke at the Linking Your Thinking conference about what I do with notes I take, and how I manage to use them to actually create stuff.

If you’d like to see the result for yourself, here are the slides!

TestOps Ask Me Anything on Devops Toolkit

Thu, 07 Apr 2022 18:00:30 +0200

I joined my friend Viktor Farcic on his channel, DevOps Toolkit, to do an Ask Me Anything on the topic of TestOps.

We ended up talking a lot about k6.io, different Grafana Labs open-source projects, and what performance testing has to do with DevOps and Site Reliability Engineering.

In search of the best Pokémon: Browser automation and load testing in one script with k6 (Automation Guild 2022)

Thu, 10 Feb 2022 12:01:53 +0200

Being a tester is like being a pokémon trainer: we want to choose the right pokémon for the job. The problem with doing different types of testing is that we also end up with multiple scripts in different languages and testing tools. We end up choosing a squad of pokémon: one for unit testing, one for automation testing, one for load testing, and so on. Making sure they all work together is a task in itself!

But what if we could just use one? What if we only needed one Mega-evolved pokémon that could fulfill multiple roles? Is it really possible to write a browser automation script and then reuse it for load testing?

In this presentation, I talk about my search for the elusive one pokémon that does multiple jobs, the advantages of hybrid load testing, and how to pick pokémon that make our life as trainers and testers easier.

Basics of load testing with k6 and Grafana in 20 minutes

Tue, 12 Oct 2021 01:04:02 +0200

This was a video I pre-recorded in preparation for the first Grafana EMEA virtual meetup.

In mid-June (just three months ago), k6 joined the Grafana Labs family. Many people didn’t quite understand what the connection was between k6 and Grafana, but I thought it made a lot of sense.

Google’s Site Reliability Engineering book talks about 4 golden signals that we should measure in a system: latency, traffic, errors, and saturation. Load testing provides 3 of those (latency, traffic, and errors), but it doesn’t show saturation, and it doesn’t give us visibility into the system.

Grafana brings observability. k6 brings data. They work really well together, which is why we’ve been using each other’s products for a few years now, before there was ever any talk of an acquisition.

In this video, I talk about why k6 and Grafana are better together and also demonstrate:

how to run a simple script in k6 OSS
how to make sense of the results of a load test
test execution options: running locally, sending results to k6 Cloud, and running a test from k6 Cloud– all triggered using the CLI
two different integration options to bring k6 and Grafana together: the k6 Cloud Grafana data source plugin and Prometheus Remote Write

TTRPG Obsidian Showcase

Sat, 09 Oct 2021 00:43:25 +0000

Here’s a presentation I participated in about a bunch of nerds who like both Obsidian and D&D. My part begins around 0:45:20, but I thought the other speakers’ parts were really fascinating as well!

Testing With k6 + Grafana: Better Together (Grafana virtual meetup EMEA)

Tue, 05 Oct 2021 21:49:13 +0200

In this first EMEA session of the Grafana Labs virtual meetups, I talked about how to use k6 and Grafana together to get the full performance picture for a software application. Observability, using tools like Prometheus and Grafana, lets us see what’s going on inside the application, and load testing, using k6, helps us get those insights earlier on in software cycles, such as during pre-production.

Intro to Load Testing With k6 and Grafana (webinar)

Thu, 16 Sep 2021 22:11:16 +0200

Tonight I did a webinar with Grafana Labs– the first k6-related webinar since the acquisition. I was more than a little nervous to give it, because I knew I was introducing k6 to a lot of new users, but also to Grafana for the first time. I think it went pretty well! I was still quite relieved when it was over, however. :)

In the webinar, I talk about:

why it’s still a good idea to do load testing in pre-production
what k6 is
why k6 and Grafana together make sense
and how to integrate k6 and Grafana

There’s quite a bit of demo in it, and I also had the k6 CEO, Robin Gustafsson, join me for Q&A as a nice treat.

Check out the full webinar here– it’s already up on the Grafana site. You can also go through my slides here, if you’d like.

Schrödinger's Pokémon: k6 and Grafana edition

Wed, 15 Sep 2021 18:22:31 +0200

Earlier this year, I gave a presentation at New Relic’s FutureStack conference about chaos engineering with k6 and New Relic. A few months ago, k6 got acquired by Grafana Labs, and so I wanted to redo it, but with the Grafana stack this time.

Here’s the result, and I think I like this iteration even better:

I presented this talk at TestCon Europe 2021 today, although the audio sync issues were not great, so I recommend you watch the video above instead.

Slides are here.

Schrödinger's Pokémon: Observability for chaotic load testing (New Relic FutureStack)

Sun, 01 Aug 2021 18:54:27 +0200

This is a presentation I gave at New Relic FutureStack 2021.

Deploying an application to production is often like putting a pokémon in a box and hoping it manages to stay alive despite real-world chaos and potential outages. What can we, as testers, do to prepare applications for these situations before they go live? In this talk, I discuss chaos engineering concepts and demonstrates how to apply them by incorporating load tests with chaos experiments and setting up observability tools to watch it all happen. Because without observability, how can we tell whether Schrödinger’s pokémon is alive or dead?

Convergence of Code and Speech

Wed, 09 Jun 2021 22:23:19 +0200

As part of the Women Tech Global 2021 conference, I presented a short talk called “Convergence of Code and Speech”, under the theme of “Future of Work”. In hindsight, the title was a little pretentious, but what are you going to do?

I haven’t gotten the recording of the live version yet, but I always like to do a recorded test run by myself, and here’s what that was like:

Programming code and human speech are often seen as being positioned at different ends of the spectrum. At one end, code is terse, strictly logical, and limited. On the other end, speech is unbounded and often verbose. However, the gap between the two is narrowing.

Code is moving towards speech. Good code these days is code that exceeds simply being functional; good code is concise, but expressive; good code is easily comprehensible; good code refers to human concepts rather than technical components.

Shrinking global borders and the internet are making collaborating on code easier than ever before, but contributors may come from different cultures and languages. Remote work has significantly increased the likelihood of a multilingual devleopment team, so code style guides have emerged that are similar to writing style guides. Artificial intelligence is learning to interpret intention, and many apps now employ fuzzy search algorithms. Our expectations for code have increased such that we now expect code to understand us even when we’re being imprecise or inaccurate. Modern programming languages like React and Python use hierarchical abstraction to hide complexity, moving towards code that prioritizes being understood over being thorough.

On the other side of the spectrum, speech is also moving towards coding. Multilingual teams also encourage simplicity in written communication, removing colloquialisms and cultural references so that a more global audience can understand us. Tools like Grammarly correct for simplicity and conciseness. Verbal speech, too, is becoming increasingly programmatic with the rise of “smart” speakers and personal assistants that enforce a syntax for voice commands. Advances in artificial intelligence and machine learning have enabled us to interact with machines in natural language, and machines to interact with us back. OpenAI’s GPT-3 can produce convincingly human text about a variety of topics with little prompting. Writing tools like Notion, Roam Research, and Obsidian are growing ecosystems that mix human language with programming keywords to enable backlinks or access other features.

What are the implications of this convergence of code and speech? Writers will need to learn to code, and coders will need to learn to write. The two will become synonymous, and equally essential especially for knowledge workers. Tools that enable both coding and writing will become more valuable, so it’s worth investing the time to learn them. Both code and speech have inherent linguistic or cultural biases, so using inclusive language will become even more important.

What are coding and speech converging into? Their intersection is simply expressing thought. She who is able to express thought, both programmatically and naturally, will be best poised to take advantage of this convergence.

Load Tests as Code: An introduction to k6

Thu, 18 Mar 2021 20:36:41 +0100

Today, I attended the Cambridge University Press QA Week 2021 to talk about load testing with k6. Here’s a recording of the presentation.

And here are the slides I used!

I had fun discussing the advantages of the load test as code approach and why k6 is a great way to implement it.

Cuándo JMeter no alcanza: Pruebas de carga simulando navegadores

Wed, 16 Dec 2020 18:18:23 +0100

Aquí está el video de mi presentación en JMeter Scripting Day, un evento organizado por JMeter en Español el 31 de Oktubre, 2020.

Y la versión audio:

Introducción

Hola a todos! Yo soy Nicole van der Hoeven, y soy performance tester. JMeter es una de mis herramientas favoritas porque es gratis, simple, y por muchos casos hace exactamente lo que quiero y no más. Sin embargo, como performers es parte de nuestro trabajo elegir la herramienta adecuada para cada proyecto, y hay que admitir que hay situaciones en que JMeter simplemente no es suficiente.

Amazon - UI + Dev Tools

Esto es el sitio web de Amazon.com. Tengo abierta la pantalla de Chrome Developer Tools, que me muestra las solicitudes (o requests) que componen la página. Cuándo vamos a la página, el navegador (Chrome en este caso) envia estas solicitudes para nosotros, pero eso occure en el fondo. Un usuario normal no las ve.

JMeter - Creando un script

Lo que JMeter hace es grabar estas solicitudes y replicarlas. Aqui vemos una ejecución de un usuario virtual de un script en JMeter. Cada solicitud tiene la respuesta y el tiempo de respuesta. Pero eso no nos da una idea completa del rendimiento de la aplicación. ¿Por qué?

Por que JMeter, como otras herramientas de nivel protocolo, tiene unas limitaciónes. La primera es que si hay javascript o otros script que deben estar ejecutado por el navegador, JMeter no los ejecuta. JMeter solo los descarga. El tiempo de respuesta mide la descarga, no la duración de la ejecución. Entonces, siempre habrá una diferencia entre los tiempos de respuesta en una prueba de JMeter y los tiempos de respuesta en práctica.

A veces esta diferencia no es tan importante. Pero hay más y más aplicaciones hoy en día que se componen principalmente de estos javascript. Por esas aplicaciones, simplemente no es posible hacer un script en JMeter porque la mayoría de la lógica de la aplicación occure en el navegador, no en el servidor.

Flood Challenge

La segunda razón es que el aumento de la seguridad de aplicaciones conduce a solicitudes más complejas que incluyen parámetros dinámicos. Estos parámetros hacen que sea más difícil para un intermediario lanzar un ataque, pero también hacen que sea más difícil para nosotros scripters, porque tenemos que extraer información desde la última respuesta y ponerla en la próxima solicitud. Es posible, pero es la pesadilla de cada performance tester.

Podemos verlo en la aplicación de muestra Flood Challenge. Ya no trabajo para Flood, pero todavía es un buen ejemplo.

Correlación en JMeter - Flood challenge script

Esto es mi script en JMeter, y vemos que hay muchos parámetros que he tenido que correlacionar. La solicitud para simular hacer clic en el botón “Start” necesita los parámetros authenticity token, step ID, y step number, que son dinámicos. El contenido de los parámetros cambian cada vez que usamos esta aplicación.

La experiencia del usuario

La tercera razón es que JMeter tampoco mide la verdadera experiencia del usuario. Solo mide la duración de la comunicación entre el navegador y el servidor, pero no mide cosas como la representación de elementos en la página web, ni el efecto de un script que dura mucho tiempo para terminar.

Eso es importante porque estas cosas afectan significativamente la experiencia del usuario. De hecho, Steve Souders, un conocido el la industría de performance, dice que hasta 90% del tiempo de respuesta que experimenta el usuario está en la interfaz. Con este numero podemos estar ciertos que una prueba con una herramienta de nivel protocolo no va a poder encapsular el rendimiento final de una aplicación.

Developer Tools - Lighthouse

Entonces, ¿qué podemos hacer?

Lo más fácil es intentar de medir la interfaz durante una prueba de cargas. Mientras ejecutamos una prueba con JMeter, podemos ir manualmente a la aplicación web en Chrome para usar Developer Tools. Podemos usar la pestaña Lighthouse, que nos da más métricos del perspectivo del usuario y sugerencias para mejorar el rendimiento.

WebPageTest

También uso WebPageTest a veces. Eso tiene más opciones para ejecutar la prueba automáticamente varias veces desde un lugar que puedo elegir. Aún podemos elegir un dispositivo para simular la experiencia de un usuario en un iPad, por ejemplo.

Herramientas alternativas para pruebas de carga

Tambien podemos elegir herramientas alternativas para ejecutar nuestras pruebas de carga. Lo que tienen en común es que fueron diseñadas para pruebas de automatización, y utilizan un navegador real.

Selenium es probablemente la más popular de esas. Selenium WebDriver en particular. Es compatible con Ruby, Java, Python, C# y Javascript. El problema con Selenium es que puede ser muy pesada. En una machina probablamente podemos ejecutar cinco usuarios de Selenium, más o menos. Eso es importante si estamos tratando de simular miles de usuarios.

Hay muchas herramientas basadas en Javascript que también podemos usar. Hay varias framework que se utilizan para adaptar Javascript y nodeJS.

Puppeteer fue desarollado por Google. Tricentis Flood también ha diseñado Element que utiliza Puppeteer, pero con un enfoque en el rendimiento.
Playwright es una creación de Microsoft.
Jest fue desarrollado por Facebook
Mocha es otro framework que es la base para Cypress.

Todos esas herramientas son open source y gratis. También hay alternativas propietarias, pero prefiero las soluciones open source porque en la mayoría de casos hacen la misma cosa y se actualizan con más frecuencia.

Element script

Esto es un ejemplo de la misma acción, hacer clic en el botón Start en la aplicación Flood Challenge.

Otra ventaja de simulando navegadores en vez de simulando las solicitudes es que es más simple crear un script. No tenemos que saber cada parámetro. Solo tenemos que describir lo que queremos que cada usuario virtual haga.

Podemos ejecutar scripts usando estas herramientas al mismo tiempo que un script de JMeter o Gatling o lo que sea, y también tenemos la opción de reemplazar los script de protocolo con estas.

Conclusión

Para resumir, JMeter y otras herramientas de nivel protocolo tiene algunas limitaciones.

No ejecutan scripts del lado del navegador
Pueden ser más complicado correlacionar parámetros dinámicos
No miden la experiencia final de un usuario

Para solucionar estos problemas, podemos complementar pruebas de carga tradicionales, de nivel protocolo, como JMeter, con herramientas simulando navegadores. De este modo podremos tener una comprensión más completa del rendimiento total de la aplicación.

Bueno, se acabó el tiempo, entonces los dejo aquí. Gracias por escuchar, y me disculpo por mi español. Gracias a todos!

How to Become a Load Tester

Wed, 07 Oct 2020 14:15:45 +0100

Originally posted here.

Load testing can be a pretty intimidating field to get into, but it’s easy to get started. Here are some concrete steps you can take to start learning how to load test from scratch.

In this video, Flood co-founder Tim Koopmans and I teamed up at Tricentis User Conference 2020 to talk about the essential ingredients for load testing.

Background required for load testing

Load testing can be quite technical. It involves not just the ability to use different programming languages to create load testing scripts but also familiarity with software engineering practices, DevOps, application infrastructure, site reliability, and networking. A good load tester needs to be able to understand the basics of how an application works and how different components interact with each other. It’s common for a load tester to work on applications employing a variety of technologies, so staying up to date with recent technological trends and switching seamlessly from one language to another, or from one platform to another, is of utmost importantance. This technical side of load testing might make you think that a load tester needs to have formal studies in computer science and several years of experience in software testing. The reality is that there are many load testers who stumbled into the field from wildly unexpected backgrounds. So you might think load testing looks like this:

‍ Photo by Christina @ wocintechchat.com on Unsplash

And sometimes it does. But it more often looks something like this:

‍ Photo by You X Ventures on Unsplash

You see, load testing is an interdisciplinary field. So much of the job involves talking to people. Sometimes the requirements for a project are not clear, and someone needs to talk to the both the business and the developers to make sure that nonfunctional requirements are clarified. A load tester is a detective, tracing a request as it goes from one application component to another, coordinating with different application teams. A load tester is a scientist, using the scientific method to design experiments that yield the results. A load tester is a statistician, making sense of a jumble of data and seeking correlations between variables. A load tester is a public speaker. Even when the actual load testing has been completed and the report has been sent, a load tester makes sure all stakeholders understand the report so that the results can be used to make meaningful changes to improve application performance. All these tasks require a mishmash of skills and competencies that can be difficult to find in just the field of computer science.

Let’s take our backgrounds as anecdotal examples.

Tim grew up and currently lives in a small country town in Australia, and he joined the military straight after school. He later developed a love of automating things, which led him first to use tools like perl to scrape websites and try to win bids with early versions of ebay, and then later to starting a performance engineering consulting career and co-founding Flood itself.

I, Nicole, grew up in Metro Manila in the Philippines (which has 14,000 more people per km2 than Mumbai does). I learned English as a foreign language in school, and I have a university degree in Economics. When I discovered load testing, I had a job in a completely different field– I was working for an airline and making sure engine components were properly maintained and repaired. My love for gadgets and technology took me down an 8-year rabbit hole involving doing performance consulting for clients across a variety of industries in Australia and the Netherlands. At Flood, I help customers around the world improve their application performance.

As you can see, Tim and I came from very different backgrounds and different parts of the world. Neither of us knew what load testing even was when we started working. Neither of us got formal instruction in load testing. But somehow, we still both ended up becoming load testers at Flood.

Reasons to become a load tester

So why become a load tester? Here are some practical reasons.

Load testing is cool

A little-known fact about load testing is that it can be really satisfying to be involved in big-name projects that end up going live.

You may have heard of Flood. But even if you haven’t, we’re sure you’re familiar with the results of load testing done on Flood. In the last few years, we have been helping Hotstar, one of the largest streaming services in the world, load test their systems in preparation for the Indian Premier League.

Hotstar uses an open-source tool that we support, Gatling, to create scripts to simulate users tuning in to watch cricket games. Then, they use Flood to create a “Game Day” simulation. They scale up those load tests and run 5 million users on thousands of machines in the cloud. We help them coordinate execution, spot problems quickly with real-time results from all those machines while the test is still running, and identify bottlenecks so that they don’t occur in production, when India’s playing.

As a load tester, you’ll definitely be working with brands you recognize and love.

Load testing can pay well

‍ Source: Payscale

I said this would be a practical list, right?

Here are 261,768 practical reasons. The average performance engineer’s salary is 71% more than that of an average software tester in India, according to Payscale.com. We all know that money isn’t everything, but this is a clear signal from the market about the importance of performance-related skills. Adding load testing to your CV as a tester makes you more attractive to employers. It’s a great way to differentiate yourself in an over-saturated market.

Load testing can literally take you places

‍ Source: Shamim from Flood

Then there are the side benefits of being in load testing. The photos above are real ones from the Flood team. That’s me on the upper left, exploring a castle in Belgrade, Serbia, and that’s Tim in the bottom right, taking a photo in San Francisco, USA.

We’re a fully distributed team on Flood, which means we can work from anywhere. Flood is a web app, so you don’t need to install anything– I’ve been known to start load tests from my iPad in a café. Load testing on the cloud frees you from being tied to running tests from the office. You can start load generators when you need them and shut them down when you don’t– all from a web browser. It’s a job that can come with a great work/life balance, if you find the right company.

Load testing can also be a great way to move to another country. It’s what’s allowed me to move to other countries– twice. In Australia, the average performance engineer’s salary is AU$108,053, or over 5.6 million rupee.

‍ Source: Payscale

So it’s clear that there are some pretty good reasons for you to become a load tester. I’m going to walk you through the practical steps to become a load tester. You may not be an expert by the end of this, but I hope I can convince you that you can get started in load testing whether your background looks like Tim’s or mine.

Load testing adds real value

And the trite, but absolutely true reason: Everyone fails.

As a load tester, you are a gatekeeper of quality. What you do directly affects revenue and customer experience.

Performance is an aspect of software that is often overlooked, but it’s only becoming more important in a world where everything is going online. People are being born who have never known what it’s like not to have the internet, and consumers in general expect applications to be responsive and stable. Would-be customers who can’t get an e-commerce site to load within 3 seconds may switch to a competitor’s site, and we all know there are many of them. Performance-related outages can be seriously damaging to a company’s reputation, and social networking amplifies customer dissatisfaction by effectively giving everyone a megaphone to complain into.

Software functionality is still important. However, if an online store goes live with no critical functional defects, but it fails when the 100th person logs into it, or it takes 10 seconds to load the homepage, can you really say that it was ready for production?

Load testing is a crucial part of ensuring software quality, and quality is the responsibility of everyone on a team, even if the word “performance” isn’t in your job title.

The essential ingredients of a load test

Load testing is like cooking a meal. This is one of Tim’s favorite dishes, aloo matar.

‍ Source: Cubes n Juliennes

You can approach cooking as a science. There are recipes with a list of ingredients, a precise measurement of how much you need for each, and a procedure for creating the dish.

But you can also look at cooking as an art. Cooking the same dish over and over again will give you an experience in working with those ingredients that a novice cook won’t have. There’s room in cooking for adding one ingredient, changing the quantity of another, or just trying something new.

A master chef knows how to use both the science and the art to create something delicious. But you don’t HAVE to be at that level in order to cook.

Anyone can cook. If you know how to follow a basic recipe, you may not create a masterpiece, but if you’re like me and you haven’t had lunch yet, you’ll still end up with something you’ll eat.

So here’s the recipe for load testing.

I prefer kadai paneer myself, but aloo matar requires, among other things, peas. And so does load testing.

P is Preparation. To prepare for a load test, you need to get your ingredients ready. You need to decide what it is you want to test, why you’re testing it, and how. You write load testing scripts that will simulate real users as closely as possible.

E is Execution. This is when you do the actual cooking. You run your load tests and scale them up according to your requirements.

A is Analysis. You figure out whether the dish you created is edible. You determine how your application servers responded during the load test, and decide if that’s acceptable.

Preparation – Execution – Analysis.

Preparing for a load test

“As far as the World Cup is concerned, it is a process. We don’t want to jump to the 50th floor straight away. We must start on the ground floor.” - Sachin Tendulkar, via ESPN

Load testing doesn’t start with writing a script. Load testing starts with building a foundation for later tests.

‍ Source: Inside Sport

Part of that foundation is asking WHY. Why do you want to do a load test? Has there been a performance-related issue before? Are you trying to determine whether an application is ready to go live, or is it already live, and you just want to improve its performance? What type of application do you want to test?

WHOM are you cooking for? I think you’d cook differently for yourself than you would for Sachin Tendulkar. What do your end users typically do on your site? Where do they come from?

These are questions that need to be answered before you think about scripting. Here are some resource for planning your load tests:

Start with Why: How to write requirements for API load testing How to write a test plan for load testing Selecting a tool for load testing Generating a load testing script without coding: Getting Started with Test Builder

Execution of a load test

So now we’re ready to start cooking. This is the Execution phase. You’ll need a machine to use as a load generator. Depending on the number of users you want to run with, you may need more than one machine. You might be tempted to run a load test on your laptop, or on a bunch of machines you have lying around, but that introduces new variables that may affect the results.

‍ Source: Ashwini Chaudhary on Unsplash

Like any good cook, you’ll also need to make sure you have all your ingredients handy. In load testing, that means making sure your scripts and test data files are ready to be executed on every load generator.

Both of these problems – provisioning a load generator and transferring files over to them – are solved pretty easily by running on the cloud using something like Flood.

Designing and executing your load test

Where should you run your load tests? A look at demand, hosted, and on-premise infrastructure

Analysis of Results

The last part is Analysis.

‍ Source: Fine Dining Lovers

This is a pretty crucial moment, because this is when you figure out whether what you cooked was delicious or not. Unfortunately, in load testing, it’s not always easy to tell how your test went. It’s like this photo of pakora (onion is my favorite kind). It’s kind of hard to say what expression that is in their faces. Is it disgust? Or is the pakora so yummy that they just can’t wait to eat more?

Load testing typically generates a LOT of data. You’d have to go copy every single file from every load generator you used. But can you imagine the hundreds of load generators that Hotstar used? Of course they didn’t manually copy those files. Instead, they used Flood.

On Flood, we show real-time graphs as you’re running the test so that you can tell immediately whether transactions are failing or whether a server has stopped responding.

After the test, you can also compare this data to previous tests to spot trends, or even create your own dashboards using an analytics engine.

Real-time metrics during your load test

Downloading results and logs after the test

Is that it?

You’ve prepared for a load test, you’ve executed a load test, and you’ve analysed the results from the load test. Congratulations, you’re a load tester! By following this recipe, you’re already doing better than many people, and you’re certainly doing a lot better than those who do not run any load tests at all.

The cool thing about load testing, though, is that this is just the beginning. There’s always room for improvement.

‍ Source: Pop Up City

Dabbawallas are people who deliver hot lunches, and while the concept is not new, the implementation in India is particularly interesting.

For these dabbawallas, having the meals prepared is not the end of the job. Once the food is ready, it needs to be packed into lunchboxes, picked up, separated according to destination, put on trains, and delivered. Later, the empty lunchboxes are picked up, and the system works in reverse.

It turns out that this is actually a pretty sophisticated system. In fact, it’s been estimated that only one tiffin every two months is not delivered. That’s equivalent to a Six-Sigma quality rating of 99.9999999% (that’s nine nines) or more.

Dabbawallas didn’t get that rating just by delivering quickly. They did it by being consistent and reliable. They did it by having a process.

Once you see the benefits of running a load test, you may also start to understand that the real value of load testing is not in a test that you run once whenever you think of it. Imagine having the benefits of a load test tied to your development process, so that it is automated.

As an advanced topic, adding your load testing to your team’s Continuous Integration/Continuous Development pipelines such as sets up a repeatable process that reaps benefits far into the future, not just whenever official load testing activities are carried out.

Continuous Integration with Flood: Integrating Travis CI, Circle CI, and Jenkins

Conclusion

So that’s the recipe for becoming a load tester.

Preparation – Execution – Analysis.

In Preparation, you determine why you’re testing and what tool best fits your needs. We showed you how to use Element, but there are many open source tools out there. Don’t get overwhelmed by scripting– a lot of times it can just mean replacing a URL with the one YOU want to test.

In Execution, you run a baseline test that will serve as a point of comparison for future testing, choose the number of users and the number of regions you want, and then you run your test. Running on the cloud is the easiest (and usually cheapest) way to get started.

In Analysis, you view the results in real-time, compare historical performance to see trends, and share reports to stakeholders so that your team can make changes to your application.

We also talked about the bonus topic of CI/CD and how you can incorporate load testing into your pipeline to get a continuous feedback during your development cycle.

Tim and I started in very different places and had different experiences before we discovered load testing. But eventually, we ended up on the same path, and that’s what we’ve outlined for you here. We hope that this convinces you that load testing doesn’t have to be complicated.

You absolutely don’t need to be an expert to run a load test. Even if you don’t think you’ll get all these steps right, a load test that is less than ideal is still better than no load testing at all.

In the end, anyone can be a load tester with the right attitude and the right tools.

Learning about load testing

High Performance Websites book by Steve Souders

Ministry of Testing/PerfBytes list of resources

PerfBytes podcast (EN and ES)

PerfGuild conferences by Joe Colantonio

Señor Performo’s YouTube Channel: Señor Performo ENG

Scott Moore’s Performance Tour on YouTube

Lifting the lid on performance testing

Thu, 24 Sep 2020 23:38:56 +0100

I joined other performance and testing engineers in this online panel at Testing Talks Online about load testing best practices.

From the Testing Talks Online site:

Performance testing is an essential part of the quality assurance function. However, across the industry the great majority of companies still struggle to master and execute a successful performance testing strategy that produces measurable results. In this panel discussion, Cameron Bradley will host some of the best performance engineers in the world. The panel will discuss how we as quality assurance experts and performance engineers can better understand performance testing, the value of performance testing and how world leaders are approaching and succeeding with performance testing as part of the delivery life cycle.

Hybrid load testing: the best of testing using protocols and real browsers

Tue, 02 Jun 2020 22:21:11 +0100

At PerfGuild 2020, I spoke about hybrid load testing and how to mix two different approaches to load testing: protocol-based and browser-based.

Here’s the unlisted video from that presentation:

TestCon Moscow 2019: Dismantling Cognitive Biases

Wed, 03 Apr 2019 23:14:31 +0100

Originally posted here.

Developers and testers alike prefer to think of themselves as entirely rational beings, because both software development and software testing do require systematic rigor and logical reasoning. However, the reality is that the human element still comes into play when building and testing software.

In this year’s TestCon conference, held in Moscow, Russia, I delivered a talk on how cognitive biases affect web performance outcomes.

In the talk, I discuss:

two types of thinking: fast thinking and slow thinking (from Daniel Kahneman’s book Thinking, Fast and Slow)
how to influence the sentence duration that judges give perpetrators
what cognitive biases are and how they can affect performance testing results
a real-life production incident that occurred on Flood and how our team’s cognitive biases got in the way of its resolution
what to do to avoid cognitive biases when testing software

Here’s the video of my full talk:

Download the presentation in PDF format.

Presentation on Nicole van der Hoeven

Asimov's Zeroth Law of Robotics: Testing for AI (HUSTEF 2026)

Resources

The Great Doubt: What Building an AI Agent Taught Us About Trust (KubeCon Japan 2026)

Resources

Related reading

Asimov's Zeroth Law of Robotics: Observability for AI (ExpoQA Madrid 2026)

Resources

Asimov's Zeroth Law of Robotics: Observability for AI (Newcrafts 2025)

The Spice Must Flow: The Fremen Guide to Sustainable Observability (Øredev 2025)

One Year Later: How I Use Visuals in My PKM (Sketch Your Mind Conference)

Asimov's Zeroth Law of Robotics (Dutch Cloud Native Day 2025)

Asimov's Zeroth Law of Robotics: Observability for AI (KubeCon EU 2025)

Cartography of the Deep: Distributed Tracing for Testing (Romanian Testing Conference)

Abstract

Adobo & Avocados: Intersectionality in Tech and DevRel (Agile Testing Days 2023)

Emergent Load Testing - TestCon Europe 2023

Taking Notes for Work With Obsidian

The Lost Art of Taking Good Notes (EuroSTAR 2023)

Load testing and tracing with k6 and Tempo

Resources

How to Avoid the Most Common Kubernetes Monitoring Mistakes

Resources

Emergent Load Testing - rules for organized chaos

Resources

Abstract

How to create things with your notes

How to Be a Gish - API, browser, and chaos testing in one script (ExpoQA 2022)

Use It or Lose It - Linking Your Thinking Conference

TestOps Ask Me Anything on Devops Toolkit

In search of the best Pokémon: Browser automation and load testing in one script with k6 (Automation Guild 2022)

Basics of load testing with k6 and Grafana in 20 minutes

TTRPG Obsidian Showcase

Testing With k6 + Grafana: Better Together (Grafana virtual meetup EMEA)

Intro to Load Testing With k6 and Grafana (webinar)

Schrödinger's Pokémon: k6 and Grafana edition

Schrödinger's Pokémon: Observability for chaotic load testing (New Relic FutureStack)

Convergence of Code and Speech

Load Tests as Code: An introduction to k6

Cuándo JMeter no alcanza: Pruebas de carga simulando navegadores

Introducción

Amazon - UI + Dev Tools

JMeter - Creando un script

Flood Challenge

Correlación en JMeter - Flood challenge script

La experiencia del usuario

Developer Tools - Lighthouse

WebPageTest

Herramientas alternativas para pruebas de carga

Element script

Conclusión

How to Become a Load Tester

Background required for load testing

Reasons to become a load tester

Load testing is cool

Load testing can pay well

Load testing can literally take you places

Load testing adds real value

The essential ingredients of a load test

Preparing for a load test

Execution of a load test

Analysis of Results

Is that it?

Conclusion

Learning about load testing

Lifting the lid on performance testing

Hybrid load testing: the best of testing using protocols and real browsers

TestCon Moscow 2019: Dismantling Cognitive Biases