Testing on Nicole van der Hoeven

Asimov

Wed, 02 Apr 2025 00:00:00 +0000

Asimov is the companion project to my talk Asimov’s Zeroth Law of Robotics. It’s an AI app built to be observed and tested: The Silent Enterprise, a single-player Star Trek text adventure. You play Data, and Claude is the game master. Every model call, dice roll, and turn shows up as telemetry in a Grafana that you can run on your own machine or in Grafana Cloud, and a k6 test suite, some of it judged by Claude, checks whether the AI follows the rules.

The game master has some known flaws, kept on purpose. It sometimes narrates a rescue that didn’t happen, or “destroys” a drone that’s still standing. The point of the demo is catching those flaws with telemetry, tests, and online evaluators.

The idea

Isaac Asimov’s three Laws of Robotics govern how robots behave. The talk argues for a law that comes before all of them: a robot must be observable, and, in its newest version, testable. AI systems fail differently from traditional software. There’s no single correct output to assert against, a response can be fluent and completely wrong, and models drift. You can’t test what you can’t see, and you can’t trust what you can’t test.

What’s in it

The game. A Go program with a command-line interface and an HTTP API. The game engine owns the rules and the state. Claude interprets the player’s input, makes the game master’s own rolls through a tool, and narrates, so a narration that contradicts the saved state is a defect that can be caught.
Telemetry. Traces, metrics, and logs go over OpenTelemetry to a local Grafana stack (Tempo, Prometheus, and Loki, with the game’s dashboard already loaded) or to Grafana Cloud. Generations can also go to Grafana Cloud’s Agent Observability, where online evaluators judge a sample of them.
Tests. A k6 suite with code checks, LLM judges, dice-trajectory evals, and whole playthroughs. All of it can run for free against a mock game master and judge.
No account needed to start. You can play offline with no API key, and see the local monitoring stack with one command and Docker.

The screenshot above is the game’s local Grafana dashboard.

The talks

I’ve given the talk, in several versions, at KubeCon Europe 2025, Dutch Cloud Native Day 2025, Newcrafts 2025, ExpoQA Madrid 2026, and HUSTEF 2026. The write-ups are linked below. The earlier talks used a Python and Flask version of the app, which has since been rewritten in Go.

Try it

The repository has a quick start for playing offline, a Docker-only path, and release binaries for Windows, macOS, and Linux.

Roundtable: How We Really Use AI in Testing (HUSTEF 2026)

Thu, 08 Oct 2026 14:00:00 +0200

On the final day of HUSTEF 2026 in Budapest, I joined the conference’s roundtable on AI in testing — “AI Open Space Experiences: a roundtable with speakers on their experiences with AI testing.” It was five of us on stage: me plus Marta Firlej, Kristoffer Nordström, Callum Akehurst-Ryan, and Maryia Tuleika, with Marta moderating.

The format was refreshingly honest — “how we really use AI, with no bullshit.” Marta had run the same “AI open space” exercise with the Polish testing community, crowdsourcing the actual, un-hyped ways people use AI (information gathering, functionality discovery, generating tests and code, TDD, recruitment, presentation prep, even designing furniture and planning holidays), and then opened it up to the panel and the room. Here’s roughly how I answered her questions:

What do you use AI for? Mostly two things at work: analysing codebases and test code to understand what something actually does, and filling gaps on teams without a dedicated quality engineer — baking my testing knowledge into an AI the team can ask questions of. On the personal side, I’ve run some weird experiments: I’m a lifelong note-taker with decades of context (journals, therapy notes, eight years of WhatsApp history with my partner), so I’ve fed it all into a local, open-weight model and asked things like “what do I claim to believe, and where have I acted contrary to that — cite your evidence?” It’s a surprisingly honest mirror. I’m in a privileged spot with effectively unlimited AI access right now, and I know it won’t last, so I want to find out what’s safe and what isn’t while I can.

How do you stay up to date? Honestly, I don’t — not by chasing model releases or news. I keep up with how other people, especially in testing, are using AI, and I lean on my coworkers (we’re at the forefront of this at work, so stepping outside that bubble is a useful reality check). I do have a scheduled Claude Code task that scrapes my ~34 AI-related Slack channels into a daily digest of just the things I care about.

One piece of advice? Don’t trust AI too much, and don’t believe you’re as far behind as you think. Almost no one has really tried this stuff; it’s genuinely early, everyone’s learning, and your foundational testing skills transfer. Just pick it up, play, and if it’s not for you, move on.

Anything you’d never use AI for? Communicating with people as a proxy. I’ll use it to translate or to help me sound more fluent (I live across multiple languages — Portugal, Dutch, English), but I never want it to craft a whole message I didn’t think of. I don’t want to lose the discipline of actually talking to a human.

Ethical dilemmas? Yes — which is exactly why I keep my use mostly in the context of work, and try not to generate imagery or anything that’s obviously lifting from artists who deserve to be paid for it. There’s no hard line; it’s a sliding scale, and I have to keep questioning myself. (A Mastodon line stuck with me: the strange future where the humans do the menial tasks and the robots write poetry and paint. I try not to let AI stop me from making art.)

The best part of these panels is always the disagreement and the texture — five people who use AI daily and still land in very different places on trust, ethics, and what it’s actually for. Thanks to Marta for moderating and to Kristoffer, Callum, and Maryia for a genuinely good conversation.

Resources

HUSTEF 2026
My HUSTEF keynote: Asimov’s Zeroth Law of Robotics: Testing for AI

Asimov's Zeroth Law of Robotics: Testing for AI (HUSTEF 2026)

Wed, 07 Oct 2026 10:00:00 +0200

This week I gave a keynote at HUSTEF 2026 in Budapest, Hungary. The talk is called Asimov’s Zeroth Law of Robotics: Testing for AI — a testing-focused evolution of a talk I’ve been giving (and rewriting) for a while, previously as Observability for AI at KubeCon EU 2025 in London, Dutch Cloud Native Day 2025 in Utrecht, NewCrafts in Paris, and ExpoQA Madrid 2026. Here’s the abstract:

A robot may not harm humans. A robot must obey humans. A robot must protect its own existence. These are Isaac Asimov’s three Laws of Robotics, created to govern the ethical programming of artificial intelligences. From the Butlerian Jihad to Skynet to cylons, we’ve been immortalizing our collective nightmares about artificial intelligence for years. But there’s an unmentioned law that comes as a prerequisite to all of that: a robot must be testable.

AI systems fail differently than traditional software. There’s no single correct output to assert against. A response can be fluent and completely wrong. Models drift silently over time. And yet we’re shipping AI into production — often without knowing what “passing” even looks like.

Observability is what makes testing possible. When we can see inside our AI systems — trace the chain of reasoning, measure what changed, surface what failed — we can start to write evaluations, define what “good” looks like, and catch regressions we didn’t know to look for.

In this talk, I explore what it means to test AI in the real world, drawing on lessons from building and evaluating an AI assistant for a large-scale observability platform. I’ll cover how evaluations work as the AI equivalent of a test suite, what to measure, how to catch hallucinations, and how to know when your model has regressed.

You cannot test what you cannot see — and you cannot trust what you cannot test.

The big shift in this version is right there in the Zeroth Law itself. In earlier iterations I argued that a robot must be observable. That’s still true — observability is the only way to see the reasoning behind a system and find out what actually happened. But I’ve come to think that pure knowledge isn’t the same as trust. You can know exactly what a system did and still not trust it. The only way to trust something is to put it through its paces: to know how it behaves in the situations you foresee, and even the ones you don’t. So the Zeroth Law has become a robot must be testable.

I framed the whole thing around Asimov’s short story Runaround (from I, Robot) and its robot, Speedy. Speedy’s smuggler owners sent him to collect selenium on Mercury, and he got stuck in a loop — circling the selenium pool, unable to resolve the conflict between obedience (the second law) and self-preservation (the third). His owners couldn’t predict his behaviour and couldn’t reason about it, so they had to do the only thing left: go out into the danger zone and fix him in production. Speedy wasn’t untestable because his owners were incompetent or lacked the tools — he was untestable because he was never built to be tested. That’s the trap I think a lot of AI systems are walking into right now.

The through-line for a testing audience: AI is rewriting how software gets built, but it doesn’t rewrite the discipline of trust. Trust that a system does what we say it does, and that when it breaks we know why and can fix it — that’s always been the job, and it isn’t going anywhere. There’s a whole generation of AI engineers shipping software they haven’t tested and can’t explain, and when someone says “the agent does this,” our job is still to ask how do you know? Testing isn’t just going to survive the age of AI — testing is the reason AI gets to survive, if we make it testable.

I demoed this with my TTRPG side project: an AI Dungeon Master, instrumented and put under test. It turns out that writing evals for something non-deterministic forces you to confront your real requirements. I thought I knew exactly when a dungeon master should call for a dice roll — I GM myself! — but trying to put it into words (“in this situation I’d roll, but in that one I wouldn’t, because I want to keep the story moving”) surfaced decisions I’d never had to make explicit. Those are things we’d normally discover much later, in production. Having to figure them out up front is a win.

The Q&A was genuinely great and ranged well beyond the slides:

“When has an AI been tested enough?” Same answer as for anything else — it’s a mix of risk and budget. If it were a life-critical robot like Speedy, you test every scenario the people relying on it need. For my little demo app, less so.
Testers vs. developers in the age of AI. I said I’d fire developers before testers — provocatively, but the point stands: anyone can write code now. What made a good developer good was never the coding; it was communicating with business and engineers, translating requirements, finding creative-but-safe solutions. Testers already think at that level and translate across teams, which makes them more important in the age of AI, not less.
Defining expected results for non-deterministic systems, multi-turn evals (I give the simulated user a personality — adversarial, cooperative, beginner — rather than a script), and the broader shift from imperative to declarative: we no longer instruct step by step, we declare the end state we want, the way we do with Kubernetes.
Convincing colleagues not to fear AI. I don’t, really. AI is here to stay; the question isn’t whether you approve of it, it’s whether you want to stay in this industry and help make it safe and trustworthy — or sit on the sidelines.

I also recommended the book Slow AI, which lands where I do: not an AI apologist, not a boycott, but a clear-eyed list of what AI is and isn’t good for — and the reminder to keep checking whether a given use of AI is making you more capable or less.

The demo app (the instrumented AI DM) lives on GitHub, and I keep refining it with each iteration of this talk.

I’ll update this post with the recording once HUSTEF publishes it.

Resources

Slides
Demo app on GitHub
Slow AI (book)
Previous versions of this talk: KubeCon EU 2025 · Dutch Cloud Native Day 2025 · NewCrafts 2025 · ExpoQA Madrid 2026

Investigations Arena: how we test Assistant investigations

Thu, 30 Jul 2026 09:00:00 +0100

How do you test an AI agent whose whole job is to investigate incidents — when every investigation is different and “correct” isn’t a single deterministic answer? In this interview I sit down with Will, a software engineer at Grafana who’s spent the last year focused on AI: the Grafana Assistant, investigations, and memory features.

The story here is how the team got from Investigations V1 to a much better V2. V1 hit a ceiling, and the demo environment they’d been testing against turned out to be the problem — the cases were too easy and too far from the messy reality of a production cluster. So they built an arena: think competitive bot programming, but the “game” is investigating real incidents in Grafana’s own production environment. Different AI agents, each with its own strategy and tools, all take a swing at the same incident, and an LLM-as-judge scores their answers against a human-verified root cause.

What’s interesting is that the score wasn’t even the most valuable output at first — the insights were. Watching which strategies worked, which tools returned so much noise that they poisoned the context, and which agents wandered off is what let the team optimize bit by bit into the V2 release. The arena also compares more than models: different harnesses running Sonnet vs. Opus, Claude Code with the Grafana MCP, even just hitting the public API directly with curl.

A couple of my favorite moments: Will explains the guardrails that stop agents from “cheating” (they can only look at data from within the incident’s time window, so they can’t peek at the fix commit that resolved it) — and how the models mostly play fair because they treat it like a game. And there’s a lovely example of emergent creativity: an agent reached for tools the team hadn’t even considered, they turned out to be genuinely useful for investigation, and those tools got promoted into the real harness. Will’s hope for the future is to open this up — an OliBench-style shared benchmark where the community can throw their own investigation agents into the arena and raise quality across the board.

Transcript

Will: Hey, my name is Will. I’m a software engineer at Grafana. I’ve been there for three years, and for the last year I’ve been focusing on AI — the Grafana Assistant, investigations, and memory features.

Nicole: And what’s the latest thing you’ve been working on?

Will: So we developed Investigations V1, and we saw that we hit a ceiling at some point. We came up with a different approach, but this time we needed a good evaluation environment — the demo environment wasn’t enough, because those cases were too easy and too far from the reality of a production environment. So we decided to build an arena.

It’s a little bit like the idea of competitive bot programming, where bots fight against each other on specific tasks or games. In this case, the topic was investigations in our production cluster — real incidents. We started building bots that had different strategies and different tools; these were AI agents. Comparing a diversity of strategies let us surface the best approaches, and from there we could iterate, bring in more test cases from more incidents, and reach higher and higher quality.

The idea is that the bots try to solve the same case, and we use an LLM as a judge to compare the different approaches. We have different criteria, because for each incident we have a root cause that’s been verified by a human. That gives us our ground truth, and the LLM-as-judge compares that ground truth against the answers the agents came up with.

The important part at first wasn’t even the score — it was the insights. This tool was better with this strategy; this one had noise in the results that poisoned the context; the results returned from this specific tool were just too much. So we managed to really optimize bit by bit and come up with Investigations V2, which we just released. It’s a massive improvement compared to our V1 approach.

Nicole: So is it comparing just different models for investigations, or is it also comparing things like Claude Code and the Grafana MCP versus Assistant?

Will: Both, actually. We have different versions of our harness for investigations that we run with Sonnet and with Opus, but we also use the Claude Code harness with the Grafana MCP — and even just using curl to directly hit our public API.

Nicole: Do the models ever cheat?

Will: So far it’s been quite good. We have guardrails. Because we investigate past incidents, we need to make sure the LLM only looks at a specific time window — only from when the incident started — otherwise it would be able to find the pull request that was used to solve the incident. But they’re pretty good at following the rules; they take it as a game a little bit. We even have an agent that checks afterward, with the judge, to make sure they followed every rule.

It’s funny, actually — we had a case using Claude Code where it started to use some tools we hadn’t thought about in the first place, and they turned out to be very useful for investigation. We’d never considered them, so we brought them into our investigation harness and it helped a lot. Giving the agents freedom and creativity in this kind of environment is really useful. That diversity sponsors new ideas that we can put into our own work.

Nicole: Cool. Do you have any plans to publish this and share it so that other people can do it too — kind of like OliBench?

Will: Yeah, I think, why not? I think we can go this way too. I’m sure it would be useful for other agents. And it would be very nice if we could compare agents from other people and other companies — maybe with different harnesses, maybe even different tools — because right now it’s mostly about our MCP, our CLI, and our investigation harness. But if the community comes up with their own investigation agents — and I think I’ve seen a few — it would be fun to put them in the arena and help not just our product, but any product improve in quality.

Resources

My notes on Grafana Assistant

Asimov's Zeroth Law of Robotics: Observability for AI (ExpoQA Madrid 2026)

Tue, 26 May 2026 10:00:00 +0200

This week I gave the opening keynote at ExpoQA Madrid 2026 in Madrid, Spain. I presented a talk called Asimov’s Zeroth Law of Robotics: Observability for AI, which I had previously given at KubeCon EU 2025 in London, at Dutch Cloud Native Day 2025 in Utrecht, and at Newcrafts in Paris. Here’s the abstract:

A robot may not harm humans. A robot must obey humans. A robot must protect its own existence. These are Isaac Asimov’s three Laws of Robotics, created to govern the ethical programming of artificial intelligences. From the Butlerian Jihad to Skynet to cylons, we’ve been immortalizing our collective nightmares about artificial intelligence for years. But there’s an unmentioned law that comes as a prerequisite to all of that: a robot must be observable.

In this talk, I discuss the different types of AI, the factors that make observing AI different from observing applications, and the telemetry signals specific to AI that we might want to listen to. How do we deal with large data sets? How do we observe for model drift? How do we take into account the costs of LLMs? How can we use distributed tracing to follow event sequences? Part cautionary tale and part technical demo, this talk shows how to instrument and monitor AI apps using OpenTelemetry, Prometheus, OpenLit, and more.

This was a keynote slot at a testing-focused audience, which gave me the chance to lean more into the evals vs. observability framing — what evals can and can’t tell you about an AI system in production, and how distributed tracing fills the gap that benchmarks leave behind. The demo app (a Flask-based “play.py” instrumented with OpenTelemetry, flowing into Tempo for traces, Prometheus for metrics, and Loki for logs) is still on GitHub, and I’ve been refining it with each iteration.

Here’s the recording of the talk:

Resources

Demo app on GitHub

Asimov's Zeroth Law of Robotics: Observability for AI (Newcrafts 2025)

Thu, 06 Nov 2025 12:23:19 +0100

Today I presented at Newcrafts in Paris, France. I gave a talk I’ve given before, called Asimov’s Zeroth Law of Robotics: Observability for AI, which I had previously presented at KubeCon EU in London and at Dutch Cloud Native Day 2025 in Utrecht. I was happy to present it again, as it gave me the chance to further refine my demo application, which you can find on GitHub.

The main change I made was to use the “AI as judge” approach that I had previously only talked about in theory. In this version of the demo app, I use AI in both the game as well as in the testing of the game, showing how AI can be integrated into both the application and its observability/testing stack. It was really interesting to note that testing with AI ended up exposing more defects than my hardcoded test cases did, which goes to show the potential of AI-driven testing.

Asimov's Zeroth Law of Robotics (Dutch Cloud Native Day 2025)

Thu, 03 Jul 2025 15:01:24 +0200

Today I presented at the Dutch Cloud Native Day 2025 in Utrecht, the Netherlands. I gave a talk titled Asimov’s Zeroth Law of Robotics: Observability for AI, which I had previously presented at KubeCon EU earlier on in the year, in London. I was very happy to be able to present it again, because it gave me the opportunity to improve the demo app I used, which you can find on GitHub.

Asimov's Zeroth Law of Robotics: Observability for AI (KubeCon EU 2025)

Wed, 02 Apr 2025 13:46:42 +0200

Today I gave a talk at KubeCon EU 2025 in London called “Asmiov’s Zeroth Law of Robotics: Observability for AI”. Here’s the abstract:

A robot may not harm humans. A robot must obey humans. A robot must protect its own existence. These are Isaac Asimov’s three Laws of Robotics, created to govern the ethical programming of artificial intelligences. From the Butlerian Jihad to Skynet to cylons, we’ve been immortalizing our collective nightmares about artificial intelligence for years. But there’s an unmentioned law that comes as a prerequisite to all of that: a robot must be observable.

In this talk, I discuss the different types of AI, the factors that make observing AI different from observing applications, and the telemetry signals specific to AI that we might want to listen to. How do we deal with large data sets? How do we observe for model drift? How do we take into account the costs of LLMs? How can we use distributed tracing to follow event sequences? Part cautionary tale and part technical demo, this talk shows how to instrument and monitor AI apps using OpenTelemetry, Prometheus, OpenLit, and more.

Cartography of the Deep: Distributed Tracing for Testing (Romanian Testing Conference)

Thu, 13 Jun 2024 12:24:02 -0700

I presented a talk today at the Romanian Testing Conference 2024 in Cluj-Napoca, Romania! My talk was called “Cartography of the Deep: Distributed Tracing for Testing” and it covered how to tie in distributed tracing into tests to get a better understanding of your system under test.

Here are my slides!

Abstract

From the Scandinavian Kraken that drags entire ships to their early graves to the Philippine Bakunawa said to rise from the waters to swallow the moon, our existential fears about what lurks at the bottom of our own oceans has inspired us to create hundreds of folkloric monsters across cultures. We’ve charted the surface of our planet, but the vast majority of the ocean floor remains a mystery.

The mysterious depths of modern software architectures makes it difficult for us to test a system for performance and reliability. The easy solution? Unmanned underwater probes: send inputs to the system, and judge its state based on the outputs that return. But this method is often too limited to tell us what’s really going on when something goes wrong. What if there were a way to make our tests themselves more observable?

In this talk, you will learn how to use open-source tools to instrument performance tests, identify bottlenecks at the code level using distributed tracing, and map out a system using a type of software bathymetry. You’ll also learn about the Mariana Trench expeditions, the difficulties of operating in a high-pressure underwater environment, and how to come face to face with monsters of our mythologies.

Mental Health as a Superpower with Karen Todd (Adobo & Avocados #28)

Thu, 25 Jan 2024 15:12:44 +0100

My friend Marie Cruz and I interviewed Karen Todd about neurodivergence and neurodiversity, and how her ADHD can be a positive thing in her career as a tester.

Why People Leave Software Testing, with Maria Kedemo (Adobo & Avocados #27)

Thu, 18 Jan 2024 11:54:46 +0100

In this episode of Adobo and Avocados, Maria Kedemo talks to my friend Marie Cruz and me about why she left software testing– and why she came back.

The Power of Community, with Tara Walton (Adobo & Avocados #26)

Wed, 10 Jan 2024 13:48:34 +0100

What does a community builder do? In this episode of Adobo and Avocados, Tara Walton talk to my friend Marie Cruz and me about what community is, its benefits, and ways to misuse it.

Learning about Glue Work and Quiet Quitting (Adobo & Avocados #25)

Wed, 20 Dec 2023 19:36:04 +0100

What is glue work, and how is it linked to quiet quitting and quality engineering? In this episode of Adobo and Avocados, Vernon Richards talks to Marie Cruz and me about what glue work is, the dangers of quiet quitting, its implications with quality engineering, and what we can do about it.

DevOps Culture from a Tester's Perspective (Adobo & Avocados #24)

Fri, 15 Dec 2023 17:50:03 +0100

We’ve all heard of DevOps, but what is DevOps from a tester’s perspective? How can we embrace this culture? In this episode of Adobo and Avocados, the one and only Lisa Crispin talks to my friend Marie Cruz and me about DevOps culture from a testing mindset, what holistic testing is really about, and her journey to being one of the well-respected voices in the testing community.

Performance Testing With Nicole van der Hoeven (Quality Bits Podcast)

Tue, 28 Nov 2023 14:01:35 +0100

My friend Lina Zubyte invited me to her podcast Quality Bits to talk about performance testing. We talked about what performance testing is, why it is important, how to get started, and how to get better at it. We also talked about the foundation of any type of testing: empathy.

Check out the episode here.

Navigating through Unemployment with Renata Andrade (Adobo & Avocados #22)

Tue, 28 Nov 2023 13:16:15 +0100

With the rise of redundancies in the tech industry, how do you navigate through unemployment calmly? In this episode of Adobo and Avocados, Renata Andrade, a Quality Engineer Lead, talks to Marie Cruz and me about her experience navigating unemployment and the steps she followed to bounce back.

Minimalism in Software Testing with Lina Zubyte (Adobo & Avocados #21)

Wed, 15 Nov 2023 13:13:52 +0100

How can we apply minimalism in software testing? Lina Zubyte, an Independent Quality Consultant, talks to Marie Cruz and I about what really matters in testing, why empathy is at the root of the testing attitude, and how we can incorporate principles from minimalism to improve quality.

Learning about experience design with Jenna Charlton (Adobo & Avocados #20)

Wed, 08 Nov 2023 13:01:12 +0100

In this episode of Adobo and Avocados, my friend Marie Cruz and I chat with fellow Developer Advocate Jenna Charlton, where they talk about their career journey, what can we learn from experience design, and the impact of different job titles to testers.

Becoming an accessibility advocate with Erin Crise (Adobo & Avocados #19)

Wed, 01 Nov 2023 12:22:40 +0100

In this episode of Adobo and Avocados, Marie Cruz and I speak with Erin Crise about her tech career journey, what led to her accessibility advocacy, and why empathy drives innovation.

(TL) Ano ang relasyon ng testing at observabilidad?

Wed, 01 Nov 2023 12:19:46 +0100

Ano ang relasyon ng testing sa observabilidad? Sa video na ito, pinag-uusapan namin ng kaibigan kong si Marie Cruz kung ano ang testing, ano ang observabilidad, at bakit dapat nating gamitin pero para pabutihin ang ating mga aplikasyon.

Emergent Load Testing - TestCon Europe 2023

Wed, 25 Oct 2023 20:05:36 +0300

I presented at TestCon Europe 2023 in Vilnius, Lithuania about emergent load testing, and how to use it to find the limits of your system. My talk was called Emergent load testing: Rules for Organized Chaos.

You can find the slides here!

Inspiring others and fuelling your passion with Jonah Andersson (Adobo & Avocados #18)

Tue, 24 Oct 2023 20:00:53 +0300

In this episode of Adobo & Avocados, my friend Marie Cruz and I chat with Jonah Andersson, a Software Developer, Cloud & DevOps Engineer, and the founder of Azure User Group Sweden. We spoke about her career journey and how she is fuelling her passion and inspiring others.

Having a coaching mindset with Kulas Angeles (Adobo & Avocados #17)

Wed, 18 Oct 2023 12:27:16 +0200

In this episode of Adobo and Avocados, my friend Marie Cruz and I chat with Kulas Angeles, a DevOps coach and Quality Engineering leader, about his career journey and how he transitioned from engineering to coaching mindset.

Investing in yourself with Nicola Lindgren (Adobo & Avocados #16)

Wed, 11 Oct 2023 11:52:37 +0200

My friend Marie Cruz and I talked with Nicola Lindgren about different ways we’ve all invested in ourselves and our career– and how you can do the same.

The Role of AI in Gender Equality with Nicola Martin (Adobo & Avocados #15)

Wed, 04 Oct 2023 17:26:14 +0200

In this episode of Adobo and Avocados, my friend Marie Cruz and I chat with Nicola Martin, Head of Quality Engineering at Adarga, to talk about the role of AI, technology, and education in gender equality.

Nicola is a Council and Committee Member for the British Computer Society and WomenTech Global Network. She is also a mentor, speaker, and panelist speaking on subjects such as AI and quality, software testing, and diversity in tech.

The benefits and dangers of public speaking (Adobo & Avocados #14)

Tue, 19 Sep 2023 21:40:43 +0200

In this episode of Adobo and Avocados, my friend Marie Cruz and I chat with Maaret Pyhäjärvi, a Development Manager and Testing Specialist who has shared countless resources about exploratory testing. We talk about her journey to tech, the benefits and dangers of public speaking, and her approach to testing.

Navigating parenthood with Laveena Ramchandani (A&A #11)

Mon, 07 Aug 2023 21:30:33 +0200

My friend Marie Cruz and I speak to Laveena Ramchandani, a Digital and Data Science Test Manager at easyJet and also a Digital Star finalist, speaker, blogger, and coach. Laveena talks about her career journey and her journey to parenthood.

Why monoculture is problematic for quality with Callum Akehurst-Ryan (Adobo & Avocados #9)

Wed, 19 Jul 2023 21:16:32 +0200

My friend Marie Cruz and I speak to Callum-Akehurst Ryan about his experience as an LGBTQIA+ software tester, the importance of diversity and inclusion, and how it can improve testing and quality.

Resources

Marie Cruz and Callum Akehurst-Ryan’s joint presentation on Testing in Story Shaping

Quality Engineering, DevOps, and work-life balance (Adobo & Avocados #07)

Thu, 06 Jul 2023 21:55:46 +0100

Developer Advocates Marie Cruz and I talk to Katja Obring, an experienced Quality Engineering Coach, about her experience in tech and moving to freelance for better work-life balance. We also talk about her book, “Testing in DevOps: A Guide to Optimised Testing for Speedy and High Quality Delivery,” and how Testing fits in the overall DevOps landscape, the importance of automated testing, and the cultural changes crucial to DevOps.

Why Representation Matters (Adobo & Avocados #06)

Sat, 01 Jul 2023 15:08:51 +0100

My friend Marie Cruz and I talk to Parveen Khan about her experience in tech and the importance of diversity and inclusion. Parveen is a Senior QA Consultant at Thoughtworks, a quality advocate, and an international speaker.

Do Appearances Matter in Tech (Adobo & Avocados #05)

Fri, 23 Jun 2023 15:03:33 +0100

My fellow Developer Advocate Marie Cruz and I talk to Imma Valls, a community support engineer who works with us at Grafana Labs, about image biases that people might have, whether that’s piercings, tattoos, or hair dye!

10 tips for starting (or progressing) a career in tech (Adobo & Avocados #04)

Wed, 07 Jun 2023 08:00:25 +0200

Marie Cruz and I did another Adobo & Avocados livestream, this time about 10 tips (five from each of us) about starting a career in tech.

Intersectionality in Tech (Adobo & Avocados #03)

Wed, 10 May 2023 21:35:00 +0800

Marie Cruz and I talk about what intersectionality is and how intersecting identities can affect how we relate to the world and our experiences.

Getting started in testing (Adobo & Avocados #02)

Wed, 03 May 2023 21:22:18 +0800

Marie Cruz and I interviewed Rafaela Azevedo about her experiences in getting started in testing, what it’s like to be a woman in tech, cultural differences between Brazil and the UK, and how she became a founder of a new Blockchain startup.

How we got into tech (Adobo & Avocados #01)

Wed, 29 Mar 2023 14:54:43 +0100

My friend Marie Cruz and I have been wanting to talk about a few topics related to diversity in tech, and over the few years we’ve known each other, we’ve only added to that list. We finally decided to take a leap and start a new YouTube channel/video podcast just so we have a space to have those discussions. Here’s the first one, where we talk about how we got into tech.

Below is a transcript of our conversation.

Why “Adobo & Avocados”?

MARIE: All right, we’re live. So, hey everyone, my name is Marie Cruz and with me I have my colleague today.

NICOLE: Hi, I’m Nicole van der Hoeven, I’m her colleague. And I’m a little sick, but I’m here.

MARIE: So we, today we’re actually gonna be doing the first ever recording for the Adobo and Avocados podcast. So we thought about this name and initially like Nicole and I wanted to, you know, ever since before we wanted to launch a podcast where we can talk about, you know, our experiences working in tech and all other things. And then we thought about the name Adobo and Avocado. So you know, to anyone who doesn’t know, so we’re both from the Philippines, and Adobo is the national dish of the Philippines. So that’s why we thought we’ll add the name of Adobo in our podcast title. Avocados, ‘cause we’re both Developer Advocates. But yeah, I think, why don’t we discuss like, why was it called avocados in the first place? Do you have any idea?

NICOLE: Yeah, well, we don’t really know for sure. So I thought that it was something to do with the language of it. Like, you know, an advocate is usually like a, could be like a lawyer in common non-tech senses. And the translation for that is abogado in Spanish or Tagalog actually,

MARIE: Even in Tagalog.

NICOLE: our native language.

MARIE: Yeah.

NICOLE: So it’s like kind of similar to avocado. I don’t know, that’s kind of what I thought.

MARIE: Yeah, that’s actually an interesting thought. Like, ‘cause the abogado does sound very, you know, similar to avocado, but I actually read this one article where I think their project manager had a hard time saying Developer Advocate, they ended up saying developer avocado. So who knows what the origin is. But yeah, I think it’s a great, you know, you know, term in terms of like, you know, the whole play, like playful, you know, meaning behind the word Developer Advocate.

How we got into tech

MARIE: So yeah, I guess first things first, like you know, who we are? Like, why are we doing this podcast? How, you know, we got into tech? So we’ll talk first about, I guess a bit of some introductions about ourselves. So Nicole, why don’t you start, like how did you get started? You know, what’s been your, I guess, background so far? Did you have a traditional, you know, computer science background or did you do like a different, I guess course back when you were still studying?

Nicole’s background

NICOLE: I did not have a traditional background. I grew up in the Philippines. I moved away when I was 18. And I don’t know what your experience was, Marie, but mine was that in the Philippines there are still, I think we do a pretty good job at being equal between genders, but I think there’s still gender roles. And when I was growing up, I really loved both the maths and sciences and like literature and languages, but I was very heavily pushed towards the humanities, ‘cause that’s what girls do. Girls don’t go into tech.

MARIE: Mm-hmm.

NICOLE: And so I really struggled at uni and in college. I shifted majors six times. I had no idea what I wanted to do.

MARIE: Wow.

NICOLE: And I went through, and that was over two countries. I went to the US to study as well. None of them was computer science because my brother was a developer. And I don’t know, he’s a little too introverted for me. And I just had this idea that, if I go into computer science, I have to be a developer, and if I’m a developer, I won’t talk to anyone. I’m just going to be

MARIE: Yes.

NICOLE: with a machine. Like I had that in mind, which now I know is completely wrong,

MARIE: Yeah.

NICOLE: but that’s why I never went into it. So finally my parents were just like, we don’t care what you finish,

MARIE: Yeah.

NICOLE: we don’t care if it’s basket weaving, just pick a major and finish it.

MARIE: Yeah.

NICOLE: And that was economics,

MARIE: Oh.

NICOLE: which I have never used in a job.

MARIE: Yeah.

NICOLE: And then so-

MARIE: That’s really interesting. Yeah.

NICOLE: And then I, how did I get into IT? I was applying for different economics related ones, and then I met my now husband and he,

MARIE: Hi Rob.

NICOLE: he was a test manager. Yeah, hi Rob! He was a test manager and I said, well what’s a test manager? And then he told me about the world of testing.

MARIE: Yeah.

NICOLE: And I didn’t understand it at first, but I wanted to know more. And then I thought the way that he described it sounded so interesting, like the idea that,

MARIE: Mm-hmm.

NICOLE: that there is a job, there is a person whose job it is to be the one using applications or products first. And then trying to see what’s wrong with it for the purposes of improving it. I just, I really loved that idea and it was another branch of tech that I didn’t know existed.

MARIE: Yeah.

NICOLE: So I started to, I did an ISTQB certification just to like have something

MARIE: No background.

NICOLE: ‘cause I didn’t have any background. Yeah. And I started job hunting, and somehow managed to land a performance testing job under someone who became my mentor.

Marie’s background

MARIE: Hmm, nice. Yeah, it’s really relatable because like similar with myself, I fall into tech accidentally. Like it was never my intention, you know, okay, I’m gonna study computer science, although I did actually have a computer science degree.

NICOLE: Oh, wow.

MARIE: But the story, the story behind that, I don’t share it too much to other people ‘cause I think the story behind it is a bit funny. But my first experience with tech is actually updating my MySpace profile. Do you remember MySpace?

NICOLE: Yes.

MARIE: Yes so, before you can customize your profile with a bunch of HTML, CSS codes and like I think before I was just trying to, you know, create something even though it didn’t look very accessible. I remember I had lots of animations, just lots of like

NICOLE: Oh.

MARIE: striking fonts, background music, and it was just something that I was doing for fun. I did a mass communications course back home in the Philippines. So I was majoring in digital media. And then I had to go back here in the UK. So I was working first for about two years before I decided to go back to uni. And during that time when I was looking at, you know, courses catalog, ‘cause I wasn’t really sure what to pick. So I saw a course which basically has three different things. So it’s computer science with business management and financial accounting.

So I thought,

NICOLE: Oh wow, okay.

MARIE: I was like thinking in my head I think that sounds like a really cool name for a course, so I’m gonna go for it. So to me I just found the name like very interesting. And it means that-

NICOLE: So when did you

MARIE: Yeah?

NICOLE: move out of the Philippines?

MARIE: So I, so we went back 2008. So I was in the UK since 2004. Actually no, we went back 2006, and then I went back in the UK 2008. So 2006 I enrolled, you know, to study mass communications. I didn’t finish the course ‘cause I had to go back in the UK after two years because of like visa, like status. And then when I got back, I just put that into a pause, you know, did some work experience. So I was working as an office assistant for about two years and then my friend and I just decided, okay, I think we should go back to uni. So I actually applied as a mature student ‘cause I didn’t, you know, had any like actual qualifications ‘cause I never graduated from my degree in the Philippines. I never finished high school as well, ‘cause I kept going back and forth between the UK and the Philippines. So the last time I actually graduated was during middle school.

NICOLE: Oh, wow.

MARIE: but applying as a mature student, they just looked at my, you know, work credentials, like my work attitude. And I submitted some of my grades like back, you know, when I was still in the Philippines ‘cause like my first two years I had some, you know, grades from like the different subjects that I was doing. But yeah, that’s how I got into tech. I just found the name like really interesting and I thought before I could mix and match it with business and accounting. So even if the computer science bit didn’t work out, like I can still maybe utilize some of the things that I’ve learned on the business side and on the accounting side. And then, when I was about to look for graduate jobs, I got approached by a testing consultancy company. I didn’t, I wasn’t aware that there’s a graduate job for, you know, just doing testing ‘cause most of the graduate jobs that, you know, we were being offered is, you know, how to, you know, do like development, or like database like management roles. So initially I wasn’t really sure if I wanted to do the developer route. Like I enjoyed the coding, I was doing the coding bit, but like back then I wasn’t like fully 100% sure that it was the right path for me. So I found out that, yeah you can have a career in testing. Again this is all accidental ‘cause if that testing company didn’t approach me, I would’ve probably looked at like other roles. So it was like really great to hear that, yeah, there’s like a graduate jobs for just doing testing. And yeah, we were doing the ISTQB as well. So we were doing some training. And yeah, the rest is history. It’s funny because I didn’t intend to continue on a developer route, but then now I work as a Developer Advocate.

How did we become Developer Advocates without being developers?

NICOLE: Yeah, maybe we should talk about that a bit ‘cause I also don’t consider myself as a developer even though I can code.

MARIE: Mm-hmm. Yeah.

NICOLE: And yet somehow we’re both Developer Advocates. How did that happen?

MARIE: Yeah. It’s like there’s some sort of, I guess, interesting link between software testing and developer advocacy. I don’t know like if it’s the same for you, but like with me, I really enjoy the teaching part, the learning and public part as well. And when I was doing my role as a software tester, I felt that I was doing the advocacy, but maybe, you know, not really in a way that it’s very out there. So I was, you know, helping my colleagues, I was talking about tools that I’ve learned, you know, with them, I was showing them how to use it. And then while I was working as well, like I started doing the learning in public on the side. So I started with my personal blog and then getting involved, you know, with meetups to talk about, you know, the different tools that I’ve learned. So I guess for me, like being a software tester also, you know, encouraged me to, I guess be a Developer Advocate ‘cause I was just telling people what I’ve been doing as a software tester. I’ve been telling them what different tools I’ve been using, you know, different concepts that I’ve been learning as well on like a day-to-day. So is that the same for you?

NICOLE: For me it was more because I was a consultant,

MARIE: Mm-hmm.

NICOLE: which means I was going on different projects all the time. And when you’re really new in an industry, I think it’s really exciting and you’re learning a lot. And certainly every project has like its own framework and different languages and different people, which is interesting for a while.

MARIE: Yeah.

NICOLE: But I don’t know, after several years of it, I started to feel like, people are actually asking the same questions. You know, there’s always, there’s some things that I always had to explain. You know, I always had to, for example, fight against the fact that performance testing is often seen as just something where you run the test, okay, you check the box, you’ve done performance testing.

MARIE: Uh-huh.

NICOLE: There’s always a certain amount of education where I’m like, no, it’s not just about checking the box, it can actually provide real value, here’s how.

MARIE: Yeah.

NICOLE: And I just found myself repeating a lot of the same things. And so I actually started to create videos also because load testing results are kind of difficult to explain sometimes.

MARIE: Yeah.

NICOLE: And I would create this like test summary report that lots of people don’t read.

MARIE: Yeah.

NICOLE: And so, as kind of an alternative way to explain things, I started creating video summaries to go along with it. And then I just thought, well maybe I can, ‘cause I can’t publish those, because it’s very specific to that company.

MARIE: Yeah.

NICOLE: But then I, I thought maybe I can pull out, I can abstract away some things and then publish those videos where I’m not talking about a specific project. And I guess that’s kind of how it started. I started, I put it into text first, because that’s just easier than video.

MARIE: Mm-hmm.

NICOLE: And then eventually found myself into video anyway.

MARIE: Yeah.

NICOLE: So I guess for me, I got into Developer Advocacy because I was tired of, I was trying to automate a part of my life, the explanation part where I kept having to say the same things and I just wanted to be like, hey, read this blog post where I explain why you need think time in a much more coherent and considerate way.

MARIE: Yeah.

NICOLE: So that’s, that’s how it naturally happened. I think you and I are the same in that regard that we were already doing it before we got the title.

MARIE: Yeah. I totally relate to that. Like, especially when you said that like, we’re just trying to solve the same problems. I started, you know, my blogging journey just trying to communicate like, you know, questions that people have been asking me on a day-to-day basis. And rather than just, you know, speaking to individual people and repeating the same things like again and again, like just put everything into a post or like you said like a video, and then just share that, you know, like to the public, ‘cause you’re not only gonna be able to help like one person but you know, maybe if someone else stumbled upon your blog or your video, then like, you might help them as well. I think like one of the, like, one of the best reward that I experienced when doing this like learning in public and you know, sharing more content out there is if you just get a message from someone or like a comment and then they say that, oh, like, I’ve been stuck with the same issue, I found your blog post or your video and it really helped me. And just that feeling that you know, you’ve helped someone is like a really great feeling. ‘Cause at first I wanted the blog as well just so that I can have something that I can look back to in case I, you know, forget like what I’ve done on a previous project. Like I can document it on a blog post, but then like now it’s also helping like other people who are experiencing the same like issues at work.

NICOLE: Yeah, I definitely see that. At first I was doing these things for myself because you know, you don’t really know that you know a subject until you can teach it to some degree. And I really believe that. And I, for me there were so many things ‘cause as a consultant, you have to switch tools very quickly, you have to switch languages, you have to be a polyglot on a lot of different levels and it’s impossible to keep all that in your mind. So after every project I tried to like, distill my learnings from it, so that I,

MARIE: Yeah.

NICOLE: future me would come back to that framework, or tool, or whatever, and not be starting from zero. But yeah,

MARIE: Yeah.

NICOLE: the kind of other people finding it was an unexpected boon.

MARIE: Yeah. So let’s just move on a bit. So we’ve talked about, you know, how we transition from software testing to developer advocacy. But I guess in terms of like the process, like did you apply to a lot of, you know, companies? Was developer advocacy something that, you know, you were aware of already, or like you just found a job description and then you’re like, oh, this is what I do already. Okay, let me apply to that job. So what’s been like the process for you?

NICOLE: So I was, this is only, this is my first job as an actual, like the first job where I have Developer Advocate in my title.

MARIE: Yeah.

NICOLE: My previous job was with a, well at the time it was a startup, it was Flood.io, which is now acquired by Tricentis.

MARIE: Yeah.

NICOLE: And when I joined, I think I was like, I think the fifth person that they hired in the team.

MARIE: Oh, wow.

NICOLE: So it was very small. And it was, I really love that environment because you can do a lot of things, you can wear multiple hats. And that’s really where it started. On a typical day, I might be answering questions in support, or presenting at a conference or ducking into a sales call or like,

MARIE: Yeah.

NICOLE: there were so many things that I could do. I was like part of the rotation for things going down basically, for things exploding. And I had to learn a lot about infrastructure as well to be competent in that role. And I started doing a lot of advocacy type things, mainly from a support perspective

MARIE: Yeah.

NICOLE: because I got the same questions there as well. And I wanted to be able to focus on the crunchier topics.

MARIE: Yeah.

NICOLE: And you can’t do that unless you address some of the more basic or common questions. And then my manager at the time told me about Developer Advocacy

MARIE: Yeah.

NICOLE: and my immediate response was, but I’m not a developer.

MARIE: Yeah.

NICOLE: He said that that’s just the name for the whole industry that I think you would be really good at and that you’re already doing. And with his encouragement, this is Tim Koopmans, the one of the co-founders of Flood, with his encouragement, strong encouragement by the way, things like, he said, oh, I’ve got this conference in Moscow that I’m speaking at. Do you wanna come watch me?

MARIE: Right.

NICOLE: And I said yes. And he backed out so I had to do it, things like that. Because of things like that that he did, I started to understand what developer advocacy is

MARIE: Mm-hmm.

NICOLE: and when it was, when I felt the time to change directions, I specifically looked at k6 actually. I didn’t apply to any other jobs because I was pretty happy where I was.

MARIE: Mm-hmm.

NICOLE: But I really loved k6. I had already not used it in a project, but I’d already, I knew what the tool was and I used it, I tested it a bit. I love the site, I love the principles behind it. And I saw that they had a Developer Advocate role. So,

MARIE: Yeah.

NICOLE: It just kind of happened. How about you?

MARIE: Well the first time I heard about the term was from you, when you said that, hey we’ve got this role for, you know like, you know like, we’ve got a Developer Advocate role for one of the tools that, you know, we’re gonna launch. Like, you’ll be like the great fit. So I didn’t really send any CV. I don’t think I sent any CV or like, yeah, I think, you just like, ‘cause we just connected already like via this Automation Guild conference. So like we were already connected via Twitter. And I think this is where like one of the benefits of like learning in public really came into play, ‘cause I was just, you know, publishing constantly. I was, you know, talking at various meetups and conferences already. But I was just doing all of this like as part of my, you know, side projects. Like it wasn’t part of my day-to-day role on my previous job. So when you explain to me what a Developer Advocate is, I’m like, oh yeah, I’ve been doing that on the side. So that was my first actual, you know, experience of like hearing what a Developer Advocate, you know, does on a day-to-day. ‘Cause I had the same perception that I’m not a developer, how can I be a Developer Advocate? But I think because as like as part of our day-to-day jobs, like even if we’re not writing application code, we’re still writing, you know, test scripts. we’re creating test automation frameworks. So I think that’s really helped as well transitioning into this role. But yeah, like I never sent any CV. I just remembered like we were talking a lot like back and forth and then I met up with, you know, a bunch of our colleagues now. And yeah, the rest is like history again.

NICOLE: Yeah, I think that there are just some people that when you meet them, you click or you don’t.

MARIE: Mm-hmm.

NICOLE: And when that click happens, I’m just like, okay, I’m gonna write her name down. I think I told you about it like for two years,

MARIE: Yeah, I just joined-

NICOLE: we were talking, and I was like, hey, this could happen…

MARIE: Yeah.

NICOLE: Yeah, I don’t know when. I think you’d be perfect for it. And you’re like, okay,

MARIE: Yeah.

NICOLE: but it’s kind of early to be telling me, right?

MARIE: Yeah.

NICOLE: I was like, Yeah, yeah, but let’s just keep in touch. You know, and it worked out. I can’t believe it actually did.

MARIE: Yeah. And now we’re, doing.

NICOLE: We didn’t actually-

MARIE: Yeah, go on Nicole.

NICOLE: We haven’t actually defined what a Developer Advocate is.

What’s a Developer Advocate?

MARIE: Yeah, I guess like we can both, you know, share what our, you know, view is of a Developer Advocate. So to me when it comes to developers, I guess trying out, you know, new tools or learning about new things, they feel most comfortable talking to someone who’s actually used, you know, the product, or you know, the tool that they want to use. So, we are, you know, that person. So we’re helping developers, we’re helping testers when it comes to, I guess learning about best recommended practices, learning about different tools. So we’re kind of like a bridge between the community and like the company that we work for. There’s a lot of misconception that people think that we’re salespeople but we’re actually not, like it’s a completely different, you know, role. We are focused more on the community side, helping them grow, helping them, you know, I guess in some ways, like be some sort of champions as well, so they can also promote like, you know, the different tools, the different practices that we have been communicating to them. But yeah, what about you? Is that like sort of the same concept?

NICOLE: Yeah, I agree with everything you said, especially about being a bridge.

MARIE: Mm-hmm.

NICOLE: Because as a Developer Advocate you’re not quite a developer, that’s not our primary responsibility,

MARIE: Yeah.

NICOLE: but we’re also not like marketers or sales people. Although there’s an overlap there in both. I kind of think of it as anti-marketing. So a marketer is going to be talking about something that they themselves have never used, are not the target user for.

MARIE: Yeah.

NICOLE: And might not know much about in terms of the technical details.

MARIE: Yeah.

NICOLE: A Developer Advocate is someone who actually is the target user. So for those who don’t know, Marie and I work at Grafana Labs specifically on k6, which is a performance testing tool. But we have both used k6, we have both been testers and still are. And so there are things that we can suggest. Part of it is advocating internally.

MARIE: Mm-hmm.

NICOLE: We advocate for testers everywhere to the product team so that we say, like these are the features that we would be looking for as testers. Like we would say it this way or we wouldn’t really use that feature, we want this other one. But another part of it is advocating externally. So this is the part that people think of as marketing, but I think of it as anti-marketing, because we’re not going out there and saying, hey, you should use this product that we’ve never used before.

MARIE: Yeah.

NICOLE: Instead, we build cool things and demonstrate cool things that we have done, not some other developer.

MARIE: Yeah.

NICOLE: Like we do our demos,

MARIE: Yeah.

NICOLE: And then when people ask us, then we say, oh yeah, we use k6. So I don’t know, marketing definitely has its place, but for me I am so addicted to that authenticity and genuineness that comes from just talking about a tool that you actually love, which is,

MARIE: Yeah.

NICOLE: what k6 is for me.

MARIE: Yeah. I really like that because for me, an important part of, you know, this advocacy is, like communicating something that I truly believe in. Like I’m not going to advocate for, you know, a tool that I haven’t used. Like I would just feel like, you know, I’m a fraud if I do that. So if I know that, you know, there are situations where a certain tool or a certain practice, you know, would work better, then I would advocate for, you know, that as well. So it’s not just about, you know, for example, like at the moment we can’t do k6 for accessibility and I’m an accessibility advocate as well. So I would go and, you know, recommend like other tools and practices based on what like I’ve experienced, based on what I’ve done. And I think, like for me, that’s one of the biggest like advantage, ‘cause yeah, if I’ve been put on the spot and say, hey, you should talk about this to then promote it to other people, that would just totally go against my, like one of my beliefs.

NICOLE: I think I also really appreciate that this position lets us shine a light on people like us who maybe,

MARIE: Mm-hmm.

NICOLE: who don’t look like the average person in tech. I know it’s still, I’ve been lucky to work with teams of mainly men that haven’t treated me any differently because I’m a woman, but there is still like a sense of being Other.

MARIE: Yeah And I guess that-

NICOLE: And being in a position of advocacy means that we’re, we’re also able to appeal to other women who maybe didn’t, who didn’t have as good an experience as we did.

What can you expect from Adobo & Avocados?

MARIE: Yeah. And I guess that really leads to this next question. And you know, people who will watch this in the future might also ask like, what are the expectations like for this podcast? Like why are we doing this? So do you want to maybe start.

NICOLE: Why ARE we doing this Marie?

MARIE: Yeah. Oh, yeah. So do you wanna start like from your side, like what are the expectations that you know, that we can, yeah, share to people?

NICOLE: Sure. I think that you and I both do a lot of technical advocacy. We, you know, do demos, we write scripts, and we tell people how to do things in tech, very technical things. But, I think that there’s a whole other side of tech that doesn’t get talked about and that’s like, things like interpersonal dynamics or cultural differences. You and I are both Europeans that have to work in, in our situation, predominantly American space, and we’re also Asian. So balancing the two, you know, we are on different sides of the spectrum on that. Just also the general stress that comes with tech. It is not easy, not just as a Developer Advocate, but just to be in tech to keep up with changing trends.

MARIE: Mm-hmm.

NICOLE: I feel like there are so many emotional aspects and psychological aspects that don’t get talked about that I would really like to cover.

MARIE: Yeah.

NICOLE: Then also, I kind of alluded to it. I would like to make it a little bit more open, an open discussion for particularly underrepresented people

MARIE: Mm-hmm.

NICOLE: like ourselves. We identify with several minority groups.

MARIE: Yeah.

NICOLE: And maybe help them along a bit by telling them things that we had to find out the hard way early on in our career. How about you?

MARIE: Yeah. Yeah, I agree with all of those. And I guess it’s also to just have a safe space, you know, for everyone just to share their experiences in tech. Like you said, there’s this whole other side of tech that we don’t talk about that maybe other women are also uncomfortable to talk about, you know, with like, I guess other colleagues, especially if it’s just predominantly male. So we want to, you know, have a safe space for them, share, you know, I guess from their side, like their unique experiences as well. And maybe like we can all have, I mean maybe we also have those shared experiences that we don’t talk about, but then when someone says, oh yeah, I’ve experienced that too. So it’s just really finding, you know, shared experiences that we can all relate to. And like you said, just helping them, you know, or empower them to, I guess move forward in terms of like the next stage in their career. Like, you know, share what’s worked for us, share what hasn’t worked for us. ‘cause I think it’s all about like being authentic as well and not just talking about the good things, but we could also talk about the bad and the negative things as well.

NICOLE: I’m excited to see where we go from here.

MARIE: Yeah.

NICOLE: I think we have a lot of ideas about where we’re going to take this and I hope people will come along for the ride.

MARIE: Yeah. So, yep. I guess this is it for the first episode. We are just, you know, extremely excited to do this on a regular basis. So we’ll talk about different, you know, topics in every episode. And we might, you know, have like really awesome, like guests as well in future episodes. So yeah, I guess just stay tuned and yeah, look out for Adobo and Avocados.

MARIE: Thanks everyone. See you next time.

NICOLE: Bye-bye.

Frontend vs Backend Performance Testing

Wed, 01 Feb 2023 22:59:18 +0000

Which should you do: frontend vs. backend performance testing? The short answer is both. Each has its pros and cons, but hybrid load testing combines them and gives you the most holistic view of performance, including frontend concerns like how quickly a page renders or perceived application performance as well as backend concerns like response times of individual components.

Introduction to Performance Testing

Wed, 25 Jan 2023 22:28:43 +0000

In this introduction to performance testing, we talk about how performance is part of the functionality of an application, why you should do performance testing, and some myths about performance that discourage teams from testing for it and make applications more likely to fail and fail miserably.

How to Be a Gish - API, browser, and chaos testing in one script (ExpoQA 2022)

Wed, 01 Jun 2022 14:06:49 +0200

For the ExpoQA 2022 conference in Madrid, Spain, I presented a talk about how to be a gish. In this talk, I covered the following topics:

what a gish is, and how it applies to the field of testing
how testing is a multidisciplinary field, and why generalists triumph over specialists in the face of uncertainty
how the tool we choose to do our testing informs the type of testing and the type of results we get
a hands-on example of how to do API load testing, browser automation, and chaos engineering in one script using a tool like k6

There was no recording of the conference talk, but here is a video of me going through my presentation beforehand:

You can find the slides here.

And here are some photos from the event!

Real Browser Testing With xk6-browser

Wed, 16 Mar 2022 16:55:56 +0100

Transcript:

Without a doubt, my favorite k6 extension is xk6-browser. xk6-browser finally brings browser test automation to k6. So you don’t need to learn a new tool or figure out how to integrate it if you already have load tests. You can create JavaScript-based, browser-driven tests with k6. Here’s how.

Since xk6-browser is an extension, we’re gonna need to build a custom version of k6 that includes that extension. And the thing that builds that is called xk6. What I’m doing here is I’m instructing xk6 to create a build using version 0.35 of k6, that includes xk6-browser. And this is a link to the repository where xk6-browser is stored.

Writing a browser testing script

So now that we have a version of k6 that includes xk6-browser, the next step is to create the browser script. Now you can do that in the one script, but just for ease of debugging I’m gonna put it in another script for now. We’re not going to be using the HTTP requests anymore so I’m going to import launcher from k6/x/browser. And I’ll save that as browser.

I’m definitely going to be using sleep, so let me go ahead and import that now as well. We’ll also need a default function for this. We’re setting up an instance of the launcher that’s going to launch Chromium in particular, and we do want to see it so I’m going to say headless: false. If you set this to true, that means that you won’t actually see the web browser being driven by the script, which is handy to see during demos. So we’ll do that, and then we have to set up the context.

So browser, oops, new context. And then we set up a page. The first one is go to the homepage. The second one is to view a product. Third, add the product to the cart. Fourth is go to the cart and then last, I want to check out. So to do that, I’ve already defined the browser context in page. Now I need to do page.goto http://ecommerce.k6.io and I will also want to wait for waitUntil: networkidle.

Great. I wanted to add some sort of check to make sure that I’ve actually gotten to a place so, let’s open up DevTools here and I wanna make sure that some results are displayed. I don’t necessarily need, you know 1-12 of 17 but, I want this one. Okay, so I’ll also say page.waitForSelector and that’ll be this.

And then let’s do a screenshot, because we just for, just in case we wanna debug later we wanna make sure that we will be able to have a look at at this screenshot to see what exactly went wrong. And I’ll put that in screenshots and I’ll say 01_homepage. Then let’s do a sleep. And I always like to do the random one. Random, and let’s say it from zero to four. This time I’m doing the custom k6 version browser.js.

For browser-based tests, I find it much more useful to actually see what’s going on rather than to look at the output. Because as you can see there are all sorts of errors here and I believe that they’re actually application errors so they can be a little bit misleading. It looks like that happened but let’s just look at the screenshots. Screenshot, yeah. That, that loaded fine. So we can continue.

Next step is to view a product. Okay, let’s go back here. This time I want to click this thing. It doesn’t really matter which, which item it is. I’ll just choose the first one. Looks like this is the actual product link. So let element = page. And this is where we actually paste the selector which is what I got from DevTools.

And then we haven’t done anything yet with it. We want to actually click it. So element.click. After that though, I’m going to want to make sure that we get to the right page. So, let’s see what happens when we click it. What can we look for? I guess we can look for this add to cart button, cuz if the item didn’t show up then we wouldn’t be able to add it to the cart. So let’s grab that. Button, button name is add to cart. Copy that.

So then I’ll say page.waitForSelector, the selector with a button name there. Take another screenshot because why not? I’ll say 02_view-product and then another sleep. From here it was pretty much more of the same. Here’s me scripting adding the product to the cart, viewing the cart and then finally going to the checkout page. And here’s what happened when I played it all back. Here it is going to the homepage, and then clicking on the first product, and then adding that product to the cart, and then clicking view cart, and then proceeding to checkout.

The best part about this is that you don’t have to choose between the browser test and the protocol test. They’re both in k6 and written in JavaScript so you could include them in the same script. You can use stages to schedule when one starts and when one ends. You can change the load profile, add checks and thresholds. And you can also use all of the output options that are available for protocol tests for the browser tests as well.

The catch is that xk6-browser is really new. We’ve just announced it. It’s still in beta so you are likely to come across more than a few bugs. However, I’d still recommend that you give it a go. I’ve been playing around with it and I’ve found it more than serviceable for a lot of my use cases. Plus it’s just really cool to be able to test front-end performance along with back-end performance in the same testing script.

Thanks for watching. I will put a link to the xk6-browser repository below and happy testing.

Private Load Zones for Load Testing (k6 Office Hours #44)

Fri, 04 Mar 2022 10:01:57 +0100

Load testing on the cloud definitely has its advantages, but what if you need to load test behind a firewall? Here to talk about using private load zones in k6 are k6 CTO Pawel Suwala and k6 Backend Lead Samuel Regandell. Private load zones are a new feature of k6 Cloud that will allow you to run load tests against applications that are not publicly accessible and also use your own AWS account to generate the load.

Basics of Chaos Engineering for Testers

Wed, 02 Mar 2022 09:59:52 +0100

What is chaos engineering? And what does it mean for testers? In this video, I go through the basics of chaos engineering from the perspective of a software tester. How can we go about incorporating principles of chaos into our testing activities?

Señor Performo's Week of Load Testing (k6 Office Hours #39)

Fri, 28 Jan 2022 23:27:47 +0100

This week for k6 Office Hours, I talked to one of the newest members on our team, Leandro Melendez, aka Señor Performo, about his experience in joining k6/Grafana Labs. We ended up talking about performance “deadly sins” and how the practice of performance engineering has changed in the last decade.

Why All Roads Lead to Testing

Tue, 25 Jan 2022 23:59:03 +0100

Why all roads lead to testing

Testing is a skill every software engineer should develop.

It’s easy to dismiss testing as a responsibility only some carry, but it’s best thought of as a hat that anyone can put on. How well you learn to wear this hat affects your career whether or not you’re a tester. Here’s why.

Ideas are cheap.

In tech, having an idea isn’t what sells– the success is in the execution.

iPhones didn’t become popular because it was the first mobile with a touchscreen. The Nintendo Switch isn’t ubiquitous because of its revolutionary hardware. Behind every successful product is a lot of testing.

Testing isn’t just for testers.

Every good software engineer does testing.

Debugging is a form of testing. Questioning requirements is a form of testing. Testers don’t (and shouldn’t) have a monopoly on testing. The reality is that everyone is doing it, often without calling it “testing”.

Testing is about people more than it is about machines.

The fundamental goal of any app is to address a human need.

Testing is finding new ways to represent and understand this need at all stages of development. Testing is a people problem, not a technical problem.

Humans are irrational.

The biggest challenge in building apps is human irrationality.

We don’t always want what we need, we don’t always know what we need, and even when we’re building for others, we don’t always work towards what we need. Testing is an attempt to make sense of this irrationality.

Testing is an attitude.

Getting good at testing extends to aspects beyond work.

Testing is a discipline of questioning. It’s having curiosity as a default state and trying to meet people where they are. These are things every project, and every community, needs in spades.

Testing is rarely the destination.

But some time, some how, all roads lead to testing.

How to Actually Remember What You Learn, for Engineers with Long Reading Lists

Mon, 24 Jan 2022 21:47:59 +0100

Who says you need the terminology of the Zettelkasten process? Here’s how I’ve learned how to learn and how to apply new ideas to my daily work.

If you’re reading this, you probably make your living on what you’ve learned that other people haven’t. Unfortunately, if you don’t have a written system to keep track of new ideas, you’ll probably unlearn everything in a few years.

Most people think reading = learning.

Here’s why reading isn’t enough.

Our brains are best at linking concepts, not at remembering them.
Writing doesn’t help us learn; it IS how we learn.
You can’t learn to swim by reading about it.

If reading isn’t enough, then what should you do instead?

Step 1: Consume

The first step is to consume content, but selectively so.

Content could be a book, a video, a podcast, an article, or even a conversation. Take notes on the author’s main points.

Example: Shape Up is a methodology emphasising iterative cycles, “betting” on work to be done, and building fully integrated slices of work.

Step 2: Redefine

Next, go beyond facts and add your thoughts.

Think about the broader picture of what the author might be saying, and supplement it with your opinions. Example: Shape Up is an Agile methodology concerned with the limits we should apply to different stages of work.

Step 3: Contextualize

Think about what the content means to you, specifically– how does it stack with other things you’ve learned? Where does it fit into your interests? Accept that your context, and your notes, will change.

Example: Shape Up’s idea of fully integrated work supports the notion of context-switching (see: Continuous integration, Multithreading) but could be in contention with productivity concepts like deep work (David Allen), batching, and Productivity Momentum.

It’s your brain!

In the end, nobody can tell you the “right” way to learn.

How to Do What Kind of Testing When: Testing Types by Scope

Sat, 22 Jan 2022 16:42:49 +0100

Correctly identifying when you test could be just as important as what. Below are types of testing according to scope.

Static testing

Static testing occurs when requirements are written. Question requirements and spell out what each one means.

Concerns: Setting up a system to track and link requirements to test cases, negotiating business needs with technical feasibility, clarifying scope.

Unit testing

Unit testing occurs when coding begins. Practice Test-Driven Development by translating requirements into tests before writing the code.

Concerns: Treating test code like application code, getting developer buy-in, communicating technical implementation, performance, pair programming.

System testing

System testing occurs when a build is released. Employ black-box testing: step back from the code and assess the feature as a whole for consistency.

Concerns: Adherence to common design standards, equivalence partitioning, boundary value analysis, fighting scope creep, usability, creating test data.

System integration testing (SIT)

SIT occurs when integrating multiple components. Use white-box or grey-box testing to target the interactions and traffic between components.

Concerns: API testing, stubbing and mocking, identifying request paths throughout the system, performance benchmarking, observability and analysis.

Business acceptance testing (BAT)

BAT occurs when the code is assessed against overall business goals. Seek out SMEs who can help you verify expected outcomes.

Concerns: Adherence to business requirements and regulations, process testing, simplifying request flows, identifying potential customer pain points.

User acceptance testing (UAT)

User acceptance testing occurs before code is released to production. Map test cases to requirements and test real-world scenarios.

Concerns: Testing for superuser workflows, data cleansing, verifying real user behavior from historical logs, exploratory testing, end-to-end performance.

Regression Testing

Regression testing occurs between builds. Supplement a robust automation test suite with nuanced manual test cases that address known problem areas.

Concerns: Setting up a CI/CD pipeline, deciding what to automate, consolidating test and development tools, tracking failures over time.

Question everything

Remember: testing is an attitude.

An Overview of Software Testing

Wed, 19 Jan 2022 17:10:11 +0100

Transcript

NICOLE: What is the difference between good software and bad software? Is it features or user interface or marketing? In my totally biased opinion, what actually makes a difference is how well the software was tested. But software testing encompasses a lot of different areas and it doesn’t even always have to be someone with a word tester in their job title that does the testing. So whether you’re looking to get started in the industry, or just trying to cross skill, here’s an overview of the main types of software testing.

Is it just me, or does your desk get that clutter too after a week of work?

Anyway, software testing is the process of validating whether an application works as intended, but there’s a lot of vagueness in that statement. One of the broadest ways to categorize testing types though, is whether it’s functional or operational.

Functional testing is more about application features. Is there a form to capture user email addresses? Is there a button next to it that submits those addresses? Once the button is clicked, where do the user details go? Do they go to the database? Are they stored correctly? Those are the sort of ways in which functional testers think. Operational testing traditionally called non-functional testing, is a way of verifying software quality and aspects that extend beyond the features of the application.

Operational testing sometimes involves concerns that are more subjective or qualitative. So it can be a little bit more difficult to test, because those things only arise in certain conditions. I’m saying operational testing here, instead of non-functional testing, because I was talking to Paul Bruce who made an excellent video just on this topic. He talks about why the term nonfunctional requirement is really a bit of a misnomer.

PAUL: It relegates all these other things, security, performance, installation, deployment, accessibility. It relegates them to a nice to have, so it should be functional versus operational.

NICOLE: So I’m using his term operational testing instead.

Here are the different forms of operational or non-functional testing.

First there’s accessibility testing. It is involved with verifying that end users can consume or navigate content. Usually through assistive devices like screen readers. While most forms of testing begin with a statistically normal user in mind, accessibility testing seeks out outliers in terms of location, linguistic background and disability.

Usability testing is designing with user experience in mind, specifically concerning the placement of on-screen elements and the intuitiveness of the process flow of an application.

Localization testing is a way of making sure that applications are usable in the local context, in the sense of being colloquially appropriate for users in different countries and from different cultures.

Security testing verifies that a system has no flaws that could be exploited, to gain access to confidential information or features that would not otherwise be accessed by expected means.

So that was me talking from my home in the Netherlands, but this week I am in Portugal. The remote working life, am I right?

But anyway, performance testing is my favorite topic and my favorite type of testing because that’s the one that I’ve had the most experience with. Performance testing is more qualitative. So it’s more about how well the application works. And there are two main types of performance testing and that’s front-end performance, and back-end performance testing.

Front-end performance testing seeks to verify the user experience at the interface level. So that’s everything that the end user sees on their local machine.

There are different approaches to front-end performance testing, and the first of them is performance profiling. Performance profiling involves running your application through a series of automated checks. And there are a lot of sites for this, where you just put in your URL, if it’s a web app, and you get a score in the end. That tells you how performance your application is, the most popular example for this and also the easiest to get started with, is DevTools because that’s already in Chrome and Firefox based browsers. But you can also go to sites like Webpagetest, or GTMetrix, YSlow, which is a plugin, or you can use Google Lighthouse, which is now included with Chrome Dev Tools.

The next approach to front-end performance testing is called RUM or Real User Monitoring. So the difference is that with performance profiling, that’s often done manually, and as a one-off thing, like maybe at the beginning of a test or during, but Real User Monitoring is not usually done within testing environments. As the name suggests, these tools measure the experience of real users. So they have to be run in production. Examples of tools in this space are; New Relic’s RUM, Elastic has a RUM as well, there’s Catchpoint, SpeedCurve Uptrends and many more.

And the third approach to measuring front-end performance is by using some sort of browser-based automation tool. So typically this has been something like Selenium, but that’s falling out of favor in terms of more modern solutions like Playwright, Puppeteer, Cypress and so on.

So front-end performance is about everything that the user sees on their end, but back-end performance testing is about everything that they don’t see. It’s about the network. It’s about latency, the application servers, the way the infrastructure is set up and all of those things that are a little bit more difficult to look into, but can also have severe bottlenecks. Back-end performance testing seeks to verify aspects of an application such as scalability, elasticity, availability, reliability and responsiveness.

Scalability is the application’s ability to respond to changes in demand. This could mean scaling up, which means increasing the resources available to a server, but it could also mean scaling out, meaning increasing the number of servers that are available to handle the application load.

Elasticity is a related topic to scalability, except where scalability is all about scaling up or out. Elasticity also emphasize a scaling down or in, and that’s because having these services or servers available on the Cloud that are dynamically adjusting can also be quite expensive. So elasticity makes sure that the infrastructure that’s available or the resources available are always appropriate for the size of the demand. So when demand increases, elasticity also checks whether the application decreases it servers or resources in response to that.

Availability is how long the application stays up, despite changing circumstances in production. So this is typically measured by uptime. Availability can also be tested using disaster recovery procedures, and you’re not just testing the application itself, but also the processes around it. So are there enough people that are on call so that they would be able to come in and troubleshoot when there’s a production incident? All of those things are included in application availability.

Reliability checks to see how your application behaves in response to unexpected errors. Reliability basically takes it as a given that your application is going to fail somehow, sometime, maybe you don’t know exactly when that will be or how it will fail, but it’s going to happen. So a big part of reliability and site reliability engineering is trying to expose your application to those failures beforehand so that you can build confidence about how it will behave when those things actually happen in production.

And the last aspect to back-end performance testing is responsiveness. And this is usually equated with load testing. Load testing is more of a technique for testing though. And really load tests can be used to test not just responsiveness, but also reliability or availability or scalability, basically any of the “ilities” in this part of back-end performance testing.

Protocol-based load testing verifies the backend performance of an application, not by simulating real users accessing the application, but by simulating the underlying requests that go back and forth between the client or the user and the application servers. One of the benefits of protocol-based load testing is that it takes up fewer resources on the side of the load generator, and therefore is less expensive to run. Some popular protocol-based load testing tools are k6, JMeter, Gatling LoadRunner, NeoLoad, and Silk Performer.

Browser-based load testing verifies the application by simulating real users accessing it. So it also has the benefit of including these front-end performance metrics. Browser-based testing can also be called browser-driven or UI-based testing. Some popular tools for browser-based load testing are Playwright, Puppeteer, Flood Element and as of a few weeks ago, k6.

Then there’s hybrid load testing, which takes the best of protocol and browser-based load testing tools by combining them in one test. Now this can present some challenges, especially if you’re using different tools for the browser-based script, than you’re using for the protocol-based script. The ideal is to use your protocol based load testing tool, to simulate majority of the load to cut down on costs for the execution and scaling out of your load test. And then you run a handful of users using your browser based load testing tool. So you still get some metrics for front-end performance while that load is being applied by the protocol based tool.

Testing software can be really daunting because there are many aspects of an application that you can test. But that’s because there’s also many aspects of an application that can fail. And that’s why all roads lead to testing and why the success of an app hinges on how well it’s been tested.

If you’d like to zero in, on load testing in particular, check out this video that I made on what load testing is. Thanks for watching. And like they say, here in Portugal, Obrigada!

Six Performance Testing Hats

Fri, 14 Jan 2022 16:30:34 +0100

I was reading Six Thinking Hats by Edward de Bono when I realized that the premise could be applied to performance testing:

Each hat represents a different way of thinking, which can be useful when working through issues. Sometimes, wearing only one hat can cause us to fixate on incorrect assumptions, and “putting on another hat” can help shake things up and get to actually fixing what went wrong.

What Tabletop Games Teach Us About Application Perfomance

Wed, 12 Jan 2022 16:30:29 +0100

Tabletop games and application performance have more in common than you might think.

See, it’s all about optimization. Both involve juggling several variables: some that you can control, and some that you can’t.

Here’s what I’ve learned about application performance from playing tabletop games.

#1: Get more workers

In worker placement games, you should always spend your first worker to train more workers.

Similarly, one of the most basic ways to increase concurrency is multithreading: using more than one thread at a time. (See: Golang) This applies to testing tools as much as to systems. The 1 Thread: 1 Virtual User paradigm is flawed.

#2: Spend time on getting your engine going

Get your big combos in place before you tap to attack.

Things like setting up automation frameworks or CI/CD pipelines DO set you back in the beginning, but they are worth their mana cost in the long run.

#3: Draft with others in mind

The trick to drafting is knowing what you’re leaving for others as much as what you’re taking for yourself.

The short game is getting your team to do the testing. The long game is getting everyone in the company involved in testing.

#4: Hope for a nat 20, plan for a nat 1

Sometimes proficiency, expertise, and a luckstone just aren’t enough.

Instead of wondering whether an app will fail, take for granted that it will, and start planning for it. Keep that Action Surge in your back pocket.

#5: Sometimes, you just have to trade for sheep

The 2:1 port is valuable, but not always the best option.

Learn what you can, but accept that you can’t learn it all. Reach out to others with compatible skillsets and ask the “stupid” questions that nobody else wants to ask.

Testing is a team sport, and everyone gets a chance to play.

Happy gaming!

The Tester's Toolbox Is Flawed

Tue, 11 Jan 2022 16:30:49 +0100

Every tester has a toolbox.

In the toolbox rests a variety of tools for every conceivable purpose. Part of our job, as testers, is to choose the right one for the job. Sounds reasonable, right?

That kind of thinking could be dangerous.

“The right tool for every job” fails for multiple jobs.

You end up with multiple tools. One for accessibility testing. Another for API testing. Another for browser testing. Another for load testing. This is the modern tester’s dilemma: we spend more time working with our tools than with the application under test.

In the real world, including a new tool in your testing stack means a new framework, usually a new programming language, and sometimes even new colleagues with experience in those tools. And what about continuous testing? All those tools have to talk to each other at some point.

More tools = more time, effort, and money.

When choosing tools, prioritize multitools.

A multitool can be used for different purposes and prioritizes flexibility.

Here are a few ways to spot a multitool:

#1: It is a polymath. A multitool is good at many things and is limited in use case only by the imagination of those that use it.

#2: It uses what you know. A multitool is built on languages and frameworks you and your team already know, reducing ramp-up time.

#3 It plays well with others. A multitool doesn’t force you to use it to the exclusion of everything else. It integrates well with best-in-class tools.

When you optimize for individual jobs, you’re missing the big picture. You’re ultimately left holding the ends of multiple tools and creating a kraken of a test suite with all these tools cobbled together.

Instead, prioritize multitools.

Obsidian for the Uninitiated

Mon, 10 Jan 2022 23:07:06 +0100

Obsidian for the Uninitiated

You’ve installed Obsidian and created a new vault. Now what?

Unfortunately, it can be more than a little daunting to get started with. It doesn’t help that there are terms like Zettelkasten and Maps of Content and fleeting notes and digital garden that are all just trying to structure what we all learned to do in primary school.

Start by logging what you’re doing.

Having a rigid structure does wonders for curtailing creativity and plain fun. Instead, I recommend starting small.

Here are some small steps to take after installing Obsidian:

Step 1: Enable the Daily Notes plugin

The most game-changing feature of Obsidian is plugins.

Go into Settings > Core Plugins, and enable the Daily notes plugin.

Then, exit out of the dialog boxes and hit CMD P (Mac) or CTRL P (Windows) to bring up the command pane. Search for and select Daily notes: Open today’s daily note. That creates a new note in your vault with today’s date.

From now on, do this every day and log things you do during the day: things to do, appointments to keep, videos you watched.

Step 2: The next time you consume content, take notes.

Now, the next time you read or watch something interesting, type its title into Obsidian with brackets:[[Cool video I watched]]

It’s now a link. CMD+click on that link, or click the glasses icon on the top right to switch to Preview mode, and then click on the link. You should now have a new note. Write about what you thought of the video.

Step 3: Start identifying and fleshing out ideas.

Keep going with these steps and every now and then, start to identify cool ideas you come across. Create new notes from them just like you did in Step 2. Don’t worry about getting it all; just do it when and until you feel like it.

These three simple steps, over time, will be the foundations of your own Personal Knowlege Management system with Obsidian.

5 Ways to Use Obsidian as a Software Engineer

Sun, 09 Jan 2022 19:20:49 +0100

Yesterday, I wrote about why I think taking notes is the surprising foundation of a tester’s career.

But it’s not just testers. Every software engineer is a knowledge worker, and that means we get paid for what we know. Using a Personal Knowledge Management (PKM) system such as Obsidian.md helps us keep up with the constant barrage of new ideas in tech.

Here are 5 things I use Obsidian for.

#1: My very own Stack Overflow

If you code, you’ve used Stack Overflow.

What if you had your own Stack Overflow, one that contained cheat sheets of every language you’ve ever tried to hack something up in, including only commands relevant to your job? This is exactly what I’ve done for Python recently, and it’s helping me solidify my knowledge.

#2: Tools database

There’s a tool for everything.

To keep them straight, I have a page for every load testing tool I try, with tutorials for how to get started, an assessment of its features, and observations I had while attempting to learn it. Here’s what I have for the load testing tool k6.

Bonus: I also use the Dataview plugin to query tools that fit my criteria.

#3: An Agile workspace

If Agile is good enough for my professional work, it’s good enough for my personal work.

I use the Kanban plugin to keep track of projects, create weekly and monthly review/retro templates, and work iteratively using the Incremental Writing plugin.

#4: A public learning log

Learning in public is the fastest way to improve.

When I’m trying to figure something out, I document my attempts. With Obsidian Publish, I can quickly push this log out to my domain and share them with colleagues to get a quick review so that they can easily follow my train of thought and tell me where I went wrong.

#5: A digital garden of software ideas

Build libraries, not feeds.

I use Obsidian to save things I read/watch/listen to but also to save space for ideas I’ve learned to germinate. I publish notes before they’re fully fleshed out, with the goal of growing everything over time.

Today, I have 4,567 notes, all about things that matter to me.

Every engineer should have a PKM system.

Take Notes, Not Tools

Sat, 08 Jan 2022 17:00:44 +0100

When I first got into testing, I thought that testing was about learning different tools and frameworks.

I learned how to use Neoload, then LoadRunner, then JMeter, and then SeleniumSilkPerformerVisualStudioGatlingPuppeteerPlaywrightk6 in short succession. I took courses on JavaScript, on C, on Java. I ticked off tools like a grocery list.

It didn’t take me long to realize that I still wasn’t a good tester.

Taking notes, not tools, is what makes a good tester.

When you think about it, a lot of things a tester does revolves around taking good notes.

Requirements are just notes on what the business wants to achieve. Test scenarios are notes on situations that the team wants to simulate. Test cases are notes on how a user would use an application. Defects are notes on issues.

The quality of a tester’s notes dictates the quality of their work.

Listen to people, not tool tutorials

Take notes on what users do. Talk to the business. Pore through Google Analytics logs. Piece together a story of the kind of people who use your application, and how they use it.

Take notes on problem areas. Talk to customer support about the most common questions they get– they know more about pain points than anyone else in the company. Talk to the ops engineers who were on call during the last production incident. These problems are what you’ll need to test later.

Take notes on what you’ve learned. If you don’t like learning, you’re in the wrong field. Document everything, even things you think you’ll never forget, because one day, you WILL forget.

Taking notes, not tools, is the foundation every testing career is built on.

Performance Objective: Intro to Load Testing With k6 and Grafana

Tue, 23 Nov 2021 17:05:06 +0100

I spoke at the Performance Objective meetup to talk about how to use k6 and Grafana for beginners. I always enjoy talking to like-minded engineers, and I think sharing ideas in a meetup context is probably the best way to do it.

Everyone Can Contribute: Load Performance Testing With k6

Tue, 23 Nov 2021 16:57:26 +0100

This is a recording of a live meetup I spoke at, the Everybody Can Contribute Café hosted by Michael Friedrich, Senior Developer Evangelist at GitLab. This was a more casual conversation and show-and-tell about k6’s capabilities, and I really enjoyed it!

Performance Testing with k6 and Cypress (Ministry of Testing)

Tue, 23 Mar 2021 19:52:48 +0100

My friend Marie Drake and I teamed up to give a presentation about performance testing at a Ministry of Testing (Philippines) meetup, and we talked about using Cypress for front-end performance testing and k6 for back-end performance testing.