Your AI wrote a great plan. It's probably wrong. Here's how to catch that before you build.

        ┌──────────────────────────────────────────────┐
        │                                              │
   You  │                                         refine
    │   │                                              │
    ▼   ▼                                              │
┌─────────────────────┐                               │
│      Plan Mode      │  ← Model A (e.g., Sonnet)     │
│  (Read · Reason ·   │                               │
│    Propose)         │                               │
└─────────────────────┘                               │
           │                                          │
           │  Draft Plan                              │
           ▼                                          │
┌─────────────────────┐                               │
│    Plan Reviewer    │  ← Model B (e.g., Opus)       │
│  (Critique · Flag · │    Fresh context              │
│    Rank by Risk)    │    Adversarial prompt         │
└─────────────────────┘                               │
           │                                          │
    ┌──────┴──────┐                                   │
    │  Blockers?  │ ──── yes ─────────────────────────┘
    └──────┬──────┘
           │ no
           ▼
┌─────────────────────┐
│      Implement      │
└─────────────────────┘

There's a failure mode buried inside plan mode that nobody talks about.

The model that wrote the plan is also the most likely to think the plan is good. It built the reasoning. It owns the assumptions. When you ask the same model to review its own output — same context, same priors — it's not reviewing. It's confirming.

This is a well-known problem in AI research. The paper that formalised it — Reflexion (Shinn et al., 2023) — showed that giving agents a verbal feedback loop dramatically improved task performance. The core insight: critique before committing to an action is structurally better than acting and recovering. The catch: the critique has to be genuine. A self-reinforcing loop is worse than no critique at all, because it gives you false confidence.

The fix is surprisingly simple. You use a second agent — different model, fresh context, adversarial instructions — whose only job is to find what's wrong with the plan before you build it.

In this post you'll learn:

• What the Reflexion pattern is and why it matters for agentic coding
• How plan mode fits into the loop — and where it falls short on its own
• How to build a plan reviewer sub-agent that actually finds problems
• Why the model family you pick for the reviewer changes what it catches
• How GitHub Copilot CLI validated this exact pattern with their Rubber Duck feature — and the numbers behind it

💡 This is Part 7 in my Context Engineering series. Part 1 covered the four pillars. Part 2 introduced agent skills. Part 3 argued for lean over Scrum. Part 4 put skills to work with git worktrees. Part 5 showed why Markdown is the native format of agentic context. Part 6 explained why spec-driven development is waterfall in a trench coat.

🧠 The Reflexion Pattern

The original Reflexion paper introduced a deceptively simple idea: rather than asking an agent to act directly on a task, give it a loop.

1. Generate — the agent produces an output (a plan, a solution, an action)
2. Evaluate — a critic assesses that output against some standard
3. Refine — the agent incorporates the critique and produces a better output

Repeat until good enough or resource limits are hit.

What made the results striking wasn't the loop itself — iteration is obvious. It was the quality of the critic. When the evaluator had a fresh context and adversarial instructions, agents caught mistakes they repeatedly missed on direct attempts. The same model could find what it wouldn't have generated.

The lesson for agentic coding isn't that you need a research paper's setup — it's that a second perspective, properly framed, is structurally different from self-review. The critique has to come from somewhere the original reasoning didn't.

📋 Plan Mode: The Generator Half

If you've used Claude Code's plan mode, you've seen the first half of the Reflexion loop already.

You describe the task. Claude Code reads your codebase — your CLAUDE.md, the relevant files, the existing patterns — and produces a structured plan before touching anything. It identifies which files change, what functions are affected, what the implementation sequence looks like, and flags anything it's uncertain about.

This is genuinely valuable. It surfaces the reasoning before you're committed to it. You can redirect, push back, or add constraints before a single line of code is written. That's the right gate.

But plan mode has a blind spot. The model that wrote the plan has been primed to think it's correct. It read your codebase through the lens of solving your task. Its context window is full of reasoning that supports the approach it just proposed. Asking that same context to critique the plan is like asking someone to proofread their own essay five minutes after writing it — the eye slides over what the brain already decided was fine.

What you need is a second reader who wasn't in the room when the plan was written.

🔴 Enter the Skeptic: The Plan Reviewer Sub-Agent

The plan reviewer is a sub-agent whose only job is to find problems.

Not to validate the approach. Not to suggest alternatives. Not to summarise what the plan does well. To find what will break it.

Here's what the output looks like in practice:

## Plan Review

### Blockers
- Step 3 calls `migrate_schema()` but this function does not exist in the codebase.
  Grep confirms no match. Plan assumes it was already implemented in a prior ticket.

### Significant Risks  
- The migration has no rollback step. If the schema change fails mid-run on the
  50M-row table, there is no documented recovery path.

### Minor Concerns
- Step 7 says "update accordingly" — underspecified. Which callers are affected
  and in what order?

That's the output you want before implementation, not after. The blocker in that example — calling a function that doesn't exist — would have caused an immediate runtime failure. The plan reviewer caught it in thirty seconds by grepping the codebase. The planning model didn't, because it assumed the function was there.

In practice, a single review pass catches the high-severity issues. If the plan changes substantially after refinement, run the reviewer again — a revised plan can introduce new assumptions just as easily as the original.

🎯 Why the Model Family Matters

The single most important decision in setting up a plan reviewer is which model it uses.

This isn't about one model being "better" than another in some general sense. It's about cognitive diversity. Different model families have different training, different blind spots, different tendencies toward optimism or caution. A plan generated by Claude Sonnet reviewed by Claude Sonnet in the same session is likely to surface the same risk profile the plan itself assumed.

A plan generated by Sonnet, then passed to Opus in a fresh context window with adversarial instructions, is different. Opus has different priors about what constitutes a risk. It hasn't been warmed up by the same planning conversation. It's reading the plan cold, the same way a senior engineer would if you dropped it in their inbox.

The principle generalises beyond Claude. If your planning agent uses GPT-4o, review with Claude. If it uses Claude Sonnet, review with Opus. The point isn't that one is smarter — it's that the review is structurally independent. You're not asking the same chain of reasoning to audit itself.

💡 Model diversity in the review step is the difference between a rubber stamp and a genuine gate.

🦆 The Industry Agrees: Copilot's Rubber Duck

This isn't a niche pattern. GitHub just shipped exactly this in GitHub Copilot CLI under the name Rubber Duck — and the numbers make the case better than theory does.

When you select Claude as the orchestrator in Copilot CLI, Rubber Duck automatically brings in GPT-5.4 as the reviewer. Cross-model by design. The system fires at three checkpoints:

1. After planning — catches suboptimal early decisions before they compound
2. After complex implementation — identifies edge cases in intricate code
3. After test writing — flags coverage gaps before execution

The performance result is striking. Testing on SWE-Bench Pro found that Claude Sonnet 4.6 paired with Rubber Duck running GPT-5.4 closed 74.7% of the performance gap between Sonnet and Opus — approaching Opus-level resolution rates at Sonnet cost. On the hardest multi-file problems, the pairing scored 4.8% higher than the baseline.

The bugs Rubber Duck caught in real-world testing are instructive too: a scheduler that immediately exited due to an architectural flaw, silent data loss from overwritten dictionary keys, cross-file conflicts breaking Redis references. These aren't hypothetical risks — they're the exact category of problem that compiles, passes linting, and only breaks in production.

💡 You don't get near-Opus performance by using a better model. You get it by adding a cross-model reviewer to your loop.

The name is a deliberate nod to the classic rubber duck debugging technique — explaining your code to an inanimate object to surface problems you couldn't see while writing. The insight is the same: a second perspective that wasn't involved in producing the output finds what you normalised away.

GitHub built it into the tool. You can build it into your workflow today with a single agent definition.

⚙️ Building the Plan Reviewer

You don't need a complex setup to implement this. In Claude Code, a plan reviewer is a standard agent definition — a markdown file in your .agents/agents/ directory with a system prompt and a model assignment.

The key elements:

1. Make it adversarial by default. The reviewer's job is to find problems, not to acknowledge what the plan does right. Any praise is wasted context.

2. Give it verification tools. The most common plan failure is an assumption about the codebase that turns out to be false. The reviewer needs Read, Grep, and Glob access to check those assumptions against actual code — not just against what the plan claims.

3. Order findings by severity. A list of concerns is useful. A list ordered by blast radius is actionable. Blockers (will cause failure) first, then significant risks (likely to cause production issues), then minor concerns (underspecified steps, optimistic assumptions).

4. Make it trigger proactively. The description field determines when Claude Code invokes it automatically. Phrase it to trigger after any plan is produced, not just when explicitly asked — because the cases where you forget to ask are exactly the cases that go wrong.

5. Set color: red. It's a small touch, but it visually marks the reviewer as the adversarial voice in your agent list — a useful reminder that its job is to find fault, not to help.

Here's the full definition from my langchain-anthropic-pdf-support project:

---
name: plan-reviewer  
description: |  
  Use this agent when a plan has been written and needs critical review before
  implementation begins. Trigger proactively after producing an implementation plan,
  migration plan, refactor proposal, or architectural design. Also trigger when
  the user asks to "review the plan", "critique this approach", or "find problems
  with this".
model: opus  
color: red  
tools: ["Read", "Grep", "Glob"]  
---

You are a principal engineer reviewing a plan. Your job is to find problems,  
not validate assumptions.

Systematically identify:  
1. Edge cases not accounted for  
2. Performance implications (N+1, blocking, scale issues)  
3. Backwards compatibility risks  
4. Missing rollback steps  
5. Anything underspecified or optimistic

Verify plan assumptions against actual code using Read/Grep/Glob.  
Order findings by severity: blockers first, then significant risks,  
then minor concerns.  
Do not praise or summarise what the plan does well.  

Note on model: opus: The repo uses model: inherit by default, which picks up whatever model the parent session is running. For the reviewer to provide genuine cognitive diversity, set it explicitly — opus if you're running Sonnet day-to-day, or swap model families entirely if you're on a non-Claude tool.

If you're not on Claude Code, the same principle applies. In Cursor, add a rule in .cursorrules that instructs a second agent pass to critique the plan before acting. In GitHub Copilot CLI, Rubber Duck does this automatically. The pattern is tool-agnostic — what matters is the structural separation: a different model, a fresh context, and an adversarial framing.

The Bottom Line

Plan mode is valuable. But a plan reviewed only by the model that wrote it is a plan that's checked its own homework.

The Reflexion pattern gives you a genuine quality gate: a second model, fresh context, explicitly tasked with finding what the plan got wrong before you build it. The reviewer reads cold. It verifies assumptions against real code. It ranks what it finds by severity so you know what to fix before you act.

• ✅ Use plan mode to generate a structured plan before any implementation
• ✅ Spawn a plan reviewer agent with a different model family and adversarial instructions
• ✅ Give the reviewer code access so it can verify assumptions, not just critique the document
• ✅ Trigger it proactively — the cases where you forget to ask are exactly the cases that go wrong
• ✅ Re-run after significant refinement — a revised plan can introduce new assumptions just as easily as the original
• ❌ Don't use the same model and context for both planning and review — you're not getting a second opinion, you're getting a confirmation

The plan reviewer doesn't replace judgment. You still decide which findings to act on. But it gives you something plan mode can't: a perspective that wasn't in the room when the plan was written.

That's the difference between a plan you reviewed and a plan that's been stress-tested.

The AI tooling ecosystem keeps rediscovering waterfall. Here's why you should resist it.

When did you last finish a requirements document and think: yes, this is exactly what we'll build, and it'll still be accurate when we ship?

If you've been in software long enough, you already know the answer. The spec is wrong the moment you stop writing it. Software is discovery — you learn what the right solution is by building it, not by describing it in advance.

So why is the AI tooling industry trying so hard to bring specs back?

In this post you'll learn:

• What spec-driven development is and which tools are pushing it
• Why it's structurally identical to waterfall — just with better marketing
• Why an AI-empowered team with agentic context engineering outships a spec-heavy team every time

💡 This is Part 6 in my Context Engineering series. Part 1 covered the four pillars. Part 2 introduced agent skills. Part 3 argued for lean over Scrum. Part 4 put skills to work with git worktrees. Part 5 showed why Markdown is the native format of agentic context.

🔍 What is Spec-Driven Development?

The premise sounds reasonable: before asking AI to write code, write a detailed specification. The AI reads the spec, understands the intent, and generates an implementation that matches it. No vague prompts. No hallucinated architecture. Just a clear document that drives predictable output.

Several tools and frameworks have formalised this into a full workflow.

OpenSpec is probably the most talked-about right now. It installs via npm and works across Claude Code, Cursor, GitHub Copilot, and 30+ other tools. When you run /openspec:proposal, it generates a structured change proposal — requirements doc, implementation tasks, design decisions — before a single line of code is written. It markets itself as "lightweight" and "brownfield-first," and to be fair, it's more considered than a blank requirements doc. But the fundamental shape is the same: spec first, code second.

💧 It's Waterfall. Here's the Proof.

Let's put the two workflows side by side.

Classic waterfall:

1. Requirements phase → document everything
2. Design phase → spec everything
3. Implementation phase → build to spec
4. Testing phase → verify against spec
5. Release

Spec-driven AI development:

1. Discovery / inception → gather requirements
2. Write spec (BA, PM, or AI-assisted) → document everything
3. AI implements to spec
4. Review and validate

Same shape. Different tools.

The spec-driven workflow also sneaks back in the roles that lean teams have been actively trying to eliminate:

• 📋 Business Analysts to write the requirements
• ✍️ Product Owners to sign off on the spec
• 🏛️ Architects to validate it's implementable
• 🔍 Dedicated QA to verify against the spec

These are hand-off roles. Each one is a seam where context gets lost, where timelines stretch, and where "it wasn't in the spec" becomes the answer to every inconvenient question.

This isn't a new failure mode. Behaviour-Driven Development made the same bet a decade ago — that BAs and product owners could write tests in Gherkin, the human-readable format behind tools like Cucumber, without deep engineering involvement. The promise was that non-technical stakeholders could define acceptance criteria directly. In practice, the Gherkin files were either written by engineers anyway, or they drifted from reality the moment the code changed. The handover assumption was the problem then, and it's the problem now.

Modern testing has moved in the opposite direction. Shift-left testing brings quality thinking into the development loop rather than delegating it downstream. Test automation tooling — MCPs and CLIs like Playwright — lets engineers write and run tests as part of the build cycle, not as a separate gate. The assumption that non-technical people should own the test layer was always backwards. What works is engineers who think about quality from the start.

This is about more than methodology. It's about team composition. In the AI era, the answer isn't a better handover process — it's not needing one. What you need are AI-empowered engineers with strong engineering instincts and the systems thinking to own a problem end to end.

But there's a subtler problem beneath the process overhead: the methodology and the headcount justify each other in a loop that's very hard to break from the inside. You need BAs to write the specs. You need QAs to validate against them. You need sign-off ceremonies before work begins and acceptance cycles before it ships. Heavyweight spec processes don't just slow teams down — they create an organisational incentive where writing a good spec becomes more valued than shipping good software.

And here's the deeper problem: spec-driven development is solving the wrong bottleneck. Generating code was never the problem. Deciding what to build — and staying adaptable as that changes — is the hard part. A spec locks you into an answer before you've done the learning.

💡 The future is T-shaped engineers who can own the full problem. Not specialists producing hand-off artefacts.

And there's another problem: spec rot. The spec is a snapshot of what you thought you needed at the moment you wrote it. The moment you start building, reality diverges. You're now maintaining two artefacts — the spec and the code — and they drift apart from day one.

No feedback loop in the build means you don't discover what's wrong with your design until you've already built to it. That's exactly waterfall's fatal flaw — and spec-driven development inherits it wholesale.

🚨 Hacker News Called It Immediately

I'm not alone in this read. When marmelab published a critique of spec-driven development on Hacker News, it hit 225 points and 191 comments — and the title said it all: Spec-Driven Development: The Waterfall Strikes Back.

The thread reactions were sharp:

"Managers appreciate SDD for the same reasons they favored waterfall initially." — constantcrying

"The distinction between waterfall and agile is merely how much spec you write before implementing — longer feedback loops increase risk of major mistakes." — pydry

"Really, we are doing waterfall, but with AI, now?" — multiple commenters

Engineers who tried Kiro — AWS's spec-driven agentic IDE — were equally blunt:

"Kiro generated massive task lists (12+ tasks with 4+ sub-tasks each) with unpredictable code deletion — a sledgehammer to crack a nut." — hatmanstack

"Kiro, your new corporate project manager." — iamsaitam (on Kiro treating a small bug fix as a full project management exercise)

When you show experienced engineers a spec-driven workflow, the reaction is consistent regardless of platform: we tried this already.

⚡ The Right Context is Lightweight, Not Heavyweight

Here's what I think people are actually reaching for when they advocate for spec-driven development: they want AI to have enough context to produce good output. That's a legitimate goal. The mistake is thinking a formal spec document is the right way to deliver it.

The Agile Manifesto articulated this argument in 2001:

"Working software over comprehensive documentation."

25 years later, spec-driven AI development is rediscovering the same mistake the Manifesto was written to correct.

What does good agentic context engineering actually look like? It's three things:

1. Context files — always-on institutional knowledge that lives in the repo, versioned with your code. CLAUDE.md covers project conventions, architecture decisions, gotchas, and commands. agents.md (covered in Part 5) gives the agent ground truth about your agent architecture — what each agent does, how they relate, known quirks. Neither of these is a spec. They describe what is, not what should be. Both are plain Markdown, readable by any agentic tool, owned entirely by your team.

2. Plan mode — an interactive dialogue with the agent to scope the work before acting. Not a document. Not a hand-off artefact. A conversation. Here's what that looks like in practice:

You type: "Add user notification preferences to the API — get and set per-user settings."

Claude Code reads your codebase and responds in under a minute: "I can see your Express router in src/api/routes.js. I'll add GET /users/:id/preferences and PATCH /users/:id/preferences, extend the User model with a JSON preferences column, and add a migration. I'll reuse your existing auth middleware. Want to proceed, or adjust anything?"

You refine. You ship. The spec-driven equivalent of that exchange is a requirements document, a BA-facilitated discovery session, an architect sign-off, and a sprint of writing before anyone touches the code.

The reason plan mode works is the same reason context engineering works — you're doing the thinking that a spec tries to capture, but you're doing it in a living, adaptive way rather than freezing it in a document that will be out of date before the first PR is merged. The plan evolves with the codebase. The context evolves with understanding. The output is software, not documentation.

Plan mode produces a plan.md — a structured, readable artefact you can review, edit, and pass to a sub-agent. Before execution I run a principal engineer review: a sceptical sub-agent persona whose job is to find problems with the plan before a line of code is written. A fresh context window sees what your planning session normalised. This is the quality gate — lightweight, fast, and embedded in the loop rather than bolted on at the end as a separate role.

OpenSpec's own FAQ raises this objection: "Plan mode is great for a single chat session — but what about longer work that spans multiple sessions?" Their answer is a spec. The actual answer is simpler: a progress.md file. The agent maintains it as work proceeds — completed steps, decisions made, what's next. At the start of the next session it reads its own notes and picks up where it left off. You don't need a platform, a structured workflow, or a vendor's artefact format. You need a text file and a habit.

3. A prototype — build something real, show it to a stakeholder, learn. The fastest way to discover what you need to build is to build it.

👥 The Team You Actually Need

In Part 3, I argued that Scrum's ceremony overhead doesn't survive contact with AI development speeds. Spec-driven development has the same problem, and it introduces an additional one: it justifies bloated team structures.

If your workflow requires someone to write the spec, someone to implement it, someone to verify it, and someone to maintain it as it drifts — you've described a four-person process for work one T-shaped engineer with an AI agent could handle in an afternoon. (A T-shaped engineer is someone with deep expertise in one area and enough breadth to own the problem end to end — from requirement clarity to production — without a handoff chain.)

Importantly, the BA and QA functions don't disappear in this model — they just stop being separate roles with separate artefacts and handoff ceremonies. A strong engineer with good AI tooling and a disciplined approach to context engineering can own requirement clarity as part of how they think about a problem, not as a separate phase before they're allowed to start. The translation between business need and technical implementation is a skill, not a job title. Engineers who develop that skill — and AI tooling makes it significantly easier — are more valuable than engineers who hand that responsibility to a BA and wait.

The same applies to quality. Quality is not a function — it's a standard. When it's owned by a specialist role, it becomes that role's problem to manage rather than every engineer's responsibility to maintain. When quality thinking is embedded in the agentic development loop rather than delegated to a gate at the end, failure rates don't increase. In many cases they improve, because the person closest to the code is also closest to the risk.

The Claude Code team at Anthropic is a well-documented example of this model working at scale. They built one of the most serious AI coding tools in the industry without traditional PRDs — using plan mode and rapid prototyping instead.

That's the team composition the AI era calls for:

• 🔷 Flat — minimal hierarchy, no specialist silos
• 🔷 T-shaped — deep in one area, broad enough to own the full stack
• 🔷 Outcome-oriented — shipping working software, not producing documents
• 🔷 AI-empowered — each engineer multiplied by an agent

In an interview with The Pragmatic Engineer, Boris Cherny — who led the development of Claude Code — put it plainly:

"There's just no way we could have shipped this if we started with static mocks and Figma or if we started with a PRD."

If the teams building the tools aren't writing PRDs, the argument that PRDs are necessary for serious product development becomes very hard to sustain. The question isn't whether this model scales — the question is whether your organisation has the hiring standards and culture to make it work.

The Bottom Line

Every generation of tooling rediscovers the same heavyweight process and gives it a new name. Waterfall became RUP became Scrum became SAFe. Now it's spec-driven development. The values the Agile movement fought for — working software over comprehensive documentation, responding to change over following a plan — are exactly what spec-driven development trades away.

You don't need a spec. You need good agentic context engineering principles — CLAUDE.md, plan mode, T-shaped engineers, prototypes. What you don't need:

• ❌ A BA writing requirements before the team has touched the problem
• ❌ A spec document to maintain in lockstep with your code
• ❌ An inception workshop before a single line is written

Spec-driven development frameworks are solving a problem that better hiring, better tooling, and better engineering culture have already made obsolete. They're not a step forward — they're a formalisation of constraints that high-performing teams have already moved past.

The spec was never the source of truth. The software always was.

It's not even a technology. It's a convention. And that's exactly why it wins.

Markdown is a writing convention — a set of rules for formatting plain text that John Gruber wrote a spec for in 2004. No runtime. No SDK. No API. Just text with agreed-upon symbols that any human or machine already understands. While the AI industry has shipped transformers, vector databases, multi-agent orchestration frameworks, and billion-parameter models, Markdown has just quietly remained what it always was: the lowest common denominator of structured information.

And that turns out to be exactly what agentic context engineering needs.

I've been a Docs as Code person for years. Markdown in version control, reviewed in PRs, rendered by static site generators — it's the default format for everything I write. When I started building agentic workflows, that habit turned out to be directly valuable in a way I hadn't anticipated. The contrast isn't simple technology versus complex technology. It's a convention versus the most sophisticated AI systems ever built — and the convention wins on cost, portability, and clarity every time.

This post is about that insight — and how it played out across every stage of a real migration: upgrading five LangChain agents from 0.3 to 1.2. The migration is the example. Markdown is the point.

In this post you'll learn:

• Why Markdown is the highest-signal, most token-efficient format for AI context — and the numbers that prove it
• How Docs as Code thinking (MkDocs, Backstage, Marp) maps directly onto agentic context engineering
• Three ways to get any documentation into your context window as Markdown automatically
• How every artefact in an agentic workflow — context files, plans, reviews, progress logs — is Markdown doing its job
• Why Markdown files that survive context resets are the key to multi-day agentic tasks

💡 This is Part 5 in my Context Engineering series. Part 1 covered the four pillars. Part 2 introduced agent skills. Part 3 argued for lean over Scrum. Part 4 put skills to work with Git worktrees.

📖 Markdown as Context: Why Format Matters for AI

I've been using Markdown for years — not just as a formatting convenience, but as a philosophy. Docs as Code: documentation that lives in version control, travels with your codebase, and gets reviewed alongside your PRs. ADRs (Architectural Decision Records) capturing why a technical choice was made. RFCs proposing and debating changes before implementation. And this blog — written in Markdown, published via Ghost, specifically because Ghost keeps it simple and stays out of the way. If you've worked with MkDocs to build static documentation sites, or Backstage's TechDocs — which uses Markdown as the foundation for an entire developer portal — you already know the power of treating docs as first-class artefacts. Even presentations: Marp turns Markdown into slides, which means I can write a deck in the same format as everything else. Markdown all the way down.

So when I'm handing context to an AI agent, Markdown is the obvious format. It's not a new habit — it's the same discipline I've used for years, applied to a new domain. The difference is that with AI, the choice of format carries a measurable, direct cost.

When you're migrating across major library versions, your agent needs documentation — lots of it. Release notes, migration guides, changelogs, API references. The naïve approach is to point your agent at the docs website and let it browse. The problem: documentation websites are HTML, and HTML is expensive.

Cloudflare put some hard numbers on this recently. Their Markdown for Agents post measured a real documentation page: 16,180 tokens as HTML, 3,150 tokens as markdown. An 80% reduction.

Their framing nails it:

"Feeding raw HTML to an AI is like paying by the word to read packaging instead of the letter inside."

A simple heading like ## About Us costs 2–3 tokens in markdown. The equivalent HTML — <h2 class="section-title" id="about">About Us</h2> — runs 12–15. Every element, every wrapper div, every ARIA attribute, every script tag is a token your agent has to process without gaining any understanding of the actual content.

Markdown strips all of that. The same page in markdown: clean headings, code blocks, links. No wasted tokens. As Microsoft's MarkItDown project puts it:

"Markdown conventions are also highly token-efficient."

And LLMs don't just tolerate markdown — they're optimised for it. They were trained on vast quantities of it. Feeding your agent markdown isn't just cheaper; it's the format the model reasons about most fluently.

Context windows are finite. Every token you waste on HTML scaffolding is a token you can't use for understanding your codebase.

Getting LangChain Docs as Markdown

My first step was pulling down the LangChain release notes and migration guides as markdown — and it turns out the LangChain docs site makes this remarkably easy. Every page has a built-in "View as Markdown" option that returns the page as clean plain text, purpose-built for LLMs.

The docs site also ships with a built-in MCP server. Hit the dropdown on any page and you'll see "Copy MCP Server" — one click gives you the config to paste directly into your .mcp.json:

{
  "mcpServers": {
    "langchain-docs": {
      "type": "http",
      "url": "https://docs.langchain.com/mcp"
    }
  }
}

Add that and your agent is connected to the full LangChain docs knowledge base — searchable, queryable, without fetching a single HTML page.

This is the ecosystem catching up to how agents actually work. More documentation sites are building markdown-first, AI-native access directly into their platforms — and when they do, use it.

The Cloudflare Trick: Markdown on Demand for Everything Else

For documentation sites that don't have built-in markdown export, Cloudflare has shipped something genuinely useful. Their new Markdown for Agents feature lets any site served through Cloudflare return markdown instead of HTML — automatically, based on a single request header.

The mechanism is simple: your agent sends Accept: text/markdown in the HTTP request. Cloudflare intercepts it, converts the HTML to markdown in real time, and returns it. The response even includes an x-markdown-tokens header so you can see exactly how many tokens you're working with.

curl -H "Accept: text/markdown" https://some-docs-site.com/page  

No conversion tooling required on your end. No scraping, no parsing. For any documentation that isn't LangChain — release notes, third-party libraries, framework guides — this is the most frictionless path to token-efficient context.

If you're using Claude Code, you don't need to do anything at all. I tested it: Claude Code's WebFetch tool sends Accept: text/markdown, text/html, */* as a request header automatically. Fetching the Cloudflare blog post itself returned:

content-type: text/markdown; charset=utf-8  
x-markdown-tokens: 4023  
content-signal: ai-train=yes, search=yes, ai-input=yes  

Markdown served directly, token count included, usage permissions declared. No extra configuration. If the site is on Cloudflare and has the feature enabled, you're already getting the token savings just by using the built-in fetch tool.

MarkItDown MCP: The Fallback for Everything Else

For anything that's neither LangChain docs nor Cloudflare-hosted — PDFs, Word documents, downloaded changelogs — MarkItDown has you covered. It's Microsoft's open-source Python tool that converts virtually any file format to clean markdown: HTML, PDF, Word, Excel, PowerPoint, and more.

Crucially, it ships with an official MCP server — markitdown-mcp — so your agent can convert any file on demand without leaving the session.

pip install markitdown-mcp  

Wire it into your MCP configuration and it's available immediately:

{
  "mcpServers": {
    "markitdown": {
      "command": "uvx",
      "args": ["--python", "3.13", "markitdown-mcp"]
    }
  }
}

I used this extensively throughout the migration. Whenever I hit a document that wasn't available as clean markdown — a PDF of the old API reference, a downloaded changelog, some Word doc from an internal wiki — MarkItDown MCP converted it in-session, inline, without me having to leave the conversation or run a separate script. It became the default fallback so reliably that I stopped thinking about it as a separate tool and started treating it as part of the context pipeline.

Between the three approaches — native markdown export (LangChain), Accept: text/markdown (Cloudflare), and MarkItDown MCP (everything else) — you can get any documentation into your context window in token-efficient form. The bottom line is the same in every case: your agent gets the content it needs, without the packaging it doesn't.

🗺️ Agentic Plan Mode: Building the Migration Blueprint

With the documentation loaded as markdown, the next step wasn't writing code — it was planning. This is where agentic plan mode earns its keep. And it's where you start to see that Markdown isn't just for docs fetching — it's the format of every artefact the agent produces and consumes.

Giving the Agent Codebase Context

Before entering plan mode, I made sure the agent had a solid understanding of the existing architecture via an agents.md file I maintain in the codebase. This is Docs as Code applied directly to AI context: a Markdown file, version-controlled alongside the code, that describes each of the five agents, what they do, how they relate to each other, their shared dependencies, and any known quirks.

# agents.md

## Agent Overview

This codebase contains five AI agents:

### 1. IngestAgent
Responsible for document ingestion and chunking...

### 2. RetrievalAgent
Handles vector search and context retrieval...

[etc.]

This kind of structured context file is critical. Without it, your agent has to reverse-engineer the architecture from code — which it can do, but it wastes time and tokens, and the inferences can be wrong. agents.md gives the agent ground truth about intent, not just implementation. It's also readable by humans, diffable in git, and costs a fraction of what an equivalent HTML document would. Markdown doing exactly what Markdown does best.

The Plan Mode Conversation

With the migration guides loaded as markdown and the codebase context established, I entered plan mode and walked through the migration with the agent. This wasn't just "here's what needs to change" — it was a structured conversation:

• What are the breaking changes between 0.3 and 1.2 that affect each agent?
• What's the safest migration order given agent dependencies?
• Where are the highest-risk changes?
• What backwards compatibility concerns do we need to manage?

Plan mode keeps the agent in a reasoning-and-documenting state rather than immediately writing code. The output was a plan.md — a structured, step-by-step migration plan covering each agent, the specific changes required, the order of operations, and the testing checkpoints.

This is the Markdown flywheel in action: you fed the agent Markdown documentation, and it produced a Markdown plan. The format flows through the entire pipeline. And because it's plain text in a file, it's reviewable, editable, and — crucially — portable to the next step.

🔬 The Principal Engineer Review

Here's the move that changed the quality of my migrations: before implementing anything, I spin up a sub-agent with a fresh context window to review the plan.

Why a Separate Context Window Matters

When you've been in a planning session for an hour, your agent has accumulated context — your assumptions, your framings, the reasoning path you took to arrive at the plan. That accumulated context can be a blind spot. The agent is unlikely to challenge the premises it helped establish.

A sub-agent starts fresh. It has no memory of the planning conversation. When you hand it the plan as a Markdown document and ask it to critique, it reads it the way a new engineer would read it — without the benefit of your shared assumptions, but also without the baggage of them.

This is where the portability of Markdown pays off directly. plan.md is a self-contained artefact. No conversation state, no special format — just a text file you paste in. The sub-agent gets everything it needs from the document itself.

I use a "principal engineer" sub-agent persona for this: an experienced, sceptical reviewer whose job is to find problems. The prompt is straightforward:

You are a principal engineer reviewing a migration plan.  
Your job is to find problems, not validate assumptions.  
Review the following plan and identify:

1. Edge cases not accounted for  
2. Performance implications  
3. Backwards compatibility risks  
4. Missing rollback steps  
5. Anything that seems underspecified or optimistic  

Then I paste in plan.md as the sole input.

What It Catches

It found areas to improve — which is exactly what you want. A fresh context window will always see things the planning session normalised, and because the plan lives in a Markdown file, handing it over is as simple as pasting the document in.

📋 Progress Tracking Across Days

Large migrations rarely happen in one sitting. I worked on this across several days in whatever time I had — and I was running it using the Git worktree agent skill from Part 4, which keeps the migration isolated in its own branch while other work continues in parallel. That introduced a new problem: how does the agent pick up where it left off across sessions?

The progress.md Pattern

This is Docs as Code for agent state. Inspired by the note-taking patterns used in deep agent frameworks, I gave the agent a progress.md file to maintain alongside plan.md. The format is simple — the example below is illustrative, not the exact steps from my migration:

# Migration Progress

## Status: In Progress

## Completed
- [x] IngestAgent — LangChain imports updated
- [x] IngestAgent — Chain interface migrated to new API
- [x] IngestAgent — Tests passing

## In Progress
- [ ] RetrievalAgent — Updating retriever interface

## Pending
- [ ] RetrievalAgent — Memory migration
- [ ] SummaryAgent
- [ ] RouterAgent
- [ ] OutputAgent

## Notes
- Remember: shared utility in utils/chains.py updated once in IngestAgent step — do NOT repeat
- Existing memory objects in DB need migration script before deploying RetrievalAgent

At the start of each session, the agent reads progress.md to understand current state. As it completes steps, it updates the file. It also cross-references plan.md to ensure it stays on the planned path rather than improvising.

This is low-tech but extraordinarily effective. The agent doesn't need to re-understand the entire codebase at the start of each session — it reads its own notes and picks up exactly where it left off. The context window investment at the start of each session stays small. And it's version-controlled, human-readable, and costs almost nothing in tokens to load.

💡 Think of progress.md as your agent's working memory that persists across sessions. The context window resets; the file doesn't. This is the Docs as Code principle applied to agent state: if it matters, write it down in Markdown and commit it.

🎯 Key Takeaways

Agentic workflows feel complex until you notice the common thread running through every step: it's all Markdown. The documentation you load, the context files you maintain, the plans you generate, the reviews you pass between agents, the state you persist across sessions — Markdown all the way down. Here's what that looks like in practice:

• 📄 Markdown is 80% cheaper than HTML. Cloudflare measured this on a real documentation page: 16,180 tokens as HTML, 3,150 as Markdown. LangChain docs has native Markdown export and a built-in MCP server. Use Accept: text/markdown for Cloudflare-hosted sites. Use MarkItDown MCP for everything else. Every token you save on packaging is a token your agent can spend on your codebase.
• 📝 Your context files are Markdown too. agents.md gave the agent ground truth about the codebase architecture. It's version-controlled, human-readable, and token-efficient. Docs as Code isn't just for documentation sites — it's for everything you want your agent to know.
• 🗺️ Plans are Markdown artefacts. Agentic plan mode produces a plan.md you can read, edit, and hand to a sub-agent. The format flows through the entire pipeline: Markdown in, Markdown out.
• 🔬 Markdown makes context portable across agents. The principal engineer review works because plan.md is a self-contained document. No conversation state, no special format — just a text file passed to a fresh context window. A fresh context window is a fundamentally different perspective; it catches what your planning session normalised.
• 📋 progress.md is your agent's external memory. The context window resets; the file doesn't. Markdown state files let your agent pick up exactly where it left off across sessions, across days, across whatever interruptions real work involves.

The entire migration — five agents, a major version jump, breaking changes throughout — completed successfully. But the bigger insight isn't about LangChain. It's that Markdown, treated as a first-class engineering artefact, is what makes complex agentic workflows tractable. Token-efficient context in, structured context out. The format does more work than you'd expect.

Enjoyed this? The full Context Engineering series:

• 📝 Part 1 — The four pillars of context engineering
• 🛠️ Part 2 — Agent skills and the Agentic AI Foundation
• 🏃 Part 3 — Why AI-powered teams need lean, not Scrum
• 🌳 Part 4 — Git worktrees and agent skills for parallel workflows

From solo agent to parallel powerhouse: how Git worktrees and agent skills turn your agentic coding setup into a multi-tasking machine

When you're using agentic coding tools seriously, you've hit this wall:

• 🤖 Agent working on Feature A in one terminal
• 🤖 Need to start Feature B right now
• 💥 Can't — same working directory, same files, chaos ensues

Whether you're running Claude Code in the terminal, using Cursor's Composer, or GitHub Copilot's agent mode in VS Code, the problem is identical: one working directory, one set of files, one task at a time.

That's frustrating when you're paying for AI horsepower and want to parallelise.

In this post, I'll show you how Git worktrees solve the isolation problem, how agent skills teach your AI to manage worktrees autonomously, and how the entire workflow — from branch creation to PR — can be streamlined across multiple parallel sessions.

💡 This is Part 4 in my Context Engineering series. In Part 1, we covered the four pillars of context engineering. In Part 2, we introduced agent skills as portable capabilities. In Part 3, we argued why AI-powered teams need lean methodology over Scrum ceremonies. Now we put skills to work.

🌳 Git Worktrees: The Parallel Sessions Solution

Git worktrees let you check out multiple branches from the same repository into separate directories. Each worktree has its own working copy with isolated files while sharing the same Git history and .git directory. Unlike cloning the repo multiple times, worktrees are lightweight — they share the Git object store, refs, and hooks, so there's no duplicated history or divergent state to manage.

# Create a new worktree with a new branch
git worktree add ../project-feature-auth -b feature/authentication

# Create a worktree from an existing branch
git worktree add ../project-hotfix-123 hotfix/critical-bug-123

# List all your worktrees
git worktree list

# Remove when done
git worktree remove ../project-feature-auth  

Why this matters for agentic coding:

Each worktree is a fresh checkout. Depending on your stack, you'll need to install dependencies — pnpm install, uv sync, bundle install, etc. That's a small price for having three AI agents working simultaneously without stepping on each other.

Claude Code's own documentation recommends exactly this pattern for running parallel sessions. But the docs stop at the manual git worktree commands — what if we could teach the agent to handle worktree management for us?

🛠️ Agent Skills: Teaching Your AI Git Worktrees

In Part 2 of this series, I introduced agent skills — folders of instructions, scripts, and resources that agents can discover and use. Skills are an emerging convention adopted by Claude Code, Cursor, VS Code, GitHub, Goose, and others. The idea is simple: you drop a skill folder into your project, and the agent automatically discovers it and knows when to use it — no manual configuration required.

Here's the thing: you could teach your agent the git worktree commands every session. Or you could teach it once and have it just know how to manage worktrees.

That's exactly what I built: a git-worktree skill that gives Claude Code (or any skill-aware agent) full worktree management capabilities.

The Skill Structure

.claude/skills/git-worktree/
├── SKILL.md          # Instructions the agent reads
└── scripts/
    └── worktree.py   # The actual tooling

The SKILL.md tells the agent what the skill does and — critically — when to activate. The description field acts as a trigger: when you ask something that matches, the agent discovers and loads the skill automatically:

---
name: git-worktree  
description: Git worktree management for parallel development workflows.  
  Use when the user wants to work on multiple branches simultaneously
  without stashing, create isolated environments for features/bugfixes,
  or manage worktrees (create, list, remove, prune). Triggers on
  requests like "create a worktree", "work on branch X in parallel",
  "set up a new feature branch environment".
---

The Python script handles the heavy lifting — creating worktrees, auto-detecting package managers, installing dependencies, and even opening your preferred editor:

# Auto-detects and installs dependencies for any stack
def install_dependencies(path: Path) -> None:  
    if (path / "package-lock.json").exists():
        subprocess.run(["npm", "ci"], cwd=path, check=False)
    elif (path / "pnpm-lock.yaml").exists():
        subprocess.run(["pnpm", "install", "--frozen-lockfile"], cwd=path, check=False)
    elif (path / "uv.lock").exists():
        subprocess.run(["uv", "sync"], cwd=path, check=False)
    elif (path / "go.mod").exists():
        subprocess.run(["go", "mod", "download"], cwd=path, check=False)
    # ... also supports yarn, bun, poetry, pipenv, bundler, cargo

Quick Reference

These are the commands the agent runs under the hood. You don't need to type them — just describe what you want in natural language and the skill handles the rest.

Supported editors: VS Code, Cursor, IntelliJ IDEA, WebStorm, PyCharm, Zed, Sublime Text, Vim, Neovim.

⚡ The Skill in Action

Here's what a parallel agentic workflow looks like with the skill installed.

Scenario: Feature work + urgent hotfix

You're halfway through a feature when a critical bug lands. Instead of stashing or committing half-finished work:

# Terminal 1: You're working on the feature
claude  
> "Create a worktree for fix/critical-bug-123 from main, install deps"

# The agent recognises the request, activates the git-worktree skill,
# and runs the script:
# ✓ Created worktree at ../project-fix/critical-bug-123
# ✓ New branch 'fix/critical-bug-123' from 'main'
# ✓ Running pnpm install --frozen-lockfile...
# ✓ Worktree ready: ../project-fix/critical-bug-123

# Terminal 2: Open a new Claude Code session in the hotfix worktree
cd ../project-fix/critical-bug-123  
claude  
> "Fix the null pointer in the payment handler"
# Agent works in complete isolation from your feature branch

# Terminal 1: Meanwhile, your feature work continues uninterrupted
> "Continue implementing the user auth flow"

Scenario: Reviewing a PR without context switching

> "Create a worktree from origin/feat/someone-elses-work for code review"

# Agent:
# ✓ Created worktree at ../project-feat/someone-elses-work
# ✓ Using existing branch 'feat/someone-elses-work'
# ✓ Worktree ready

No stashing. No committing half-finished work. No branch juggling. Each agent gets its own isolated directory, its own dependencies, its own context.

🚀 The Full Loop: Commit Commands Plugin

Git worktrees handle the beginning of the workflow — creating isolated environments. But what about the end? Committing, pushing, and opening a PR.

This is where Claude Code's commit-commands plugin completes the loop.

A quick distinction: skills teach the agent domain knowledge (like worktree management) and are activated automatically when relevant, while plugins are installable packages that bundle slash commands, hooks, and other extensions. You install plugins from Claude Code's marketplace:

# Browse available plugins
> /plugin

# Or install directly from the official marketplace
> /plugin install commit-commands@claude-plugins-official

Plugins are namespaced, so the commit-commands plugin provides commands like /commit-commands:commit. For details on the plugin system, see the Claude Code plugins documentation.

The commit-commands plugin provides three slash commands that work beautifully in tandem with the worktree skill:

/commit-commands:commit — Context-Aware Commits

> /commit-commands:commit

# Claude will:
# - Review staged and unstaged changes
# - Match your repo's commit message style
# - Draft an appropriate message
# - Stage and commit

The AI reads your recent commit history and generates messages that match your team's conventions. No more generic "fix stuff" commits.

/commit-commands:commit-push-pr — Branch to PR in One Step

This is the real power move. After finishing work in your worktree:

> /commit-commands:commit-push-pr

# Claude will:
# - Create a feature branch (if needed)
# - Commit with a meaningful message
# - Push to origin with tracking
# - Create a PR via GitHub CLI with auto-generated description
# - Return the PR URL

The PR description includes a summary of all commits in the branch (not just the latest), a test plan checklist, and proper attribution.

/commit-commands:clean_gone — Worktree Cleanup

After your PRs are merged and remote branches deleted, cleanup is one command:

> /commit-commands:clean_gone

# Claude will:
# - Find branches marked as [gone]
# - Remove associated worktrees
# - Delete stale local branches
# - Report what was cleaned up

The Complete Workflow

Here's the full loop in practice:

# 1. Create isolated worktree (git-worktree skill)
> "Create a worktree for feat/user-auth, install deps, open in VS Code"
# ✓ Worktree ready: ../project-feat/user-auth

# 2. Open a new Claude Code session in the worktree
cd ../project-feat/user-auth  
claude  
> "Implement OAuth2 integration"

# 3. Work is done — commit and create PR (commit-commands plugin)
> /commit-commands:commit-push-pr
# ✓ Committed: Add OAuth2 provider integration
# ✓ Pushed to origin/feat/user-auth
# ✓ PR #142 created: https://github.com/...

# 4. PR is reviewed, approved, and merged on GitHub
# Back in your main worktree, clean up stale branches:
> /commit-commands:clean_gone
# ✓ Removed worktree: ../project-feat/user-auth
# ✓ Deleted branch: feat/user-auth

From worktree creation to PR to cleanup — streamlined across your parallel sessions.

🔄 Transferable to Other Tools

I'm showcasing this with Claude Code because it's my primary agentic tool, but the concept is transferable. The worktree workflow isn't tied to any single tool — it's a pattern you can bring to any agentic coding environment.

If you're using GitHub Copilot's agent mode, you can achieve the same worktree workflow with a reusable prompt. Create a file at .github/prompts/git-worktree.prompt.md:

---
description: Create and manage Git worktrees for parallel development  
tools: ['run_in_terminal', 'get_terminal_output']  
---

# Git Worktree Management

When the user asks to create a worktree, work on a branch in parallel,  
or set up an isolated environment:

1. Determine the branch name and base branch (default: main)  
2. Create a sibling directory: `git worktree add ../repo-name-branch -b <branch>`  
3. Detect and install dependencies (check for package-lock.json, pnpm-lock.yaml,  
   uv.lock, go.mod, etc.)
4. Report the worktree path so the user can open a new agent session there  

Then invoke it in Copilot's agent mode with /git-worktree.

The key difference is structural: Claude Code skills can bundle executable scripts alongside the instructions (like the worktree.py script), while Copilot reusable prompts rely on the agent generating and running commands from the prompt's instructions alone.

The key insight from Part 2 holds: agent skills and reusable prompts are two implementations of the same idea — teaching your AI once so you never repeat yourself.

📊 Why This Matters for Agentic Workflows

The combination of Git worktrees + agent skills + commit commands creates a workflow that's more than the sum of its parts:

💡 The key insight: Git worktrees solve the parallelisation problem at the file system level. Agent skills solve the repetitive teaching problem at the cognitive level. Commit commands solve the Git ceremony problem at the workflow level. Together, they let you focus on directing the AI agents, not managing the infrastructure around them.

🎬 Getting Started

Prerequisites: Claude Code installed and authenticated, Python 3.10+, and GitHub CLI (gh) installed and authenticated (needed for PR creation).

1. Add the git-worktree skill to your project

Copy the skill into your project's .claude/skills/ directory. You can grab it directly from the repository:

# From your project root
mkdir -p .claude/skills/git-worktree/scripts

# Download the skill files
curl -sL https://raw.githubusercontent.com/shavo007/langchain-anthropic-pdf-support/main/.claude/skills/git-worktree/SKILL.md \  
  -o .claude/skills/git-worktree/SKILL.md

curl -sL https://raw.githubusercontent.com/shavo007/langchain-anthropic-pdf-support/main/.claude/skills/git-worktree/scripts/worktree.py \  
  -o .claude/skills/git-worktree/scripts/worktree.py

Claude Code automatically discovers skills in .claude/skills/ — no configuration needed.

The full skill source is at: github.com/shavo007/langchain-anthropic-pdf-support/.claude/skills/git-worktree

2. Install the commit-commands plugin

Install it from Claude Code's official marketplace:

> /plugin install commit-commands@claude-plugins-official

Once installed, /commit-commands:commit, /commit-commands:commit-push-pr, and /commit-commands:clean_gone are available in any session. See the commit-commands README for full details.

3. Create your first parallel worktree

claude  
> "Create a worktree for feat/my-first-parallel-task, install deps"

The agent will activate the skill, create a sibling directory, set up the branch, and install your project's dependencies. Then open a new terminal, cd into the worktree path the agent reports, and run claude to start a parallel session.

4. For Copilot users

Create .github/prompts/git-worktree.prompt.md in your repository using the reusable prompt example from the Copilot section above. Then invoke it in agent mode with /git-worktree.

🎯 Key Takeaways

1. Git worktrees provide complete file isolation for running multiple agentic sessions in parallel
2. Agent skills teach your AI worktree management once — no more repeating instructions
3. The concept is transferable across tools: Claude Code skills, Copilot reusable prompts, Cursor rules
4. Plugins like commit-commands complete the loop: from worktree creation through PR and cleanup
5. Teach once, reuse forever — the core principle of context engineering applied to your Git workflow

🔗 Resources

• 🌳 Git Worktree Skill — The skill showcased in this post
• 🚀 Commit Commands Plugin — Claude Code's commit, push, and PR commands
• 💻 Claude Code: Parallel Sessions with Git Worktrees — Official Claude Code docs on worktree workflows
• 📖 Git Worktree Documentation — Official Git docs
• 📝 Context Engineering Part 1 — The four pillars
• 🛠️ Context Engineering Part 2 — Agent skills and the Agentic AI Foundation
• ⚡ Context Engineering Part 3 — Scrum is Dead: why AI-powered teams need lean methodology

Why AI-powered teams need lean methodology, not ceremonies

Let me ask you something.

How many hours did you spend in meetings this week? Sprint planning. Daily standups. Backlog grooming. Sprint review. Retro.

Now ask yourself: did any of those meetings help you ship faster?

• 🗓️ Are your sprints constantly disrupted by changing priorities?
• 📝 Do you spend more time writing Jira tickets than writing code?
• 🔄 Does "Agile" feel anything but agile?

In Part 1 and Part 2 of this series, I showed how Context Engineering—giving AI the right information at the right time—can cut project costs by 98%. But there's a problem: your process can't keep up.

When a Spring Boot migration takes 15 minutes instead of 2-3 days, when AI agents can execute complex workflows autonomously—the 2-week sprint starts to feel... quaint.

Here's the uncomfortable truth:

💡 Scrum has become the very thing the Agile Manifesto rebelled against.

In Parts 1 and 2, we covered the technical side of Context Engineering—AGENTS.md, MCPs, Custom Agents, and Prompts. But here's what I didn't address: what happens to your team's process when AI can complete a sprint's worth of work in an afternoon?

🎯 What the Agile Manifesto Actually Said

Let's go back to 2001. The Agile Manifesto was a rebellion against heavyweight, document-driven development. It valued:

• 👥 Individuals and interactions over processes and tools
• 💻 Working software over comprehensive documentation
• 🤝 Customer collaboration over contract negotiation
• 🔄 Responding to change over following a plan

Notice what's NOT in the Manifesto:

• ❌ Two-week sprints
• ❌ Story points
• ❌ Jira
• ❌ Daily standups
• ❌ Sprint ceremonies
• ❌ Velocity tracking
• ❌ Backlog grooming

The Manifesto was about principles, not processes. Scrum added the process layer—and over two decades, that layer calcified into the very bureaucracy Agile sought to escape.

📜 We replaced waterfall documentation with Jira stories. We replaced status meetings with daily standups. We swapped one bureaucracy for another.

⚡ Why AI Breaks Scrum

The fundamental assumption of Scrum is that software development happens at a predictable, human pace. Sprint planning assumes you can estimate work. Velocity tracking assumes consistency. Two-week iterations assume that's a reasonable feedback loop.

AI development breaks all of these assumptions.

The Speed Mismatch

In my Context Engineering series (Part 1 and Part 2), I documented real results:

When tasks complete in minutes instead of days:

• 📅 Sprint planning becomes obsolete before the sprint ends
• 🎯 Estimates are meaningless when AI can complete tasks 10x faster than expected
• 📋 The backlog churns faster than you can groom it

Paul Dix calls this The Great Engineering Divergence. He applies Amdahl's Law—a principle from computer performance theory—to software delivery:

📐 "If coding is 20% of the end-to-end cycle, making it 10x faster only yields ~1.25x overall speedup. Once coding speed jumps, everything around it becomes the constraint."

Developers report 20-50% productivity gains from AI, but Dix argues they could achieve 10x, 100x or more—if their pipelines could handle it. The bottlenecks aren't in the code. They're in code review bandwidth, testing capacity, release confidence, security gates, and sprint ceremonies.

In other words: Scrum is the constraint.

🗣️ What the Industry Leaders Are Saying

I'm not alone in this observation. The developers building with AI every day are reaching the same conclusion.

Geoff Huntley (Creator of Ralph Wiggum Technique)

"Agile and standups doesn't make sense any more."

"The days of being a Jira ticket monkey are over."

Huntley's Ralph Wiggum technique runs AI in continuous loops:

while :; do cat PROMPT.md | claude-code ; done  

This deceptively simple script feeds a prompt file into Claude Code repeatedly—the AI attempts a task, evaluates its own errors, and retries until it succeeds. At ~$10/hour in compute costs, this technique has cloned Atlassian products and created entire programming languages. Humans stay "in the loop" but enter later and less frequently than traditional workflows demand.

Peter Steinberger (PSPDFKit Founder)

From his Pragmatic Engineer interview:

"Pull requests are dead, long live 'prompt requests.'"

"Code reviews are dead for this workflow—architecture discussions replace them."

"I ship code I don't read."

Steinberger runs 5-10 AI agents simultaneously, maintains flow state throughout development, and deliberately rejects remote CI in favour of local validation. Why wait 10 minutes for a pipeline when agents can validate in seconds?

🔄 The Lean Alternative

So what replaces Scrum? Return to the Manifesto's principles—but with AI-native practices. If you've been following this series, you'll recognise Agentic Context Engineering at work here.

From Ceremonies to Continuous Flow (Agentic Context Engineering in Action)

📋 AGENTS.md: Project Context for AI

In my langchain-anthropic-pdf-support project, I use AGENTS.md to document everything an AI agent needs to work on the codebase.

What AGENTS.md replaces:

• 📚 Confluence documentation
• 🎓 Onboarding wikis
• 🧠 Tribal knowledge
• 📝 Acceptance criteria patterns (the reusable parts)

What AGENTS.md doesn't replace: You still need to capture the task itself—what you're working on right now. That might be a Kanban card, a GitHub issue, or a prompt to the agent. Tools like OpenSpec take this further with spec-driven development—you write specs before code, and AI agents read them to understand intent. The difference is that task definitions become lightweight because all the project context lives in AGENTS.md.

Example structure:

## Development Environment
- Run agent: `uv run poe dev`
- FastAPI server: `uv run poe serve`

## Testing Strategy
- Unit tests: `uv run poe test`
- Full CI locally: `uv run poe ci`

## Key Architecture Decisions
- Default model and why
- Caching strategy
- API design patterns

## Code Patterns
- How to add new tools
- Testing conventions

## Important Gotchas
- API key requirements
- Coverage targets (80%)

💡 An AI agent can read AGENTS.md and immediately know how to run, test, and contribute to the project. Give it a task—via a Kanban card, GitHub issue, or prompt—and it has all the context it needs.

⚡ Local CI: Shift Left to Seconds

Why wait for remote CI when you can validate locally in seconds?

In my projects, I use pre-commit hooks with 9 automated checks:

• 🔍 ruff — Linting and formatting
• 🔒 detect-private-key — Security check
• 📝 mypy — Strict type checking
• ✅ check-yaml, check-merge-conflict — File hygiene

The workflow comparison:

Traditional:  
Code → Commit → Push → Wait 10min for CI → Fix → Repeat

AI-native:  
Code → Agent validates locally → Commit → Push → CI passes first time  

The command uv run poe ci mirrors exactly what remote CI would do, but runs locally in seconds. Combined with pre-commit hooks, problematic code never leaves your machine.

There are two ways to achieve this with Claude Code:

Approach 1: Hooks. Configure hooks to automatically run linting and formatting after every file change. The agent validates its own work in real-time, catching issues before you even see them.

Approach 2: Document commands in AGENTS.md. Tell the agent what validation commands exist so it knows how to verify its changes:

## Validation
- After editing files: `uv run poe check` (lint + format + typecheck)
- Before committing: `uv run poe ci` (full validation suite)

The agent reads this, runs the checks, and self-corrects.

🎬 Making the Transition

You don't have to abandon everything overnight. Start here:

Week 1: Add AGENTS.md

Document your project's development workflow, testing strategy, and architecture decisions. Make it executable—commands, not prose.

Week 2: Set Up Local Validation

Configure pre-commit hooks. Add a poe ci or equivalent command that mirrors your remote pipeline. Stop waiting for GitHub Actions.

Week 3: Question Each Ceremony

For every meeting on your calendar, ask: "Does this help us ship faster?" If the answer is "we've always done it this way," that's not a reason.

Week 4: Measure

Track time spent in ceremonies vs. time shipping. The numbers will speak for themselves.

🎯 Key Takeaways

• 1️⃣ The Agile Manifesto never prescribed Scrum — ceremonies were added later
• 2️⃣ AI development happens at inference speed — 2-week sprints can't keep up
• 3️⃣ AGENTS.md replaces scattered documentation — project context lives in the repo, not Confluence
• 4️⃣ Local CI replaces remote waiting — validate in seconds, not minutes
• 5️⃣ Architecture discussions replace code reviews — review prompts, not generated code
• 6️⃣ Context Engineering gave you the tools — now remove the constraints that slow them down

✨ The days of being a Jira ticket monkey are over. The question is: will you adapt—reviewing architecture instead of syntax, shipping continuously instead of in sprints—or will you keep attending standups while the industry moves on?

🔗 Resources

This Series:

• 📖 Context Engineering Part 1 — The four pillars
• 📖 Context Engineering Part 2 — AAIF and real examples

References:

• 📐 Paul Dix: The Great Engineering Divergence — Amdahl's Law applied to delivery
• 📰 Pragmatic Engineer: I Ship Code I Don't Read
• 📰 The Register: Ralph Wiggum Technique
• 📜 The Agile Manifesto — What it actually says
• 🐙 langchain-anthropic-pdf-support — AGENTS.md and pre-commit example

From Theory to Practice: How the industry is converging on context engineering standards

In Part 1, we explored the fundamentals of context engineering—the four pillars, the hybrid model, and why teaching AI once beats fighting it every time.

Now let's get practical. In this post, I'll show you:

• 🏛️ How the Agentic AI Foundation validates this approach
• 📋 Why AGENTS.md is becoming the new standard
• 🚀 A real case study: 98% cost reduction on Spring Boot 4.0 migrations

🏛️ The Agentic AI Foundation: Industry Convergence

Something significant happened in December 2025. The Agentic AI Foundation was announced—a directed fund under the Linux Foundation ensuring agentic AI evolves transparently and collaboratively.

The founding members might surprise you:

Supporting members include Google, Microsoft, AWS, Cloudflare, and Bloomberg.

💡 The key insight: The industry is converging on the same patterns I covered in Part 1. AAIF formalises what forward-thinking teams have already been doing.

📋 The Three Core Contributions

AGENTS.md (from OpenAI)

A standardised format for AI agent instructions—think of it as a "README for AI agents".

# AGENTS.md

## Dev environment tips
- Use `pnpm install` for dependencies
- Run `pnpm dev` for local development

## Testing instructions
- Run `pnpm test` before committing
- Ensure 80% coverage minimum

## PR instructions
- Title format: [JIRA-123] Description
- Always run linting before push

This maps directly to the Instructions pillar from Part 1. The difference? Now it's an open standard adopted across the industry.

MCP (from Anthropic)

The Model Context Protocol is the "USB-C for AI tools"—an open standard for connecting AI to external systems.

• 📊 10,000+ active MCP servers
• 📦 97M+ monthly SDK downloads
• 🔧 Adopted by VS Code, Cursor, ChatGPT, Gemini

This is the MCPs pillar formalised. Instead of copying documentation into prompts (which gets stale), MCP gives AI direct access to authoritative, always-current sources.

Goose (from Block)

An open-source autonomous AI agent that "actually does the work".

• 🏗️ Builds entire projects
• 🐛 Executes and debugs code
• 🔌 Native MCP integration
• ⭐ 24.8k+ GitHub stars

This maps to the Custom Agents pillar—role-based AI behaviour for different tasks.

🗺️ Mapping AAIF to the Four Pillars

Here's how the Agentic AI Foundation contributions extend the framework from Part 1:

🛠️ Agent Skills: Portable Capabilities

In December 2025, Anthropic published Agent Skills as an open standard—folders of instructions, scripts, and resources that agents can discover and use.

---
name: spring-boot-migration  
description: Migrates Spring Boot 3.x apps  
  to 4.0 with dependency modularisation,
  Jackson 3 updates, and Jakarta EE 11.
license: Apache-2.0  
metadata:  
  author: your-name
  version: "1.0"
---

## Instructions
1. Analyze current dependencies...  
2. Update starter modules...

## References
See [migration guide](references/GUIDE.md)  

Skills enable:

• 🧠 Domain Expertise - Package specialized knowledge into reusable instructions
• 🔄 Interoperability - Reuse the same skill across different agent products
• ⚡ New Capabilities - Give agents abilities like creating presentations or analysing datasets
• ✅ Repeatable Workflows - Turn multi-step tasks into consistent, auditable workflows

Adopted by: Cursor, VS Code, GitHub, Claude Code, Goose, OpenAI Codex, and Factory.

📖 Case Study: Spring Boot 4.0 Migration

Let me show you context engineering in action with a real migration project.

The Meta-Instruction Pattern

I used a meta-instruction pattern—having AI generate the migration instructions, then use those same instructions to execute the migration.

The secret weapon? The instructions.instructions.md file from the awesome-copilot repository—a meta-instruction that teaches AI how to write high-quality instructions.

Phase 1: Generate Migration Instructions (5 minutes)

Using #file:instructions.instructions.md  
and examples like  
#file:java-21-to-java-25-upgrade.instructions.md

I want to define new instructions for  
Spring Boot 4 migration guide.

You can find the guide here:  
https://github.com/spring-projects/  
spring-boot/wiki/  
Spring-Boot-4.0-Migration-Guide

User Feedback:  
"Can focus on gradle and kotlin
as that is what we mostly use"  

What the AI does:
1. Fetches official Spring Boot 4.0 guide
2. Analyzes existing instruction patterns
3. Extracts key migration themes
4. Incorporates project preferences (Gradle/Kotlin)
5. Generates springboot-4-migration.instructions.md

Result: 1,400+ line comprehensive migration guide created in 3 minutes.

Phase 2: Execute Migration (10 minutes)

Migrate this project to Spring Boot 4.0  
using the generated instructions in  
#file:springboot-4-migration.instructions.md

What Plan Agent Mode does:

✅ Read migration instructions
✅ Analyze current pom.xml
✅ Update dependencies:
   spring-boot-starter-web →
   spring-boot-starter-webmvc
✅ Add spring-boot-starter-webmvc-test
✅ Verify Jackson property paths
✅ Run: ./mvnw clean compile
✅ BUILD SUCCESS

🎯 The key innovation: AI generates the migration guide... then AI uses that guide to execute the migration.

Giving Back to Open Source

The Spring Boot 4.0 migration instructions I generated have been contributed back to the community. You can use them right now in your own projects—no need to recreate the wheel.

The ROI: 98% Cost Reduction

The numbers speak for themselves. What used to cost $20,000 across 10 applications now costs $350—and you get better quality with zero errors.

👨‍💻 The Developer's Role Evolves

This isn't about replacing developers. It's about changing what we do.

The shift is clear: fewer repetitive tasks, more strategic thinking. AI handles the implementation grind while you focus on what matters—architecture, requirements, and business value.

🎓 Validation from the Source

Here's something that caught my attention. Boris Cherny, the creator of Claude Code at Anthropic, shared his actual workflow on X:

What's fascinating is how his practices map directly to the context engineering pillars:

💡 "Every mistake becomes a rule." — Boris Cherny

💰 On using Opus (the best model): "Paying the 'compute tax' upfront eliminates the 'correction tax' later."

The exact same patterns work in production at one of the world's leading AI companies. Independent validation that this approach works at scale.

🚀 Getting Started

Essential MCP Configuration

Here's a practical MCP setup for local development:

{
  "servers": {
    "atlassian/atlassian-mcp-server": {
      "type": "stdio",
      "command": "pnpm",
      "args": ["dlx", "mcp-remote",
        "https://mcp.atlassian.com/v1/sse"]
    },
    "github": {
      "type": "http",
      "url": "https://api.githubcopilot.com/mcp/"
    },
    "context7": {
      "type": "stdio",
      "command": "pnpm",
      "args": ["dlx", "@upstash/context7-mcp@latest"]
    }
  }
}

What this enables:

• 📋 Atlassian MCP - Query Jira issues, search Confluence, create tickets from chat
• 🐙 GitHub MCP - Search repositories, read files, create pull requests
• 📚 Context7 MCP - Fetch latest library documentation, always up-to-date API references

The Awesome Co-Pilot Repository

The awesome-copilot repository is a goldmine for context engineering. It's a community-driven collection of 163+ instruction files covering virtually every technology stack you might work with.

What's inside:

• 💻 Programming languages - Python, Java, TypeScript, Go, Rust, and more
• 🔧 Frameworks - React, Vue, Spring Boot, Django, FastAPI
• ☁️ Infrastructure - Terraform, Docker, Kubernetes, GitHub Actions
• 🔒 Security - OWASP standards, secure coding practices

The meta-instruction magic:

The real power comes from instructions.instructions.md—a meta-instruction that teaches AI how to write high-quality instructions. It defines:

• 📋 Required frontmatter structure - Description and file pattern matching
• 📝 Organized content sections - Guidelines, best practices, code standards
• ✅ Concrete examples - Both recommended and discouraged approaches
• 🧪 Validation requirements - Build commands, linting, testing steps

This is the bootstrap for context engineering—use it to generate domain-specific instructions for your own projects, then use those instructions to guide AI through complex tasks.

🔗 Resources

• 🛠️ awesome-copilot MCP Server - github.com/github/awesome-copilot
• 🍃 Agent Skills Specification - github.com/anthropics/agent-skills
• 📖 Official Spring Boot 4.0 Migration Guide - Spring Boot Wiki
• 🏛️ Agentic AI Foundation - agenticaifoundation.org

🎯 Key Takeaways

• 1️⃣ AAIF validates the approach - The industry is standardising on context engineering
• 2️⃣ AGENTS.md standardises instructions - One format, every agent
• 3️⃣ Meta-instructions accelerate adoption - Have AI generate the guides, then use them
• 4️⃣ 98% cost reduction is achievable - This isn't theoretical, it's measured

✨ The future isn't about better prompts. It's about better context.

How I went from repeating instructions every conversation to having AI that actually understands my codebase

Let me ask you a few questions:

• 😤 Are you frustrated with AI giving inconsistent results?
• 🔁 Do you find yourself repeating the same instructions over and over?
• 🤔 Do you wish AI just knew your project better?

If you answered yes to any of these, you're not alone. And here's the thing—it's not the AI's fault. It's how we're feeding it information.

I spent months fighting with AI assistants, carefully crafting prompts, only to get generic code that didn't follow our architecture. Then I discovered context engineering, and everything changed.

🎯 The Fundamental Distinction: Prompt Engineering vs. Context Engineering

Most developers focus on prompt engineering—crafting the perfect question to get the right answer. But there's a more powerful approach.

Here's the key insight:

💡 Prompt engineering is fighting the AI each time. Context engineering is teaching the AI once.

Think about onboarding a new developer. You don't explain your coding standards every time they write a function. You point them to documentation, show them examples, and let them absorb the patterns. Context engineering does the same thing for AI.

🧠 Understanding Context Windows: The Hybrid Model

Every AI has a context window—its working memory. Claude 3.5 Sonnet has 200,000 tokens (roughly 150,000 words). That sounds like a lot, but it's finite and precious, like RAM in a computer.

The challenge: you can't include everything. So you need to be strategic.

The solution is a hybrid approach:

┌──────────────────────────────┐
│  ⬆️  UPFRONT CONTEXT         │
│  • Standards & conventions   │
│  • Architecture patterns     │
│  • Team workflows            │
├──────────────────────────────┤
│  ⚡ JUST-IN-TIME             │
│  • File contents (tools)     │
│  • External docs (MCPs)      │
│  • Real-time data (APIs)     │
└──────────────────────────────┘

🎯 The principle is simple: Give agents the RIGHT information at the RIGHT time, not ALL information ALL the time.

Your project structure itself becomes a signal. Folder hierarchy teaches purpose. Naming conventions show role. Even timestamps signal patterns. A well-organized codebase is already teaching the AI.

✨ The Power of Context: A Real Example

Let me show you what this looks like in practice.

❌ Without Context Engineering

Request: "Create a new user management API endpoint"

Result:

from flask import jsonify  
from models import User

@app.route('/api/users', methods=['GET'])
def get_users():  
    users = User.query.all()
    return jsonify([u.__dict__ for u in users])

The problems? Direct database access in the route, no service layer, no validation, exposes ORM objects directly, no error handling, no tests. It's generic code that doesn't follow your architecture.

✅ With Context Engineering

Same request: "Create a new user management API endpoint"

Result:

# src/api/routes/users.py
from fastapi import APIRouter, Depends  
from src.application.users.queries import GetUsersQuery  
from src.application.users.schemas import UserListResponse

router = APIRouter(prefix="/api/users", tags=["users"])

@router.get("/", response_model=UserListResponse)
async def get_users(query: GetUsersQuery = Depends()):  
    return await query.execute()

# src/application/users/queries.py
class GetUsersQuery:  
    def __init__(self, user_service: UserService = Depends()):
        self.user_service = user_service

    async def execute(self) -> UserListResponse:
        users = await self.user_service.get_all()
        return UserListResponse(users=[UserDTO.from_entity(u) for u in users])

⏱️ Time saved: 30-60 minutes per feature.

The AI didn't get smarter. It just had the right context.

🏛️ The Four Pillars of Context Engineering

I've distilled context engineering into four pillars. Each serves a distinct purpose, and together they create an environment where AI becomes a true collaborator.

📜 Pillar 1: Instructions

Purpose: Project-wide persistent rules that the AI should always know.

.github/
  copilot-instructions.md
  instructions/
    typescript.instructions.md
    react.instructions.md
    testing.instructions.md

This is where you document:

• ✅ Coding standards and conventions
• ✅ Architecture patterns your team follows
• ✅ Testing requirements
• ✅ Common pitfalls to avoid

📚 Think of instructions as the "employee handbook" for your AI assistant.

🎭 Pillar 2: Chatmodes (Personas)

Purpose: Role-based AI behavior for different tasks.

.github/chatmodes/
  principal-engineer.chatmode.md
  code-reviewer.chatmode.md
  test-engineer.chatmode.md

Example chatmode:

---
description: 'Thorough code reviewer'  
tools: ['editFiles', 'search']  
---
# Code Reviewer Mode

## Review Criteria
- Code quality and readability
- Performance implications
- Security vulnerabilities
- Test coverage

Different tasks need different mindsets. A code reviewer thinks differently than someone implementing a feature. Chatmodes let you switch the AI's perspective.

📝 Pillar 3: Prompts

Purpose: Standardize common tasks into reusable templates.

.github/prompts/
  create-api-endpoint.prompt.md
  write-unit-tests.prompt.md
  refactor-component.prompt.md

Example prompt:

---
mode: 'agent'  
description: 'Create REST API endpoint'  
---
# Create API Endpoint
1. Route handler in src/routes/  
2. Service layer in src/services/  
3. Add types in src/types/  
4. Unit + integration tests  
5. Update API docs  

Prompts ensure consistency. Every API endpoint follows the same structure. Every test file uses the same patterns. No more variation based on how you phrase the request.

🔌 Pillar 4: MCPs (Model Context Protocol)

Purpose: Connect AI to external tools and authoritative sources.

Popular MCPs include:

• 🐙 GitHub → Repository data, PRs, issues
• ☁️ AWS → Well-Architected Framework, best practices
• 📋 Atlassian → Jira tickets, Confluence documentation
• 🐘 PostgreSQL → Direct database queries
• 📚 Context7 → Up-to-date library documentation

🔑 The key advantage: Instead of copying documentation into your prompt (which gets stale), MCPs give AI direct access to authoritative, always-current sources.

📖 A Real-World Story: The Docker Challenge

Let me share how these four pillars came together for a real task.

🎯 The Challenge

I needed to document a business case for Docker hardened images across our infrastructure. The requirements:

• 🔧 Technical depth for engineers
• 💼 Business justification for leadership
• ☁️ Alignment with AWS Well-Architected Framework
• 📄 Delivered in Confluence
• ⏰ Timeline: 2-3 days

😓 The Traditional Approach

Day 1: Research (6-8 hours)  
Day 2: Draft document (6-8 hours)  
Day 3: Format and publish (4-6 hours)  
────────────────────────────────
Total: 16-22 hours 😱  

Plus 15-20 context switches, manual copy-paste between tools, and the inevitable documentation drift.

🚀 The Context Engineering Approach

My setup:

• 🎭 Chatmode: Principal Engineer (strategic thinking, architectural focus)
• 📜 Instructions: Business case template with our standard structure
• 🔌 MCPs: AWS Knowledge Base, Atlassian integration

My single request:

"Create business case for Docker hardened images using AWS Well-Architected Framework. Publish to Confluence."

What happened:

1. 🔍 Agent queried AWS MCP for framework requirements
2. 📋 Structured content using my business case template
3. ✍️ Generated comprehensive documentation
4. 📤 Published directly to Confluence via Atlassian MCP

✨ Result: 2 days of work compressed into 4 hours of focused execution.

No context switching. No manual copy-paste. No gaps in framework coverage. Flow state maintained throughout.

🎓 Key Principles for Effective Context Engineering

Through trial and error, I've identified six principles that make context engineering work:

1️⃣ Design for Discoverability

Structure your project so both humans AND agents can navigate intuitively.

src/features/auth/  
  components/  ← Purpose is clear
  services/    ← Logic lives here
  types/       ← Definitions grouped

💡 Good structure is good context.

2️⃣ Separate Persistent from Dynamic

• Persistent (upfront): Standards, patterns, workflows—things that rarely change
• Dynamic (just-in-time): File contents, external docs, real-time data—things that change frequently

Don't bloat your context window with information that should be fetched on demand.

3️⃣ Leverage Specialized Tools

Don't make AI memorize documentation. Connect it to the source.

4️⃣ Iterate and Measure

Track what matters:

• ⏱️ Time saved per task
• ⭐ Quality improvements
• 🔄 Number of iterations needed
• 😊 Developer satisfaction

📊 What gets measured gets improved.

5️⃣ Show, Don't Just Tell

Few-shot prompting works. Instead of writing a laundry list of rules, provide 3-5 high-quality examples showing expected behavior.

🖼️ For AI, examples are worth a thousand words.

6️⃣ Context Compounds

Better context leads to better results. Better results help you refine context. The cycle accelerates.

     ┌─────────────┐
     │   Better    │
     │   Context   │
     └──────┬──────┘
            │
            ▼
     ┌─────────────┐
     │   Better    │
     │   Results   │
     └──────┬──────┘
            │
            ▼
     ┌─────────────┐
     │  Refined    │
     │   Context   │◄──── 🔄 The cycle continues!
     └─────────────┘

This is why the initial investment pays dividends over time.

💰 The ROI: Why This Matters

Let's talk numbers:

⏱️ Time Investment:
├─ Initial setup: 2-3 hours
├─ Weekly maintenance: 30 minutes
└─ Quarterly review: 1 hour

💵 Weekly Savings: 8-17 hours recovered

🎉 Annual ROI: 400-800 hours saved

🚀 That's 10-20 weeks of productive time returned to you every year!

But it's not just about time. It's about:

• 🎯 Consistency: Every feature follows your patterns
• ⭐ Quality: Production-ready code from the start
• 🧘 Flow: No more context switching between docs and tools
• 👥 Onboarding: New team members (human and AI) ramp up faster

🌟 The Paradigm Shift

Here's what I've come to believe:

"Context engineering represents a fundamental shift in how we build with LLMs. As models become more capable, the challenge isn't just crafting the perfect prompt—it's thoughtfully curating what information enters the model's limited attention budget at each step."

This isn't about more prompting. It's about better context.

This isn't about AI as a tool. It's about AI as an informed collaborator.

The developers who master context engineering won't just be more productive—they'll be working in a fundamentally different way. One where AI doesn't fight them, but works alongside them with full understanding of the codebase, the patterns, and the goals.

🎬 Your Next Step

Start with one instruction file. Document your coding standards. Watch how the AI's output improves.

Then iterate.

✨ The best context is the one that evolves with your team.

We all know how vital shifting security left is and how engineering needs to work more closely with security. According to the Cloud Security Alliance (CSA), 70% of security professionals and engineering teams struggle to “shift left,” with many unable to recognize the formation of anti-patterns and understand/appreciate the wider impact to development, cost, governance, culture, etc.

Recognizing the Paradigm Shift

Engineering teams that implement DevSecOps practices and automate security tooling will discover security risks earlier, saving developers time, accelerating release cycles, and shipping more secure and compliant code.

As I have previously discussed, dependency management is an important part of securing your software supply chain. This post covers how to roll out Dependabot internally within your company to keep your dependencies up to date.

NB: There are other tools such as Renovate and Snyk, but as Dependabot is native to the GitHub platform I am focusing the talk on this tool.

Keeping your dependencies up to date is one of the easiest ways to keep your systems secure. The issue of supply chain security has become increasingly obvious in the past number of years, from the malicious flatmap-stream package to the most recent log4shell vulnerabilities. Dependabot will alert developers when a repository is using a software dependency with a known vulnerability. By rolling out Dependabot internally to all of your repositories, you can measure, and significantly reduce, your usage of software dependencies with known vulnerabilities.

Everyone will remember the security breach at Experian in 2017. Hackers stole the personal details of 143 million Americans from the Equifax credit report company after exploiting a security flaw in the Apache Struts framework. That is why it is so important to keep track of your dependencies and any known vulnerabilities.

OWASP published last year their top ten, and vulnerable and outdated components moved up to sixth position. Every organization must ensure an ongoing plan for monitoring, triaging, and applying updates or configuration changes for the lifetime of the application or portfolio. There are over 20,000 common vulnerabilities and exposures (CVEs) discovered per year in open-source and third-party code.

Empowering developers to be security minded

According to the SANS 2022 DevSecOps Survey: Creating a Culture to Significantly Improve Your Organization’s Security Posture, “management buy-in” was the number one factor contributing to DevSecOps security programs’ success.

What is Dependabot?

There are a number of guiding principles when evaluating tools and designing a rollout plan. For example, Does the security benefit of this new process outweigh the impact on engineering teams? How do we roll this out incrementally and gather feedback? What are our expectations for engineers, and how do we clearly communicate these expectations?

For Dependabot in particular, some of these questions are easy to answer. Dependabot is a native feature of GitHub, meaning, that it integrates with your engineers’ current workflows on GitHub.com. By better tracking the security of your software supply chain, you will keep your software secure, which outweighs any potential impact on engineering teams.

When Dependabot raises pull requests, these pull requests could be for security or version updates:

• Dependabot security updates are automated pull requests that help you update dependencies with known vulnerabilities.
• Dependabot version updates are automated pull requests that keep your dependencies updated, even when they don’t have any vulnerabilities. To check the status of version updates, navigate to the Insights tab of your repository, then Dependency Graph, and Dependabot.
• Dependabot security updates may include compatibility scores to let you know whether updating a dependency could cause breaking changes to your project.

Caveats

• Dependabot alerts - Owners of private repositories, or people with admin access, can enable Dependabot alerts by enabling the dependency graph and Dependabot alerts for your repositories.
• Secrets - When a Dependabot event triggers a workflow, the only secrets available to the workflow are Dependabot secrets. The simplest solution is to store the token with the permissions required in an action and in a Dependabot secret with identical names.
• Semantic versioning – don’t trust that every third party dependency aligns with this!
• Vulnerability database - GitHub's security features do not claim to catch all vulnerabilities and malware. They actively maintain GitHub Advisory Database and generate alerts with the most up-to-date information. However, they cannot catch everything or tell you about known vulnerabilities within a guaranteed time frame. These features are not substitutes for a human review of each dependency for potential vulnerabilities or any other issues, and they recommend consulting with a security service or conducting a thorough dependency review when necessary.

Beware of dependency fatigue

You may have heard of alert fatigue before when you are on a support roster. Well, the same can be said for dependency management: I call it dependency fatigue! The last thing you want is for teams to not configure dependabot properly and start to ignore the PRs and alerts created or turn them off completely. So what can we do to help alleviate this? Below I provide some tips and tricks I have used in the past and some sample configs.

Tips

• Design config with your team - Get the team to review the configuration and agree on the setup.
• Understand what package-ecosystem(s) to enable (docker, maven, actions, terraform etc)
• Add labels to your PRs to allow easier filtering
• Prioritise the order and schedule
• Assign ownership to who is on call
• If your dependencies are very outdated, you might want to start with a daily schedule until the dependencies are up-to-date, and then drop back to a weekly schedule.
• Review the changelog (look for any breaking changes announced) and make sure checks pass on your branch
• It's good practice to have automated tests and acceptance processes in place so that checks are carried out before the pull request is merged.
• Test and learn! 😅

Sample config

You can place this file dependabot.yml under your .github directory in your repo.

This example below showcases a repo that is interested in two eco-systems: docker and actions. I have added in a schedule for docker updates in my local timezone of Melbourne. And I have added some labels. This helps to filter your PRs if you want to focus on specific updates. The open-pull-requests-limit default is 5, so in this instance, I have reduced it to 2.

The next example showcases a sample NodeJS project. Again you are interested in updates from two ecosystems: npm and actions. What this config is also showcasing is how you can ignore certain dependencies. You can see that it ignores packages that start with aws, ignores updates to express and patch updates for all dependencies.

Lastly, is a sample config for a JVM-based project. Here we are covering three package ecosystems as it's a cloud-native microservice. It covers updates to maven, docker and actions.

Below is a sample PR that was created by dependabot based on the config above. You can see it has added the labels of dependencies and maven. It provides the release notes and changelog for you to review.

If you want to know more about the configuration options for dependabot, check out the doc here

Rollout

Dependabot, like other GitHub Advanced Security features, can be enabled for all repositories within an organization from the organization’s administration page. Depending on how you create new GitHub repositories (infra as code anyone??), you can also enable upon creation by default. That way there is no major impact on teams and a staged rollout is not needed. Frequent company-wide comms are recommended.

The aim is to answer, clearly and succinctly, the most important questions in your communications: What are we doing? Why are we doing this? When are we doing this? Lastly, what do I need to do? The last question was key. Make it clear that you are rolling out Dependabot organization-wide to understand your current risk and that, while we encourage service owners to upgrade dependencies, you are not expecting every Dependabot alert to be fixed right away.

Running an engineering workshop on dependency management and dependabot can help educate teams on the importance of this initiative and shifting security left. Showcasing how to manage security and version updates for your services via dependabot will set the foundations and kick-start the conversation. Like any change management framework, it is important to communicate/educate, build the capability and change the mindsets.

Remediation

Once you had Dependabot enabled for all your GitHub repositories, you could measure the general trend of Dependabot alerts across the company. At an org level under the insights tab, you have a bird's eye view of dependencies and open security advisories. You can filter by most critical and then switch your focus from measuring the current state to working with repository owners to upgrade their dependencies. By leveraging insights, you can ensure that you focus your remediation efforts on repositories that are running in production, where a vulnerable dependency could present a risk to your company and your users. As I mentioned earlier, "management buy-in" is extremely important in incorporating dependency management into your security program. Otherwise, you will end up competing with feature delivery and it will fall further down the pecking order. Security trumps customer centricity.

If you have a platform engineering team you could go even further and build or use a nice internal service catalog tool like backstage and hook into GitHub graphQL API to track this. As part of measuring system health and running an internal security program, you can work with your stream-aligned teams to define SLOs and SLIs regarding timeframes to resolve. An example could be:
"Critical and high severity vulnerable dependencies are fixed within 30 days".

It is important to acknowledge that not all Dependabot alerts can be actioned immediately. Instead, assign a realistic grace period for service owners to remediate Dependabot alerts before marking a metric as failing.

Outcomes

The goal here is to reduce the no. of Dependabot alerts over time for your services. But do not regard this as a once-off task. Rather, you need to follow the dependabot alert metrics continuously over time and intervene if you see them trending in the wrong direction. This will help you have a better picture of your system health across the company and prioritise work.

Happy updating! 😀

As part of adopting CD (Continuous Delivery) and embracing a DevOps culture, shifting security left is one of the capabilities needed to drive high software delivery and organisational performance.

Tech companies have identified that security is EVERYONE's responsibility. ie. Security is a distributed ownership model. What shifting left means is including concerns such as security earlier in the development lifecycle.

The principles of shifting left also apply to security, not only to operations. It’s critical to prevent breaches before they can affect users, and to move quickly to address newly discovered security vulnerabilities and fix them.

In software development, there are at least these four activities: design, develop, test, and release. In a traditional software development cycle, testing (including security testing), happens after development is complete. This typically means that a team discovers significant problems, including architectural flaws, that are expensive to fix.

After defects are discovered, developers must then find the contributing factors and how to fix them. In complex production systems, it's not usually a single cause; instead, it's often a series of factors that interact to cause a defect. Defects involving security, performance, and availability are expensive and time-consuming to remedy; they often require architectural changes. The time required to find the defect, develop a solution, and fully test the fix are unpredictable. This can further push out delivery dates.

Research from Accelerate State of DevOps Research and Assessment 2022 (DORA) shows that teams can achieve better outcomes by making security a part of everyone's daily work, instead of testing for security concerns at the end of the process. One key finding is that the largest predictor of an organization's software security practices was not technical but instead cultural. Leveraging Westrum's organizational topology, high-trust, low-blame cultures focused on performance were significantly more likely to adopt emerging security practices than low-trust, high-blame cultures that focused on power or rules. For more on organisational culture and my assessment of the book Accelerate check out my previous blog post

How to implement improved security quality

Get Infosec involved in software design

The Infosec team should get involved in the design phase for all projects. As you can imagine, the ratio of engineers/builders to security within a company rates pretty high. So companies these days need to rethink how to approach shifting security left. I found this great article on how a company adopted the team's topology approach and looked to re-frame security as an enablement team.

This mode helps reduce gaps in capabilities and is suited to situations where one or more teams would benefit from the active help of another team facilitating or coaching an aspect of their work. This is the primary operating mode of an enabling team and provides support to many other teams. It can also help discover gaps or inconsistencies in existing components and services used.

Educating teams on OWASP 10 and API security is extremely important. Understanding AuthZ, excessive data exposure and insufficient logging and monitoring remain high on the list.

Education Programs

Companies like AWS have created a guardian program whereby every stream-aligned team needs a security guardian (or what I like to call a security “champion”). So you cannot deliver value to your customers (release to prod) until these criteria are met. No guardian, no shipping software! 😛

The program consists of workshops and training from the security team (OWASP, threat modelling, AWS Well-Architected framework (security pillar) etc). Due to the ratio of engineers to security, they regard the security team as an enablement function. This ties back to team topologies again and its four-team types (enablement/stream aligned/platform/complicated sub-system).

Instead of “us versus them,” make security part of development from the start, and encourage day-to-day collaboration between both teams.

Security and compliance

Threat modelling

I have observed companies adopting a threat modelling framework as part of the assessment. Frameworks such as STRIDE and PASTA are quite popular. OWASP for example has a modelling tool called Threat Dragon that supports frameworks like STRIDE.

Threat modelling is a set of techniques to help you identify and classify potential threats during the development process — but I want to emphasize that this is not a one-off activity only done at the start of projects.

This is because throughout the lifetime of any software, new threats will emerge and existing ones will continue to evolve thanks to external events and ongoing changes to requirements and architecture. This means that threat modelling needs to be repeated periodically — the frequency of repetition will depend on the circumstances and will need to consider factors such as the cost of running the exercise and the potential risk to the business. When used in conjunction with other techniques, such as establishing cross-functional security requirements to address common risks in the project's technologies and using automated security scanners, threat modelling can be a powerful asset.

AWS provides workshops on threat modelling and educating teams on the approach. GitHub talks about its threat modelling process and how it brings more improved communication between security and the engineering teams and how they are able to "shift left".

Bug bounty programs

Many companies are now adopting and investing in bug bounty programs. Bug bounty programs give you continuous, real-time vulnerability insights across your expanding digital attack surface so you can eliminate critical threat “blind spots” and strengthen your security posture.

Check out this interview with one of the security researchers in the Github Security Bug Bounty program to understand more about their background and how they keep up with the latest vulnerability trends. One program I have used in the past is BugCrowd.

CTF (Capture The Flag) competitions are also a great way to keep engineers engaged (security mindset).

Adopting an NFR (Non-Functional Requirements) checklist in the design phase helps to deliver quality at speed; incorporating performance, security and reliability. One example of compliance is the 4-eyes principle where at least two approvers are needed before merging the PR. CODEOWNERS helps automatically assign individuals (or teams) to review PRs. Below I will elaborate further on the tooling that can enable and automate compliance as part of the development lifecycle early.

Security-approved tools

Security-approved tools help to integrate and automate many tooling into your day-to-day life as an engineer. I have noticed over time the rapid rise of Devsecops and tooling that can help achieve this. The areas that I used and have observed include (non-exhaustive list):

Infra as code scanning

• Tfsec - Uses static analysis of your terraform code to spot potential misconfiguration. Can be easily incorporated into your CI pipeline.

• Checkov by Bridgecrew - Can scan results across platforms such as Terraform, CloudFormation, Kubernetes, Helm, ARM Templates and Serverless framework.
• Snyk supports many formats and can also detect drifts post-deployment.
• Policy enforcement for terraform using conftest - Doordash have a great article on how they ensure reliability and velocity through conftest and Atlantis.

Bots

I am a big fan of chatbots! I watched a presentation from a company recently that built multiple chatbots on shifting security left and helping with remediation.

They had an intern with an interest in python and he created some very nifty chatbots.
Behind the scenes, they use Cloud conformity from Trend Micro. Now when an engineer is naughty and maybe clickops and opens a security group to the world or an s3 bucket, the bot will alert the channel and notify of the issue in real-time (they only broadcast on high and mediums). You can then ask the bot also who made the change. It will then scan Cloudtrail, find the IAM user in question and link it back to the slack handle of the user and post back to the channel who's to “blame” (more in the sense of git blame than to literally blame!). The bot could also perform auto-remediation. This is just an example of the opportunities that exist with chatbots.

Linting (Compliance)

Linters are tools used to flag programming errors, bugs, stylistic errors and suspicious constructs. Examples include:

• Eslint - is an open-source project that helps you find and fix problems with your JavaScript code.
• TFlint is a pluggable terraform linter that finds possible errors (like invalid instance types) for Major Cloud providers (AWS/Azure/GCP) and enforces best practices.
• Actionlint is a static checker for GitHub workflow files. I use the VSCode extension extensively.
• yamllint is a linter for YAML files.
• oaslint - Spectral is an OAS linter to enforce best practices for API design of Open API Specs. I have introduced this linter before at organisations to align with companies' API guidelines and catch any inconsistencies early in the development lifecycle.

If you are using GitHub actions check out this awesome list of linters that the community recommends.

Security & Version updates

OWASP published last year their top ten, and vulnerable and outdated components moved up to sixth position

Dependabot and Renovate are two tools I have used in the past for dependency updates and vulnerability management. Dependabot is part of the GitHub ecosystem now and is maturing every day. But I do miss the nice presets that Renovate has and how you can group certain dependencies too.

One lesson I have learnt from automating this process is to be aware of dependency fatigue. Work with the team on your schedule and what dependencies are the highest priority. Otherwise, you could end up becoming overwhelmed with PRs and start to ignore updates!

Github talks about how they use dependabot internally to secure their own platform.

OWASP dependency checker is also a useful tool to run as part of your CI pipeline to identify security vulnerabilities.

Code scanning (SAST)

SonarQube is a self-managed, automatic code review tool that systematically helps you deliver Clean Code. As a core element of their solution, SonarQube integrates into your existing workflow and detects issues in your code to help you perform continuous code inspections of your projects. The tool analyses 30+ different programming languages and integrates into your CI pipeline and DevOps platform to ensure that your code meets high-quality standards.

Container scanning

• Twistlock is now part of Palo Alto’s Prisma Cloud offering and is one of the leading container security scanning solutions.

• Snyk also provides a similar product offering to scan images.

I hope you have found this list of security tools useful. Head over to CNCF to find more.

Suffice it to say are we still shifting left? Is it realistic to expect developers to take on the burdens of security and infrastructure provisioning, as well as writing their applications? Is platform engineering the answer to saving the DevOps dream?? I will leave you with that thought! 😉

In the book, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, Dr. Nicole Forsgren, Jez Humble, and Gene Kim studied what made separated strong performing technology organizations from their less effective counterparts.

The book summarises 4 years of rigorous research from years of State of DevOps Reports, built upon 23,000 datasets from over 2,000 unique companies all around the world. The organizations studied included start-ups and enterprises, profit and not-for-profit organisations, and companies that were born digital alongside those that had to undergo digital transformation.

Four key metrics

The research identified that just Four Key Metrics distinguish the performance of various organisations. These “North Star” metrics serve as indicators of overall software engineering health.
These metrics aren’t “The Goal” of a business, but organisations that did well against these metrics had higher rates of profitability 💵, market share, and customer satisfaction. In other words; they allowed organisations to experiment faster 🚀, ship reliably, and prevent burnout.

The Four Key Metrics were as follows:

• Cycle Time (Change Lead Time) - Time to implement, test, and deliver code for a feature (measured from first commit to deployment)
• Deployment Frequency - Number of deployments in a given duration of time
• Change Failure Rate (CFR) - Percentage of deployments which caused a failure in production
• Mean Time to Recovery (MTTR) - Mean time it takes to restore service after production failure

Why these metrics

Where is the evidence that these metrics help organisational performance though? How do you convince your boss that these things matter??

Martin Fowler on Accelerate metrics

The picture they paint is compelling. They describe how effective IT delivery organizations take about an hour to get code from "committed to mainline" to "running in production", a journey lesser organizations take months to do. They, thus, update their software many times a day instead of once every few months, increasing their ability to use software to explore the market, respond to events, and release features faster than their competition. This huge increase in responsiveness does not come at a cost in stability, since these organizations find their updates cause failures at a fraction of the rate of their less-performing peers, and these failures are usually fixed within the hour. Their evidence refutes the bimodal IT notion that you have to choose between speed and stability instead, speed depends on stability, so good IT practices give you both.

So, as you may expect, I'm delighted that they've put this book into production, and I will be recommending it willy-nilly over the next few years. (I've already been using many bits from its drafts in my talks.) However, I do want to put in a few notes of caution. They do a good job of explaining why their approach to surveys makes them a good basis for their data. However, they are still surveys that capture subjective perceptions, and I wonder how their population sample reflects the general IT world. I'll have more confidence in their results when other teams, using different approaches, are able to confirm their reasoning. The book already has some of this, as the work done by Google on team cultures provides further evidence to support their judgment on how important a Westrum-generative organizational culture is for effective software teams.

Nicole Forsgren On How DevOps Metrics Correlate With Organisational Performance

Rigorous appraisal of these Four Key Metrics has shown that higher performers are 2x more likely to meet their commercial goals (productivity, profitability, market share, number of customers) and their non-commercial goals (quantity of products or services, operating efficiency, customer satisfaction, quality of products or services and achieving organisational or mission goals). Indeed, companies which do well under these DevOps metrics have a 50% higher market cap growth over 3 years. 📈

NOTE: These findings will apply whether you’re using a traditional “waterfall” methodology (also known as gated, structured, or plan-driven) and just beginning your technology transformation, or whether you have been implementing Agile and DevOps practices for years.

Flaws in previous attempts to measure performance

Before I jump into what capabilities a company needs to focus on to achieve these metrics, I wanted to explain how companies have in the past measured performance.

There have been many attempts to measure the performance of software teams. Most of these measurements focus on productivity. In general, they suffer from two drawbacks.

• First, they focus on outputs rather than outcomes.
• Second, they focus on individual or local measures rather than a team or global ones.

LOC (lines of code)

Measuring productivity in terms of lines of code has a long history in software. Some companies even required developers to record the lines of code committed per week. However, in reality, we would prefer a 10-line solution to a 1,000-line solution to a problem. Rewarding developers for writing lines of code leads to bloated software that incurs higher maintenance costs and higher costs of change.

Ideally, we should reward developers for solving business problems with the minimum amount of code and it's even better if we can solve a problem without writing code at all or by deleting code (perhaps by a business process change). However, minimizing lines of code isn't an ideal measure either. At the extreme, this too has its drawbacks. Accomplishing a task in a single line of code that no one else can understand is less desirable than writing a few lines of code that are easily understood and maintained.

Velocity

With the advent of Agile software development came a new way to measure productivity: velocity. In many schools of Agile, problems are broken down into stories. Stories are then estimated by developers and assigned a number of "points" representing the relative effort expected to complete them. At the end of an iteration, the total number of points signed off by the customer is recorded; this is the team's velocity.

Velocity is designed to be used as a capacity planning tool; for example, it can be used to extrapolate how long it will take the team to complete all the work that has been planned and estimated. However, some managers have also used it as a way to measure team productivity, or even to compare teams.

Using velocity as a productivity metric has several flaws. First, velocity is a relative and team-dependent measure, not an absolute one. Teams usually have significantly different contexts which render their velocities incommensurable. Second, when velocity is used as a productivity measure, teams inevitably work to game their velocity. They inflate their estimates and focus on completing as many stories as at the expense of collaboration with other teams (which might decrease their velocity and increase the other team's velocity, making them look bad). Not only does this destroy the utility of velocity for its intended purpose, but it also inhibits collaboration between teams.

I have seen this in the past myself, "gaming" the system and engineers aiming to deliver as my story points as you can within a sprint. Teams celebrating burn down charts and how many points they delivered compared to other squads...

This obfuscates the underlying quality of the code. And what tech debt or improvements you could have made but decided not to. If you don't have the right leadership in place or a peer review process, then this can unfold very quickly. And not being a good girl scout or boy scout. They talk about this more in "system thinking" and not having that growth mindset.

Utilisation

Finally, many organizations measure utilization as a proxy for productivity. The problem with this method is that high utilization is only good up to a point. Once utilization gets above a certain level, there is no spare capacity (or "slack") to absorb unplanned work, changes to the plan, or improvement work. This results in longer lead times to work. Queue theory in math tells us that as utilization approaches 100%, lead times approach infinity-in other words, once you get to very high levels of utilization, it takes teams exponentially longer to get anything done. Since lead time is a measure of how fast work can be completed, and is a productivity metric that doesn't suffer from the drawbacks of the other metrics we've seen, it's essential that we manage utilization to balance it against lead time in an economically optimal way.

Performance metrics

Below you can see the different groupings of teams and how they perform. You can see the elite teams, they're able to deploy on-demand. Their Cycle Time is less than one day. Their MTTR, mean time to restore, is less than one hour. Change failure rate is in the zero to 15% range. You can really see how the Cycle Time, in particular, the lead time for changes, how varies throughout those four columns. It's less than one day for the elite teams. For the high-performance teams, it goes down to that one day and one week. Medium teams one week and one month and the low performing teams are really between that one month and six months mark.

I will highlight next the main capabilities and provide real use-cases that I have observed in my experience:

Capabilities

Thoughts and feedback

As highlighted above, there is a lot of investment needed to achieve CD (Continuous Delivery) and become high-performing. Whenever you are asked about what makes a high-performing team, people say trust and a sense of purpose are important. And they are, don't get me wrong. But you also need engineers with a growth mindset, discipline, and technical acumen to make it a reality.

But regarding trust and a sense of purpose, there are numerous ways to achieve this within your team. Team topologies go in-depth into this discussion. They talk about Dunbar's trust number and having two size pizza teams (normally 5-8 depending on the complexity of the domain).

Then there is also Tuckman's model approach to team development: forming-storming-norming-performing.

Regular social events with your team and team playbooks are a great way to break the ice and start to understand eachothers personalities. Especially when you work in a culturally diverse environment. Working in Europe and the Asia Pacific, I have worked with many nationalities and am always keen to understand more about their culture.

Sense of purpose - having that clear vision and objective for the team and getting alignment is crucial to the success of the team. Running exercises such as team charter and RACI model help with this.

But now onto the fun part; the technical aspects. But I will let you in with a little secret first:

Engineers love shipping software! 😛

I remember years ago working in London for a mobile apps company. There was an engineer who I worked with that hadn't shipped code in over six months! (due to indecisiveness on the client's part). I could see how demoralized she looked not able to see her hard work go live. So always remember, engineers, love to ship software.

I have worked in high-performing teams in the past where change lead time was < 90 mins and deployment frequency daily. I have seen firsthand how it has a direct impact on job satisfaction. Having engaged employees doing meaningful work drives business value.

Deployment pipeline

Continuous delivery does not happen overnight. You need to methodically design your CI pipeline, and focus on your technical capabilities to make it a reality. Working with CI servers such as buildkite, GitHub actions or circleCI allows you to design your pipeline as code. We have moved on from the clickops days of TeamCity or bamboo where engineers are not across how CI/CD works. To be able to visualise it as code, can really help even new team members understand and get up to speed so much faster.

A lot of these CI servers now support plugins and integrations to expedite your delivery and simplify the work needed in the pipeline. Accessing secrets from secret manager, running docker-compose, generating test reports, and more. Continually looking to optimise and improve your pipeline to run as fast as possible through certain software techniques/enhancements or agent upgrades is an ongoing task.

Test Automation

Software quality and test automation are the next phases. Working in cross-functional teams can be a mindset change for a lot of engineers. It is well publicised now the mandate at AWS of decoupling architecture and building two size pizza teams that build, run, and support your workloads. Working on teams with 24/7 support, your mindset changes into how you perceive the quality of your code. Getting a change "out the door" to achieve a deadline is frowned upon. And those traditional measures of velocity and high utilisation.

If you are on 24/7 support, you don't want to be woken up at 3 AM in the morning right! So your mindset changes. You are more aware of the quality of your code and of others. Working on trunk-based development makes engineers' life also easier when reviewing PRs.

There is a lot of talk about shifting left and specifically shifting testing left. Having a healthy automated test suite and being aware of anti-patterns such as ice cream cones and cupcake is very important.

You cannot achieve CD by having a heavy layer of e-2-e tests. This will delay your change lead time and overtime those tests will become flaky and less reliable.

Shifting left to integration tests (that can run on your local machine) is the key. QAs focus less on manual tests and free them up to look at exploratory testing or chaos engineering for example. But many high-performing teams these days hire more engineers to focus on test automation and shifting testing and performance left rather than hire QAs. Working with the teams to educate them on the benefits of shifting left and changing that mindset of how to test a feature is extremely valuable.

Now companies are given the space to revisit their environments and look to consolidate. Not only does this improve change lead time and MTTR. But also the maintenance overhead of trying to keep envs in parity. Cost savings on your cloud provider can be significant too.

Feature toggles

Feature toggles are a great way to separate deploy from release when looking to consolidate envs. There are several commercial options out there such as launch darkly and other open source options to help with this. Github provides a great example of how they ship frequently and safely to production with the use of feature toggles.

Cloud

Cloud infrastructure is adopted quite heavily now by companies. And there is an expectation now for engineers to upskill in cloud providers such as AWS or Google Cloud. Working in cross-functional teams in the past we managed our own AWS accounts and infrastructure. There are many tools of choice for infra as code such as cloudformation, terraform, AWS CDK, and pulumi.

Companies also these days are identifying areas of the platform to design and build to provide self-service capabilities to the delivery teams. I have worked with teams that treated the platform as a product and its internal customers as delivery teams and looked to streamline their development process by providing custom tooling with well-documented APIs and support.

Deploying static sites and deploying app workloads to Kubernetes are examples I have seen. The key is to balance autonomy with alignment and still allow teams to choose their own tooling when needed. They call it providing a "thinnable viable" platform.

Slack has a great article on how they adopted this approach.

Proactive monitoring

Once you are happy with your pipeline, code quality, and test automation, the next area to invest in is monitoring. I am not going into the specifics here but you have many tools at your disposal now for logging and monitoring namely Splunk, sumologic, datadog, new relic, and appdymanics.

In a loosely coupled architecture having distributed tracing and a common log format really helps when troubleshooting. Investing time in an incident management playbook and alert notifications too. Just be aware of alert fatigue

Lean Management and product development

With this investment in CD, your team can now afford to streamline its change approval process. Gone are the days that you need to present to CAB for production deployments. This allows the team to adopt a peer-review process via pair programming or intra-team code review without sacrificing speed or quality.

One common requirement I hear from companies is segregation of duties which is a common regulatory requirement. This can now be achieved by using peer review to meet the goal of segregation of duties, with reviews, comments, and approvals captured in the team's development platform as part of the development process.

The time and attention of those in leadership and management positions are freed up to focus on more strategic work. This transition, from gatekeeper to process architect and information beacon, is consistent with the practices of organizations that excel at software delivery performance.

CAB idea is a form of risk management theater: we check boxes so that when something goes wrong, we can say at least we followed the process. At best, the process only introduces time delays and handoffs.

Lean product management and visibility of work from the business all the way through to customers is a new approach that I have seen. Team experimentation, visibility into customer feedback, and continous discovery are ways for the team and engineers to feel empowered to contribute on what opportunities to work on next, and become more engaged in product discovery and engaging with customers.

Shifting security left

Security is another CFR that can be shifted left. Areas I have worked on in the past are the automated management of dependencies via tools such as dependabot, snyk or renovate.

We devised a schedule on what frequency and priority we needed to be notified of, for our repos to prevent dependency fatigue. Having multiple automated PRs created every day for patches is not an efficient use of time! Tools such as snyk also provide infra scanning and container scanning. Engaging teams such as architecture and security earlier in the development lifecycle is also recommended.

Culture and transformative leadership

And lastly, but most importantly culture. Creating a climate for learning is a key investment within a company and investing in your people.
Some steps that I have advocated for and adopted in the past include:

• Creating a training budget
• Ensuring the team has the resources to engage in informal learning and space to explore ideas
• Blameless culture and making it safe to fail
• Creating opportunities and spaces to share information through lightning talks or guilds
• Make resources available for continued education such as meetups and attending conferences

As this survey has highlighted, investing in culture that values learning contributes significantly to software delivery performance.

As a leader, it is important to really invest in culture and be able to measure it. Your work is never done, always looking to continually improve. Providing the technical vision and where the company is going, supportive leadership, and challenging team members (intellectual stimulation) are the areas I continually focus on.

Pitfalls to watch out for

• Focus on capabilities and not maturity The key to successful change is measuring and understanding the right things with a focus on capabilities - not on maturity. Maturity models focus on helping an org "arrive" at a mature state and declare themselves done with their journey. Alternatively, capability models focus on helping an org continually improve and progress, realising that the technological and business landscape is ever-changing.

• Treating transformation as a one-time project. in high-performing organizations, getting better is an ongoing effort and part of everybody's daily work. However, many transformation programs are treated as large-scale, one-time events in which everyone is expected to rapidly change the way they work and then continue on with business as usual. Teams are not given the capacity, resources, or authority to improve the way they work, and their performance gradually degrades as the team's processes, skills, and capabilities become an ever poorer fit for the evolving reality of the work. You should think of technology transformation as an important value-delivery part of the business, one that you won't stop investing in. After all, do you plan to stop investing in customer acquisition or customer support?

• Treating transformation as a top-down effort. In this model, organizational reporting lines are changed, teams are moved around or restructured, and new processes are implemented. Often, the people who are affected are given little control of the changes and are not given opportunities for input. This can cause stress and lost productivity as people learn new ways of working, often while they are still delivering on existing commitments. When combined with the poor communication that is frequent in transformation initiatives, the top-down approach can lead to employees becoming unhappy and disengaged. It's also uncommon for the team that's planning and executing the top-down transformation to gather feedback on the effects of their work and to make changes accordingly. Instead, the plan is implemented regardless of the consequences.

Google project

Google provides a very useful project on github showcasing how to capture these metrics. They use services on GCP such as Pub/Sub and BigQuery to capture this data in real-time. I would love to see an example of using Kafka and ksqlDB to mirror this approach (hack day idea!)

To find out where your company sits, run this quick survey

I hope you have found this post insightful and helped create a conversation in your organization about where you stand today. And where you want to go.

As my main man Kelsey said:

I started looking into API performance testing again recently. As part of delivering an API to production, performance testing is crucial to see how your API performs under load ( more on that shortly)

You might have heard of the ridiculous load that Shopify handled over BFCM (Black Friday/Cyber Monday) weekend. They executed and ran performance testing on their systems well in advance as far back as July to make sure they were well prepared for "game-day".

⏪ Rewinding for a moment — preparing for an event of this scale doesn't happen overnight. In anticipation of BFCM 2021 we began load testing back in July! To better simulate real global traffic we spread out our load generation across @GoogleCloud's global network. pic.twitter.com/5oXqFOadae

— Shopify Engineering (@ShopifyEng) November 30, 2021

Now there are many tools that are out there in the wild. I have worked with JMeter quite extensively, Gatling and Artillery. But as always, I like to do some research and see what is happening in that space.

Enter k6!

I follow ThoughtWorks tech radar and they identified this new tool. So I decided to have a play around.

K6

• Created in 2016 by loadImpact
• Acquired by Grafana in 2021
• 14.6k Github stars ⭐️
• Built in golang, test scripts using Javascript Typescript 😉
• Multiple choices for storage (datadog/kafka/cloudwatch/Prometheus/JSON/CSV)
• Converters (HAR/Postman/OAS)
• Github action
• Test builder/browser recorder (low code options)
• Aligns with “API first” approach

I mention "API first" here as that is a common practice for organizations now. In my previous post, I talked about another API first tool to use when mocking APIs.

Installation

There are several ways to install k6 depending on your operating system or system environment. But in this article, I will only touch on two of them, MacBook and Docker.

You can install k6 on a MacBook by using brew

brew install k6

or via docker

docker pull loadimpact/k6  

You can find the full explanation here https://k6.io/docs/getting-started/installation/

Typescript template

As I mentioned above, the test scripts can be defined in javascript. But I have decided from the beginning to use typescript instead for static type checking. I have created a typescript template on Github that can be easily re-used by anyone. It is built on the good work that k6 did with some enhancements.

Rationale

While JavaScript is great for a myriad of reasons, one area where it falls short is type safety and developer ergonomics. It's perfectly possible to write JavaScript code that will look OK and behave OK until a certain condition forces the executor into a faulty branch.

While it, of course, still is possible to shoot yourself in the foot with TypeScript as well, it's significantly harder. Without adding much overhead, TypeScript will:

• Improve the ability to safely refactor your code.
• Improve readability and maintainability.
• Allow you to drop a lot of the defensive code previously needed to make sure consumers are calling functions properly.

Lifecycle

The four distinct life cycle stages in a k6 test are "init", "setup", "VU" and "teardown"

Init and VU stages

Scripts must contain, at the very least, a default function - this defines the entry point for your VUs, similar to the main() function in many other languages.

Code inside default is called "VU code", and is run over and over for as long as the test is running. Code outside of it is called "init code", and is run only once per VU (Virtual User).

A VU will execute the default function from start to end in sequence. Nothing out of the ordinary so far, but here's the important part; once the VU reaches the end of the default function it will loop back to the start and execute the code all over.

Setup and teardown stages

Beyond the required init and VU stages, which is code run for each VU, k6 also supports test-wide setup and teardown stages, like many other testing frameworks and tools. The setup and teardown functions, like the default function, need to be exported functions. But unlike the default function setup and teardown are only called once for a test. setup is called at the beginning of the test, after the init stage but before the VU stage (default function), and teardown is called at the end of a test, after the VU stage (default function).

You might have noticed the function signature of the default function and teardown function takes an argument, which we here refer to as data.

This data will be whatever is returned in the setup function, so a mechanism for passing data from the setup stage to the subsequent VU and teardown stages.

Further below I will showcase how in setup you can inject in an access token to the default function for an API secured by OAuth2.

Basic example

import { sleep, check } from "k6";  
import { Options } from "k6/options";  
import http, { Response } from "k6/http";  
import { padStart } from "lodash";  
import { textSummary } from "./helper";  
import { Trend } from "k6/metrics";

// 1. init code

console.log(padStart("Hello TypeScript!", 20, " "));

//custom define metric
const durationInSeconds = new Trend("duration_in_seconds");

export let options: Options = {  
  vus: 5, //no. of concurrent virtual users
  duration: "5s",
  discardResponseBodies: true, //discard response bodies to improve perf
  //if you want to fail the whole load test use thresholds
  thresholds: {
    http_req_failed: ["rate<0.01"], // http errors should be less than 1%
    http_req_duration: ["p(95)<350"], // 95% of requests should be below 350ms
  },
  // httpDebug: "true",
  userAgent: "K6GreetingsDemo/1.0",
};

export function setup() {  
  //setup is called once off
  // 2. setup code
}

export default () => {  
  // 3. VU code

  let baseUrl = __ENV.BASE_URL ?? "http://localhost:8090";
  let url = `${baseUrl}/greetings`;
  const res: Response = http.get(url, {
    tags: { team: "team-label", api: "greetings" },
  });
  check(res, {
    "status is 200": () => res.status === 200
  });
  // we know that the duration is in millisecond
  // but for demonstration purposes, we convert it to second
  durationInSeconds.add(res.timings.duration / 1000);
  sleep(1);
};

export function handleSummary(data: any) {  
  console.log("Preparing the end-of-test summary...");

  // Send the results to some remote server or trigger a hook

  return {
    stdout: textSummary(data, { indent: " ", enableColors: true }), // Show the text summary to stdout...
  };
}

First, we imported the dependencies on top. k6 has types (@k6/types) that are nice when developing in typescript.

Note that in the background, k6 doesn’t run in NodeJS, since in general JavaScript is not well suited for high performance. It’s written in Go to achieve the desired high-performance testing.

The test itself is running inside the exported default function. This part of the code is what we usually called as VU Code. So, the test is running once and uses only one virtual user (think of this as a real user, but simulated) by default, but you can change that using options. We discussed the lifecycle earlier.

So in the example above, we are simulating 5 users over 5 secs. We have set thresholds for the test in relation to response time and error rate. We have added in an additional trend to showcase how that is defined. No setup is involved here. We have one check that the status is 200.

To run this we can run the script yarn bundle. This will transpile to JS using babel and bundled with webpack. The outputted file will be located in the dist dir.
Now run locally:

k6 run dist/greetings.js  

You’ll see the result of the test right away on the terminal. Something similar to this.

You can see the built-in metrics that were outputted and also your custom metrics duration_in_seconds. To read more about the metrics go to https://k6.io/docs/using-k6/metrics/

Now I have only outputted this to stdout. But you may decide you want to pipe it into a time series database like influxdb, APM such as datadog or appdynamics which is all possible. For more on the supported outputs check out https://k6.io/docs/getting-started/results-output/. If you want to look at an example of using influxdb and grafana check out here.

API first tools

Now I mentioned earlier that it supports the "api first" approach. In the past, I have looked to use tools such as open api generator to generate server stubs based on the OAS.

Slack have talked about this approach recently regarding their APIs. And stoplight has some great resources on the API first approach too. So I looked into open api generator and saw they actually supported k6. Let's try that out next.

OAS + k6

I have designed a sample OAS already, called greetings API and I wanted to generate the k6 script using open api generator. You can follow along by cloning this repo

docker pull openapitools/openapi-generator-cli

docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli generate \  
    -i /local/oas3.yaml \
    -g k6 \
    -o /local/k6-oas3/ \
    --skip-validate-spec

Here I pull down the docker image and then run the cmd by mounting my oas file and using the generator type k6. This then generates some boilerplate test script code inside the dir k6-oas3.

NB: This will need to be refined and enhanced afterward but is a great starting point.

open api to typescript

Another tool I found useful is this node module open api typescript.

I read an article recently on how etsy migrated from JS to TS and they used this tool to generate the types which can save a lot of time and aligns with the "API first" approach.

So in the example below I will showcase how to use the types when running a load test against bpay API. This API is secured by OAuth2 client credentials grant type also. You will see that I use the setup function to inject in the access_token that's needed when running the test.

As I am defining my script in typescript and this API method is a post, I can use openapi-typescript to generate the types for me from the OAS. This saves me a lot of time not having to define my own interfaces.

To generate the types:

npx openapi-typescript bpay/oas3.yaml --output src/bpay/schema.ts  

This generates the types in the bpay dir.

Now let's have a look at the test script for bpay API.

import http from "k6/http";  
import { group, check, sleep } from "k6";  
import { PaymentPaymentMethodEnum } from "./bpay";  
import { Options } from "k6/options";  
import { getToken, Options as BpayOptions } from "./helper";  
import { components } from "./bpay";

type Payment = components["schemas"]["Payment"];  
type PaymentItem = components["schemas"]["PaymentItem"];

const BASE_URL = "https://sandbox.api.bpaygroup.com.au/payments/v1";  
// Sleep duration between successive requests.
// You might want to edit the value of this variable or remove calls to the sleep function on the script.
const SLEEP_DURATION = 0.1;  
// Global variables should be initialized.

export let options: Options = {  
  vus: 1, //no. of concurrent visual users
  duration: "1s",
  httpDebug: "true",
};

export function setup() {  
  const options: BpayOptions = {
    clientId: `${__ENV.CLIENT_ID}`,
    clientSecret: `${__ENV.CLIENT_SECRET}`,
    domain: "https://sandbox.api.bpaygroup.com.au",
  };
  try {
    return getToken(options);
  } catch (error) {
    return "";
  }
}

//desctructing assigment here and explicit type annotation
export default function ({ access_token = "" }: { access_token: string }) {  
  group("/validatepayments", () => {
    let url = BASE_URL + `/validatepayments`;

    const payment: Payment = {
      billerCode: "565572",
      crn: "651234567890123",
      amount: 234.83,
      paymentMethod: PaymentPaymentMethodEnum.Debit,
      settlementDate: "2017-10-23",
    };
    const payment1: Payment = {
      billerCode: "1313",
      crn: "1230",
      amount: 1045.98,
      paymentMethod: PaymentPaymentMethodEnum.Debit,
      settlementDate: "2017-11-06",
    };
    const paymentItem: PaymentItem = { tid: "1", payment };
    const paymentItem1: PaymentItem = { tid: "2", payment: payment1 };
    const payments: Array<PaymentItem> = [];
    payments.push(paymentItem);
    payments.push(paymentItem1);
    const payload = { payments };
    let params = {
      headers: {
        "Content-Type": "application/json",
        Accept: "application/json",
        Authorization: `Bearer ${access_token}`,
      },
    };
    let request = http.post(url, JSON.stringify(payload), params);
    check(request, {
      "ok":
        (r) => r.status === 200,
    });
    sleep(SLEEP_DURATION);
  });
}

Let's look at the imports first. You can see we are importing bpay types that were auto-generated. These types are subsequently used when building up the payload inside the
default function. I'm running against bpays sandbox env and it's a simple smoke test.

I will shed some light on the different types of performance tests you can run later.

Now in the setup function, you can see I've defined a helper fn to get the access token. This performs a request against the sandbox env with your client id and secret and returns the access token. I can then return this value which is then passed into the default fn to be used in the Authorisation header.

NB I am using env variables here for client_id and secret as they should not be committed into src code.

The default fn constructs the payload and submits the post request to the API with the bearer token.

I have not defined any thresholds in this example but just a simple check on the HTTP status response.
You can see how easy it is to define your smoke test and utilize the tools at your disposal if you adopt the "api first" approach.

Browser builder

Now, let's say I'm a bit lazy 😅 and want to go the low-code approach! k6 does have a browser recorder extension for chrome and firefox that can record your interactions and download a har (HTTP archive file). This then can be imported into k6 cloud and provide a nice GUI to construct the script. Once you are happy with the script you can copy this locally and use it with the CLI.

Builder:

Corresponding script:

Types of testing

The examples above showcased smoke tests running against your environment(s) (or what is now called "shifting perf left"). But it is possible to perform many types of tests using k6, each type serving a different purpose.

Smoke test

Smoke test is a regular load test, configured for a minimal load.

You want to run a smoke test as a sanity check every time you write a new script or modify an existing script.

• can be run as part of CI pipeline - "shifting perf left"
• More ideal for microservices

k6 options for smoke test:

Load test

Load Testing is primarily concerned with assessing the current performance of your system in terms of concurrent users or requests per second.

If you need some understanding about the amount of traffic your system is seeing on average and during peak
hours. ie. How to configure performance thresholds.

If your system crashes under a load test, it means that your load test has morphed into a stress test!

K6 options for load test:

Stress test

Stress testing is to assess the availability and stability of the system under heavy load (think HA).

To execute a proper stress test, you need a tool to push the system over its normal operations, to its limits, and beyond the breaking point.”

You typically want to stress test an API or website to
determine:

• How your system will behave under extreme conditions.
• What the maximum capacity of your system is in terms of users or throughput.
• The breaking point of your system and its failure mode.
• If your system will recover without manual intervention after the stress test is over.

Companies use this form of testing for black Friday sales for example.

K6 options for stress test:

Soak test

A soak test uncovers performance and reliability issues stemming from a system being under pressure for an extended period.

You typically run this test to:

• Verify that your system doesn't suffer from bugs or memory leaks, which result in a crash or restart after
  several hours of operation.
• Verify that expected application restarts don't lose requests.
• Find bugs related to race-conditions that appear sporadically.
• Make sure your database doesn't exhaust the allotted storage space and stops.
• Make sure your logs don't exhaust the allotted disk storage.
• Make sure the external services you depend on don't stop working after a certain amount of requests are
  executed.

K6 options for soak test:

Conclusion

If you are comfortable developing in javascript or typescript, k6 is a breeze to use. There are so many options you could use, scenarios if you need advanced user behavior, saving the test result in a CSV or JSON file, having a dashboard for presentation using grafana, etc.

As I demoed earlier, if you adopt the "API first" approach there are tools you can use such as open API generator and openapi-to-typescript that can help improve the development experience.

Happy testing!

Recently, I have spent considerable time researching and analyzing the tooling available for "API first". At the core of this eco-system is the OAS (Open API specification) or interface as we normally like to call it.

OAS (Open API Specification)

OAS (or what was commonly known as swagger spec) is the industry standard for defining REST interfaces. I had previously worked with many others such as RAML and Blueprint, but it's good to see we have a winner! Latest release of OAS (V3.x) includes webhook support and the latest JSON schema draft. Open API is now under the governance of the Linux Foundation. The OpenAPI Specification was originally based on the Swagger Specification, donated by SmartBear Software.

Mocking

The term "mock" for a lot of developers will have unit-testing connotations. In unit-testing, a mock is a fake implementation of a class or function, which accepts the same arguments as the real thing. It might return something pretty similar to the expected output, and different test cases might even modify those returns to see how the code under test works.

This is almost exactly the concept here, just at a HTTP level instead. This is done using a "mock server", which will respond to the expected endpoints, error for non-existent endpoints, often even provide realistic validation errors if a client sends it an invalid request.

So today, I am going to talk about mocking REST APIs. Anyone that has worked in cross-functional teams before would be very used to mocking APIs for local development. If for example, your team consisted of front and back-end devs, normally the BE devs would aim to design the interface upfront (API first) and provide a mock API to FE devs to commence development in parallel.

Why?
Both streams of work should occur in parallel, rather than sequentially. Less waterfall!

But there are many other benefits of mocking APIs such as:

• Showcasing to stakeholders the interactions as part of the API design process
• Showcase to external consumers
• Use as a sandbox on the dev portal
• Integration testing on CI ("shift testing left")
• Performance testing on CI ("shift perf left")

Tools

I have used many frameworks and tools in the past to mock APIs. When I developed with typescript or nodeJS, a framework I used heavily was expressJS. Other tools out there include mountebank or wiremock. But now there is a new breed of mock API tools that are OAS compliant. Two that I have found recently are prism from stoplight and Mockoon.

Mockoon

Mockoon lets you mock an API in seconds

Some features include:

• Intuitive interface to create your mock API and run anywhere via CLI
• Integrates with your workflow - Compatible with the OpenAPI specification, Mockoon integrates perfectly with your existing applications and API design workflow.
• Advanced features and tackle the most complex situation with HTTP requests recording, proxying, integration testing, etc.
• Complex rules system and dynamic body templating
• Powerful forwarding and debugging

Creating our first API with Mockoon

Step1. Installation

Mockoon is available on the three major operating systems: Windows, macOS, and Linux.
You can install the native app here.

Step 2. Create your first mock API

After launching the application for the first time, you will find a demo mock API, also called "environment" in Mockoon. You can keep it and build from here or create a new one. To create a new mock API, open the collapsible environments menu on the left and press the blue "plus" button:

Step 3. Create your first API route

The newly created mock API already includes a route on /. You can modify it by setting up the method and path of your choice.

You can also create a new endpoint by clicking on the blue "plus" button at the top of the endpoint list:

Step 4. API endpoint configuration

You can further customize your endpoint by adding a custom header and the following sample body (which makes use of Mockoon's templating system).

Step 5. Run and call Mock API

The last step is to run your mock API. For this, click on the green "play" arrow in the header:

Your mock server is now available on http://localhost:3001 (but also on http://127.0.0.1 and all your local network adapters).

You can do a test call to the following URL http://localhost:3001/tutorials using your favorite tool (here using Insomnia) and see the returned response:

curl --request POST \  
  --url http://localhost:3001/tutorials

So now you have got the hang of it, let's look at its more advanced features and its "API first" support.

API first approach

So let's import a sample OAS file with Mockoon.

Step 1 Import OAS

Open the app and go to Import/Export > Swagger/Open API > Import Swagger v2/Open API v3

I have defined a sample OAS file here that you can use to follow along. Once you import, it now should look like this.

You can see that for each route, it has already been set up for you all the responses you have defined in the spec and the examples. You can easily toggle on random responses or sequential responses based on the route (NB: This will disable the rules tho)

It is even smart enough to convert certain fields weatherType and status to its templating language. How cool is that! And again, based on previous steps you can start the server and test it out via Insomnia.

Insomnia is such a powerful app also as it supports the "API first" approach. I can import in the exact same OAS file and test the endpoints. (You will start to see a recurring trend here of "API first" tools 😅)

Step 2 Run mock anywhere

Ok, so now we are happy with the mock responses, how can I run this anywhere?

Introducing Mockoon CLI 🎉🎉🎊🥳

Mockoon's perfect complement for all your headless and automated environments.

Mockoon CLI Supports all Mockoon's features, Lightweight and fast, and allows you to Run your mocks everywhere.

NB: Also available as a Docker image, run your mock APIs in Github Actions or on your favorite CI platform!

The CLI is a companion application to Mockoon's main interface designed to receive an exported Mockoon data file.

It has been written in JavaScript/TypeScript and uses some great libraries like oclif and PM2. One of the benefits of using PM2 is that you can easily manage your running mock APIs through the CLI or by using PM2 commands if you are used to them.

Step3. Install mockoon CLI

I installed using nodeJS

npm install -g @mockoon/cli

Step 4. Export your mock API to a JSON file

To export your environment, open the "Import/export" application menu and choose "Mockoon's format" -> "Export all environments to a file (JSON)" or "Export current environment to a file (JSON)".

You can then select a location to save the export data file. Let's name the file Greetings_Mockoon.json.

Step 5. Start your mock API

After exporting your data file, you are ready to run your API mock with the CLI.

In your terminal, navigate to the folder where your export data file is and run the following command:

mockoon-cli start --data ./Greetings_Mockoon.json

If you want to use a remotely hosted file, you can also provide a URL to the --data flag like this:

mockoon-cli start --data https://domain.com/data-export.json

Step 6. Manage your API mock

After running one or more API server mock, you might want to check their health and statuses. To do so you can type mockoon-cli list:

shanelee at shanes-MacBook-Air in ~/projects/mockoon-demo on main [?]  
$ mockoon-cli list
 Name                 Id   Status    Cpu    Memory    Hostname       Port
 ──────────────────── ──── ───────── ────── ───────── ────────────── ──────
 mockoon-greeting-api 0    online    0.5    71 MB     0.0.0.0        3002

You can also stop all running servers at once with mockoon-cli stop all

Step 7. View a running mock's logs

Mockoon CLI log all events like requests and errors in your user folder in the following files: ~/mockoon-cli/logs/{process_name}-out.log and ~/mockoon-cli/logs/{process_name}-error.log.

Step 8. Deploy Mockoon CLI using Docker

Now to the fun part! The CLI can containerize your mock API for you. The docker base image is node:14-alpine which is very lightweight.

Using the dockerize command

mockoon-cli dockerize --data ./Greetings_Mockoon.json --port 3000 --index 0 --output ./Dockerfile

Now build the image:

    docker build -t mockoon-greeting-api .

And then run the container:

    docker run -d -p 3000:3000 mockoon-greeting-api

Step 9. Use Mockoon CLI in a CI environment: GitHub Actions

Mockoon CLI being a Javascript application, it can run on any environment where Node.js is installed, including continuous integration systems like GitHub Actions, Buildkite, or CircleCI. It is useful when you want to run a mock server while running integration tests on another application. For example, you could mock the backend when running React front-end application tests. Or when you are running IT tests for a java based application that integrates with Greetings API.

Here is an example of a GitHub Action running a mock API (via docker) before running some tests:

name: Run mock API server

on: [push]

jobs:  
  run_integ_tests:
    name: Run integ tests
    runs-on: ubuntu-latest
    services:
      greetings:
        image: shanelee007/mockoon-greeting-api:latest
        ports:
          - 3000:3000

    steps:
    - uses: actions/checkout@v2
    - name: Set up JDK 11
      uses: actions/setup-java@v2
      with:
        java-version: '11'
        distribution: 'adopt'
    - name: Cache Maven packages
      uses: actions/cache@v2
      with:
        path: ~/.m2
        key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
        restore-keys: ${{ runner.os }}-m2
    - name: Build with Maven
      run: mvn --batch-mode -ff -V --update-snapshots verify

So just to summarise with the "API first" approach I can:

• import an OAS to mockoon
• configure the routes (if needed)
• set some rules
• add dynamic templating to responses (more on this later)
• export this file, containerise using the CLI and run it as part of my CI build.

All within a matter of seconds!

Generating dynamic data

Mocking an API can save you time. By faking the backend responses early, you don't have to worry about whether an endpoint is ready or not. You are up and running in no time and can start implementing your application. However, your mock should still be realistic. And the examples provided in the OAS are often not enough to surface UI layout problems, container overflowed by text, etc.

When mocking using Mockoon, you can easily customize your endpoints to make them look like real ones and even behave realistically, thanks to the dynamic templating system.

Generate random fake data

Nowadays, most developers work with JSON. Generating a massive amount of fake JSON data with Mockoon is a breeze thanks to the powerful templating system based on Handlebars syntax.

Mockoon also offers multiple helpers and embarks the Faker.js library, which can generate localized random data as various as cities, addresses, first names, phone numbers, UUID, etc.

Complete JSON example: posts list

So let's revisit our Greetings API again. The API provides a GET request to return all greetings. So let's override the examples provided from OAS with much richer content.

By using a combination of repeat, image.avatar, lorem.sentences, etc. you can quickly get a massive amount of random data. Combined with the latency option, you can even simulate a slow server and check how your application behaves under stress.

To use the templating system, you only have to use the response body editor and start adding your content. Remember to use the double curly braces to delimit your helpers {{ helperName }} Let's have a look at what such a body could look like:

[
  {{#repeat (queryParam 'total' '5')}}
  {
    "id": {{@index}},
    "message": "{{faker 'lorem.sentence' 3 5}}",
    "creationDate": "{{date '2020-11-20' '2020-11-25' "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"}}",
    "label": "key",
    "isFriendly": {{faker 'random.boolean'}},
    "weatherType": "{{oneOf (array '1' '2' '3')}}",
    "status": "{{oneOf (array 'SMILEY_FACE' 'SAD_FACE')}}"
  }
  {{/repeat}}
]

After a call to Mockoon, this would be the kind of body generated from this template:

[
  {
    "id": 0,
    "message": "Earum veritatis est.",
    "creationDate": "2020-11-24T17:32:33.293Z",
    "label": "key",
    "isFriendly": false,
    "weatherType": "1",
    "status": "SMILEY_FACE"
  },
  {
    "id": 1,
    "message": "Rerum ipsa autem.",
    "creationDate": "2020-11-23T10:30:00.526Z",
    "label": "key",
    "isFriendly": true,
    "weatherType": "3",
    "status": "SAD_FACE"
  },
  {
    "id": 2,
    "message": "Porro aut dolores.",
    "creationDate": "2020-11-22T21:03:58.452Z",
    "label": "key",
    "isFriendly": true,
    "weatherType": "1",
    "status": "SMILEY_FACE"
  },
  {
    "id": 3,
    "message": "Qui repudiandae quibusdam.",
    "creationDate": "2020-11-22T09:13:08.923Z",
    "label": "key",
    "isFriendly": true,
    "weatherType": "2",
    "status": "SAD_FACE"
  },
  {
    "id": 4,
    "message": "Qui et voluptatem.",
    "creationDate": "2020-11-22T08:36:19.770Z",
    "label": "key",
    "isFriendly": true,
    "weatherType": "1",
    "status": "SMILEY_FACE"
  }
]

This example makes extensive usage of what Mockoon and Faker.js have to offer. First, it generates as many "greetings" items as provided in the total query parameter (or default to 5) when calling GET /your/endpoint?total=140. It is especially useful when you want to request a specific number of items depending on the pagination or a "number per pages" user setting. Second, you can see that multiple properties are defined, and random mock data is generated like sentence, date-time, boolean etc.

There are a lot of possibilities and combinations you can try. You can also make your template react to a lot of parameters from the entering request by using Mockoon's helpers. We've already seen queryParam above, but you will find many more in the templating documentation. They allow you to query the request information like body, urlParam, header, method, etc.

Mockoon does not limit you to JSON. The templating language based on Handlebars is compatible with any content type. It means that you can generate CSV, HTML, XML, etc. You will find below some examples of what can you can achieve with the templating system.

Generate dynamic templating depending on the request

We just saw some interesting use-cases but still quite simple. When working on your application, you may want to go a little bit further by making the template react to the request sent to Mockoon. This is possible by using various helpers that you will find in the templating documentation: body, queryParam, urlParam, cookie, header, hostname, ip, method, etc.

They allow you to access the entering request's information. Combined with other helpers like repeat, switch, or if, you will be able to dynamically generate more complex content.

You will find below some examples:

New greeting after a POST request

We will reuse in the response the various parameters present in the request:

{
  "id": "{{faker 'random.uuid'}}",
  "message": "{{body 'message'}}",
  "creationDate": "{{date '2020-11-20' '2020-11-25' "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"}}",
  "label": "{{body 'label'}}",
  "isFriendly": {{body 'isFriendly'}},
  "weatherType": {{body 'weatherType'}},
  "status": "{{body 'status'}}"
}

After a call to this endpoint with the following body:

POST /greetings  
Content-Type: application/json

{
  "message": "Hello Docker",
  "label": "key",
  "isFriendly": true,
  "weatherType": 0,
  "status": "SMILEY_FACE"
}

We would receive this kind of response content, containing the request information plus some new fields (id and creationDate):

{
  "id": "6df3c0c6-bce8-4094-ae29-5cb637fc15a3",
  "message": "Hello Docker",
  "creationDate": "2020-11-24T14:49:47.139Z",
  "label": "key",
  "isFriendly": true,
  "weatherType": 0,
  "status": "SMILEY_FACE"
}

For more complex cases or to test various error handling scenarios, you could also create multiple responses for the same route, with different bodies, and trigger them by defining some rules. To learn more about using multiple responses combined with rules, you can have a look at the related documentation.

Src files used in this post can be found on github

I said jibbing not jiving! 😆

Cloud native architecture

When it comes to microservices and cloud native architecture you first think about containers. Now you can of course compose your own docker file. But with jvm based microservices there is a tool from google called jib that can simplify containerization.

We'll take a simple Spring Boot application and build its Docker image using Jib. And then we'll also publish to GCR and deploy on google cloud run.

Jib

Jib builds optimized Docker and OCI images for your Java applications without a Docker daemon - and without deep mastery of Docker best-practices. It is available as plugins for Maven and Gradle and as a Java library.

You don't even need docker installed. Just use maven or gradle plugin and away you go! It uses distroless base image under the hood, but as you expect all of this is configurable.

Jib supports multiple container registries, can change the base image, jvm flags, tags, volumes and much more.

Greeting App

I use VSCode heavily for development so I wanted to see what support they have for java and spring boot. In fact they have many extensions. The extensions I installed are:

• Gradle Extension Pack
• VS Code for java
• Spring boot tools
• Spring Initializr Java Support
• Spring Boot Dashboard
• Lombok Annotations Support for VS Code
• OpenAPI (Swagger)
• Settings Sync very important to sync if I change laptop in the future...

I used Spring Initializr to create a simple project in vs code.
It'll expose a simple GET endpoint:

http://localhost:8080/greeting  

Deployment

Now I wanted to deploy this on cloud run and not worry about defining a docker file. So I used the gradle jib plugin and configured the credentials helper to deploy to GCR (Google cloud registry)

Cloud run

You can think of cloud run as CAAS (container-as-a-service) or serverless containers. It allows you to run your stateless HTTP containers without worrying about provisioning machines, clusters or autoscaling. The main difference with app engine flexible is that it can scale to zero - only pay per request.

If you are interested to learn more about cloud run check out the unofficial faqs

There is even a cloud run button if you want to deploy your API publicly.

There is a fantastic VSCode extension called cloud code that can help you deploy to cloud run or GKE on google cloud. It can even pick up if you are using skaffold or jib under the hood.

This command will build, push your image to GCR and deploy to cloud run. And that's it!

Now you can access the public url and test out the endpoint.

The source code for the example is over on github

API design

In my previous post I discussed graphQL. Graphql provides a schema for introspection and type safety.

Now for REST, swagger or open API spec is the standard for designing and documenting your API. Even kubernetes API supports open api spec. Every time you call kubectl describe <resource> it calls this endpoint for info.

Tools such as insomnia designer and api gateways now support open api spec.

I decided to design and document my greetings API using redoc

Redoc

OpenAPI/Swagger-generated API Reference Documentation

Redoc supports open api spec v3 and provides responsive documentation with code samples. There is many ways to deploy but again Ill go the docker way and deploy on cloud run.

Redoc generator

Redoc has a generator which provides a docs-like-code approach to OpenAPI definitions. It allows you to validate your spec before bundling it. I documented my greetings API above, bundled it and served it as static content from the API itself

One thing to note is you need to enable CORS in your API to serve the static file. I also had an issue with @EnableMVC annotation so I removed that.

To deploy, I built redoc on GCR and deployed to cloud run. You can run this in cloud shell if you like

export PROJECT_ID=$(gcloud config list --format 'value(core.project)')  
#change CLOUD_RUN_SPRING_BOOT_BASE_URI to the location of your API deployed on cloud run

gcloud run deploy redoc-greetings-api --project $PROJECT_ID --image gcr.io/$PROJECT_ID/redoc --platform managed --region us-central1 --port 80 --cpu 1 --memory 256Mi --concurrency 80 --timeout 300 --update-env-vars SPEC_URL=https://<CLOUD_RUN_SPRING_BOOT_BASE_URI>/swagger.yaml  

Again, the src code can be found on github

And thats it. Happy jibbing! 🕺🏻

Well, the Google Cloud Platform (GCP) is behind the 8 ball for sure in comparison to AWS. But with the huge spike in companies looking to reduce their Capex and move to the cloud, its offerings in the data analytics space for example is very enticing. Also in my view, they have the best managed kubernetes service GKE. Which makes sense, when you understand that kubernetes originated from googles own internal cluster orcherstration service called borg.

I have worked with AWS for many years professionally. Gaining exposure to other cloud providers is important when you see companies adopting a multi-cloud approach but fundamentally many of the underlying patterns and managed services commonly exist across the top 3 (AWS, GCP, Azure).

Up until recently I did not fully understand where Anthos fit into the picture. But now I do. By bringing compute closer to the data, existing cloud users can now use the likes of GKE and BigQuery. Nicely played Google! If the mountain won't come to muhammed... ☺️

Certification

This certification allowed me to analyse and understand the majority of GCP services. But also provide a solid understanding of architectural best practices when it comes to design considerations, security, reliability and cost optimisation.

Exam Preparation

There are several training providers out there that provide courses specific to this exam: Coursera, Udemy, Linux academy to name but a few.

Coursera’s:

• Architecting with Google Compute Engine

• Preparing for the Professional Cloud Architect Examination

Linux Academy’s:

• Google Cloud Certified Professional Cloud Architect

I took the updated course from Linux Academy’s Google Cloud Certified Professional Cloud Architect.

Linux Academy’s updated course was a good choice as it covers the majority of exam related GCP based topics and its practice sessions are an added advantage associated after each lesson. Their master builders guide was a very important document that I referred back to many times.

I also like to be hands on, so I signed up to qwiklabs and completed many of the quests they provide in relation to GCP. It's extremely helpful especially if you do not have your own google cloud account.

Google also provides a learning path as an alternative option.

Exam registration

Register for the exam when you are about 1 week into your study. This allows you to book in a slot not too far in the future and keeps you motivated.

Exam layout

The exam is two hours long. You can take it remotely or at a test centre ( I took it at home). There are 50 questions - multiple choice and multiple select. The GCP case studies take up about 12 questions with the rest, GCP in general.

Exam score

The passing marks are not shared by Google in any of GCP exam except it tells about pass or fail. However it has been assumed from various blogs and training providers that it’s roughly around 80% for GCP exams though no official confirmation about it.

Exam readiness

Check your readiness by taking the google practice exam and Linux academys (if you have signed up for the course). If you are not scoring 90% or above, then keep studying!

Tips

• You will have plenty of time to go back and review marked questions. So make sure if you are stuck, mark question and move on.

• Eliminate answers that you know are incorrect

• Look out for main keywords in the question - speeds, low latency, failover, serverless etc etc

Final suggestions

• Remember to deeply learn Kubernetes, Bigdata services(Big Query, BigTable), NoSQL options, App engine, Storage, IAM, BigQuery Roles, Case studies..

• Googles new architecture framework is a very useful resource too.

Still waiting on my bag!! Best of luck.

This is an overview of my journey to reducing my carbon footprint and making my home as energy efficient as possible.

Energy efficiency

WFH myself since March, I did not realise how cold and under insulated australian households can be! I'm european and I feel colder here than winter time in Vienna at minus 10 degrees! 🥶

We’re lagging between 10-15 years behind in Europe, parts of US and Canada,” says Trivess Moore, a senior lecturer in construction at RMIT and member of the Sustainable Building Innovation Laboratory.

You can read more on this topic where a housing estate project looked at building homes with at least an energy rating of 7.5 and the outcomes that came from this in relation to cost savings but also wellbeing
https://onestepoffthegrid.com.au/even-with-rooftop-solar-boom-consumers-are-paying-dearly-for-what-lies-underneath/

So what can we do about it?? Well, one recommendation I would like to see is for state and local councils to work with property developers and provide incentives to build energy efficient homes. But in the meantime, what can a homeowner do to rectify the situation... 🤔

Renewables

Last year, I installed solar panels on my house. This is something I had always planned to do when I purchased my own home. With Victorias rebate program, there is a huge uptake in residents installing solar. And why wouldn't you! Reduce your energy costs over time and become less reliant on the grid and our coal-burning plants. 🌍

Solar

Before I chose a solar installer to go with, I researched online to understand the best solution for my home.

Clean energy council (CEC) has a great guide on solar here. I choose not to get a battery now as they are still quite expensive. But with the rapid innovation in this space, it won't be long in my view before it becomes attainable.

Victoria energy council has expanded the battery rebate to certain postcodes if applicable.

Finn at solar quotes has invaluable information on choosing the right panels, inverters and provides quotes for local reviewed installers.

I contacted around five or six installers (all CEC accredited) and was surprised by the results. One actually refused to do a house inspection and focused solely on nearmap satellite imagery. Another stated that it was not worth installing based on their panel placement!

System choice

So the system I decided on was 6.27KW Solar System with:

• 19x Q-Cells Dual Cell Q.Peak Duo G5-330W
• 19x Micro-Inverters Enphase IQ7+
• 1x Envoy S-Metered + DRM
• 1x Sub-board installation to fit the Envoy relays and allow room for future use

All panels were installed west facing.

Enphase Micro-Inverters make each panel run independently, which is a big advantage compared to a string inverter system where when one panel starts to act faulty, it shuts down the entire array of panels.

It also comes with a monitoring system, the Enphase enlighten view, which allows you to monitor not only the PV production of your solar system but also your home electricity consumption.

More information on micro inverters can be found here

There are some myths about installing on the south for example but it really depends on where you live as Finn explained, but it will inevitably still produce energy! It all comes back to self consumption

I have the monitoring setup using enlighten and have a cheeky look every day to see my usage and production.

Below is the energy produced for a typical summers day in Melbourne.

There is a great sense of satisfaction knowing I am using clean energy to power my house during the day.

Self consumption

I mentioned earlier about self-consumption. As my wife and I both work during the day (pre-covid), we try to utilize solar power when we can.

For example, I turn on the dishwasher before I leave for work in the morning and set the washing machine on a timer to turn on at peak time around 12-1 pm during the day.

Here is a great case study of how a young family self consume.

It is important to understand that the feed-in tariff (FIT) most likely will fluctuate in the future (in VIC Jul 2020 reduced from 12c to 10.2c) so try and use up as much clean energy as possible.

Lastly, here is a fantastic write up of a resident using solar for a year

An energy efficient home

After installing solar, I looked into other areas I could make my home more energy efficient.

I stumbled across vic energy saver site that provides savings for energy efficient products.

I had existing halogen downlights that I wanted to replace with LEDs and this initiative allowed me to replace all for FREE!

I used an accredited provider and it was pretty painless. They replaced all 40 of my downlights (apart from my dimmers). #winning

The program even supports showerheads, weather sealing, and hotwater systems. Buying energy efficient appliances such as your oven, heat pump dryer, washing machine will make a difference too.

Get an inspection

Finally, if you want you can look into an energy assessment of your house.

Waste

I was happy to see the Vic government bring in the plastic bag ban. Definitely well overdue. I believe in Ireland it came into play about ten years ago!

Since China stopped accepting alot of our waste the government has been scrambling to fix the waste management issue. I am happy to see Vic government outlining their plan. Highlights are:

• Four bin system 🙌🏻
• Container deposit scheme
• Circular economy

I actually saw new popup bicycle lanes in fitzroy north recently using recycled plastic 👍

More info on using recycled materials can be found here

Food waste

Each year in Victoria households throw out 250,000 tonnes worth of food – enough wasted food to fill Melbourne's Eureka Tower. 😱

My wife and I have made a joint effort when shopping to buy fruit and veg loosely if possible. And also buy organically too. We have trialed growing some herbs and vegetables at home. Trial and error! But a great learning experience.

I listened to a podcast recently with Dr. Sandro Demaio (CEO of VicHealth) on food and its impact on society, our ecosystem and health. Less red meat people!! #votewithyournote

We also bring our scrunchy plastic to the supermarket. Both coles and woolworths support the initiative.

Transport

After energy, transport is the biggest carbon emitter.

Transport in Australia contributes around 100 million tonnes of greenhouse gasses into our atmosphere every year.

The main reasons for transport emissions trending upwards are an over-dependence on cars with high average fuel use and an over-reliance on energy-intensive road freight.

So it comes to my favourite topic of all active transport and cycling!

Active transport

Anyone who knows me, knows how I love cycling. I learned to cycle from a very young age in the irish countryside before you had to worry about cars on the road. Wherever I have worked and lived, whether in europe or australia my preferred mode of transport is the bike. For me, I love the independence and time alone on the bike. Plus it's a great way to wake up in the morning (especially if you need to cycle down chapel st 🤣)

Anytime my wife and I go abroad, I always go check out a city's shared bicycle scheme if available and explore the city by bike. This to me is the best way to see a city. On a warm summer evening cycling down along the banks of the river seine in Paris listening to some jazz and watching the world go by is pure bliss. 🇫🇷

Since covid there has been a huge uptake in cycling. I notice it myself at the weekend and on the trails. Governments like ireland and the UK have invested heavily in protected cycleways and pedestrian infrastructure.

Change is coming, there is no doubt about it. The new paris mayor Anne Hidalgo has called it the 15 minute city idea. That is to have all services within a 15 minute walk/ride from your home. The days of commuter towns and work in the city after covid will not be the same. Working from home and even coworking spaces in rural areas will become more mainstream I predict.

“We need to reinvent the idea of urban proximity,” Moreno says. “We know it is better for people to work near to where they live, and if they can go shopping nearby and have the leisure and services they need around them too, it allows them to have a more tranquil existence.

Electric vehicles

Just a quick mention on EVs. I am still waiting for the national EV strategy to be released in Australia. It has been delayed many times. Nudge, nudge Angus Taylor...

Now take Norway for example - The Norwegian Parliament has decided on a goal that all new cars sold by 2025 should be zero (battery electric or hydrogen) emission vehicles.

Unfortunately for alot of aussies, EVs are just too expensive right now and there is no incentive to switch.

Investing for the future

I just finished Ross Garnuat book recently Superpower: Australia's low carbon opportunity and our renewable investment needs to increase dramatically. The IMF says Australia should be spending more on infrastructure, but this should be on rail, airports and seaports, rather than roads.

With low interest rates for the forseeable future, Deloitte, Grattan institute, and many others have stated now is the time to invest and invest smartly. The IMF has called this "great reset" an opportunity for fiscal stimulus that is smarter, greener and fairer. So lets push ahead and be bold for a change.

Finally

I know reducing our carbon footprint sounds challenging and some feel overwhelmed by it all. But I hope this article helps educate others and know there is support out there. Albert Einstein once said:

“In the middle of difficulty lies opportunity.”

We all have an opportunity to make a difference and create a more sustainable future.