Skip to content
Corentin GS

TDD in the Age of AI: Who Tests the Tests?

№40 · · ·1619 words ·8 min read
TDD in the Age of AI: Who Tests the Tests?
In this piece

The AI Coding Paradox

I use Codex and Pi daily. They write code faster than I can read it. Lines appear on my screen in seconds that would have taken me twenty minutes to type.

Over the past year, I have learned one rule: the code AI writes is only as good as the specification I give it.

Vague prompts produce plausible code that misses the point. Precise prompts produce code close to what I intended. The difference is the specification.

AI makes code faster to produce; it does not improve your understanding of what the code should do. Without a definition of correct behavior, you get code that compiles without proving anything.

TDD gives AI-assisted development a specification layer.

You write tests that describe intent. AI writes implementation that satisfies them. You review. The tests verify.

TDD still matters in the age of AI because it defines the behavior the implementation must satisfy.


My TDD Workflow With AI

My day-to-day workflow:

  1. I write a test that describes the behavior I need.
  2. AI writes the implementation.
  3. The tests run and verify the behavior.
  4. I review the implementation.
  5. I refactor with tests protecting me.

A PasswordValidator illustrates the process.

Step 1: I write the spec (tests)

type Result<T, E> = { ok: true; value: T } | { ok: false; error: E };

function ok<T>(value: T): Result<T, never> {
	return { ok: true, value };
}

function err<E>(error: E): Result<never, E> {
	return { ok: false, error };
}

class PasswordError {
	constructor(
		public readonly code: string,
		public readonly message: string
	) {}
}

describe("PasswordValidator", () => {
	it("should accept strong passwords", () => {
		const result = PasswordValidator.validate("Str0ng!Pass#2026");
		expect(result.ok).toBe(true);
	});

	it("should reject passwords under 12 characters", () => {
		const result = PasswordValidator.validate("Sh0rt!");
		expect(result.ok).toBe(false);
		if (!result.ok) {
			expect(result.error.code).toBe("TOO_SHORT");
		}
	});

	it("should require uppercase, lowercase, digit, and special character", () => {
		const noUpper = PasswordValidator.validate("alllowercase1!");
		expect(noUpper.ok).toBe(false);
		if (!noUpper.ok) {
			expect(noUpper.error.code).toBe("MISSING_UPPERCASE");
		}

		const noDigit = PasswordValidator.validate("NoDigitHere!");
		expect(noDigit.ok).toBe(false);
		if (!noDigit.ok) {
			expect(noDigit.error.code).toBe("MISSING_DIGIT");
		}

		const noSpecial = PasswordValidator.validate("NoSpecialChars9");
		expect(noSpecial.ok).toBe(false);
		if (!noSpecial.ok) {
			expect(noSpecial.error.code).toBe("MISSING_SPECIAL");
		}
	});

	it("should reject common passwords", () => {
		const result = PasswordValidator.validate("Password123!");
		expect(result.ok).toBe(false);
		if (!result.ok) {
			expect(result.error.code).toBe("TOO_COMMON");
		}
	});

	it("should reject passwords with repeated sequences", () => {
		const result = PasswordValidator.validate("abcabcABCd1!");
		expect(result.ok).toBe(false);
		if (!result.ok) {
			expect(result.error.code).toBe("REPEATED_SEQUENCE");
		}
	});
});

The tests define behavior, expected error codes, and the edge cases I care about. That is the specification AI needs.

Step 2: AI generates the implementation

I paste the tests into my AI assistant and say: “Make these tests pass.”

It produces:

class PasswordValidator {
	private static readonly COMMON_PASSWORDS = [
		"password",
		"password123",
		"123456789",
		"qwerty123",
		"admin123",
		"letmein1",
		"welcome1",
		"password1!",
		"iloveyou",
		"monkey123",
	];

	private static readonly MIN_LENGTH = 12;

	static validate(password: string): Result<string, PasswordError> {
		if (password.length < PasswordValidator.MIN_LENGTH) {
			return err(
				new PasswordError(
					"TOO_SHORT",
					`Password must be at least ${PasswordValidator.MIN_LENGTH} characters`
				)
			);
		}

		if (!/[A-Z]/.test(password)) {
			return err(
				new PasswordError(
					"MISSING_UPPERCASE",
					"Password must contain at least one uppercase letter"
				)
			);
		}

		if (!/[a-z]/.test(password)) {
			return err(
				new PasswordError(
					"MISSING_LOWERCASE",
					"Password must contain at least one lowercase letter"
				)
			);
		}

		if (!/[0-9]/.test(password)) {
			return err(
				new PasswordError(
					"MISSING_DIGIT",
					"Password must contain at least one digit"
				)
			);
		}

		if (!/[!@#$%^&*()_+\-=\[\]{};':"\\|,.<>\/?]/.test(password)) {
			return err(
				new PasswordError(
					"MISSING_SPECIAL",
					"Password must contain at least one special character"
				)
			);
		}

		const lowerPassword = password.toLowerCase();
		const isCommon = PasswordValidator.COMMON_PASSWORDS.some(
			(common) =>
				lowerPassword.includes(common.toLowerCase()) ||
				common.toLowerCase().includes(lowerPassword)
		);
		if (isCommon) {
			return err(
				new PasswordError("TOO_COMMON", "Password is too common or predictable")
			);
		}

		if (/(.{3,})\1{1,}/i.test(password)) {
			return err(
				new PasswordError(
					"REPEATED_SEQUENCE",
					"Password contains repeated sequences"
				)
			);
		}

		return ok(password);
	}
}

These tests would now pass against this implementation.

Step 3: I review and refine

The structure and validation logic are reasonable, but the review finds three problems:

  • COMMON_PASSWORDS is hardcoded and small. Production needs a larger dataset. A bloom filter can reduce memory use, but its false positives can reject valid passwords.
  • The special-character regex has unnecessary escapes.
  • The repeated-sequence regex /(.{3,})\1{1,}/i needs edge-case checks.

The tests let me review the implementation against a contract I wrote. Without them, I would be judging code quality without an objective measure of correctness.


What AI Gets Wrong About Tests

Letting AI write both the tests and the implementation creates an appearance of verification without an independent check. AI-generated tests often become implementation-aware: they test how code works rather than what it does.

This is exactly the trap from the permission slip article: testing structure instead of behavior.

Watch what happens when I ask AI to generate tests for a UserService:

// ❌ AI-generated tests (implementation-aware)

describe("UserService", () => {
	it("should call userRepository.save with user object", async () => {
		const mockRepo = {
			save: jest.fn().mockResolvedValue(undefined),
			findByEmail: jest.fn().mockResolvedValue(null),
		};
		const service = new UserService(mockRepo as any);

		await service.register("[email protected]", 25);

		expect(mockRepo.save).toHaveBeenCalledWith(
			expect.objectContaining({
				email: "[email protected]",
				age: 25,
			})
		);
	});
});

These tests verify internal method calls and their arguments. A refactor that preserves behavior can still break them.

Now here’s what I write:

// ✅ Human-written tests (behavior-aware)

describe("UserService", () => {
	it("should register a new user with valid data", async () => {
		const repo = new InMemoryUserRepository();
		const service = new UserService(repo);

		const result = await service.register("[email protected]", 25);

		expect(result.ok).toBe(true);
		const found = await repo.findByEmail("[email protected]");
		expect(found.ok).toBe(true);
	});

	it("should reject duplicate email registration", async () => {
		const repo = new InMemoryUserRepository();
		const service = new UserService(repo);

		await service.register("[email protected]", 25);
		const result = await service.register("[email protected]", 30);

		expect(result.ok).toBe(false);
		if (!result.ok) {
			expect(result.error.code).toBe("DUPLICATE_EMAIL");
		}
	});

	it("should reject invalid email during registration", async () => {
		const repo = new InMemoryUserRepository();
		const service = new UserService(repo);

		const result = await service.register("not-an-email", 25);

		expect(result.ok).toBe(false);
		if (!result.ok) {
			expect(result.error.code).toBe("INVALID_EMAIL");
		}
	});
});

The tests assert observable behavior with a fake repository rather than internal calls.

Human-written tests describe intent; AI-generated code must satisfy it.


The New TDD Cycle

AI changes the roles in TDD:

Traditional TDD:
  Human writes test → Human writes code → Human refactors

AI-Augmented TDD:
  Human writes test → AI writes code → Human reviews → Human refactors

                                    Tests protect this step

Red, Green, and Refactor remain the same phases:

  • Red: The human writes the test; it specifies the behavior.
  • Green: AI writes the implementation under supervision.
  • Refactor: The human refactors, with AI assistance and tests as a safety net.

The human moves from implementation toward specification and review. Test quality bounds the quality of AI-assisted code.


Practical Patterns for AI-Augmented TDD

These patterns have held up in my workflow:

Pattern 1: Test-First Prompting

Start with tests, then prompt AI: “Make these tests pass.” The tests act as the specification and acceptance criteria.

describe("Email", () => {
	it("should normalize to lowercase", () => {
		const result = Email.create("[email protected]");
		expect(result.ok).toBe(true);
		if (result.ok) {
			expect(result.value.getValue()).toBe("[email protected]");
		}
	});
});
// Prompt: "Implement Email to make these tests pass"

Pattern 2: Specification Testing

Write tests that describe domain rules. Let AI figure out the architecture.

describe("Order", () => {
	it("should reject orders exceeding credit limit", () => {
		const order = Order.create(customerId, CreditLimit.usd(100)).value!;
		order.addLine("Widget", 1, Money.usd(75));

		const result = order.addLine("Gadget", 1, Money.usd(50));
		expect(result.ok).toBe(false);
	});
});
// AI decides: class-based? function-based? How to structure it.
// You decide: what the rules are.

Pattern 3: Mutation Testing for AI Code

Change the implementation slightly. Do the tests catch it? If not, your tests aren’t specific enough. Your AI-generated code could silently regress.

Pattern 4: The Trust Boundary

I write tests for domain logic and critical paths. I let AI write tests for utility functions and boilerplate. The rule: the more important the behavior, the more human involvement the tests need.


What This Means for Everything We’ve Built

The patterns from this series gain value when AI writes more of the implementation:

Value Objects are constraints for AI output. When I tell AI to create an Email value object, the private constructor and Result-returning factory constrain what AI can produce. It can’t generate an Email that bypasses validation because the type system won’t allow it. Value Objects are guardrails for AI.

Aggregates protect invariants that AI might accidentally violate. Without the Order aggregate boundary, AI might generate code that adds order lines without checking the credit limit. With it, the aggregate enforces the rule regardless of who writes the calling code.

The Result pattern makes AI-generated error handling auditable. When every function returns Result<T, E>, I can see exactly what errors AI decided to handle, and which ones it missed. Explicit error types are a review tool.

Behavior-focused testing, domain modeling, and explicit errors give you the vocabulary to direct AI. They turn vague prompts into precise specifications you can verify.


What Developers Still Decide

I keep coming back to one question: if AI can write the implementation, what’s my value as a developer?

My job is to decide what to build.

AI can draft a password validator, a registration flow, or a payment pipeline quickly. It lacks the domain context to decide which rules matter: Password123! may meet a complexity rule but remain common; duplicate registration may be a domain error; a credit limit may cover every pending order.

Those decisions come from domain knowledge, stakeholder conversations, experience with the system, and an opinion about what good looks like.

TDD requires those decisions before implementation begins. AI makes that requirement harder to ignore.


FAQ

Can AI write tests for you?

AI can draft tests, but critical behavior needs human-written tests. AI-generated tests often verify implementation details, break during refactoring, and miss real bugs. Write the specification; let AI implement it.

How does TDD work with AI coding assistants?

The cycle shifts from “human writes test, human writes code” to “human writes test, AI writes code, human reviews.” Red-Green-Refactor remains intact; AI handles Green while you specify intent and review the output.

Do you still need TDD with AI?

Yes. Tests provide an objective check on AI-generated code. TDD turns tests written before implementation into acceptance criteria and prevents false confidence from weak tests.


Summary

TDD gives AI-assisted development a contract: you define behavior in tests, AI implements it, and you review and refactor with a safety net.

Kent Beck’s name still matters. Tests drive design by stating desired behavior before code exists. When AI writes more implementation, those decisions have to be explicit.

Explore this subject

More on Software design & testing