Part 2 of our Agentic Engineering Playbook series: How we designed the engineering system that allows AI coding agents to work within organizational standards, validate their implementations, and continuously improve.
Introduction: Building the Engineering System Around the Agent
In the first article of this series, I shared our seven-month journey at Madgicx, where we moved from experimenting with AI coding agents to establishing a structured approach to agentic engineering.
That journey taught us something important.
The challenge is not simply getting AI agents to write good code. It is creating an engineering environment in which they can consistently make good decisions.
When an experienced engineer joins an organization, they gradually learn its architecture, engineering practices, business terminology, security policies, and governance requirements.
They also learn where to find information, which decisions require approval, and how to validate their work.
An AI coding agent needs access to much of the same knowledge.
But there is an additional challenge.
Unlike an engineer who gradually develops organizational understanding, an AI agent often begins a new session with limited context.
It may not know what happened in the previous session.
It may not understand why an architectural decision was made.
And unless we provide the necessary information, it may confidently implement something that violates our existing engineering contracts.
We initially tried to solve this problem through extensive prompts and planning documents.
That approach quickly became too heavy.
We then moved toward smaller, focused standards, persistent project memory, shared organizational knowledge, and deterministic validations.
Over time, these capabilities evolved into our Agentic Engineering Playbook (AEP).
Today, the AEP operates as a local engineering framework that guides agents through research, planning, implementation, and validation.
It integrates with our organizational governance standards and complements the centralized security and quality controls we enforce through GitHub.
It also captures feedback from engineering sessions so that important lessons can become improvements to the shared playbook.
In this article, I want to move beyond the journey and explain the architecture itself.
I will cover the seven layers of our AEP, the engineering decisions behind them, how governance and enforcement work, and how we are building a continuous learning mechanism around agentic development.
This is not intended to be a universal architecture that every organization should adopt unchanged.
It is the approach we developed through practical engineering experience, and the principles we believe are transferable to other organizations.
1. The Architectural Principle: Separate Intelligence from Enforcement
Before discussing the individual layers, I want to establish the most important principle behind our architecture.
We do not rely on the AI agent to enforce every engineering requirement.
AI agents are good at reasoning about requirements, investigating systems, evaluating alternatives, and generating implementations.
But they are probabilistic systems.
They can overlook instructions, misinterpret context, or incorrectly determine that a rule does not apply.
This makes them useful for engineering judgment, but unsuitable as the sole mechanism for enforcing deterministic requirements.
For example, consider a rule that prohibits committing credentials.
We can document that rule in the agent’s instructions.
We can remind the agent during planning.
We can even ask the agent to review its own changes for secrets.
All of those measures are useful.
But none provides the same assurance as executing a secret scanner against the actual changes.
The same principle applies to linting, package restrictions, protected directories, formatting, and other machine-checkable requirements.
Our architecture therefore separates three responsibilities.
| Responsibility | Purpose | Examples |
|---|---|---|
| Agent intelligence | Understand problems, make proposals, and execute engineering work | Research, planning, implementation, design review |
| Local AEP enforcement | Guide and validate work during development | Gitleaks, package checks, code quality, workspace standards |
| Centralized enforcement | Independently verify organizational requirements | GitHub CI/CD, security scanning, required checks, review controls |
Organizational governance sits above these responsibilities, providing the authoritative policies and standards they must follow.
This separation is fundamental.
The agent can reason about whether a design complies with a security requirement.
The local tooling can detect violations that are mechanically verifiable.
And the centralized engineering platform can independently enforce critical requirements before changes progress through the delivery pipeline.
The result is not a single control.
It is a system of complementary controls.
A simplified view of the architecture
This is a conceptual view rather than a complete deployment diagram.
The AEP operates within the developer’s or agent’s local workflow. GitHub remains an independent enforcement layer.
The feedback mechanism allows the playbook to improve based on actual engineering experience.
With that foundation established, let us examine the seven layers
2. Layer One: A Role-Based Engineering Pipeline
Our first layer defines how agents participate in the engineering lifecycle.
One of the mistakes we made early was allowing a single agent session to move from a loosely defined requirement directly into implementation.
Sometimes that worked.
But on more complex tasks, the agent would begin coding before fully understanding the problem.
It might make assumptions about business logic, overlook an existing architectural pattern, or implement a solution that later required significant rework.
Our initial response was to introduce a comprehensive planning process.
As I explained in the first article, that became too heavy.
We eventually simplified the approach into four responsibilities:
Research → Plan → Implement → Validate
The distinction is important.
These are engineering responsibilities, not necessarily four separate agents that must be invoked for every task.
The workflow should remain proportional to the complexity and risk of the change.
Research: Understand before changing
The research responsibility is to establish what is actually true about the system.
The agent investigates the relevant codebase, existing functionality, dependencies, and architectural constraints.
It should identify the files and components involved, determine whether similar functionality already exists, and understand the contracts that the implementation must respect.
Research should be grounded in the actual repository and authoritative organizational knowledge.
An agent should not invent an API contract simply because it seems reasonable.
If the contract exists, it should be retrieved and examined.
If the contract is unclear, the uncertainty should be surfaced rather than silently resolved through assumption.
Plan: Decide what should be built
The planning responsibility is to translate the requirement and research findings into an implementation approach.
A useful plan should explain the intended changes, identify affected components, highlight important risks, and define how success will be validated.
For larger changes, this may include architectural alternatives and trade-offs.
For smaller changes, a concise plan may be sufficient.
The objective is not to produce documentation for its own sake.
It is to ensure that implementation begins with a reasonable understanding of the problem and the intended solution.
Implement: Build in verifiable increments
The implementation responsibility is to execute the approved approach.
We prefer smaller, independently verifiable increments rather than large changes that are difficult to review.
This allows the agent to validate progress and correct problems earlier.
It also makes the resulting changes easier for engineers to understand.
Validate: Prove that the work meets its requirements
Validation is not simply asking the agent whether its implementation is correct.
It involves executing relevant tests, code-quality checks, security validations, and other applicable controls.
The agent should compare the implementation against the acceptance criteria and report what has actually been verified.
An important distinction is between a test that was executed successfully and a validation the agent merely believes would pass.
We want evidence, not confidence.
3. Challenging Plans with GrillMe-with-Docs
One of the more interesting additions to our planning workflow is GrillMe-with-Docs.
We introduced it to address a recurring problem.
An agent could produce a technically plausible implementation plan that failed to account for an important business, governance, or security requirement.
Traditional code validation often catches problems after implementation.
But by that stage, the agent may already have built a solution around an incorrect assumption.
We wanted a way to challenge those assumptions earlier.
How we use GrillMe-with-Docs
During planning or architectural design, we use GrillMe-with-Docs to challenge the proposed approach against relevant organizational documentation.
This includes:
- Business requirements and domain standards.
- Security policies.
- Governance requirements.
- Architectural decisions.
- Other applicable engineering standards.
The objective is to expose gaps in the design before implementation begins.
For example, imagine an agent proposing a new service that processes customer data.
The proposed architecture might be technically sound.
But does it follow the organization’s established data-handling requirements?
Does it introduce a new integration pattern when an approved one already exists?
Does it respect the relevant business definitions?
Are there security constraints that affect the design?
GrillMe-with-Docs introduces a structured challenge to these assumptions.
It is not a replacement for architectural review or human judgment.
Instead, it helps ensure that relevant organizational requirements are considered during the planning process.
Why this matters
In agentic engineering, we should not concentrate all our controls at the end of the development lifecycle.
We should also improve the quality of the decisions that lead to implementation.
Finding a design problem before coding is usually less expensive than correcting it after implementation.
There is an important balance, however.
We do not want every small change to require a heavyweight architectural interrogation.
The depth of planning and design review should reflect the risk and complexity of the work.
That principle emerged directly from our earlier experience with excessive process.
4. Layer Two: Engineering Standards as Focused Skills
Our second layer provides the engineering standards that agents need to follow.
Initially, we experimented with large instruction documents.
These contained coding conventions, architecture guidance, security requirements, testing expectations, and numerous other rules.
The problem was that much of this information was irrelevant to the task being performed.
A frontend agent did not need every backend standard.
A Python project did not need React-specific instructions.
And a minor configuration change did not require the entire organization’s engineering handbook.
Loading unnecessary information increased token consumption and could introduce conflicting instructions.
We moved toward smaller, focused standards that can be activated when relevant.
Universal and technology-specific standards
We distinguish between standards that apply broadly and those that apply to particular technologies or engineering domains.
Universal standards cover principles such as maintainability, testing expectations, security practices, and avoiding unnecessary complexity.
Technology-specific standards address the conventions and requirements of particular languages, frameworks, and platforms.
We generally aim to keep individual standards concise—around 200 lines or fewer where practical.
The purpose is not to impose an arbitrary document limit.
It is to encourage focused, actionable guidance rather than comprehensive reference manuals.
Detecting the technology stack
One of our early mistakes was activating standards based on repository or folder names.
That proved unreliable.
A directory name might suggest React, for example, while the actual project dependencies indicated a different set of requirements.
We changed the approach to use evidence from the project itself.
Configuration and dependency files, such as package.json and pyproject.toml, provide more reliable information about the technologies in use.
This allows the AEP to select more appropriate standards.
Standards should guide, not duplicate tooling
We also distinguish between instructions that require judgment and requirements that can be enforced mechanically.
A standard might explain when to introduce an abstraction.
That requires engineering judgment.
But a formatting requirement should generally be enforced through a formatter.
A prohibited dependency can often be detected through package validation.
A secret should be detected through security tooling.
The principle is:
Use standards to guide engineering decisions. Use tools to enforce deterministic requirements.
This reduces the burden placed on the agent and improves consistency.
5. Layer Three: Persistent Project Memory
AI coding sessions have a fundamental limitation.
They are temporary.
Even when a tool supports long conversations, carrying an entire development history forward is not always desirable.
Context becomes expensive.
Important decisions become buried.
And outdated information may remain alongside newer decisions.
We needed a way to preserve useful project knowledge without retaining every conversation.
Our solution was to maintain a small number of living project documents.
Our three-document approach
| Document | Responsibility |
|---|---|
CLAUDE.md or AGENTS.md | Project instructions, conventions, and relevant operating guidance |
TODO.md | Current tasks, progress, and remaining work |
HANDOFF.md | Important findings, decisions, and context required by subsequent sessions |
The precise file names may differ between agent platforms.
The responsibilities are more important than the names.
Why we keep project memory small
Our initial planning-heavy workflow generated too many documents.
Some were useful for a single session but had little long-term value.
Others became outdated.
We therefore became more selective about what should persist.
A durable architectural decision belongs in project memory.
A temporary debugging hypothesis generally does not.
A critical integration constraint should survive a session handoff.
The full transcript of how the agent discovered that constraint usually does not need to.
Session handoffs
Handoffs have become particularly valuable.
Instead of asking a new session to consume the entire previous conversation, we provide a concise summary of completed work, important decisions, unresolved issues, and next actions.
This allows development to continue with less context.
It also makes the work easier for another engineer—or another agent—to understand.
Together with focused skills and smaller context windows, this approach contributed to our measured reduction in Claude token usage.
The broader lesson is that persistent memory should preserve engineering knowledge, not conversation history.
6. Layer Four: Shared Organizational Knowledge
Project memory solves part of the problem.
But many important engineering decisions extend beyond individual repositories.
Consider business terminology.
A term such as active customer may affect billing, analytics, reporting, and product functionality.
If each project independently defines it, inconsistencies become inevitable.
The same applies to architectural patterns, shared services, and external integration contracts.
We therefore introduced shared organizational knowledge.
What belongs in organizational knowledge?
We focus on information that is relevant across projects or teams.
This includes business definitions, cross-cutting architectural decisions, service relationships, and verified external API contracts.
The objective is to provide agents with authoritative information rather than forcing them to reconstruct organizational knowledge from scattered code and documentation.
Verified contracts matter
We found that agents could generate convincing but incorrect assumptions about external APIs.
A field name might look plausible.
A response structure might resemble a documented example.
But if the actual API behaved differently, the implementation would fail.
Where necessary, we verify integration contracts against actual systems.
This helps distinguish established facts from assumptions.
Knowledge must have ownership
Making organizational knowledge available to agents introduces a responsibility to maintain it.
An outdated document can be worse than missing information if an agent treats it as authoritative.
We therefore distinguish between project-specific knowledge, shared organizational knowledge, and canonical governance standards.
These categories serve different purposes.
Project memory describes the local implementation context.
Organizational knowledge describes shared facts and decisions.
Governance defines authoritative requirements that the organization expects its systems and processes to follow.
This separation becomes especially important in the governance layer.
7. Layer Five: Local Guardrails and Engineering Validation
The fifth layer is where guidance becomes enforceable engineering practice.
This is one of the most important parts of our AEP.
We learned that an agent’s claim to have followed a standard is not sufficient evidence that it has done so.
Where possible, we validate the actual changes.
Local security checks
We use tools such as Gitleaks to detect potential secrets before they are committed.
This gives engineers and agents immediate feedback.
It also reduces the likelihood that sensitive information reaches the repository.
Code-quality validation
The AEP can invoke relevant code-quality checks, including linting, formatting, type checking, and other project-specific validations.
We also examine changes for issues such as dead code, duplication, unnecessary complexity, and incomplete implementations.
Not every code-quality concern can be evaluated deterministically.
But many common problems can be detected through existing tools.
Package and dependency validation
Agents can introduce dependencies unnecessarily.
Sometimes they add a package to solve a problem already addressed by the project.
Other times, they select a dependency that does not align with organizational requirements.
Package validation helps detect prohibited or unexpected dependencies.
Workspace standards
We also protect areas of a repository that should not be modified casually.
Examples may include generated files, vendored dependencies, or applied database migrations.
These restrictions can be represented through local validation rules rather than relying solely on instructions.
Governance-related validation
Where canonical governance requirements can be expressed as machine-checkable rules, the AEP can apply relevant local checks.
Other requirements remain part of planning and design review.
This distinction is important.
Not every governance principle can—or should—be reduced to a linter rule.
The importance of installation health
Our experience with silently disabled hooks taught us that local enforcement must be verifiable.
The AEP therefore includes an installation health check.
It helps identify missing or incorrectly configured components.
Without this capability, users may believe they are operating within the required controls when those controls are not actually active.
But even with reliable local validation, we do not treat the AEP as the final enforcement boundary.
That responsibility is shared with our centralized engineering platform.
8. Local AEP Validation and Centralized GitHub Enforcement
This separation deserves particular attention because it is fundamental to our implementation.
At Madgicx, we have established centralized CI/CD, security checks, linting, and other required controls at the GitHub level.
The AEP operates locally.
It provides guidance, validation, and engineering gates during the development process.
GitHub independently enforces the requirements that must be centrally controlled.
Why maintain both layers?
Local validation provides fast feedback.
An engineer or agent can detect a problem immediately, correct it, and continue working.
This reduces unnecessary pull-request failures and review cycles.
Centralized validation provides independent enforcement.
It does not depend on the local agent having followed every instruction or executed every check.
For example, a developer may accidentally disable a local hook.
The local control may fail to run.
But a required centralized check can still detect the relevant violation before the change is merged.
The division of responsibility
| Local AEP | Centralized GitHub |
|---|---|
| Agent guidance and engineering workflow | Repository-level enforcement |
| Local secret detection | Centralized security scanning |
| Local code-quality checks | Required linting and quality checks |
| Workspace validation | Repository and branch protection |
| Package and governance validation | Mandatory CI/CD checks |
| Immediate feedback during implementation | Independent verification before integration |
Some validations may intentionally exist in both layers.
That is not necessarily duplication to eliminate.
It is defense in depth.
The important distinction is that the local and centralized controls serve different purposes.
Local validation improves the development experience. Centralized enforcement protects the integrity of the engineering process.
This also helps us remain less dependent on a particular coding-agent platform.
We initially built much of our workflow around Claude Code.
Some of our engineers now also use Codex.
The mechanisms for invoking skills, roles, and local hooks may differ, but the engineering requirements and centralized controls remain applicable.
9. Layer Six: Packaging, Distribution, and Installation Health
An engineering playbook is not useful if it works only for the people maintaining it.
As adoption expanded, we encountered several practical distribution problems.
Our early installation process required users to complete multiple steps manually.
Some users installed the standards but missed the hooks.
Others had outdated versions.
Maintainers were sometimes testing a local development checkout rather than the packaged version distributed to users.
We needed to treat the AEP as a product.
One-command installation
We moved toward a packaged plugin with a simplified installation process.
The objective was to install and register the necessary local components consistently.
This became particularly important as users outside engineering began adopting the tooling.
Installation health checks
We introduced a health-check command that reports the state of the installation.
This helps users identify missing hooks, configuration issues, and other problems that could prevent the AEP from operating as intended.
Versioning and updates
A shared engineering playbook changes over time.
Standards evolve.
New validations are introduced.
Existing tools are corrected.
Users therefore need a reliable way to receive approved updates.
Private repository distribution and authentication introduce practical challenges, particularly when users have different account configurations.
We have addressed these through controlled installation and update procedures, while continuing to improve the distribution experience.
Testing the packaged product
We also learned that maintainers should test the same artifact that ordinary users install.
A development checkout is not an adequate substitute.
This led us to introduce a separate installation profile for testing releases.
The broader lesson is straightforward.
An internal engineering framework needs the same product discipline as any other software system: versioning, installation testing, diagnostics, and controlled releases.
10. Layer Seven: Governance and Continuous Learning
The final layer connects the AEP to two sources of organizational direction.
The first is canonical governance.
The second is feedback from actual engineering work.
I think of these as complementary flows.
Governance tells the AEP what the organization has decided.
Engineering feedback tells us where the AEP is not yet working effectively.
Both are necessary.
Governance: The top-down flow
We maintain canonical governance standards separately from the AEP.
This is an important architectural decision.
The AEP should not independently define organizational policy.
Security requirements, business standards, and architectural governance should have authoritative owners and sources.
The AEP consumes those standards and incorporates applicable requirements into agent guidance and engineering controls.
What is operational today?
Our canonical governance repository and its integration with the AEP are already implemented.
The AEP team currently runs synchronization tooling to retrieve canonical standards and update the playbook.
This process is operational.
What we are developing now is the automation around detecting governance changes and notifying the AEP team when updates are required.
The objective is to reduce manual coordination and make changes more visible.
The intended process is:
Today, the synchronization tooling is initiated manually.
Automated detection and notification are the next steps.
Why we prefer consumer-initiated synchronization
Earlier, we encountered problems with cross-repository credentials and governance distribution.
In one instance, a consuming copy of governance standards remained stale for approximately ten weeks.
This highlighted the risks of synchronization processes that fail silently.
We therefore moved toward a model where consumers retrieve canonical standards using appropriately scoped access.
The AEP team can then review and incorporate the relevant changes.
This reduces dependence on the governance repository having write access to every consuming system.
It also keeps responsibility for implementation with the team maintaining the consuming framework.
Continuous learning: The bottom-up flow
The feedback loop works in the opposite direction.
During engineering sessions, the AEP captures information about issues and failures.
These may include failed validations, recurring agent mistakes, incorrect assumptions, or gaps in existing standards.
We monitor this feedback and identify improvements that should be incorporated into the shared playbook.
This process has been operational since July.
When important fixes are identified, we update the AEP rather than waiting for the same problem to recur across multiple projects.
Automating feedback curation
We are currently developing the final stage of the feedback process.
The intended workflow is:
The human approval step is deliberate.
We do not want agents automatically rewriting the standards that govern them.
Instead, we want them to help identify improvements that can be reviewed and approved by the responsible people.
Preventing rule accumulation
There is another important challenge.
Not every failure requires a new organizational rule.
If we automatically convert every agent-generated recommendation into a standard, the AEP will eventually become difficult to maintain.
We therefore evaluate whether a proposed improvement addresses a recurring or significant problem, belongs at the organizational level, and has a clear owner.
We also consider whether the issue should be addressed through guidance, deterministic validation, or changes to existing tooling.
A continuous learning system should improve the quality of its rules, not simply increase their number.
11. How the Seven Layers Work Together
The individual layers are useful, but the real value comes from how they operate together.
Consider a typical feature-development request.
An engineer asks an agent to implement a new capability in an existing application.
Step 1: Establish the project context
The AEP identifies the project and retrieves the relevant instructions.
It examines the project configuration to determine which technology-specific standards apply.
It also makes the relevant project memory and organizational knowledge available.
The agent begins with a clearer understanding of the environment in which it will work.
Step 2: Research the requirement
The agent investigates the existing implementation, identifies affected components, and examines relevant contracts.
It checks whether similar functionality already exists.
Where information is missing or ambiguous, the agent should surface the uncertainty rather than invent an answer.
Step 3: Develop and challenge the plan
The agent proposes an implementation approach.
For work requiring additional scrutiny, GrillMe-with-Docs can challenge the proposal against documented business, governance, security, and architectural requirements.
An engineer can review the plan and resolve important questions before implementation.
Step 4: Implement the change
The agent develops the implementation using the applicable engineering standards.
It works within the established project structure and uses existing components where appropriate.
The implementation can proceed through smaller, verifiable increments.
Step 5: Execute local validations
The AEP invokes the applicable local checks.
These may include Gitleaks, linting, type checking, package validation, workspace restrictions, and governance-related controls.
Detected violations are corrected before the work progresses.
Step 6: Submit the change to GitHub
The resulting change enters the established repository workflow.
Centralized CI/CD, security checks, linting, and other required controls independently verify the implementation.
Human review and approval remain applicable according to the organization’s engineering policies.
Step 7: Capture lessons
If the development process reveals a significant issue, the feedback mechanism can record it.
The AEP team can evaluate whether the issue requires a change to shared standards, tooling, or validation.
Approved improvements are incorporated into subsequent AEP releases.
This is the operating model we have been working toward.
It allows agents to perform increasingly substantial engineering tasks without treating their output as inherently trustworthy.
The agent performs the work. The engineering system establishes the boundaries and verifies the result.
12. What I Would Measure in an Agentic Engineering System
An AEP should not be judged by the number of prompts, skills, or rules it contains.
It should be evaluated by whether it improves engineering outcomes.
I would separate the measurements into two categories.
Engineering effectiveness
These metrics help determine whether agentic engineering is improving software delivery.
| Metric | What we want to understand |
|---|---|
| Development lead time | Are changes moving from implementation to production faster? |
| Deployment frequency | Can teams deliver smaller changes more frequently? |
| Change failure rate | Are faster releases introducing more production problems? |
| Escaped defects | Are quality controls preventing defects from reaching users? |
| Token usage per comparable task | Are agent workflows becoming more context-efficient? |
| Review effort | Are engineers spending less time correcting avoidable problems? |
| Code complexity | Are implementations remaining maintainable as development accelerates? |
Playbook effectiveness
These metrics help determine whether the AEP itself is working.
| Metric | What we want to understand |
|---|---|
| Installation health | Are the required local controls installed and functioning? |
| AEP version adoption | Are teams using current, approved versions? |
| Local validation failures | What problems are being detected during development? |
| Centralized CI/CD failures | Which problems are escaping local validation? |
| Recurring agent mistakes | Where do our standards or tools need improvement? |
| Governance synchronization status | Is the AEP aligned with current canonical requirements? |
| Feedback-to-release cycle time | How quickly do approved lessons reach users? |
| Standards maintenance | Are rules being reviewed, consolidated, and retired when appropriate? |
There is an important distinction between activity and effectiveness.
For example, detecting more violations does not automatically mean that the engineering system has improved.
It may mean that enforcement is working better.
But it may also mean that agents are repeatedly making the same mistakes.
Similarly, increasing deployment frequency is not necessarily a success if the change failure rate also increases.
We need to evaluate these metrics together.
What we have measured so far
As discussed in the first article, we measured an approximately 50% reduction in Claude token usage after introducing smaller contexts, specialized skills, and structured session handoffs.
Our development delivery cadence has also changed from release cycles of approximately two sprints to around four development releases per week.
These are meaningful outcomes, although the delivery improvement reflects a broader organizational transformation rather than the AEP alone.
We have also observed improvements in code maintainability and the ability of non-engineering teams to deliver software changes within established controls.
Our next challenge is to measure these outcomes more systematically.
I believe this is an important discipline for engineering leaders adopting AI.
We should measure the value of the software delivered, the reliability of the engineering process, and the resources required—not simply how much code an agent generates.
13. How I Would Implement an AEP in Another Engineering Organization
Having spent seven months building and refining our playbook, I would approach a new implementation incrementally.
I would not begin by attempting to reproduce all seven layers.
That would risk recreating the heavyweight process we initially struggled with.
Instead, I would establish a minimal foundation and expand it through actual engineering experience.
Phase 1: Establish the engineering foundation
Start with one development team and an existing production codebase.
Define a simple research, plan, implement, and validate workflow.
Introduce a small set of universal engineering standards and the relevant technology-specific guidance.
Establish how important decisions and session handoffs will be preserved.
At this stage, the objective is consistency.
You want agents to understand the project, follow established conventions, and produce changes that engineers can review.
Phase 2: Introduce deterministic validation
Identify the requirements that can be enforced mechanically.
These may include secret detection, formatting, linting, type checking, dependency restrictions, and protected workspace areas.
Implement the relevant local validations.
Then verify that the organization’s centralized CI/CD and security controls independently enforce critical requirements.
Test the controls by deliberately introducing violations.
Do not assume that a configured validation is functioning correctly.
Phase 3: Connect organizational knowledge and governance
Identify the business definitions, architectural decisions, and integration contracts that engineers repeatedly need.
Make this information accessible to the agents.
Establish authoritative sources and ownership.
Where canonical governance standards exist, introduce a controlled mechanism for consuming them.
For important design changes, use a structured review mechanism such as GrillMe-with-Docs to challenge the proposed solution against the relevant requirements.
Phase 4: Package and distribute the playbook
Once the workflow has demonstrated value, make it easy for other engineers to adopt.
Provide a consistent installation process.
Include health checks.
Version the playbook and test the packaged releases.
Avoid relying on every user to understand and manually configure the underlying tooling.
Phase 5: Establish continuous learning
Capture important failures and recurring issues.
Review them regularly.
Determine whether improvements belong in project-specific guidance, organizational standards, or deterministic tooling.
Introduce human approval for shared changes.
As the process matures, automate feedback collection, curation, and release preparation where doing so provides value.
Phase 6: Expand adoption carefully
Once the engineering workflow is reliable, consider extending appropriate capabilities to product, marketing, and operations.
This requires more than distributing an AI coding tool.
Teams need controlled access to repositories, data, development environments, and deployment mechanisms.
Their permissions must remain appropriate to their responsibilities.
The objective is to enable broader participation in software development without weakening organizational controls.
Phase 7: Measure, simplify, and improve
Finally, establish a regular review of the playbook itself.
Are the standards still relevant?
Are some rules duplicated?
Are agents loading unnecessary context?
Are validations creating avoidable friction?
Are important failures being captured and addressed?
Is the playbook actually improving engineering outcomes?
These questions should be revisited continuously.
The playbook should evolve as the organization learns.
But evolution should not automatically mean adding more complexity.
The best AEP is not the one with the most rules. It is the one that provides the necessary engineering discipline with the least unnecessary friction.
14. What We Are Building Next
Our AEP is operational, but it is not complete.
There are several areas where we are continuing to invest.
Automated governance synchronization
Our canonical governance integration is already in place.
The AEP team currently uses tooling to retrieve and incorporate approved standards.
We are working on automating change detection and alerting so that governance updates become visible without requiring the team to initiate the process manually.
Automated feedback curation
Our engineering feedback loop is also operational.
We capture and review feedback, incorporating important improvements into the AEP.
The next stage is to automate the preparation and curation of proposed improvements.
Human verification will remain part of the process before approved changes are distributed through plugin releases.
Broader agent-platform compatibility
We initially developed much of our approach around Claude Code.
Some engineers now also use Codex.
We want the underlying engineering practices to remain applicable regardless of the coding-agent platform.
The implementation mechanisms may differ, but standards, governance, validation, and engineering accountability should not depend on a particular model or tool.
An open-source Agentic Engineering Playbook
We are also preparing an organization-agnostic version of the AEP.
The intention is to separate the reusable engineering framework from Madgicx-specific knowledge, governance standards, and internal tooling.
The reusable foundation would include concepts such as the role-based engineering workflow, focused standards, local validations, installation health checks, and continuous learning mechanisms.
Organizations could then introduce their own business knowledge, technology standards, and governance requirements.
I do not expect every organization to adopt the same implementation.
But I believe the underlying principles can help engineering teams avoid many of the problems we encountered.
Final Thoughts: Engineering Discipline for an Agentic World
When we first started working with AI coding agents, we concentrated on what the models could do.
How well could they understand our code?
How quickly could they implement a feature?
Could they investigate bugs, generate tests, and propose architectures?
Those questions remain important.
But our experience has led me to believe that they are only part of the challenge.
As coding agents become more capable, engineering organizations need to think more carefully about the environment in which those agents operate.
Agents need access to reliable knowledge.
They need to understand the standards that apply to their work.
They need clear boundaries between planning, implementation, and validation.
They need mechanisms that challenge questionable assumptions.
And their output needs to pass independent engineering controls.
At the same time, the system must remain lightweight enough to preserve the productivity benefits of agentic development.
This balance has been the central challenge of building our AEP.
We have made mistakes along the way.
Our initial planning process became too heavy.
Our standards sometimes activated in the wrong projects.
Our local guardrails were not always installed correctly.
Our governance distribution encountered synchronization problems.
And our learning loop required more human judgment than we initially anticipated.
But each of these experiences helped us refine the architecture.
Looking back, I believe three principles have become particularly important.
First, separate intelligence from enforcement.
Use agents for reasoning, planning, and implementation. Use deterministic tooling and centralized controls to enforce requirements that can be verified mechanically.
Second, make organizational knowledge and governance part of the engineering workflow.
An agent should not have to rediscover established business definitions, architectural decisions, or security requirements every time it begins a new task.
Third, build a system that learns without losing accountability.
Capture feedback, identify recurring problems, and improve the playbook. But keep human judgment involved when changes affect shared engineering standards and governance.
Ultimately, I do not see the Agentic Engineering Playbook as a replacement for experienced engineers or established software engineering practices.
I see it as a way to make those practices available to a new generation of engineering participants: AI agents.
The engineer remains responsible for understanding the problem, making architectural decisions, defining acceptable outcomes, and ensuring that the resulting software is fit for purpose.
The agent can increasingly perform the work required to reach those outcomes.
And the engineering system provides the knowledge, boundaries, and verification mechanisms that make this collaboration reliable.
The future of agentic engineering will not be determined only by how intelligent our agents become. It will also depend on how well we engineer the systems in which they operate.
That is what we are trying to achieve with our Agentic Engineering Playbook.
And as the technology continues to evolve, I expect the playbook—and our understanding of agentic engineering—to evolve with it.