AWS DevOps Agent + Custom MCP: Autonomous EKS Node Diagnostics Without Shell Access

The AWS DevOps Agent stalls at the cluster boundary. Node-level data — iptables rules, containerd logs, CNI configuration, kernel messages — lives outside its native integrations, and when that is where the failure is, the agent cannot help. A custom MCP server changes that architecture: it extends the agent’s reach to the operating system without requiring direct shell access to the node.

What Happened

AWS published a reference implementation showing how to attach a custom MCP server to the DevOps Agent for EKS node troubleshooting. The pattern addresses a specific gap: node-level diagnostics that require OS-layer data the agent’s default tools cannot reach.

The reference MCP server exposes roughly 19 tools organized around log collection, pre-indexed error retrieval with severity classification, cross-file search and correlation, structured network diagnostics, and live packet capture. Investigation runs via AWS Systems Manager Automation — the agent never touches the node directly.

The workflow chains sequentially: collect → status → errors → search → correlate → read → summarize. Each step’s output feeds the next. The agent runs parallel analysis tasks — iptables inspection, error pattern matching, healthy-node comparison — and synthesizes findings into a structured root-cause report without human intervention between steps.

Why It Matters

The architectural shift here is meaningful. EKS node failures — IP allocation errors, DNS resolution failures, storage mount timeouts, node registration failures — have historically required an SRE to SSH into the node and manually correlate kernel logs with CNI state and iptables output. That process is slow, requires elevated access, and doesn’t scale across large node pools.

The MCP pattern moves that diagnostic work to an agent operating through SSM, with pre-indexed error patterns and structured correlation logic. The investigation chain is reproducible and auditable — the same sequence runs every time, which matters for regulated environments where access logs and diagnostic trails have compliance weight.

For teams that have pushed agentic patterns into their platform tooling, this reference implementation is notable because it shows AWS’s own DevOps Agent being extended via MCP rather than through native integrations. That signals a future where the agent’s capabilities are primarily determined by the MCP servers attached to it — a composable extension model rather than a fixed-function tool.

The observability angle connects directly to alert fatigue at scale. Autonomous first-response investigation reduces the number of tickets that require an on-call engineer, which changes the economics of incident response. The alert noise problem documented across enterprise environments isn’t solved by faster human diagnosis — it’s solved by moving diagnosis off the human path entirely for known failure classes.

What This Means for Platform Teams

The reference implementation is a starting point, not a production system. Teams adopting this pattern will need to define their own error-pattern libraries, tune the SSM execution boundaries for their security posture, and validate that the correlation logic handles their specific CNI and storage configurations.

The MCP extension model also means the agent’s diagnostic quality is only as good as the tools attached to it. An MCP server with shallow error indexing produces shallow root-cause reports. The investment is in the tooling quality, not just the agent itself.

Source: AWS DevOps Blog

Content created with AI assistance and reviewed for accuracy.