AI Agent Architect, Customer Experience

• Own Agent retrieval accuracy and relevance . Architect the knowledge systems that enable AI agents to surface the right answer on the first try. Measure and improve retrieval precision, contextual relevance, and hallucination rates.
• Drive automated resolution rates . Build the decision frameworks that allow agents to take confident actions. What APIs do agents need to access? When can they make account modifications? You're accountable for encoding business logic into auditable, predictable systems that resolve issues without human intervention.
• Manage AI safety and trust. Establish the guardrails that keep resolution rates high while failure rates stay low. You're responsible for what the agent doesn't do wrong: edge cases caught, prompt injection blocked, unintended behaviors prevented.
• Own the feedback loop. Monitor the observability layer that turns agent behavior into actionable insights. Instrument retrieval accuracy, action success rates, and failure patterns. Use this data to drive measurable week-over-week improvements in agent performance.
• Continuously improve agent quality. Develop and maintain the prompt architecture that governs how agents reason and respond. Build systematic approaches to versioning, A/B testing, and performance evaluation, measuring consistency, accuracy, and adaptability across scenarios.
• Drive integration strategy. Architect how agents connect to external systems—billing platforms, CRMs, internal tools, Airtable APIs. Define authentication patterns, error handling, and data transformation. Uptime, error rates, and data accuracy are your metrics.

• You understand how large language models work—not just how to use them, but how they reason, where they fail, and why. You're familiar with concepts like RAG architectures, prompt engineering patterns, chain-of-thought reasoning, and agent frameworks. You've built or significantly contributed to AI-powered systems in production.
• You think in terms of data flows, state management, error handling, and edge cases. You can design complex systems that are both powerful and reliable. You've likely worked in roles like solutions architecture, platform engineering, or technical program management.
• You can write scripts, work with APIs, query databases, and prototype solutions. You're not a full-time software engineer, but you're dangerous enough to build, test, and validate technical approaches independently.
• You instrument systems, analyze logs, and use data to diagnose issues and validate improvements. You build dashboards, define metrics, and can tie technical changes to business outcomes like resolution rates and customer satisfaction.
• You can explain complex AI system behavior to non-technical stakeholders, write clear technical documentation, and translate business requirements into system specifications. You're effective working across engineering, product, and operations teams.

Browse similar jobs