Staff AI Engineer - Conversational & Agentic AI

ServiceNow

USA, California, Santa Clara Full-Time $176,100 - $308,200 2026-06-30

Generative AI

LLM

Large Language Models

Agentic AI

Conversational AI

Prompt Engineering

Evaluation Frameworks

About ServiceNow

ServiceNow is a leading AI control tower for business reinvention, empowering 85% of the Fortune 500® to work smarter, faster, and better through its AI platform. Founded on the principle of freeing people from busywork, ServiceNow integrates AI, data, and workflows to foster an AI-native culture where technology and talent collaborate seamlessly.

About the Position

Introduction

ServiceNow is seeking a Staff AI Engineer to join their Agent Orchestration team, focusing on building a conversational AI experience that transforms enterprise intent into completed work. This role is crucial for advancing how enterprise AI reasons, remembers, and executes, specifically by owning significant parts of the agent harness and its associated infrastructure.

Responsibilities

Harness Engineering: Design and build the agent execution harness, the orchestration layer that routes inputs, manages context, invokes tools, handles retries, and surfaces execution state across multi-step agentic workflows.
Reliability at Scale: Ensure the runtime's fault tolerance, latency, and throughput, designing for enterprise workflows that demand high reliability.
Observability: Instrument the harness with tracing, cost attribution, and latency visibility to understand agent behavior in production and proactively address failures.
Prompt Infrastructure: Develop prompt management systems including versioning, templating, and systematic evaluation to maintain stable agent behavior across model updates and configuration changes.
Eval Engineering: Design and manage evaluation frameworks (unit, integration, and production monitors) to measure agent quality, identify regressions, and support data-driven decisions.
LLM Integration: Integrate with and abstract over frontier Large Language Models, managing model routing, fallback strategies, and balancing cost and latency in production.
Technical Leadership: Establish technical standards through architecture decisions, code reviews, and mentorship, with a focus on agentic design patterns and production AI discipline.
System Boundary Design: Define the scope of agent logic, distinguishing between tool calls, sub-agents, hardcoded paths, or human escalations, and standardize these design patterns across the team.

Requirements

7+ years of experience building production software systems with a strong track record in reliability, performance, and scalability.
Hands-on experience shipping generative AI products, involving ownership of AI-powered features used by production users.
Deep understanding of Large Language Models: their failure modes, context constraints, and the impact of prompt design on model behavior at scale.
Practical prompt engineering experience: systematically designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
Proven track record in eval engineering, demonstrated by designing, shipping, and utilizing evaluation suites for production AI systems.
Cost and efficiency awareness at the system level, including experience with model routing, inference cost, and latency tradeoffs in production.
Strong software engineering fundamentals: distributed systems, API design, and testing discipline.
Comfort operating in fast-moving, ambiguous, startup-like AI product environments.

Nice to Have

Experience with multi-agent coordination patterns (A2A, MCP).
Familiarity with agent frameworks (LangChain, LlamaIndex, or similar).
Prior experience shipping AI systems in enterprise software.
Experience with AI observability tooling (tracing, cost tracking, LLM-specific monitoring).
Familiarity with cloud-native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting.