Junior AI Engineer, Applied AI

About the Role

You will build, evaluate, and harden the agent itself: the orchestration, the context it sees, the tools it can call, and the evaluations that tell us whether a change made it better or just different.

Key Responsibilities

• Own end-to-end slices of the agent: prompt and context architecture, retrieval over policy and coverage data, tool and API calls into core claims systems, and handoff to a human when the agent should stop.
• Build and run the eval loop, golden sets, trajectory evaluation, LLM-as-judge, error analysis on real production call traces, and make it the thing that gates releases.
• Drive containment and resolution rates up without trading away accuracy, and instrument the system well enough that we can tell which of the two moved.
• Work directly on the voice path: turn-taking, interruption, latency budgets, and the telephony layer underneath (LiveKit, SIP).
• Sit in on carrier calls and listen to recordings.

Requirements & Tech Stack

• Building and deploying AI applications.
• Shipped an LLM application that real users depend on.
• Fluent in building blocks: LLMs, context engineering, RAG, agentic workflows.
• Real ML and deep learning depth.
• Ability to make an unpredictable system behave predictably enough to trust through disciplined evals, error analysis, and statistical measurement.
• Software engineering fundamentals: reasoning about cost, latency, reliability, and scalability as tradeoffs.
• Experience designing data stores and distributed systems.
• Writing tests.
• Taking security and privacy seriously.
• Advocating CI/CD process automation.
• Working with coding agents: managing context, defining specs, providing verifiers, and knowing when to intervene.
• Adapting workflow as tooling changes.
• Shaping the build: product sense and sufficient insurance context to contribute to product decisions.
• Continuous learning.

Nice to have

• Real-time or streaming voice systems (ASR, TTS, telephony, SBC/SIPREC).
• Experience in insurance, claims, or another regulated domain where errors have a cost.
• Experience building internal eval or observability tooling that engineers adopted.
• Experience in Machine learning principles, or experience in model experimentations, training, and deploying models in production.
• Experience in P&C Insurance or exposure to Guidewire products and services.
• Prompt or pipeline optimization frameworks (DSPy, GEPA) applied to a production system.