AI Engineer

Key Responsibilities

• Agentic Systems

• Design and build LLM agents that plan, call tools, and complete multi-step business tasks across voice, chat, and back-office channels.

• Implement task and conversation state, memory and context management, retrieval (RAG), and model routing between fast and more capable models.

• Define tool schemas and permissions, validate every tool call, and add human approval points for high-impact actions.

• Build durable workflows with checkpoints, retries, idempotency keys, timeouts, and compensation for failed actions.

• Use multi-agent designs, such as a router with specialist agents, only where they measurably beat a single agent.

• Integrate CRMs, ticketing systems, and databases for actions such as complaint registration, classification, routing, status updates, and notifications, with warm human handoff.

• Voice Agents

• Build streaming pipelines that connect telephony (SIP, Twilio, Exotel) and WebRTC audio to ASR, LLMs, and TTS over WebSockets.

• Implement voice activity detection, turn detection, barge-in, and cancellation so the agent stops talking the moment a caller speaks.

• Set and hold latency budgets per hop (ASR, LLM, TTS, network), with a target of under 1 second from end of caller speech to first agent audio.

• Adapt agent behavior for speech: short turns, confirmations before actions, and recovery from mishearing.

• Support multilingual and code-switched calls, including language detection and matching TTS voices.

• Evaluation

• Build and maintain golden test sets that define the user goal, persona, expected tool calls, and pass criteria for each scenario.

• Run trajectory evals that check each step's tool choice, arguments, and order, not just the final answer.

• Run simulated-caller evals for voice: an LLM plays the caller, TTS voices it with accents, noise, and interruptions, and the agent is tested end to end over real audio.

• Score task success, tool-call correctness, policy compliance, hallucination, word error rate, and latency, using rule checks and LLM judges calibrated against human ratings.

• Gate releases in CI on eval results, block regressions, and turn every confirmed production failure into a new test case.

• Backend and Operations

• Write async Python services (FastAPI, asyncio) with structured logging, tracing, and metrics for every agent run and conversation.

• Load-test concurrent sessions, find bottlenecks, and keep p95 latency and per-task model cost within budget.

• Protect sensitive data: authentication and authorization, audit trails for agent actions, PII redaction, and tenant isolation.

• Ship with Docker and CI using staged releases and rollback, and troubleshoot live agents in production.

• Work with product and delivery teams to turn client workflows into working agent integrations.

Requirements & Tech Stack

• Strong Python and backend skills: APIs, asyncio, WebSockets, structured error handling, and automated tests.
• Hands-on experience shipping LLM agents beyond a prompt demo: tool calling, state management, RAG, and failure handling in a real workflow.
• Sound judgment on agent design: when a fixed workflow beats an autonomous agent, single versus multi-agent, and how to keep agents safe with permissions and guardrails.
• Working knowledge of a real-time voice stack: streaming ASR and TTS, VAD, turn-taking, and the trade-offs between latency and accuracy.
• Experience designing evaluations for LLM agents: test sets, trajectory and outcome metrics, LLM-as-judge, and regression tracking — and knowing when a metric is misleading.
• Ability to debug across services using logs, traces, and recorded sessions to find where an agent failed or slowed down.
• Experience with SQL or NoSQL databases, Git, Docker, and CI pipelines.

Qualifications

• Bachelor’s degree in a technical field or equivalent practical experience, with clear examples of delivered work.

Preferred Experience

• Agent frameworks and orchestration tools such as LangGraph, OpenAI Agents SDK, Temporal, or MCP tool servers.
• Production voice agents on telephony (SIP, Twilio, Exotel) or WebRTC, with frameworks such as LiveKit Agents or Pipecat.
• Speech providers such as Deepgram, Whisper, Google STT, ElevenLabs, Azure TTS, or Cartesia, and measuring their WER and latency on your own data.
• Indian-language and code-switched speech (Hindi, Telugu, Tamil and English).
• Queues, caching, and event-driven systems (Redis, Kafka, RabbitMQ), and load-testing tools such as Locust or k6.
• Access controls, PII redaction, and on-premises or restricted-network deployments.
• Other modalities, such as vision or document understanding (images, screenshots, scanned forms), and combining them with voice or text in one agent.