Software Engineer, Inference
CoreWeave
Sunnyvale, CA0–3 yrs2h agovia LinkedIn
About the Role
As a Junior Software Engineer (IC1/IC2) on the Inference team, you’ll own a single production service end-to-end, shipping features, maintaining dashboards/alarms, and improving operational excellence. You’ll collaborate with peers across runtime, scheduling, and networking to move key SLIs.
Key Responsibilities
- Own one model-serving service end-to-end: design, implementation, tests, deployment, and ops.
- Add and tune observability (dashboards, alerts, traces); automate runbook steps to reduce toil.
- Improve latency/throughput via micro-batching, concurrency tuning, and caching.
- Participate in on-call; lead post-incident action items for your service.
- Review peers’ PRs and contribute design feedback in team reviews.
Requirements & Tech Stack
- 0–3 years of software engineering experience (or equivalent) building cloud/distributed services in production.
- Proficiency in Python or Go; familiarity with gRPC/HTTP APIs and containerized deployments.
- Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines.
- Experience with metrics/alerting (Prometheus/Grafana/OpenTelemetry) and structured logging.
- Basic understanding of GPU inference concepts (KV caching, mixed precision, streaming tokens).
Nice to have
- Experience with inference frameworks (Triton, vLLM, TensorRT-LLM, Ray Serve, TorchServe).
- Hands-on performance profiling/benchmarking (e.g., wrk, vegeta, perf, Nsight).
- Contributions to reliability practices: SLI/SLOs or postmortem improvements.