Software Engineer, Backend

About the Role

We are building an in-house team to maintain and contribute to our internal Apache Kafka distribution. This is not an operations role; you will work directly on the Kafka codebase itself: the Broker, KRaft controller, protocol layer, clients, and storage engine. Working alongside senior technical anchors, you will help maintain our internal Kafka distribution, backport critical fixes, contribute upstream to Apache Kafka, and build benchmarking and testing pipelines for a large-scale production fleet.

Key Responsibilities

• Build and maintain performance testing suites, benchmarks, and validation tools for local patches.
• Maintain our internal Kafka distribution: patch integration, bug fixes, and release testing.
• Contribute upstream to Apache Kafka through test-cases, doc-reviews, bug-reports, bug fixes, code reviews, and community design discussions (mid to long term).
• Diagnose and fix broker and controller issues related to replication, partition handling, KRaft consensus, and storage management.
• Collaborate with platform engineering teams to advise on broker behavior, configuration, and capacity decisions.
• Document internal patches, architectural changes, and upgrade procedures.

Requirements & Tech Stack

• Java or Scala with experience reading, modifying, or debugging JVM-based system code.
• Understanding of consensus protocols, replication models, log-structured storage, and distributed state management.
• Familiarity with JVM internals (garbage collection, memory layout) and operating system I/O (page cache, zero-copy, socket tuning).
• Demonstrable experience in systems or infrastructure software engineering, working with distributed middleware or database engines.

Qualifications

• Codebase & Systems Familiarity
• Distributed Systems Fundamentals
• Systems Programming
• Development Experience

Good to Have

• Prior contributions to Apache Kafka or comparable distributed systems (e.g., Pulsar, BookKeeper, ZooKeeper, etcd).
• Familiarity with KRaft consensus or ZooKeeper-less Kafka architecture.
• Experience with Kafka Streams or Kafka Connect internals.
• Experience using Kafka system tests, Trogdor, or duct tape test frameworks.

What This Role Is NOT

• Not an SRE/Operations Role: You will write and fix the software that platform teams operate.
• Not Application Development: You will work directly inside Kafka rather than building consumer applications.
• Not a Consulting Role: You will write code, review patches, and ship production-ready software.

What We Offer

• Dedicated focus and time for open-source upstream contributions.
• Opportunity to validate code against a massive production fleet and observe large-scale distributed behavior.
• Direct mentorship from senior systems engineers and technical anchors.
• Proven distribution model and strong organizational backing for core open-source development.