Associate Data Engineer

About the Role

As an Associate Data Engineer at BeyondTrust, you will contribute to building and enhancing our data lake, which consumes billions of events each day. You will leverage state-of-the-art data engineering tools to help develop cost-effective cloud solutions that enable next-level cybersecurity research. You will maintain and optimize existing infrastructure, build new data assets, and collaborate with experienced engineers to deliver reliable data solutions that drive business value.

Key Responsibilities

• Contribute to data modelling efforts that support identity security use cases and analytical requirements.
• Develop and maintain data pipelines that bring together complex information from disparate systems to support security analysis in the datalake.
• Optimize data workloads at a software level by improving processing efficiency.
• Work with senior engineers to identify areas for process improvement and help implement solutions.
• Use monitoring and observability best practices to ensure optimal pipeline performance.
• Assist with ML Operations to ensure optimal model efficiency.
• Leverage CI/CD best practices to effectively develop and release source code.
• Participate in code reviews and collaborate with team members to improve engineering practices.

Requirements & Tech Stack

• Programming and technology knowledge in cloud data processing.
• Spark experience is needed.
• Existing Databricks knowledge is a plus.
• Interest in cybersecurity; interest in identity security is preferred.
• Understanding of software engineering principles, data structures, and distributed data processing concepts.
• Experience working with distributed systems and data processing on object stores.
• Ability to work autonomously and collaboratively.

Technologies You Should Know

• Required: Data warehousing for analytics use cases.
• Required: Proficient in Python (or another object-orientated language).
• Required: Distributed processing framework experience.
• Highly Preferred: Spark.
• Highly Preferred: Realtime processing experience.
• Ideal: Some experience with graph data stores.

Qualifications

• Ability to learn fast and strong desire to develop.