Data Engineering Specialist

About The Role

• The Data Engineer supports the capture, management, storage, and utilization of structured and unstructured data from internal and external sources, translating business needs into robust data solutions that underpin BT Group’s data strategies and strategic decision making.
• Delivers the coding, testing, and deployment of data processes for the ingestion, transformation, and curation of data, ensuring compliance with data security and privacy standards.
• Contributes to the development and refinement of data architecture and design patterns, supporting the processing and storage of high volume, structured, and unstructured data sets.
• Contributes to the delivery of key engineering projects, helping shape technical approaches and driving new solutions to complex or open-ended problems.
• Delivers business-relevant and actionable data outputs, enabling the creation of reliable data assets and strengthening data engineering capabilities.
• Applies engineering best practices and Agile delivery approaches to ensure consistent, high-quality, and iterative delivery of data engineering solutions.
• Executes initiatives to develop and enhance data and analytics infrastructure, supporting scalable and reusable data engineering capabilities.
• Supports the identification and implementation of improvements to data engineering processes, using data analysis to assess issues, optimize workflows, and drive efficiency.
• Conducts quality assurance and testing of data engineering routines, models, and frameworks, ensuring accuracy, robustness, and suitability for business use.

What You’ll Be Doing

• Act as a Data Engineer Lead within their Squad, including coaching and mentoring other engineers within the Alliance, ensuring adherence to Engineering standards and best practices, facilitating workshops, and running training sessions where needed.
• Ensure both self and others take responsibility for achieving brilliant customer and business outcomes through high-quality execution, collaboration, and a focus on continuous improvement.
• Design and build data products and data pipelines that are cost-effective, performant, and resilient.
• Build both batch and NRT data pipelines.
• Ensure all data products delivered and maintained are of high quality, accurate, secure, and always adhere to data protection policies.
• Support data product documentation, including clear and comprehensive user guides and runbooks to aid control and knowledge transfer.
• Perform operational maintenance of all data products.

Requirements & Tech Stack

• Experience with CI/CD setups and infrastructure as code frameworks (ideally Terraform or OpenTofu).
• Experience with open-source big data frameworks (ideally Spark, Apache Beam, or Apache Flink).
• Experience with NRT data pipelines is highly desirable.
• High proficiency with SQL, Python, and one of Scala/Java.
• Strong experience of cloud environments (ideally GCP) in the following areas:
• SQL and NoSQL databases.
• Orchestration tools.
• Big data and ETL services.
• Serverless compute services.
• Experience with K8 and cloud networking is desirable but not required.
• Experience working on AI Applications such as Amazon Q, Kiro, or Gemini.

Qualifications

• Strong data management and data architecture experience.
• Strong understanding of data security and compliance processes.
• Strong communication skills and ability to articulate complex technical concepts to both technical and non-technical stakeholders at all levels, written and verbal.
• Excellent stakeholder management skills.
• Strong planning skills and the ability to prioritize demand in line with business and commercial requirements.
• Strong experience leading / coaching and mentoring.
• Experience working in Agile.