Data Engineer

About the Role

We are seeking a Data Engineer responsible for designing, developing, testing, and implementing new features within an existing data engineering framework. This role requires translating business requirements into technical solutions and developing standalone PySpark scripts. You will work closely with the team to deliver reliable, maintainable, and effective data solutions.

Key Responsibilities

• Design, develop, and implement new features using Python and PySpark.
• Develop efficient PySpark scripts for data transformations based on defined business logic.
• Apply Object-Oriented Programming concepts to solve complex problems and develop robust, reusable solutions.
• Collaborate with team members to understand requirements and develop solutions aligned with business objectives.
• Translate business requirements into technical solutions and data transformation processes.
• Test, debug, and troubleshoot code to ensure accurate and reliable results.
• Identify and resolve complex software and data-related issues.
• Develop standalone scripts and applications to support data processing requirements.
• Follow coding best practices and contribute to framework enhancement.
• Document programming tasks, processes, and procedures.
• Communicate effectively with team members regarding requirements, challenges, and solutions.
• Work independently with minimal supervision while contributing to the team.

Requirements & Tech Stack

• Strong hands-on experience with Python.
• Strong hands-on experience with PySpark and data transformations.
• Solid understanding of Object-Oriented Programming concepts.
• Experience developing standalone scripts based on defined business logic.
• Strong problem-solving and troubleshooting skills.
• Ability to understand complex requirements and translate them into effective technical solutions.
• Experience working with data engineering frameworks and large-scale data processing.
• Excellent written and verbal communication skills.
• Ability to work independently with minimal supervision.

Preferred Skills

• AWS / Cloud technologies
• Large-scale data processing
• Data transformation and ETL
• Software development best practices
• Framework development and enhancement
• Automated testing and debugging
• Data engineering tools and technologies