Data Engineer
Scoutit
Pune City, Maharashtra, IndiaFresher12h agovia LinkedIn
Key Responsibilities
- Develop and maintain ETL/ELT pipelines for collecting, transforming, and processing data.
- Write efficient SQL queries for data extraction, transformation, validation, and analysis.
- Develop data processing scripts and utilities using Python.
- Work with relational and NoSQL databases for data storage and processing.
- Perform data validation, cleansing, and troubleshooting to ensure data quality.
- Support integration of data from multiple sources into centralized data platforms.
- Work with cloud platforms such as AWS, GCP, or Azure.
- Monitor data pipelines and troubleshoot failures or data-related issues.
- Collaborate with software engineers, analysts, and other technical teams on data requirements.
- Follow engineering best practices for version control, testing, documentation, and deployment.
Requirements & Tech Stack
- Strong programming fundamentals in Python.
- Good understanding of SQL and relational databases.
- Understanding of ETL/ELT concepts and data pipelines.
- Familiarity with databases such as MySQL, PostgreSQL, Oracle, MongoDB, or similar.
- Basic understanding of at least one cloud platform: AWS, GCP, or Azure.
- Familiarity with Git/version control.
- Strong problem-solving and analytical skills.
Qualifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
Good to Have:
- Exposure to PySpark, Apache Spark, Pandas, or NumPy.
- Experience with cloud services such as GCP, AWS, or Azure.
- Exposure to Docker, CI/CD, or Linux.
- Experience working with APIs and data integration.
- Academic, internship, or personal projects involving data engineering, data pipelines, databases, or cloud technologies.