Python / PySpark Developer, Data Engineer

CGI

Chennai, Tamil Nadu, India21m agovia LinkedIn

Posted 16 Nov 2026, 22:42 IST

This job at a glance

Who can apply
Freshers
Must know
PythonSQLSparkBusiness analysisUnit testingGit
Nice to have
Flask
Work
On-site in Chennai
Pay
Not stated by the employer.
Listed
16 Nov 2026 · still open

Sign in free to see which of these skills your resume already shows.

About the Role

At CGI, we are a team of builders. We call our employees members because all who join CGI are building their own company. Founded in 1976, CGI is a leading IT and business process services firm committed to helping clients succeed. We have the global resources, expertise, stability, and dedicated professionals needed to achieve results for our clients and for our members.

Key Responsibilities

  • Design and develop robust ingestion, transformation, and enrichment pipelines with Python, PySpark, and SQL.
  • Write and optimize complex SQL queries, analytical UDFs, and window functions for data aggregation and reporting.
  • Collaborate with data architects, data scientists, and business analysts to translate functional requirements into technical specifications.
  • Perform unit testing, integration testing, and code reviews.
  • Maintain CI/CD pipelines (Git, Jenkins, Docker) for automated build, test, and deployment of jobs.
  • Monitor production workloads and troubleshoot performance bottlenecks, memory issues, and job failures.
  • Document data lineage, pipeline design, and operational run books in Confluence/SharePoint.
  • Stay up-to-date with the latest technologies and trends, providing input, expertise, and recommendations.
  • Contribute towards innovation (e.g., AI/ML), suggesting new technical practices for efficiency improvement.
  • Participate in Agile ceremonies (sprint planning, daily standups, retrospectives) and assist in grooming backlogs.
  • Mentor junior engineers and champion best practices in Python coding, Spark optimization, and data engineering patterns.
  • Evaluate emerging technologies and deliver proof of concepts.

Requirements & Tech Stack

  • Python
  • PySpark
  • SQL
  • ETL
  • NumPy
  • Pandas
  • Python Frameworks
  • Restful APIs
  • MS SQL or Oracle
  • PySpark DataFrames
  • Spark SQL
  • Structured Streaming
  • CI/CD pipelines (Git, Jenkins, Docker)
  • Confluence/SharePoint
  • AI/ML
  • Agile, Scrum, DevOps
  • Linux/Unix environment (basic commands, shell scripting)
  • Build tools like Maven
  • DevOps tools like Bitbucket
  • Front end technologies, preferably Flask

Qualifications

  • Good experience in technical analysis of n-tier applications with multiple integrations using object-oriented, APIs & Microservices approaches.
  • Strong knowledge of design patterns and development principles.
  • Inclination and prior experience working across SQL, Python, and ETL.
  • Strong hands-on experience with Python (NumPy, pandas, Python Frameworks, Restful APIs, MS SQL or Oracle).
  • Experience with PySpark DataFrames, Spark SQL, Structured Streaming, and performance tuning (partitioning, caching, broadcast joins).
  • Advanced SQL – complex queries, stored procedures, query optimization.
  • Good knowledge and experience using Python packages such as Pandas, NumPy, etc., for data cleaning, wrangling, analysis, and visualization.
  • Good experience in development and maintenance of code/scripts in both functional and technical specifications of all application components, bug fixing, and production support.
  • Good knowledge of Linux/Unix environment (basic commands, shell scripting, etc.), testing phases, documentation, and new frameworks.
  • Some experience working with build tools like Maven & DevOps tools like Bitbucket, Jenkins.
  • Knowledge of Agile, Scrum, DevOps.
  • Development experience in a Data Engineering environment.
  • Ability and willingness to learn and work on diverse technologies (languages, frameworks, and tools).
  • Self-motivated, good interpersonal skills, and an inclination to constantly upgrade new technologies and frameworks.
  • Good communication and coordination skills.
  • Good to have knowledge of front-end technologies, preferably Flask.
  • Ability to synthesize / simplify.
  • Ability to collaborate / Teamwork.
  • Attention to detail / rigor.
  • Ability to deliver / Results driven.

FirstHire never charges to apply, and real employers don't ask for money for tests, interviews or offer letters. How we list jobs · Report this job