Python Data Engineer

Detailed Job Description:

JD:

• 8+ years of experience in python scripting.

• Experience with python packages such as NumPy, pandas, polars, BeautifulSoup, Selenium, Requests, etc.

• Experience in building native python ETL pipelines for scraping/processing.

• Good to have experience in processing big data.

• Proficient in SQL programming, Postgres SQL. Spark knowledge is a good to have.

• Knowledge on DevOps CI/CD like Azure Pipelines, Jenkins, Git.

• Experience working with AWS services (S3, lambda, etc.) and Azure Databricks.

• Have experience in delivering projects with Agile and Scrum methodology.

• Able to coordinate with teams across multiple locations and time zones.

• Strong interpersonal and communication skills with an ability to lead a team and keep them motivated.

Key Responsibilities:

• Lead the design, development, and optimization of scalable data pipelines, data processing frameworks, and enterprise-grade Python applications.

• Provide technical leadership in Python development, including best practices for automation, data processing, and integration workflows.

• Work extensively with Python and data-centric libraries such as NumPy, Pandas, BeautifulSoup, Selenium, pdfplumber, and Requests.

• Architect and optimize SQL queries and database solutions, with strong proficiency in PostgreSQL and other relational data stores.

• Integrate DevOps practices using CI/CD pipelines, Jenkins, Git, and ensure smooth deployment of data solutions.

• Develop and manage cloud-based data workflows on platforms such as AWS (S3) and Azure Databricks.

• Drive agile delivery by collaborating with cross-functional teams under Agile/Scrum methodologies.