Job Summary
We are looking for an experienced Data Engineer with 5+ years of hands-on experience in designing, developing, and maintaining scalable data solutions. The ideal candidate should have strong expertise in Python, PySpark, SQL, and Cloud platforms such as AWS, Azure, or GCP.
Key Responsibilities
- Design, develop, and maintain scalable and reliable data engineering solutions.
- Develop and optimize data processing pipelines using Python and PySpark.
- Write complex and efficient SQL queries, scripts, and data transformation logic.
- Apply data engineering concepts including ETL/ELT, data processing, data integration, and data pipeline development.
- Work with cloud-based data platforms and services across AWS, Azure, or GCP.
- Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
- Perform data validation, troubleshooting, and issue resolution across data pipelines.
- Collaborate with data scientists, analysts, software engineers, and business stakeholders.
- Follow best practices for data quality, security, monitoring, and documentation.
- Stay updated with emerging technologies in Data Engineering, Generative AI, and Large Language Models (LLMs).
Required Skills
- 5+ years of experience in Data Engineering
- Strong hands-on experience with Python
- Strong hands-on experience with PySpark
- Strong knowledge of SQL scripting and query optimization
- Good understanding of Data Engineering concepts
- Hands-on experience with at least one major cloud platform:
- AWS
- Microsoft Azure
- Google Cloud Platform (GCP)
- Strong analytical and problem-solving skills
- Excellent communication skills – Mandatory
Nice to Have
- Knowledge of Generative AI
- Understanding of Large Language Models (LLMs)
- Experience integrating AI/ML capabilities into data engineering solutions
Education
Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
What We’re Looking For
A technically strong Data Engineer who can independently work on data pipelines, cloud-based data solutions, and large-scale data processing while effectively collaborating with cross-functional teams.