Lead Data Engineer
Date Posted: 06-Feb-2024 | Qualification: B.E/B.Tech, MCA, ME/MTech, Bsc, BCA
| Position: | Lead Data Engineer |
| Skills Required: | Python, Pyspark, SQL, Business problem solving skills |
| Employment Type: | Full Time |
| Location: | India |
| Job Mode: | Work from Home |
| Experience: | 7 - 13 (years) |
Job Description:
Job Title: Sr. Data Engineer / Data Engineer - Python / PySpark / SQL
Job Description:
We are seeking an experienced Sr. Data Engineer / Data Engineer with a strong background in Python, PySpark, and SQL to join our dynamic team. The ideal candidate should have a minimum of 6 years of overall experience, with at least 3 years of hands-on experience in Python, PySpark, and SQL. The candidate should also demonstrate expertise in data flow articulation, syntax proficiency, and optimization of PySpark and SQL for handling large and complex datasets.
Key Responsibilities:
Hands-on Expertise: Possess a minimum of 3 years of hands-on experience in Python, PySpark, and SQL, with a proven track record of successful project implementation.
Project Experience: Demonstrate 2-3 project experiences showcasing critical skills and the ability to articulate the data flow, roles, and responsibilities fluently.
Syntax Proficiency: Showcase strong proficiency in Python, PySpark, and SQL syntaxes.
Optimization Skills: Demonstrate experience in PySpark and SQL optimization techniques to handle large and complex datasets, resulting in performance improvements.
Architecture Knowledge: Possess strong knowledge of PySpark architecture and configuration.
Datawarehouse Concepts: Familiarity with Datawarehouse concepts such as Data Quality, Slowly Changing Dimensions (SCDs), Data Lineage, etc.
Mandatory Requirements:
- Minimum of 6 years of overall experience.
- Minimum of 3 years of hands-on experience in Python, PySpark, and SQL.
- 2-3 project experiences in critical skills.
- Ability to articulate data flow and roles/responsibilities fluently.
- Strong proficiency in Python, PySpark, and SQL syntaxes.
- Experience in PySpark and SQL optimization for large/complex datasets.
- Strong knowledge of PySpark architecture and configuration.
- Familiarity with Datawarehouse concepts (Data Quality, SCDs, Data Lineage).
Good to Have:
- Exposure/hands-on experience with Cloud technologies.
- Exposure/hands-on experience with Hadoop/Hive.
- Exposure/hands-on experience with Databricks.
If you meet these qualifications and are passionate about working with cutting-edge technologies in the data engineering space, we encourage you to apply and be part of our innovative team.
