As a Sr. Data Engineer you will be a key member of our team, responsible for designing, developing, and maintaining sophisticated software / data platforms in achieving the charter of the engineering group. Your mastery of a technical domain enables you to take up business problems and solve them with a technical solution. With your depth of expertise and leadership abilities, you will actively contribute to architectural decisions, mentor junior engineers, and collaborate closely with cross-functional teams to deliver high-quality, scalable software solutions that advance our impact in the market. This is a role where you will have the opportunity to take up responsibility in leading the technology effort – from technical requirements gathering to final successful delivery of the product - for large initiatives (cross-functional and multi-month-long projects).
• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
• 10+ years of experience in data engineering, with a focus on data architecture, ETL, and database management.?
• Proficiency in programming languages like Python/PySpark and Java or /Scala?
• Expertise in big data technologies such as Hadoop, Spark, Kafka, etc.?
• In-depth knowledge of SQL and experience with various database technologies (e.g., PostgreSQL, MariaDBMySQL, NoSQL databases).?
• Experience and expertise in building complex end-to-end data pipelines.?
• Experience with orchestration and designing job schedules using the CICD tools like Jenkins, Airflow or Databricks
• Ability to lead ETL migration from Talend to Databricks pyspark
• Demonstrated building capabilities, reusable utilities, and tools for speeding complex business processes.
• Ability to work in an Agile environment (Scrum, Lean, Kanban, etc)?
• Ability to mentor junior team members.?
• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services (e.g., AWS Redshift, S3, Azure SQL Data Warehouse).?
• Strong leadership, problem-solving, and decision-making skills.?
• Excellent communication and collaboration abilities.?
• Familiarity or certification in Databricks is a plus.
Skills:- Scala, Python, Java, Apache Hadoop, SQL server and PySpark