Technical Lead – Data Engineer

Location – Hyderabad or Bangalore

Job/Role Description

• Experience – 8 to 12 Years
• Leading a team of Data Engineers
• Expertise in design and development of Data pipelines and ETL/ELT using ADF, Databricks to move data from relational/structured/unstructured data from source to data lake to data warehouse, on Azure
• Creating Pipelines and integrations of multiple data sources using Azure data factory and other integration technologies
• Good to have batch and streaming (Kafka, Spark Streaming, Stream Analytics)
• Good to have Big data processing and transformation using Databricks (Scala/Python)
• Good to have Expertise in optimizing cost and performance of data pipelines

Primary Skills (Must have)

• Expertise with minimum 3+ projects (5+ Yrs. exp.)
• Azure SQL DB/DW
• Experience in implementing effective database solutions and data models to store and retrieve data
• Designing DB schema, data models on RDBMS and DW
• Programming and optimizing DB objects Views, Stored Procs, Functions
• Azure Data Lake or Blob storage
• Experience working with high volume data, large objects

Key Skills: Databricks, Azure Data Factory, ADF, Azure Synapse, Data Lake, Blob storage, Python, Pyspark