Job Title: Data Engineering Lead/Technical Lead (Databricks)

● Location: Chennai / Pune

● Experience Required: 8+ Years

Job Summary:

We are seeking a highly skilled Databricks Technical Lead with 8–10 years of experience in Big Data and

Spark technologies, including 3–4 years of hands-on experience with Databricks. The ideal candidate will

lead the architectural design, development roadmap, and implementation of scalable data pipelines and

lakehouse solutions using Delta Lake, Unity Catalog, and Databricks Workflows. Candidates must have

product building experience or mindset.

Key Responsibilities:

● Lead end-to-end architecture and implementation of modern data solutions using Databricks.

● Drive development best practices, code reviews, and performance optimization efforts.

● Design and manage scalable data pipelines using Delta Lake, Databricks Jobs, Workflows, and DLT

(Delta Live Tables).

● Implement governance and security using Unity Catalog for access control and data lineage.

● Collaborate with data engineers, analysts, and stakeholders to align business goals with technical

solutions.

● Provide guidance on cloud data architecture, especially within Azure/AWS/GCP environments

integrated with Databricks.

● Own and refine the development roadmap, ensuring timely and quality deliverables.

Required Skills & Experience:

● 8–10 years of experience in Big Data Engineering using Apache Spark and related technologies.

● Minimum 3 years of hands-on experience with Databricks, including:

○ Delta Lake for scalable and reliable data lake architectures.

○ Unity Catalog for centralized data governance.

○ Job & Workflow orchestration, including DLT pipelines.

● Strong knowledge of data modeling, ETL/ELT design patterns, and performance tuning.

● Proficient in PySpark, SQL, and Spark optimization techniques.

● Experience with at least one cloud platform (Azure/AWS/GCP).

● Familiar with CI/CD practices and DevOps integration in Databricks environments.

● Excellent communication, stakeholder management, and leadership skills.

Preferred Qualifications:

● Databricks certification (Data Engineering Associate/Professional) is a plus.

● Experience contributing to internal accelerators, utilities, or frameworks.

● Experience with MLflow or Databricks SQL is a plus.

● Exposure to enterprise data governance frameworks.