• Create Hadoop Impala SQLs and oozie workflows to get data from different data sources and integrate into the Hadoop file system.
• Create Azure pipelines to copy data from common blob storage to member specific storage accounts for serverless SQL pools and create a pipeline to copy data from common storage to Azure dedicated SQL pool external tables, tables and views.
• Work on Azure Databricks notebooks to load data from azure blob storage to deltalake using Pyspark.
• Monitor daily and weekly Oozie jobs which does ETL processing and push the latest data files to Hadoop cluster and Azure blob storage.
• Lead and monitor the execution of batch jobs post implementation to ensure the process execution is successful and accurate.
• This includes data flow from source systems to application, transformation process and target load.
• Accountable for deliverables of the GAVS India project team and will be providing an excellent technical experience, by anticipating and resolving customer issues in a manner that meets and exceeds the customer expectations.
• Act as point of contact for all technical development discussions within the project team.
• Works with Project Manager to define the project's deliverables/ requirements and conducts meetings and interactions with necessary project teams.
• Work with Scrum Master/ Technical Project Manager to manage and prioritize the work items as per the requirement and work for inclusion of Epics/Features based on team availability and facilitate discussions with the project team on scope of each sprint within an iteration.
• Create complex SQL/ T-SQL statements, stored procedures, functions, dynamic SQL and performance tuning for existing SQL queries by following Database management system and Data warehousing concepts.
• Create generic code that can be reused as part of the application using advanced techniques by making use of the available tools.
• Duties entail work with Azure, Teradata, PostgreSQL, Powershell, Python and SQL. Travel to client sites may be required. $164,195.00/year.

Requirements:

• Bachelor’s degree or equivalent in Computer Science, Engineering, Information Technology or related field.
• 5 years of experience in job offered or in related occupations.