Consulting D&A Azure Data Engineer – Staff

Tech Stacks

• Microsoft Azure
• Scala/ Java / Python on Spark/ Pyspark
• Spark SQL

Job Description

• Hands-on role on data models, data storage, cleansing, validation integration and transformation, and data migration to manage data within the organization, for a small to medium-sized project
• Responsible for building pipelines, performing data profiling, and implementing data quality rules
• Need to work as a team member to contribute to various technical streams of project implementation
• Interface and communicate with the onsite coordinators
• Completion of assigned tasks on time and regular status reporting to the lead

Requirements

• In-depth cloud domain experience with a focus on Microsoft Azure / Microsoft Azure Stack solutions
• Experience building on Azure using Databricks, ADF
• Good to have a Synapse experience.
• Sound programming, debugging/troubleshooting, and problem-solving skills with familiarity in Scala/ Java / Python on Spark/ Pyspark.
• Strong knowledge of Pyspark/ Spark SQL.
• Strong data warehousing concepts.
• Good analytical skills.
• 3+ years of work experience in a very large data warehousing environment.
• 3+ years of experience with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.
• Specialization in a specific sector/domain will be an added advantage.