Primary Responsibilities:
-
Design, develop, and maintain applications and services on Microsoft Azure, with a strong focus on Azure Databricks, ensuring scalability, performance, and security.
-
Build and manage robust data pipelines using Azure Databricks and other Azure data integration services.
-
Write, debug, and optimize Spark, Scala, and Python code to process and analyze large datasets efficiently.
-
Develop advanced SQL queries and work with Snowflake for data extraction, transformation, and analysis.
-
Implement security and access control measures; regularly audit Azure infrastructure for compliance.
-
Understand, validate, and justify design and effort estimates for assigned modules or tasks.
-
Follow and enforce best engineering practices, including design reviews, unit testing, functional test automation, CI/CD, and code quality standards.
-
Maintain clean, maintainable, and testable code to ensure long-term sustainability.
-
Monitor system performance, optimize resources, and ensure cost-effective, high-availability solutions.
-
Define, document, and promote best practices for application deployment and infrastructure maintenance.
-
Provide technical guidance and support on infrastructure and architectural questions.
-
Manage and monitor continuous integration and deployment systems effectively.
-
Take ownership and accountability for assigned features and deliverables while collaborating effectively with team members.
-
Adhere to company policies, procedures, and directives, including potential reassignments, flexible work arrangements, or other changes driven by business needs.
Required Qualifications:
-
Bachelor’s degree in Technology (B.Tech) or MCA with 16+ years of formal education.
-
Minimum 7 years of professional experience in data engineering and cloud-based application development.
-
At least 5 years of advanced SQL development experience.
-
3+ years of hands-on experience with Azure (ADF), Databricks, and DevOps practices.
-
3+ years of experience architecting, designing, developing, and implementing cloud solutions on Azure.
-
2+ years of experience writing, reading, and debugging Spark, Scala, and Python code.
-
Experience collaborating with international clients to gather requirements and deliver solutions.
-
Strong proficiency in programming languages and scripting tools relevant to cloud and data engineering.
-
Solid understanding of cloud storage, SQL, and NoSQL database technologies.
-
Excellent troubleshooting skills across diverse technologies and environments.
-
Familiarity with DevOps tools and practices, including CI/CD pipelines and Terraform.
-
Demonstrated ability to collaborate with multidisciplinary teams, including business analysts, developers, data scientists, and SMEs.
-
Proactive mindset with the ability to identify problems, optimize processes, and improve performance.
-
Excellent communication, presentation, and documentation skills.