Job Description
Position Overview
We are seeking a highly skilled Senior IT Consultant to design and implement a secure, scalable Integrated Data Hub (IDH) using Databricks and Medallion Architecture. This role will be responsible for security design, access control configuration, data modeling, metadata management, and large-scale data processing across bronze, silver, and gold layers. Experience with FHIR data standards at the gold layer is considered a strong asset.
Key Responsibilities
Architect and build robust data pipelines across Medallion Architecture (Bronze/Silver/Gold layers) using Databricks.
Implement Delta Lake optimizations, Unity Catalog, and secure data access through RBAC and ACLs.
Profile large datasets to assess structure, completeness, and quality.
Design and optimize ETL/ELT pipelines (batch and streaming) using PySpark, SQL, and Delta Live Tables.
Develop and manage metadata-driven ingestion and transformation frameworks.
Integrate and manage REST APIs with downstream systems such as MDM and FHIR.
Configure and optimize Databricks jobs, clusters, and DevOps pipelines.
Lead logical and physical data modeling across all lakehouse layers.
Collaborate with business analysts and developers to translate requirements into technical solutions.
Guide sprint planning, backlog grooming, and act as a technical SME for data platform initiatives.
Required Skills & Experience
Technical Expertise
Data engineering experience in cloud environments.
Expert in Databricks, including:
Unity Catalog
Delta Lake
Auto Loader
PySpark
Deep understanding of Medallion Architecture and data lake design.
Proficient with Azure Data Lake, Azure DevOps, Git, and Postman.
Expert in SQL (complex queries, tuning, debugging).
Experience in CDC (Change Data Capture) strategies for batch and streaming.
Skilled in managing RBAC and fine-grained access controls.
Experience building metadata-driven pipelines using Python and PySpark.
Proficient in Databricks cluster configuration, version control, and environment management.
Capable of leading data modeling, metadata definition, and data governance strategies.
General Skills
Proven ability to translate business requirements into scalable data architectures.
Strong experience conducting data discovery, profiling, and quality assessments.
Experience leading data platform initiatives from discovery through deployment.
Effective communicator capable of working with technical and business stakeholders.
Strong analytical and diagnostic skills across infrastructure, ETL, and DB layers.
Passion for mentoring and establishing best practices and reusable frameworks.
Familiarity with Agile methodologies, sprints, user stories, and iterative delivery.
Skilled at assembling and refining Product Backlog Items, and collaborating with PMs and team leads.
Preferred Qualifications
Experience with FHIR data models and services.
Knowledge of enterprise data governance frameworks and best practices.
Job Types: Full-time, Permanent, Fixed term contract
Contract length: 12 months
Pay: From $70.00 per hour
Expected hours: 36.25 per week
Schedule:
Monday to Friday
Application question(s):
How many years of hands-on experience do you have with Databricks, including Unity Catalog, Delta Lake, Auto Loader, and PySpark?
How many years have you worked with Medallion Architecture (Bronze/Silver/Gold) in Databricks?
How many years of experience do you have conducting data profiling to assess structure, completeness, and data quality?
How many years of experience do you have designing and managing ETL pipelines, including Change Data Capture (CDC)?
Work Location: Hybrid remote in Toronto, ON
Application deadline: 2026-07-16
Expected start date: 2026-07-21
We are seeking a highly skilled Senior IT Consultant to design and implement a secure, scalable Integrated Data Hub (IDH) using Databricks and Medallion Architecture. This role will be responsible for security design, access control configuration, data modeling, metadata management, and large-scale data processing across bronze, silver, and gold layers. Experience with FHIR data standards at the gold layer is considered a strong asset.
Key Responsibilities
Architect and build robust data pipelines across Medallion Architecture (Bronze/Silver/Gold layers) using Databricks.
Implement Delta Lake optimizations, Unity Catalog, and secure data access through RBAC and ACLs.
Profile large datasets to assess structure, completeness, and quality.
Design and optimize ETL/ELT pipelines (batch and streaming) using PySpark, SQL, and Delta Live Tables.
Develop and manage metadata-driven ingestion and transformation frameworks.
Integrate and manage REST APIs with downstream systems such as MDM and FHIR.
Configure and optimize Databricks jobs, clusters, and DevOps pipelines.
Lead logical and physical data modeling across all lakehouse layers.
Collaborate with business analysts and developers to translate requirements into technical solutions.
Guide sprint planning, backlog grooming, and act as a technical SME for data platform initiatives.
Required Skills & Experience
Technical Expertise
Data engineering experience in cloud environments.
Expert in Databricks, including:
Unity Catalog
Delta Lake
Auto Loader
PySpark
Deep understanding of Medallion Architecture and data lake design.
Proficient with Azure Data Lake, Azure DevOps, Git, and Postman.
Expert in SQL (complex queries, tuning, debugging).
Experience in CDC (Change Data Capture) strategies for batch and streaming.
Skilled in managing RBAC and fine-grained access controls.
Experience building metadata-driven pipelines using Python and PySpark.
Proficient in Databricks cluster configuration, version control, and environment management.
Capable of leading data modeling, metadata definition, and data governance strategies.
General Skills
Proven ability to translate business requirements into scalable data architectures.
Strong experience conducting data discovery, profiling, and quality assessments.
Experience leading data platform initiatives from discovery through deployment.
Effective communicator capable of working with technical and business stakeholders.
Strong analytical and diagnostic skills across infrastructure, ETL, and DB layers.
Passion for mentoring and establishing best practices and reusable frameworks.
Familiarity with Agile methodologies, sprints, user stories, and iterative delivery.
Skilled at assembling and refining Product Backlog Items, and collaborating with PMs and team leads.
Preferred Qualifications
Experience with FHIR data models and services.
Knowledge of enterprise data governance frameworks and best practices.
Job Types: Full-time, Permanent, Fixed term contract
Contract length: 12 months
Pay: From $70.00 per hour
Expected hours: 36.25 per week
Schedule:
Monday to Friday
Application question(s):
How many years of hands-on experience do you have with Databricks, including Unity Catalog, Delta Lake, Auto Loader, and PySpark?
How many years have you worked with Medallion Architecture (Bronze/Silver/Gold) in Databricks?
How many years of experience do you have conducting data profiling to assess structure, completeness, and data quality?
How many years of experience do you have designing and managing ETL pipelines, including Change Data Capture (CDC)?
Work Location: Hybrid remote in Toronto, ON
Application deadline: 2026-07-16
Expected start date: 2026-07-21
💡 Quick Summary
Seeking a career-building opportunity? The Senior IT Consultant position is now open for candidates interested in the IT Engineer & Developer Jobs sector. This role in Toronto offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in IT Engineer & Developer Jobs is a plus.