Job Description
Full job description
At least 8 years of hands-on experience in enterprise-level data integration and ETL (Extract, Transform, Load) development, with a significant focus on integrating with Microsoft Dynamics 365 (Customer Engagement and/or Finance & Operations) and related Azure data services.
This position is currently listed as "Onsite"; however, the Assignment under this request will provisionally be "Hybrid", working 7.25 hours per calendar day, sometime between 8:00 AM and 5:00 PM (excluding breaks) Monday to Friday inclusive, unless otherwise identified. These conditions are subject to change as the OPS reflects on its current situation. During the duration of the assignment, you will be under the discretion of the Hiring Manager's requirements for the Project you are assigned to.
Scope
The Office of the Public Guardian and Trustee (OPGT) requires a Senior Software Developer – ETL to perform activities for interfacing the new Dynamics 365 solution and data warehouse with internal (OPS) and external systems as a member of the integrations team made up of internal and vendor staff.
Assignment Deliverables
As a member of the integrations team, the ETL Developer will be responsible for integrating the new Dynamics 365 solution, the data warehouse, and various internal (OPS) and external systems. This team will comprise both internal staff and vendor resources.
A high-level list of deliverables for the ETL Developer includes:
ETL Solution Design and Development:
o Design, develop, and implement robust ETL (Extract, Transform, Load) processes for data migration and ongoing integrations between diverse source systems (e.g., internal legacy systems, external vendor platforms) and Microsoft Dynamics 365 Customer Engagement (CE) and Finance & Operations (F&O).
o Develop and optimize data transformation logic to ensure data quality, consistency, and adherence to business rules and D365 data models.
o Utilize and recommend appropriate ETL tools and technologies (e.g., Azure Data Factory, SSIS, other cloud-based ETL services) to build efficient and scalable data pipelines.
o Implement data cleansing, validation, and error handling mechanisms within ETL processes.
Data Migration Planning and Execution:
o Lead and execute all phases of data migration activities from legacy systems to D365 CE and F&O, including data profiling, mapping, cleansing, transformation, and loading.
o Develop and maintain data migration strategies, cutover plans, and rollback procedures.
o Collaborate with data owners and business users to ensure data accuracy and completeness during migration.
Testing and Quality Assurance:
o Design, develop, and execute comprehensive test plans, cases, scripts, and test data (e.g., manufactured, obfuscated) based on functional and technical specifications to validate ETL solutions and data integrity.
o Create and maintain a full test plan, testing procedures, and an associated library of reusable test cases and scripts, ensuring full traceability from requirements to test outcomes.
o Perform both manual and automated testing to validate system and integration functionality, data accuracy, performance, and scalability. This includes unit testing, integration testing, system testing, and performance testing for ETL processes.
Collaboration and CI/CD Integration:
o Actively collaborate with stakeholders across business units, development teams, and external vendors to understand integration requirements and ensure proper data flow.
o Ensure proper integration of ETL processes and tests into the continuous integration/continuous delivery (CI/CD) pipeline to support automated deployments and efficient release cycles.
Support and Documentation:
o Provide analytical, development, and testing support for ETL processes and data integrations throughout the project lifecycle.
o Develop and maintain detailed technical documentation for all ETL processes, data mappings, data dictionaries, and integration architectures.
o Assist in troubleshooting and resolving data integration issues, providing timely support and solutions.
Experience and Skillset Requirements
Mandatory Requirements
At least 8 years of hands-on experience in enterprise-level data integration and ETL (Extract, Transform, Load) development, with a significant focus on integrating with Microsoft Dynamics 365 (Customer Engagement and/or Finance & Operations) and related Azure data services.
Nice to Have Requirements
The ideal candidate for this ETL Developer role will possess a strong blend of technical expertise in data integration, a deep understanding of Microsoft's data ecosystem, and excellent collaborative abilities.
ETL Tool Proficiency:
Mandatory: Proven hands-on experience with Microsoft's primary ETL tools for enterprise data integration, specifically Azure Data Factory (ADF). This includes designing and implementing pipelines, data flows, activities, datasets, linked services, and integration runtimes.
Highly Desirable: Experience with SQL Server Integration Services (SSIS) for existing on-premise integrations or migration scenarios.
Familiarity with other relevant data integration tools and concepts (e.g., Change Data Capture - CDC, data streaming) is a plus.
Database and Data Warehousing Expertise:
Strong SQL proficiency: Ability to write complex SQL queries, stored procedures, functions, and views for data extraction, transformation, and loading across various database platforms (e.g., SQL Server, Azure SQL Database).
Solid understanding of data warehousing concepts (e.g., dimensional modeling, star/snowflake schemas, data marts) and experience designing and implementing data warehouse solutions.
Experience with Azure data services relevant to data warehousing and analytics (e.g., Azure Synapse Analytics, Azure Data Lake Storage).
Microsoft Dynamics 365 Data Acumen:
Fundamental understanding of Dynamics 365 data models for both Customer Engagement (CRM) and Finance & Operations (ERP). This includes knowledge of key entities, relationships, and common data patterns within D365.
Ability to extract data from D365 APIs and OData feeds, and load data effectively into D365 (e.g., using Data Management Framework - DMF, KingswaySoft, or custom integrations).
Data Quality and Governance:
Experience implementing data cleansing, validation, error handling, and reconciliation processes within ETL pipelines to ensure high data quality.
Understanding of data governance principles and best practices for managing data integrity and consistency.
Programming/Scripting (Desirable):
Proficiency in scripting languages such as Python, PowerShell, or C# for custom data transformations, automation of ETL tasks, and interacting with APIs.
Version Control and CI/CD:
Experience with version control systems (e.g., Git, Azure DevOps Repos) for managing ETL code and configurations.
Familiarity with Continuous Integration/Continuous Delivery (CI/CD) pipelines for automated deployment of ETL solutions.
Analytical and Problem-Solving Skills:
Excellent analytical and problem-solving skills with a keen eye for detail to identify data discrepancies, troubleshoot complex integration issues, and optimize ETL performance.
Ability to translate business requirements into technical data integration solutions.
Communication and Collaboration:
Strong verbal and written communication skills to articulate technical concepts clearly to both technical and non-technical stakeholders.
Ability to collaborate effectively within a multidisciplinary team (internal and vendor staff), including data architects, D365 functional consultants, and business users.
Demonstrated ability to document technical designs, data mappings, and ETL processes thoroughly.
At least 8 years of hands-on experience in enterprise-level data integration and ETL (Extract, Transform, Load) development, with a significant focus on integrating with Microsoft Dynamics 365 (Customer Engagement and/or Finance & Operations) and related Azure data services.
This position is currently listed as "Onsite"; however, the Assignment under this request will provisionally be "Hybrid", working 7.25 hours per calendar day, sometime between 8:00 AM and 5:00 PM (excluding breaks) Monday to Friday inclusive, unless otherwise identified. These conditions are subject to change as the OPS reflects on its current situation. During the duration of the assignment, you will be under the discretion of the Hiring Manager's requirements for the Project you are assigned to.
Scope
The Office of the Public Guardian and Trustee (OPGT) requires a Senior Software Developer – ETL to perform activities for interfacing the new Dynamics 365 solution and data warehouse with internal (OPS) and external systems as a member of the integrations team made up of internal and vendor staff.
Assignment Deliverables
As a member of the integrations team, the ETL Developer will be responsible for integrating the new Dynamics 365 solution, the data warehouse, and various internal (OPS) and external systems. This team will comprise both internal staff and vendor resources.
A high-level list of deliverables for the ETL Developer includes:
ETL Solution Design and Development:
o Design, develop, and implement robust ETL (Extract, Transform, Load) processes for data migration and ongoing integrations between diverse source systems (e.g., internal legacy systems, external vendor platforms) and Microsoft Dynamics 365 Customer Engagement (CE) and Finance & Operations (F&O).
o Develop and optimize data transformation logic to ensure data quality, consistency, and adherence to business rules and D365 data models.
o Utilize and recommend appropriate ETL tools and technologies (e.g., Azure Data Factory, SSIS, other cloud-based ETL services) to build efficient and scalable data pipelines.
o Implement data cleansing, validation, and error handling mechanisms within ETL processes.
Data Migration Planning and Execution:
o Lead and execute all phases of data migration activities from legacy systems to D365 CE and F&O, including data profiling, mapping, cleansing, transformation, and loading.
o Develop and maintain data migration strategies, cutover plans, and rollback procedures.
o Collaborate with data owners and business users to ensure data accuracy and completeness during migration.
Testing and Quality Assurance:
o Design, develop, and execute comprehensive test plans, cases, scripts, and test data (e.g., manufactured, obfuscated) based on functional and technical specifications to validate ETL solutions and data integrity.
o Create and maintain a full test plan, testing procedures, and an associated library of reusable test cases and scripts, ensuring full traceability from requirements to test outcomes.
o Perform both manual and automated testing to validate system and integration functionality, data accuracy, performance, and scalability. This includes unit testing, integration testing, system testing, and performance testing for ETL processes.
Collaboration and CI/CD Integration:
o Actively collaborate with stakeholders across business units, development teams, and external vendors to understand integration requirements and ensure proper data flow.
o Ensure proper integration of ETL processes and tests into the continuous integration/continuous delivery (CI/CD) pipeline to support automated deployments and efficient release cycles.
Support and Documentation:
o Provide analytical, development, and testing support for ETL processes and data integrations throughout the project lifecycle.
o Develop and maintain detailed technical documentation for all ETL processes, data mappings, data dictionaries, and integration architectures.
o Assist in troubleshooting and resolving data integration issues, providing timely support and solutions.
Experience and Skillset Requirements
Mandatory Requirements
At least 8 years of hands-on experience in enterprise-level data integration and ETL (Extract, Transform, Load) development, with a significant focus on integrating with Microsoft Dynamics 365 (Customer Engagement and/or Finance & Operations) and related Azure data services.
Nice to Have Requirements
The ideal candidate for this ETL Developer role will possess a strong blend of technical expertise in data integration, a deep understanding of Microsoft's data ecosystem, and excellent collaborative abilities.
ETL Tool Proficiency:
Mandatory: Proven hands-on experience with Microsoft's primary ETL tools for enterprise data integration, specifically Azure Data Factory (ADF). This includes designing and implementing pipelines, data flows, activities, datasets, linked services, and integration runtimes.
Highly Desirable: Experience with SQL Server Integration Services (SSIS) for existing on-premise integrations or migration scenarios.
Familiarity with other relevant data integration tools and concepts (e.g., Change Data Capture - CDC, data streaming) is a plus.
Database and Data Warehousing Expertise:
Strong SQL proficiency: Ability to write complex SQL queries, stored procedures, functions, and views for data extraction, transformation, and loading across various database platforms (e.g., SQL Server, Azure SQL Database).
Solid understanding of data warehousing concepts (e.g., dimensional modeling, star/snowflake schemas, data marts) and experience designing and implementing data warehouse solutions.
Experience with Azure data services relevant to data warehousing and analytics (e.g., Azure Synapse Analytics, Azure Data Lake Storage).
Microsoft Dynamics 365 Data Acumen:
Fundamental understanding of Dynamics 365 data models for both Customer Engagement (CRM) and Finance & Operations (ERP). This includes knowledge of key entities, relationships, and common data patterns within D365.
Ability to extract data from D365 APIs and OData feeds, and load data effectively into D365 (e.g., using Data Management Framework - DMF, KingswaySoft, or custom integrations).
Data Quality and Governance:
Experience implementing data cleansing, validation, error handling, and reconciliation processes within ETL pipelines to ensure high data quality.
Understanding of data governance principles and best practices for managing data integrity and consistency.
Programming/Scripting (Desirable):
Proficiency in scripting languages such as Python, PowerShell, or C# for custom data transformations, automation of ETL tasks, and interacting with APIs.
Version Control and CI/CD:
Experience with version control systems (e.g., Git, Azure DevOps Repos) for managing ETL code and configurations.
Familiarity with Continuous Integration/Continuous Delivery (CI/CD) pipelines for automated deployment of ETL solutions.
Analytical and Problem-Solving Skills:
Excellent analytical and problem-solving skills with a keen eye for detail to identify data discrepancies, troubleshoot complex integration issues, and optimize ETL performance.
Ability to translate business requirements into technical data integration solutions.
Communication and Collaboration:
Strong verbal and written communication skills to articulate technical concepts clearly to both technical and non-technical stakeholders.
Ability to collaborate effectively within a multidisciplinary team (internal and vendor staff), including data architects, D365 functional consultants, and business users.
Demonstrated ability to document technical designs, data mappings, and ETL processes thoroughly.
💡 Quick Summary
Seeking a career-building opportunity? The Software Developer - ETL - Senior position is now open for candidates interested in the Software Developer Jobs sector. This role in Toronto offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Software Developer Jobs is a plus.
