Senior Data Engineer working a decision science project – modeling migration to Databricks 74445

Position Title: Data Engineer

Line of Business: Decision Science - Personal & Business banking

Target Start Date: asap

Duration: 6 months

Potential for this contract to be extended or converted to FTE? Yes

Remote/Hybrid: Hybrid, twice a week to office

Office Location: 81st bay street, 25th floor

Job Description - Data Engineer -

· Determine methods and approaches to projects, transforming business requirements specifications into programming instructions, designing, coding and testing programs

· Work with senior management, technical and client teams in order to determine data requirements, business data implementation approaches, best practices for advanced data manipulation, storage and analysis strategies

· Perform data analysis on multiple disparate data sources and databases to identify key data elements/KPI’s that satisfy business/data requirements

· Designing and implementing data ingestion pipelines from multiple sources using SAS/SQL

· Code data quality and transformation logic for data movement from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times

· 8+ years of experience as a ETL/Data Engineer using SAS/SQL

· Demonstrated analytical, data analysis and problem-solving skills particularly those that apply to a big data environment

· Knowledge of approaches, tools, techniques and roles and responsibilities in providing technical or business guidance to business and technical partners.

· Process flow documentation experience required to maintain data lineage

· Able to document and communicate effectively to tell a ‘story’ for a technical recommendation from framework to feature level

· Microsoft Azure experience including Databricks, Python, Spark, SparkSQL an asset

· Degree in Computer Science, Engineering, Statistics, Management Analytics or a related field of study

· Team player and can effectively contribute and collaborate within a diverse, fast paced dynamic environment

· Interface with business partners to understand requirements and present solutions

· Hybrid working environment, 1 or 2 days per week in office

Must Have Requirement:

1. Experience with ETL/Data Engineer using SAS/SQL - 8 years

2. Experience with Python - 5 years

3. Experience Spark - 5 years

4. Strong communication skills

5. Cloud experience

Nice to Have:

1. Spark SQL experience