Description:
Ontario Power Generation (OPG) is looking for a dynamic, strategic and results-driven professional to join our team in the role of a Senior Data Engineer.
Reporting to the Senior Manager, Data Lake & Platform Engineering, this position is responsible to build and support the data driven applications which enable innovative, customer centric digital experiences. You will build reliable, supportable & performant data lake & data warehouse products to meet the organization’s need for data to drive reporting analytics, applications, and innovation. You will employ best practice in development, security and accessibility to achieve the highest quality of service for our customers.
KEY ACCOUNTABILITIES
- Build and optimize scalable data ELT/ETL pipelines across various organizational data sources.
- Collaborate with Data Architect to implement common data models for business intelligence and reporting.
- Ensure data security in transit and at rest with infrastructure and cyber teams.
- Clean, prepare, and optimize datasets, applying lineage and quality controls throughout the data lifecycle.
- Support Business Intelligence Analysts with data modeling and reporting optimization techniques.
- Troubleshoot issues related to data ingestion, transformation, pipeline performance, and data accuracy.
- Develop data pipelines for the data marketplace in collaboration with analysts, scientists, and engineers.
- Implement process improvements, automate tasks, and optimize data delivery for scalability and efficiency.
QUALIFICATIONS
- 4-Year University education in computer science, computer/software engineering or other relevant programs within data engineering, data analysis, artificial intelligence, or machine learning.
- 4-6 years working experience in a similar or related field
- Experience as a Data Engineer building data pipelines.
- Fluent in creating data processing frameworks using Python, PySpark, SparkSQL and SQL
- Experience with Azure Data Factory, ADLS, Synapse Analytics and Databricks
- Experience building data pipelines for Data Lakehouses and Data Warehouses
- Good understanding of data structures and data processing frameworks
- Knowledge of data governance and data quality principles
- Effective communication skills to translate technical details to non-technical stakeholders