A Data Scientist extracts knowledge and insights from data using a combination of statistical, computational, and domain expertise. Their work involves collecting, cleaning, and analysing large datasets to identify patterns, trends, and anomalies. They then build predictive models and machine learning algorithms to solve business problems, improve decision-making, and create innovative products or services. The Data Scientist shall report to the Data Analytics Manager.
Duties and Responsibilities
Data Collection & Preparation: -Identify and collect relevant data from various sources (databases, APIs, web scraping, etc.). -Clean, transform, and pre-process data to ensure data quality and consistency. This includes handling missing values, outliers, and inconsistencies. -Works mainly to format and clean data meant for the Data Scientist, which will include real-time data. -Leading to the requirement of technologies such as Apache Spark. -Develop and implement data pipelines for efficient data ingestion and processing.
Data Model Design and Implementation; The Data Engineer is responsible for collaborating with data scientists and business stakeholders to design and implement a robust and scalable data warehouse. This includes: Developing and maintaining dimensional and normalised data models. Ensuring data integrity and consistency by establishing clear schema definitions, primary keys, and foreign keys. Optimising data structures and indexing for fast query performance to support analytical needs.
ETL/ELT Pipeline Management The Data Engineer is accountable for building, maintaining, and monitoring Extract, Transform, and Load (ETL) or Extract, Load, and Transform (ELT) pipelines. This includes: Developing data ingestion jobs to move data from various source systems into the data warehouse. Implementing data cleansing, transformation, and enrichment logic to prepare the data for analysis. Establishing robust error handling and logging mechanisms to ensure pipeline reliability and data quality.
Qualifications and Experience
Bachelor's Degree: A strong foundation in a quantitative field is essential. Qualifications in one of the following: -Computer Science -Statistics -Mathematics -Physics -Engineering (especially related to data or computation) -Economics
Professional Certificates in three of the following technologies or related are a must: Apache Spark A relational database system (e.g., PostgreSQL, MySQL, or SQL Server) Cloud certifications (e.g., AWS Certified Data Analytics, Google Professional Data Engineer, or Microsoft Certified: Azure Data Engineer) Experience with data orchestration tools (e.g., Airflow, Dagster)
Skills • A minimum of one year of experience in a similar position
