Are you a Data Engineer with the expertise to orchestrate symphonies of insight from complex datasets? If you revel in transforming raw numbers into a golden tapestry of innovation, then we invite you to join us on an exciting journey. At Mukuru, we are in search of a Data Engineer extraordinaire to wield their magic wand and shape the backbone of our data architecture.
Role Overview: The primary objective of this role is to prepare and manage large volumes of data for analytical and operational purposes. As a Data Engineer at Mukuru, you will be instrumental in designing, building, and maintaining the architecture used for data processing and storage.
Duties and Responsibilities
-Data Modeling and Design: Crafting efficient, scalable data models optimized for diverse storage and retrieval needs. -Data Pipeline Construction: Building and maintaining robust data pipelines ensuring seamless data flow from various sources to storage systems, often involving ETL processes. -Database Management: Managing and optimizing databases, ensuring speed, reliability, and security, with proficiency in SQL and NoSQL databases. -Big Data Technologies: Utilizing cutting-edge technologies like Hadoop, Spark, and Kafka for large-scale data processing. -Automation and Optimization: Automating data processes and optimizing data flow to enhance efficiency and reduce errors. -Ensuring Data Quality and Governance: Implementing measures for data accuracy, consistency, and security while adhering to data privacy laws and regulations.
Duties and Responsibilities: -Providing DataOps support to Analytics Engineers, overseeing AWS services, and managing CI/CD processes for deploying DDL to a Snowflake Data Warehouse. -Managing data extraction from various source systems hosted on AWS RDS and EC2 instances. -Maintaining and optimizing the extract, stage, delta load staging process for efficient data flow and integrity. -Transitioning to and maintaining an Operational Data Store (ODS) using Change Data Capture (CDC) for data ingestion across multiple source databases. -Collaborating with cross-functional teams to understand and meet data requirements, ensuring the data infrastructure aligns with organizational needs. -Ensuring data security and compliance with financial industry regulations and best practices. -Troubleshooting and resolving issues related to data systems and processes. -Continuously evaluating and implementing new technologies and tools to enhance data systems and processes.
Qualifications and Experience
-Minimum of 3 years of experience in Data Engineering, preferably in a financial services environment. -Strong experience with cloud data warehousing, including schema change management and DDL deployments (Snowflake CDW preferable). -AWS Solutions Architect Associate certification is preferable. -Proven experience in managing AWS services, including RDS and EC2, in a professional setting. -Solid understanding and experience with CI/CD processes and tools, specifically related to data operations. -Proficiency in SQL and familiarity with programming languages such as Python or Java. -Experience with data modeling and warehousing, particularly in a Snowflake environment. -Familiarity with Snowpark, Snowflake's native integration with Spark, or similar technologies. -Experience with CDC-based data ingestion tools, particularly Debezium. -Knowledge of data security and privacy standards specific to the financial industry.
