Data Engineer Job at Palladium

We are recruiting to fill the position below:

Job Position: Data Engineer

Job Requisition ID: req20194
Job Location: FMOH headquarters, Abuja

Project Overview and Role

• Data.FI is a five-year global project funded by the U.S. Agency for International Development (USAID) to support the HIV and COVID-19 response. Data.FI partners with programs to accelerate and sustain access to high-quality data to expedite HIV and COVID-19 epidemic control and maintenance.
• By improving global, regional, national, and community/local in-depth analyses of HIV and COVID-19 epidemiologic and programmatic data, Data.FI expedites program achievements to improve client services and attain and sustain control of the epidemics.
• Data.FI supports host country governments in enhancing existing health information systems (HIS), informing management responses to gaps in programming and sustaining impact by supporting local partner transition.

Purpose of Position

• Palladium is seeking to recruit a for a skilled data engineer to join our systems development team.
• S/he will be responsible for architecting, designing, building, and maintaining the data infrastructure to support data storage, processing, and retrieval while ensuring efficient and reliable data management, processing, and integration.
• The data engineer will also be responsible for designing and building data pipelines, data warehouses, data marts and data lakes for multiple project products in line with global architectural principles and ensure that data is easily accessible, reliable, and secure.

Primary Duties and Responsibilities

• Responsible for conceptualizing, building, and maintaining optimal data pipeline architectures for the project.
• Work with the development team to conceptualize, design, and build the infrastructure required for optimal extraction, transformation, and loading of data from various data sources using SQL and other data technologies.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Provide technical support and resolution to data-related technical issues and data infrastructure needs for system users and stakeholders.
• Responsible for data governance of the Central repositories. Work with the information security and quality assurance teams to ensure data security and privacy.
• Contribute to the planning and documentation of technical specifications of database design and solution features. 
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• Engage in sprint planning and scrum meetings as part of the agile team.
• Implement Palladium’s standardized software development approach, including accompanying tools and standard operating procedures (SOPs) as part of the informatics unit.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
• Work with the data analytics team to build analytics tools that utilize the data pipeline to provide actionable insights into program performance and the provision of information for decision-making.
• Responsible for writing testable, scalable, and efficient queries
• Provide technical leadership in adoption and compliance to global best practices including the principles of digital development, applicable coding conventions, open standards, and security standards.

Required Qualifications:

• Bachelor’s Degree in Computer Science, Informatics, Information Systems or any other relevant field.
• Minimum five years of experience developing and implementing large-scale digital tools specifically relating to technology and data solutions in a data-driven environment.
• Familiarity with cloud platforms such as Azure, AWS, Google Cloud
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with object-oriented / object-function scripting languages e.g., Java.
• Experience building and optimizing data pipelines, architectures, and data sets at large scale.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Advanced working SQL knowledge and experience working with relational databases, including Postgres, query authoring (SQL), and working familiarity with various databases.
• Strong understanding of database systems, distributed computing systems, and enterprise-level data stream-processing technologies such as Kafka. 
• Strong analytic skills related to working with unstructured datasets and data exchange formats (e.g., Json)
• Experience in manipulating, processing, and extracting value from large, disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable data stores.
• Exceptional time management skills and solid interpersonal skills
• Solid technical background with understanding or hands-on experience in the development and implementation of Information Systems that support health and social protection programs will be an added advantage.
• Experience with Global Goods for digital health such as DHIS, OpenMRS and OpenHIE will be an added advantage.