Apply now

Apply for Job

Senior Data Engineer

Date:  12 May 2024
Location: 

Kuala Lumpur

Company:  StarHub Ltd

Key Responsibilities

As the Senior Data Engineer, you will serve as our technical expert and work closely with cross-functional teams to design, build, and optimise data solutions that drive business insights and decision-making. You will be responsible for defining data architecture, developing data pipelines, and ensuring the reliability, scalability, and performance of our data systems.

 

This role is an individual contributor position, with a focus on hands-on data engineering tasks.

  • Data Architecture Design: Design and implement scalable and efficient data architecture, including data models, data warehouses, and data lakes. Understand the business requirements and design appropriate data models, data pipelines, and data warehouses.
  • Data Integration: Integrate data from various sources such as databases and APIs into a unified format for analysis. Develop ETL (Extract, Transform, Load) processes and real-time data pipelines.
  • Data Modeling: Design and implement dimensional and data models to support data warehouses, and analytical and reporting needs.
  • Data Pipeline Development: Develop and maintain robust ETL processes and data pipelines for ingesting, processing, and transforming large volumes of data from various sources. Ensure data quality, reliability, and consistency throughout the pipelines.
  • Performance Optimisation: Optimise the performance of data processing, visualization, and storage systems, including database tuning, query and ETL processes optimisation, and infrastructure scaling, to ensure timely and efficient data access.
  • Data Governance and Security: Establish and enforce data governance policies and procedures to ensure data integrity, privacy, and compliance with regulations and internal policies. Manage access controls, encryption, and auditing of data.
  • Tool and Technology Selection: Evaluate and select appropriate tools and technologies for data storage, processing, and visualisation.
  • Collaboration and Communication: Collaborate with cross-functional teams such as data scientists, analysts, and business stakeholders to understand their requirements and deliver data solutions that meet their needs. Communicate technical concepts effectively to non-technical audiences.
  • Documentation and Knowledge Sharing: Document data pipelines, processes, and best practices to facilitate knowledge sharing and ensure the maintainability of data solutions. Promote a culture of documentation and knowledge sharing within the team.
  • Continuous Learning and Improvement: Stay updated with the latest trends, advancements, technologies, and best practices in data engineering through continuous learning and self-improvement.

Qualifications

  • Critical and logical thinker with keen business acumen to link the dots between data and business.

  • Independent, proactive, and self-motivated attitude.

  • Excellent problem-solving skills and attention to detail.

  • Excellent verbal and written communication skills, with the ability to collaborate with cross-functional stakeholders and communicate technical concepts effectively.

  • Appreciate the advantages and limitations of different technical solutions in meeting analytics needs.

  • Bachelor's or Master’s degree in Computer Science, Engineering, Information Systems, or a related field.

  • At least 5 years of relevant experience in data engineering roles with demonstrated experience in designing and building data infrastructure, pipelines, ETL processes, and data modelling.

  • Proficiency in SQL and experience with relational (e.g. PostgreSQL, MySQL) and NoSQL databases, and data warehousing technologies (e.g. Snowflake, Redshift) will be advantageous.

  • Experience with cloud platforms such as AWS (Amazon Web Services), Azure, or Google Cloud Platform, including services like S3, EC2, EMR, and BigQuery will be advantageous.

  • Strong programming skills in Python, Java, or Scala, with experience in building data processing applications and workflows using frameworks like Apache Spark or Apache Beam will be advantageous.

  • Experience with data visualisation tools such as Power BI, Tableau, or Looker, and proficiency in data modelling and visualisation techniques will be advantageous.

  • Knowledge of data governance principles, data security best practices, and regulatory compliance requirements will be advantageous.

  • Track record in management and working on multiple projects concurrently.

 

We regret that only shortlisted candidates will be notified.

 

To APPLY NOW, click on Skye!

Apply now

Apply for Job