Platform Engineer (Data)

Company Description

For over 20 years, Abcam has been providing tools the scientific community needs to enable faster breakthroughs in critical areas like cancer, neurological disorders, infectious diseases, and metabolic disorders.

We believe that to continue making progress, we need to work together in new ways. We need your own unique perspective as well as this of our people to make an always greater impact on the world. This community needs people like you: dedicated, agile and above all audacious so we can truly bring progress forward.

Job Description

We are seeking a highly skilled and experienced Data Platform Engineer to join our innovative team. As a Data Platform Engineer, you will play a crucial role in designing, building, and operating the data infrastructure and platforms that support our organisation's data-driven initiatives. You will be working with the Data Engineering team to help design, build and run our data platforms. As part of the platform team, you will work alongside other platform engineers to deliver best in class cloud services, ensuring adherence to standards and principles to ensure the seamless integration and availability of high-quality data across our systems. Join our team and contribute to building robust and scalable data platforms that empower our organisation's data-driven decision-making.

Key Roles & responsibilities:

• Design, develop, and maintain scalable and efficient data pipelines, frameworks, and infrastructure to support data ingestion, processing, transformation, and storage using industry best practices.

• Collaborate with data scientists, data analysts, and other stakeholders to understand data requirements and implement solutions that address their needs.

• Optimise and tune data processing and storage systems for performance and reliability, ensuring high availability, scalability and DataOps at scale.

• Implement and maintain data governance and security measures to ensure the privacy and integrity of sensitive data.

• Design and maintain data models, schemas, and data dictionaries to enable efficient and accurate data access and retrieval.

• Evaluate and implement appropriate data storage and processing technologies, leveraging cloud-based solutions such as Amazon Web Services (AWS).

• Develop and maintain automated data validation, monitoring, and alerting systems to ensure data quality and reliability.

• Collaborate with Data Engineering team to integrate data engineering workflows into CI/CD pipelines, enabling continuous integration and deployment of data infrastructure.

• Stay up-to-date with the latest industry trends and technologies related to big data, data engineering, and cloud computing, and apply them to enhance our data platforms and processes.

• Provide technical guidance and mentorship to members of the data engineering team.

• Follow and spear-head high coding quality for production level software.

• Understand and drive integrated, scalable, complex software systems, including their testing.

Qualifications

Bachelor's or higher degree in Computer Science, Software Engineering, or a related field.

• Proven experience as a Data Platform Engineer, preferably with a focus on building and maintaining data platforms and infrastructure.

• Strong proficiency in programming languages such as Python, Java, or Scala for data handling and scripting.

• Experience with infrastructure as code tools (terraform) and pipeline based deployment of infrastructure.

• Extensive experience with cloud-based data technologies, particularly AWS services such as S3, EC2, EMR, Glue, Athena, Redshift, and Lambda.

• Solid understanding of big data processing frameworks and technologies such as Hadoop, Spark, and Kafka.

• Proficiency in designing and implementing data pipelines using tools like Apache Airflow or AWS Step Functions.

• Familiarity with data warehouse/lake/mesh concepts, dimensional modelling, SQL-based query optimisation and various data types, e.g. structured vs unstructured.

• Experience with version control systems (e.g., Git), continuous integration, and deployment tools (e.g., GitLab, Jenkins, AWS CodePipeline).

• Strong analytical and problem-solving skills, with the ability to understand complex data-related challenges and develop innovative solutions.

• Excellent communication and collaboration skills to effectively work with cross-functional teams and stakeholders.

Additional Information

We know that when it comes to benefits, no one size fits all. Flexibility and choice matter which is why, in addition to market competitive salaries, we offer you a flexible benefits package which is tailored to your unique needs and support your financial, physical and emotional wellbeing. This includes our award-winning share scheme, 18 weeks fully paid maternity leave, 6 weeks fully paid paternity leave as well as highly flexible working and much more. Besides, your development will be integral to your experience here. You will grow alongside other talented minds, in ways you may often find unexpected.

When people come together, incredible things happen. Here you’ll work in a safe environment where you can be who you truly are. We’ll champion and celebrate your uniqueness throughout your journey with us. This is how we excel at partnering with the scientific community no matter the challenge, ultimately helping solve the world’s most critical diseases. Find out more about Diversity & Inclusion at Abcam.

#LI-CM1