Software Dev Engineer(Data Engineer)
Overview
Placement Type:
Temporary
Compensation:
£420 per day(PAYE Inside IR35)
Start Date:
Asap
Contract – 9 months, Location – London(Remote) Travel required to Berlin.
This project consists of transforming our AWS Redshift cluster (UDM) from a production-level system used across multiple orgs to a reporting-level system used within CF only. It consists of several tasks that focus on i) reducing the surface area, ii) deprecating and migrating core pipelines off of UDM and iii) upgrading the cluster configuration based on the new needs. In addition, third-party vendor is also expect to address company-wide mandates related to the Digital Markets Act (DMA) and security initiatives
Deliverables
UDM Migration
1. Offboard external teams
2. Offboard internal teams’ production use cases
3. Migrate key pipelines off of UDM
4. Migrate derived datasets off of UDM
5. Clean up the cluster
6. Update cluster configuration
7. Automate cluster management and maintenance
DMA and Security
1. Move UDM to private subnet
2. DMA Tagging of Customer Behavior Datasets
Skills
Candidates are expected to have 2+ years of experience in 5 out of the 6 following domains.
• Knowledge of distributed systems as it pertains to data storage and computing
• Experience with data modeling, warehousing and building ETL pipelines
• Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
• Experience programming with at least one programming language such as C++, C#, Java, Python, Golang, PowerShell, Ruby
• Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
Client Description
Our Client is a FTSE 100, multinational technology company no longer known for just one thing. Their areas of expertise include e-commerce, cloud computing, digital streaming, artificial intelligence, original entertainment… the list goes on. Despite being primarily digital, their aim is to be the most customer-centric company in the world.
Aquent is dedicated to improving inclusivity & is proudly an equal opportunities employer. We encourage applications from under-represented groups & are committed to providing support to applicants with disabilities. We aim to provide reasonable accommodation for any part of the employment process, to those with a medical condition, disability or neurodivergence.