- Industry: Finance / Banking
- Business trips: Occasional
- Remote work: possible after onboarding
- Project language: English, Polish
- Renumeration: up to 118 PLN/h + VAT
- Project length: till 31.01.2021 + possible extensions
Migration of the data pipelines from Cloudera 5 to Cloudera 6.
Data pipelines include scala/spark based projects, hive based projects, orchestrated with oozie.
The DATADB migration project started during 2020 when a newer Hadoop (Cloudera
) cluster was made available by the IT organisation within the bank. DATADB is an application with a large variety of data pipelines developed on an older version of Cloudera throughout a period of 5 years. The business cases supported by the application is mostly commercial activities, but also regulatory. For example, customer insights, KYC, and many more uses cases. PROJECT RESPONSIBILITIES:
- Making sure the current pipeline which is running in Cloudera 5 environment will also has to be running in Cloudera 6 environment;
- Testing the existing code in Cloudera 6 test environment,
- If bugs are identified, they would need a fix by development of the respective components and deploy the job in production environment.
- Required experience with: Oozie, Sqoop, Bash, and Hive;
- Nice to have: Scala and Spark.
- Very good communication skills in English
- Challenging international projects in a Scandinavian business culture.
- Transparently built relations based on trust and fair play.
- Benefits: Medicover card, Multisport card.
- Internal reference bonus.
Min. 5 years of professional IT experience.