Jobs
  • Share this Job

Java software engineer with GCP experience and DataFlow SME.

Location : Canada
Job Type : Temp/Contract
Hours : Full Time
Required Years of Experience : 4+
Required Education : Bachelor degree (BS in Computer Science) or similar
Travel : No
Relocation : No

Job Description :
GAL AeroStaff, which provides specialized staffing and recruiting services in the Business & Commercial aviation industry, is presently seeking a Java software engineer with GCP experience and DataFlow SME. based out in Canada. GAL AeroStaff is an internationally approved OEM service provider/supplier. We offer competitive salaries, as well as a professional and ethical work environment



Responsabilities

 



  • Design and develop data processing pipelines using Java/Apache Beam and Cdap and deploying into Google cloud in Google GCP Dataflow.

  • Developed various custom Cdap plugins in Java for transformations. Developed data pipelines monitoring system using Cdap Rest APi & Stackdriver API, Alerts from Java application.

  • Design and develop a solution to keep data in sync from on-premises Hadoop cluster to GCP cloud. A data change in Mapr clusters copies the data in near real-time to GCP and triggers a job in GCP.

  • Migrating Hive based data transformation pipelines from on-premises Mapr Hadoop clusters to Google GCP Dataproc clusters and converting it to Spark pipelines using Pyspark

  • Developed POC using various technologies Apache Beam/Spark/Cdap/DataFusion based data pipelines on GCP

  • Explore cloud-based new technologies/products for Marketing Analytics and Data Science team and provide data engineering consultancy


Required Qualifications :
 

 


 


 


Powered by AkkenCloud
Top