Job Description
Big Data Engineer | Data Engineer | Data Scientist (Scala, Spark, Hadoop, Python).
Fantastic opportunity for a talented Data Engineer to join a successful Tech House that provides data science services and complex software solutions to a range of industries helping them innovate and drive transformation.
As a Data Engineer, you will have a number of responsibilities including; documenting solution requirements, managing and transforming high volume data, performing advanced analytics to identify trends associated with adverse behaviour and automating analysis to proactively alert on high-risk activity. This is a challenging and interesting role where you will be using leading open source data science tools including Scala, Spark and Hadoop.
Based in Sydney CBD, you will be working alongside a world-class team of friendly, motivated and talented people, who are passionate about their craft.
Requirements:
As a Data Engineer, you will have a number of responsibilities including; documenting solution requirements, managing and transforming high volume data, performing advanced analytics to identify trends associated with adverse behaviour and automating analysis to proactively alert on high-risk activity. This is a challenging and interesting role where you will be using leading open source data science tools including Scala, Spark and Hadoop.
Based in Sydney CBD, you will be working alongside a world-class team of friendly, motivated and talented people, who are passionate about their craft.
Requirements:
- Experienced in Java, Scala and/or Python, Unix/Linux environment on-premises and in the cloud Java development and design using Java 1.7/1.8. Advanced understanding of core features of Java and when to use them
- Experience with most of the following technologies (Apache Hadoop, Scala, Apache Spark, Spark streaming, YARN, Kafka, Hive, HBase, Presto, Python, ETL frameworks, MapReduce, SQL, RESTful services).
- Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Pig, Spark, Spark SQL.
- Must have experience with developing Hive QL, UDF's for analysing semi-structured/structured datasets
- Experience with time-series/analytics db's such as Elasticsearch
- Experience with industry standard version control tools (Git, GitHub), automated deployment tools (Ansible & Jenkins) and requirement management in JIRA
As a Data Engineer (Spark, Hadoop) you can expect to earn a competitive salary plus bonus and benefits. This is an immediate need offering a quick interview process.
Social Stream