PALO IT - Data Engineer
Your profile & role on the project YOU: â€¢ Thrive on challenge. When was the last time you failed? â€¢ Are curious & always learning. What are you up to right now? â€¢ Can deal with constant change.
When were you last surprised? â€¢ Have mastered at least one skill of your trade but youâ€™re not defined by it. What can you teach us? Can you wear many hats? YOU AGAIN: You will work on acquiring, storing, governing and processing huge sets of structured/unstructured data. You will get to learn about the big data solutions landscape and choosing optimal architecture components to use for these purposes as well as a set of data engineering skills for implementing enterprise data foundation such as data lake.
Finally, you will work closely with other PALO IT experts in Data Intelligence, Research, UX Design, Digital Technology and Agile teams to enable intelligence in the digital mesh of our clients and deliver big impact to them. â€¢ Identify and implement big data frameworks / components that best fit to required capabilities â€¢ Gather and process data at scale, tackling challenges coming from unstructured data, real-time processing, data quality etc. â€¢ Enable data management capabilities to govern enterprise data / metadata and provide an easy and secure access to them â€¢ Support the definition and optimization of underlying infrastructure STILL YOU: â€¢ Possess a Bachelor, Master or PhD degree in IT and/or Computer Science, Statistics, Mathematics â€¢ Good knowledge of big data technology landscape and concepts related to distributed storage / computing â€¢ Min.
3 years of experience with big data frameworks (e.g. Hadoop, Spark) and distributions (Cloudera, Hortonworks, MapR) â€¢ Working with batch & ETL jobs to ingest and process data from multiple data sources â€¢ Experience with NoSQL databases (e.
g. Cassandra, MongoDB, Neo4J, ElasticSearch) â€¢ Experience with querying tools (e.g Hive, Spark SQL, Impala) â€¢ Experience or willingness to go in real-time stream processing, using solutions such as Kafka, AWS Kinesis, Flume and/or Spark Streaming â€¢ Experience or willingness to learn about DevOps and DataOps principles (e.
g Infrastructure as code, automating different parts of the data pipeline) â€¢ High-level understanding of Data Science (how models are built, trained and deployed) â€¢ You are passionate about technology and continuous learning comes naturally to you Your role at PALO IT You will be invited to take part in R&D works done within our Practices. You will have the chance to assist or be a speaker at must-attend international IT conferences. You will have the opportunity to write articles for our Blog or specialized press.
Genuine ambassador of PALO IT, you will present our offers and take an active role in the development of the company. â€¢ This job role is located in Singapore About PALO IT PALO IT is an innovation & agile de....