Data Engineer / Software Engineer

Closing on: Sep 20, 2021

We are a dynamic software development company, working on innovative projects utilizing modern technology.

we are seeking multiple developers who are skilled in Spark, Scala, Java, or Python to work on a BigData Project. The project is to build a BigData platform for one of our clients using Apache Spark, Apache Nifi, Apache Zeppelin, Apache Kafka, and such cutting-edge technologies.

Key Responsibilities

  • Writing Code in Scala or Python
  • Design, Develop and test BigData solutions using selected tools
  • Read, extract, transform, stage and load data
  • Writing code/scripts, web scraping, calling APIs, SQL queries, etc.
  • Process unstructured data and to generate modelled data
  • Build self monitoring and self healing/zero touch operations
  • Ability to design solutions independently based on high-level architecture
  • Collaborate with other development and research teams

Preferred Qualifications

  • 1+ years of experience as a Big Data Engineer/Software Engineer or similar role
  • Bachelor’s Degree in Computer Science or equivalent preferred
  • Programming experience, ideally in Python, Scala, or Java
  • Willingness to learn new programming languages
  • Experience with Spark/Hadoop Ecosystem
  • Experience with building stream-processing systems, such Spark-Streaming
  • Experience with integration of data from multiple data sources
  • Experience with various messaging systems, such as Kafka
  • Knowledge of statistics
  • Knowledge of data cleaning, wrangling, visualization, and reporting
  • Experience in Apache NiFi, PostgreSQL, MapReduce, TimescaleDB, data mining, machine learning, natural language processing, or information retrieval is a plus
  • Experience processing large amounts of structured and unstructured data
  • Integrating data from multiple sources
  • Experience with Big Data ML toolkits, such as SparkML, Mahout, or H2O is a plus

“You are required to have some level of skills at least of a few of the above technologies and depending on your knowledge and
skill level, you will be considered for different positions”

Company: Digital Micro Services
Company email: [email protected]
Job Location: Colombo 01
Job Category: IT-Software
Job Type: Full Time

Apply now

Allowed Type(s): .pdf, .doc, .docx