Data Engineer / Software Engineer
Expired on: Sep 20, 2021
We are a dynamic software development company, working on innovative projects utilizing modern technology.
we are seeking multiple developers who are skilled in Spark, Scala, Java, or Python to work on a BigData Project. The project is to build a BigData platform for one of our clients using Apache Spark, Apache Nifi, Apache Zeppelin, Apache Kafka, and such cutting-edge technologies.
Key Responsibilities
- Writing Code in Scala or Python
- Design, Develop and test BigData solutions using selected tools
- Read, extract, transform, stage and load data
- Writing code/scripts, web scraping, calling APIs, SQL queries, etc.
- Process unstructured data and to generate modelled data
- Build self monitoring and self healing/zero touch operations
- Ability to design solutions independently based on high-level architecture
- Collaborate with other development and research teams
Preferred Qualifications
- 1+ years of experience as a Big Data Engineer/Software Engineer or similar role
- Bachelor’s Degree in Computer Science or equivalent preferred
- Programming experience, ideally in Python, Scala, or Java
- Willingness to learn new programming languages
- Experience with Spark/Hadoop Ecosystem
- Experience with building stream-processing systems, such Spark-Streaming
- Experience with integration of data from multiple data sources
- Experience with various messaging systems, such as Kafka
- Knowledge of statistics
- Knowledge of data cleaning, wrangling, visualization, and reporting
- Experience in Apache NiFi, PostgreSQL, MapReduce, TimescaleDB, data mining, machine learning, natural language processing, or information retrieval is a plus
- Experience processing large amounts of structured and unstructured data
- Integrating data from multiple sources
- Experience with Big Data ML toolkits, such as SparkML, Mahout, or H2O is a plus
“You are required to have some level of skills at least of a few of the above technologies and depending on your knowledge and
skill level, you will be considered for different positions”
Company: Digital Micro Services
Company email: [email protected]
Job Location: Colombo 01
Job Category: IT-Software
Job Type: Full Time
Sorry! This job has expired.