Data Engineer

Technology Stack

• Java or Go
• Kafka RabbitMQ
• Apache Spark or Flink
• YAML
• JSON

Job Description

• Contribute to our joint effort of real-time analytics development and external-facing API design, including managing project deliverables and delegating work to other engineers
• Design, document, and deliver customer API specifications
• Build real-time streaming pipelines to construct and distribute analytics using Apache Flink
• Work across teams to gather requirements and understand reporting needs
• Investigate metric discrepancies and data anomalies
• Maintain high-quality documentation for our real-time metrics and API specifications
• Contribute to our data infrastructure platform and tooling
• Participate in our on-call rotation to keep production pipelines up and running

Requirements

• Expertise in building software in a high-level programming language, ideally Java or Go
• Experience designing, implementing and documenting APIs, webhooks, and callbacks
• Experience building software around a message bus such as Kafka or RabbitMQ
• Familiarity with real-time streaming engines such as Apache Spark or Flink
• Deep technical knowledge of data exchange and serialization formats such as Protobuf, Parquet, YAML, and JSON
• What we’d like to see
• B.S.+ in computer science or software engineering, or equivalent hands-on experience
• Previous experience in a data engineering or data warehousing
• Experience with complex event processing, distributed systems, and stream processing