Do you know Big Data?
As a Senior Big Data Engineer you will work with web services, messaging and Big Data technologies.
- Design and build Big Data Platform that is fault-tolerant and scalable.
- Participate in API-based integration and back-end based build-out of data architecture and pipeline.
- Develop, refine and scale data management and analytics procedures, systems, workflows, best practices and other issues.
- Development of data-driven products.
- Visualize and communicate data clearly for use both internally and externally.
- Collaborate with our engineers to produce excellent products for Plum clients as well as streamline internal processes.
- 3+ years experience in building scalable, high-performance, reliable, highly available and large distributed systems with high degree of autonomy.
- Experience in working with big data technologies such as Spark, Hadoop ecosystems, Kafka, Zookeeper, Elasticsearch, Yarn, MapReduce, Storm, HBase and Cassandra.
- Experience in SQL based technologies such as MySQL and have solid understanding of NoSQL solutions such as MongoDB and DynamoDB.
- Understanding of data flows, data architecture and processing of structured and unstructured data.
- Experience with AWS cloud services such as EC2, RDS, S3, EMR and Redshift.
- Experience with data mining concepts, machine learning algorithms and basic statistics.
- Understanding of Social API integration (Facebook, Twitter, Google).
- Proficiency in high level language such as Java, Scala or Python.
- Strong knowledge of Linux / UNIX systems.
- An entrepreneurial spirit as well as passion for solving difficult challenges through innovation and creativity, with a strong focus on results.
- Bachelor’s degree in Mathematics, Statistics, Engineering, Computer Science or related discipline.
- Legal authorization to work in the U.S. is required. We will not sponsor individuals for employment visas for this job.
- Knowledge of Text Mining and NLP techniques.
- Experience with Graph database like Neo4j or Titan and web crawler such as Apache Nutch.