Big Data Engineer

San Francisco, CA

Do you know Big Data?

As a Senior Big Data Engineer you will work with web services, messaging and Big Data technologies.

RESPONSIBILITIES

Design and build Big Data Platform that is fault-tolerant and scalable.
Participate in API-based integration and back-end based build-out of data architecture and pipeline.
Develop, refine and scale data management and analytics procedures, systems, workflows, best practices and other issues.
Development of data-driven products.
Visualize and communicate data clearly for use both internally and externally.
Collaborate with our engineers to produce excellent products for Plum clients as well as streamline internal processes.

REQUIRED QUALIFICATIONS

3+ years experience in building scalable, high-performance, reliable, highly available and large distributed systems with high degree of autonomy.
Experience in working with big data technologies such as Spark, Hadoop ecosystems, Kafka, Zookeeper, Elasticsearch, Yarn, MapReduce, Storm, HBase and Cassandra.
Experience in SQL based technologies such as MySQL and have solid understanding of NoSQL solutions such as MongoDB and DynamoDB.
Understanding of data flows, data architecture and processing of structured and unstructured data.
Experience with AWS cloud services such as EC2, RDS, S3, EMR and Redshift.
Experience with data mining concepts, machine learning algorithms and basic statistics.
Understanding of Social API integration (Facebook, Twitter, Google).
Proficiency in high level language such as Java, Scala or Python.
Strong knowledge of Linux / UNIX systems.
An entrepreneurial spirit as well as passion for solving difficult challenges through innovation and creativity, with a strong focus on results.
Bachelor’s degree in Mathematics, Statistics, Engineering, Computer Science or related discipline.
Legal authorization to work in the U.S. is required. We will not sponsor individuals for employment visas for this job.

PREFERRED QUALIFICATIONS

Knowledge of Text Mining and NLP techniques.
Experience with Graph database like Neo4j or Titan and web crawler such as Apache Nutch.
Experience with HTML/Javascript/XPath and other web technologies.