Data Platforms Technical Lead Architect & Engineer - Hadoop, Spark, Kafka, Parquet, Hive, Pig, Spark Streaming, AWS, EMR, HP Vertica - New York, NY

Our well-established ad tech client is seeking Data Platforms, Technical Lead Architect & Engineer to design and implement data systems that integrate into their existing systems that process billions of daily events, and expose hundreds of data points to their clients and partners as well as internal data science applications. You will build, coach and lead a team of engineers in this effort.

Responsibilities

  • Design, architect and implement scalable, fault tolerant distributed big data systems.
  • Deliver insightful data to clients partners systems as well as internal systems in various ways by implementing robust, scalable data delivery applications and APIs.
  • Working closely with engineering lead staff to get requirements and define interfaces. * Recruit, coach and mentor engineers.

Requirements

  • At least 3 years hands-on development experience with big data NoSQL data system
  • Experience in one or more of the following technologies: Hadoop, Spark, Kafka, Parquet, Hive, Pig. Hands on experience with Spark Streaming or other live stream processing technology
  • At least 1-2 years’ experience managing and coaching a team of Engineers
  • Advanced SQL query writing abilities and data understanding
  • Solid understanding of Data Structures, Algorithms & Object-Oriented design
  • Proven, significant experience with AWS and specifically EMR
  • Excellent communication skills and a team player
  • Experience with HP Vertica database is a plus
  • Experience mentoring and team leading skills is a big plus