Big Data Hadoop Architect Lead - job id 33732

Your Way To Work™

Big Data Hadoop Architect Lead

$175,000 F/T Employee

75 Rockefeller Plaza

How to Apply


George Konetsky


(646) 876-9562


(212) 616-4800 ext-180

A F/T position at a major U.S. financial institution.

Pay Options: F/T Employee.

Contact George Konetsky. call (646)876-9562 / (212)616-4800 ext.180 or email with the Job Code GK33732 or Click the Apply Now button ().

Location: Financial District.

Skills required for the position: BIG DATA, HADOOP, CLOUDERA, AGILE, DATA STRUCTURES.

Detailed Info: Data Scientist/ Lead to be responsible for developing, enhancing, modifying and/or maintaining a multi-tenant big data platform.

  • Functionally lead a team of developers located on and off shore and collaborate with Product Owners, Quants, and other technology teams to deliver data/applications/tools.

  • Work closely with the Business Stakeholders, Management Team, Development Teams, Infrastructure Management and support partners.

  • Use your in-depth knowledge of development tools and languages towards design and development of applications to meet complex business requirements.

  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

  • Design and implement scalable data platforms for our customer facing services.

  • Deploy and scale Hadoop infrastructure.

  • Hadoop / HDFS maintenance and operations.

  • Data cluster monitoring and troubleshooting. Hadoop capacity planning. OS integration and application installation.

  • Partner with program management, network engineering, site reliability operations, and other related groups. Willingness to participate in a 24x7 on-call rotation for escalations

Development/Computing Environment:

  • Deep understanding of UNIX and network fundamentals.

  • Expertise with Hadoop and its ecosystem Hive, Pig, Spark, HDFS, HBase, Oozie, Sqoop, Flume, Zookeeper, Kerberos, Sentry, Impala etc.

  • Experience designing multi-tenant, containerized Hadoop architectures for memory/CPU management/sharing across different LOBs.

  • 5+ years managing clustered services, secure distributed systems, production data stores. 3+ years of experience administering and operating Hadoop clusters.

  • Cloudera CHD4 /CDH5 cluster management and capacity planning experience.

  • Ability to rapidly learn new software languages, frameworks and APIs quickly.

  • Experience scripting for automation and config management (Chef, Puppet) Multi-datacenter, multi-tenant deployment experience, a plus.

  • Strong troubleshooting skills with exposure to large scale production systems.

  • Hands on development experience and high proficiency in Java / Python.

  • Skilled in data analysis, profiling, data quality and processing to create visualizations.

  • Experience working with Agile Methodology.

  • Good SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases (Hive, Impala, Kudu a plus).

  • Experience building and optimizing 'big data' data pipelines, architectures and data sets. Build processes supporting data transformation, data structures, metadata, dependency and workload management.


The position offers competitive compensation package.

Job Id: 33732