Big Data Hadoop Architect Lead - job id 33732


Your Way To Work™

Big Data Hadoop Architect Lead

$175,000 F/T Employee

75 Rockefeller Plaza



How to Apply

logo

George Konetsky


logo

(646) 876-9562


logo

(212) 616-4800 ext-180




A F/T position at a major U.S. financial institution.

Pay Options: F/T Employee.

Contact George Konetsky. call (646)876-9562 / (212)616-4800 ext.180 or email george@sans.com with the Job Code GK33732 or Click the Apply Now button ().

Location: Financial District.

Skills required for the position: BIG DATA, HADOOP, CLOUDERA, AGILE, DATA STRUCTURES.


Detailed Info: Data Scientist/ Lead to be responsible for developing, enhancing, modifying and/or maintaining a multi-tenant big data platform.

  • Functionally lead a team of developers located on and off shore and collaborate with Product Owners, Quants, and other technology teams to deliver data/applications/tools.

  • Work closely with the Business Stakeholders, Management Team, Development Teams, Infrastructure Management and support partners.

  • Use your in-depth knowledge of development tools and languages towards design and development of applications to meet complex business requirements.

  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

  • Design and implement scalable data platforms for our customer facing services.

  • Deploy and scale Hadoop infrastructure.

  • Hadoop / HDFS maintenance and operations.

  • Data cluster monitoring and troubleshooting. Hadoop capacity planning. OS integration and application installation.

  • Partner with program management, network engineering, site reliability operations, and other related groups. Willingness to participate in a 24x7 on-call rotation for escalations

Development/Computing Environment:

  • Deep understanding of UNIX and network fundamentals.

  • Expertise with Hadoop and its ecosystem Hive, Pig, Spark, HDFS, HBase, Oozie, Sqoop, Flume, Zookeeper, Kerberos, Sentry, Impala etc.

  • Experience designing multi-tenant, containerized Hadoop architectures for memory/CPU management/sharing across different LOBs.

  • 5+ years managing clustered services, secure distributed systems, production data stores. 3+ years of experience administering and operating Hadoop clusters.

  • Cloudera CHD4 /CDH5 cluster management and capacity planning experience.

  • Ability to rapidly learn new software languages, frameworks and APIs quickly.

  • Experience scripting for automation and config management (Chef, Puppet) Multi-datacenter, multi-tenant deployment experience, a plus.

  • Strong troubleshooting skills with exposure to large scale production systems.

  • Hands on development experience and high proficiency in Java / Python.

  • Skilled in data analysis, profiling, data quality and processing to create visualizations.

  • Experience working with Agile Methodology.

  • Good SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases (Hive, Impala, Kudu a plus).

  • Experience building and optimizing 'big data' data pipelines, architectures and data sets. Build processes supporting data transformation, data structures, metadata, dependency and workload management.

.

The position offers competitive compensation package.


Job Id: 33732