Big Data Engineer

Location: New York , New York, United States
Type: Full-time
Posted: 03.NOV.2018


The role drives business and technical capability definition and development in our Digital Platforms space. The candidate will play a key l...


The role drives business and technical capability definition and development in our Digital Platforms space. The candidate will play a key leadership role on Digital Cloud and Big Data Team that partners with Product Owners, Enterprise Architects, Infrastructure Engineering, Scrum Masters, business partners, and external service providers to assess the technology landscape, identify future capabilities, held define strategy, architecture and drive the implementation of the Big Data platforms.? This role will work on next generation data platform services and process that support Business to Business and Business to consumer to enable web and mobile applications.? The role is expected to challenge the teams to think differently to achieve desired business outcomes, foster culture of quality, and mentor other?employees to develop architecture expertise. ?Responsibilities

  • Architect, design and build scalable infrastructure and platform to ingest, store and process very large amounts of data (structured, semi-structured and unstructured), including streaming and real-time into the cloud based Big Data Lake
  • Drive cloud based Big Data solutions by providing expert technical guidance on architecture, design, implementation, tuning, and integration.
  • Recommend industry standards and best practices for a cloud based data lake implementation
  • Collaborate with the various data source teams both internal and external? (Oracle, Teradata, mongoDB, cloud etc.) on effective strategies on data ingestion.
  • Build best in class ETL based solutions for effective ingestion and transformation of? data
  • Work closely with the IT and the platform team to deliver technical solutions. This includes end to end data orchestration, data layering, data governance, metadata management and data cataloging
  • ?Collaborate with the various technical, platform and product development team on the use of various AWS services (including Amazon Elastic Compute Cloud (EC2), S3, Elastic Map Reduce (EMR), Glue, Athena etc.
  • Propose technical solutions used for visualization, descriptive analytics, predictive modeling, Machine/Deep learning and prescriptive real-time decisions analytics
  • Work with the various consumers of the data lake on effective end user solutioning driven by business needs and requirements
  • Be an evangelist for all things Big Data and Digital
  • Assess the current state of business processes across and recommend solutions to increase business agility and revenue
  • Financial forecasts and resource tracking
  • Champion software engineering culture
  • Recruitment of software and engineering talents to the team
  • Creating a continuous learning environment to ensure that the software engineers are trained on the technology stack being adopted
  • Drive internal developer community to encourage sharing and crowdsourcing of latest engineering best practices and
  • Drive greenfield development of the software and data assets and relevant frameworks
  • Drive improvements in our development practice of continuous delivery and promote cloud computing and open source solutions where applicable.
  • Partner with business stakeholders to deliver the necessary digital solutions
  • Partner with the technology teams of external Ecosystem partners to deliver seamless and integrated solution
  • Performs code reviews as required
  • Bachelor?s Degree in Computer Science
  • At least 12+ years of relevant Big Data and Data Visualizations experience of mission critical platforms
  • Experience in Agile development practices
  • Experience in large scale development using some of the following technology stack
  • Java, Spring, Spring Boot
  • Javascript (AngularJS, ReactJS)
  • Hadoop
  • Glue, Athena, EMR and other AWS Native data Services
  • Kafka, Redis, NoSQL,? MongoDB,
  • Experience with Blockchain technologies
  • Working knowledge of Machine Learning concepts
  • Expert in leveraging continuous integration and robust build/test automation, with a preference for cross platform stacks and containerization (Jira, Git, Jenkins, Sonar) to create a true DevOps culture
  • Ability to work in a fast-paced environment both as an individual contributor and a technical lead
  • Prior hands-on experience delivering self service big data and data visualization platforms
  • Demonstrated ability to work in a global, virtual organization
  • Driven major changes in technology architecture to one that is truly Agile which incorporates cloud native architecture patterns in a distributed environment)
  • Driven major technology team culture change to become a truly Agile team which is self-organizing, truly DevOps and believes in everything automated
  • Extensive, demonstrable experience building complex consumer facing digital products and internal facing systems across web and mobile applications in demanding environment.
  • Experience in a leadership position responsible for building, motivating and leading high performing development teams by example
  • Experience in public cloud technology stacks of Amazon Web Services, and Microsoft Azure
  • Passionate about software engineering best practice include Agile,? continuous delivery, micro services, measurement and software.
  • Very strong analytical skills to be able to manage complex problems using a number of techniques.
  • Ability to demonstrate influencing and negotiating skills to align a number of potentially conflicting views with internal and external stakeholders.
  • Challenges systems, processes and people in order to achieve business objectives

Apply Now


Free eBook

Loader2 Processing ...