Work order information:
1. Role Title - Data Engineer/Architect
2. Number of roles – 2
3. Offsite/Remote – NO
4. Location - Newark, NJ
5. Bonus skills - CI/CD, Devops and Donedo (Virtualization)
6. Client’s business domain - Financial
7. Rate - $80-$85 per hour
8. Contract to hire possibility - Yes
Critical Technologies - on premise cloudera hadoop, with mostly cyberfusion
mft, rhel, hdfs, hive, impala, sentry, kerberos, hbase, hiveql, kafka, java and python, db2 and
This role is responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. experienced data analyst, data pipeline builder and data wrangler who enjoys analyzing data, building and optimizing data systems from the ground up then this is the right role . will implement data initiatives and ensure optimal data acquisition, management and delivery architecture which is consistent throughout ongoing projects. be self-directed and comfortable supporting data needs of multiple teams, systems and products. You will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.
- Empowering our businesses with the enterprise information and data knowledge they need to achieve value
- Building and maintaining data-intensive applications utilizing modern front-end and back-end technologies to deliver value to our businesses
- Creating and maintaining optimal data pipeline architecture assembling large, complex data sets that meet functional / non-functional business requirements.
- Identifying, designing and implementing internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Building the infrastructure required for optimal extraction, transformation, and
loading of data from a wide variety of data sources using Relational, NoSQL and Hadoop technologies.
- Building and maintain data services and data consumption tools that utilize the data pipeline to deliver actionable insights into key business performance metrics.
- Creating data visualizations for analytics and assisting other team members with using our data products.
- Working with partners including the Architecture, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Preparing and maintaining physical models and implementation-level details that affect the continuum of disciplines involved in the architecture, design, implementation and management of enterprise information.
- Collaborating with other teams to design, develop data tools that support both operations and data application use cases.
- Analyzing large data sets using components from the Hadoop ecosystem.
- Evaluating big data technologies and prototype solutions to improve our data processing architecture.
- Bachelor's degree in computer science, engineering, math, etc. or equivalent experience
- Strong analytic skills related to working with structured and unstructured datasets
- Experience building processes supporting data transformation, data structures, metadata, dependency and workload management.
- Experience with big data technologies: Hadoop, Hive, Impala, Hbase, Spark, PIG, SQOOP, HDFS, Solr
- Expert-level query language skills including SQL, HiveQL and experience working with Relational, NoSQL & Hadoop systems
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- Experience with Denodo Data Virtualization Platform (nice to Have)
- Hands-on experience implementing MDM, BI or data warehouse solutions preferred
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and find opportunities for improvement.