Sr Big Data Engineer
Austin, TX 73301
Staffing Technologies is seeking a Big Data Engineer (SPARK) for a client in Austin, TX . This position will provide complete application lifecycle development, deployment, and operations support for Big Data solutions and infrastructure. In this role our client is using Google Cloud / Big Query. You will need strong Spark Development Skills (Java/Scala) as well as Solid BIG Data Cloud experience creating data streams.
You will partner with product owners, data scientists, solutions engineers, and business analysts to facilitate the development, automation, and seamless delivery of analytics solutions into Big Data clusters.
This is a W2 position, no subcontracting.
- Implement and enhance complex big data solutions with a focus on collecting, parsing, managing, analyzing, and visualizing large data sets that produce valuable business insights and discoveries.
- Determine the required infrastructure, services, and software required to build advanced analytics solutions, both on premise and in the cloud.
- Develop prototypes and proof of concepts for specified business challenges.
- Assist data scientists with exploration and analysis activities.
- Understand advanced algorithms and apply problem solving experience to build high-performance, parallel, and distributed solutions.
- Perform code and solution review activities, then recommend enhancements that improve efficiencies, performance, stability, and lower support costs.
- Quickly understand and apply new technologies and solution approaches.
- Perform other duties as assigned
- Experience with Spark with knowledge of how Spark and other big data tools work (3+ years)
- Experience with Hadoop (2+ years)
- Experience with Scala (3+ years) and Java skills
- Implement the AWS Best Practices
- Analyze the architecture and look for the optimization of the same
- Monitor/Manage the existing AWS System for possible data crunches.
- Experience with GitHub and/or other versioning tools
- Advanced experience with Software development, in an open source environment (5+ years)
- Ability to quickly learn new technologies, application domains, and adapt to changes.
- Ability to develop simple, elegant solutions to complex problems.
- Ability to handle multiple competing priorities in a fast-paced environment.