Title: Data Engineering, Big Data Analytics, and High-Performance Computing

Location: Washington, DC (Remote)

Rate:Depends on Experience

Experience Level: 10+ Years

The Contractor will provide the appropriate level of resources

Cloud Support for Hadoop, EMR, and HPC

The objective is to gain a skilled and technical individual(s) to support services for information systems hosted both, on-premise and in Amazon Web Services (AWS) and Microsoft Azure.

The ideal contractor has the:

• Ability to work with huge volumes of data so as to derive Business Intelligence
• Knowledge to analyze data, uncover information, derive insights, and propose data-driven strategies
• Knowledge of OOP languages like Java, C++, and Python
• Understanding of database theories, structures, categories, properties, and best practices
• Knowledge of installing, configuring, maintaining, and securing Hadoop
• Analytical mind and ability to learn-unlearn-relearn concepts

The day-to-day work requires someone with extensive hands-on experience in the following areas:

• Apache Hadoop and Apache Spark. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. The framework supports programming languages like Python, Scala, Java,
• Amazon Web Services/Redshift (for data warehousing). Must be familiar with the most popular data warehousing applications, including Amazon Web Services and Amazon Redshift.
• Azure. Microsoft's Azure cloud technology to build large-scale data analytics systems.
• HDFS and Amazon S3.

• This includes enforcement of operational standards to produce quality information systems and the enhancement of a shared infrastructure. The goal is to adopt the best practices of both Amazon AWS with an operational focus on uniformity and automation across the enterprise which can be measured by the following:

Consistency - The end result can be reliably achieved through step-by-step standard operating procedures with some foundational knowledge, but minimal system-specific knowledge and the procedures defined therein are largely reusable from system to system.

Repeatability - The end result is reproducible in a way that minimizes the probability of error and manual interaction. This includes eliminating steps where practicable and the need for specialized knowledge or training.

Interchangeability- Technical solutions and operational standards implemented under this contract support will enable these objectives so that support is predictable and interchangeable from system to system

Tasks include but are not limited to:

• Requirements gathering and analysis
• System architecture, analysis, and design
• Performing code reviews, security reviews, and content/accessibility reviews
• Configuring Continuous Integration (CI) systems and automated deployments;
• Setup and configuration of Python and Anaconda
• Writing automated unit and functional tests;
• Trouble-shooting and support of Python
• System optimization and performance tuning
• System documentation
• Using open source technologies including:
• Red Hat and Amazon Linux;
• Hadoop, Hive, Spark, and Impala
• Demonstrated past experience with Extract, Transfer, and Load (ETL) procedures