Data Engineer=TS/SCI- Colorado Springs
- Job Title
- Data Engineer=TS/SCI- Colorado Springs
- Job ID
- Colorado Springs, CO
- Other Location
DATA ENGINEER (DE)
We are looking for a self-motivated Data Engineer to join a growing team of mission oriented analysts and developers. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow from multiple operational domains across multiple network domains. The Data Engineer will support software developers, database architects, and data analysts and will ensure optimal data delivery architecture is consistent throughout ongoing projects. Candidate must be self-directed and comfortable supporting the data needs of multiple projects and diverse data products.
Responsibilities and Duties:
- Create and maintain optimal data pipeline architecture,
- Manage data flow pipeline across 3 networks
-Prepare data in xml format for consumption by cross-domain solution (CDS) to other networks in accordance with security requirements/policy
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources via REST, SOAP, WSDL APIs
- Create flowfiles in nifi that move data into Accumulo via Kafka
Knowledge, Skills and Ability:
- Applicants should also have a demonstrated understanding and experience using software and tools including big data tools like Nifi, Kafka, Spark and Hadoop; relational NoSQL and SQL databases including Accumulo and Postgres
- Graph technology such as Neo4j is a bonus, but not required
- Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
- Strong analytic skills related to working with unstructured datasets.
- A successful history of manipulating, processing and extracting value from large disconnected datasets.
- Candidate should have experience using the following software/tools/technology:
Nifi Kafka SQL Accumulo Python Jupyter pandas
Bachelor’s Degree (Computer Science), 3-5 years exp, Security+ (preferred)