Information Exchange - Pull data from EFT ARM then dump it to a json file
Go to file
2024-08-11 00:48:34 -05:00
.gitea/workflows Updated build.yaml 2024-08-05 11:30:43 -05:00
.gitignore moved encoder to seperate file and created .token refresh logic 2024-07-29 17:30:11 -05:00
config.toml.example made query more performance and allowed embedded override 2024-08-10 00:49:53 -05:00
inex.py fix: #14 now correctly handles 'other' transactions 2024-08-10 02:20:26 -05:00
inexConnect.py generalized db query and embedded it. 2024-08-02 20:57:20 -05:00
inexDataModel.py fix: #14 now correctly handles 'other' transactions 2024-08-10 02:20:26 -05:00
inexDataProcessing.py fix: #14 now correctly handles 'other' transactions 2024-08-10 02:20:26 -05:00
inexEncoder.py moved encoder to seperate file and created .token refresh logic 2024-07-29 17:30:11 -05:00
inexLogging.py Added logging. 2024-07-11 16:13:45 -05:00
inexSqlquery.py made query more performance and allowed embedded override 2024-08-10 00:49:53 -05:00
LICENSE changed license 2024-07-31 20:55:49 -05:00
README.md Updating documentation 2024-08-11 00:48:34 -05:00
requirements.txt moved requirements.txt 2024-07-17 02:44:43 -05:00
test.py Updated data models for download/upload, delete and login 2024-08-10 00:01:58 -05:00

Inex

Information Exchange - Ingest ARM data into EFC

This application is designed to pull data from an Microsoft SQL database, reformat it, and then send that data via the web to the Event Fusion Center. It's designed to be performant. The embedded query doesn't return unneccesary data and data is written to memory before being pushed into EFC. Initially you may have to pull a large amount of days so the system is caught up but the sweet spot would be to pull a set of data every day.

Build from source

You will need to install git and clone the repository: git clone https://git.jonb.io/jblu/Inex.git Alternitively you can download the files from the releases page.

Then follow OS specific instructions. These are taken directly from the steps used in automation to build the executables found in the releases page.

Windows

You can find the requirements.txt in the repository. python -m pip install -r requirements.txt pyinstaller --noconfirm --onefile --console path/to/inex.py

Linux

Update Linux Distro. This is Ubuntu specific. Please check the package management tool in your specific version. apt-get update You need unixodbc or else pyodbc cannot install. apt-get install unixodbc -y You can find the requirements.txt in the repository. pip install -r requirements.txt pyinstaller --noconfirm --onefile --console path/to/inex.py

Setup

Config.toml

Table Purpose
fortraPlatform Fortra Specific data
database MSSQL Configuration
immutables Data that must be included but is not provided by the source database
output If and where to write files
logging Set If, where and level of logging
fortraPlatform
Fortra Specific data.
database
MSSQL Configuration.
immutables
Data that must be included but is not provided by the source database
output
If and where to write files.
logging
Set If, where and level of logging.
Usage