Why the new name?
Yes, I've changed the name of this project from ShadowMeter to Galileo Network Analytics. There are two reasons I did this. First, stronger association with science and technology. Galileo is widely considered the "father of modern science." He made groundbreaking contributions to the scientific community, was a constant innovator, and was somewhat of a rebel. Let’s face it, Galileo was a “bad ass”.
Secondly, I renamed the project because "Galileo Network Analytics" was the name of a startup venture that I shelved earlier this year. For various reasons, I've decided to pivot to developing robotic and IoT solutions for agriculture. Therefore, it made sense for me to repurpose the brand. By the way, as I've shifted my focus to AgTech, I've realized that the Internet of Robotic Things also requires network security so this project is still relevant what I do.
What is the status of the project?
From the inception of this project, I decided to implement as much as made sense in Rust (programming language). After months of banging my head against the keyboard, I've grown to appreciate Rust's features, conventions, ecosystem, and best practices. For those unfamiliar with Rust, the language has its way of doing things that can frustrate a former C++ developer. As I learn more about the advanced features of the language, I find that writing software in Rust is truly a delight. I don’t think I’ll be going to back to C/C++ as my primary programming language.
If you take a look at the project page on GitHub, you'll see references to three Docker images associated with the project. These images are maintained on DockerHub.
1) fidelismachine/galileo_toolkit
This image bundles the core command line applications of the toolkit. First is gnat_yaf, a performant flowmeter, a modified version of YAF. Second, gnat_flow transforms (imports) YAF IPFIX files into Parquet format; conversely, it exports Parquet format to JSON, CSV, or QuestDB. And, finally, gnat_detect, which is currently work in progress. The command line gnat_detect performs 1) feature engineering, i.e., transforming flow record attributes into values suitable for deep learning, and 2) inference, i.e., scoring flow records in a streaming fashion.
2) fidelismachine/galileo_dashboard
This image is a customized version of Grafana open-source version. First, it only includes the QuestDB data source. Secondly, it omits plugins that are not relevant to the project. And it includes simple dashboards and the Galileo logo.
3) fidelismachine/galileo_proxy
This image is a customized image of nginx that functions as a reverse proxy for Grafana. This image serves as an application firewall and manages TLS connections.
Except for gnat_detect, the code base is fully functional now. Those interested in experimenting with the toolkit, please look at the README page on GitHub for more details. The docker-compose.yml is a functional example of running an end-to-end network monitoring workflow.
What is next?
My focus through the end of this year is to complete the implementation of gnat_detect. This effort includes selecting and transforming the raw flow data into suitable inputs (feature engineering) for the autoencoder neural network which I plan to implement with the PyTorch library.
As always, feedback is welcome!