Project architecture

The architecture of Hermes has been designed with a microservices approach in mind. This means that the system is composed of small, independently deployable services, each responsible for a specific within the overall workflow.

Overall, the architecture of Hermes can be represented in this way (with some simplifications):

Mobile app

This is the mobile application built using Flutter. It communicates exclusively with the Back-end API to send and receive data.

Back-end API

This component is developed using the Spring framework. It serves as a bridge between the mobile app and other back-end services. It handles requests from the mobile app and interacts with a MariaDB database to store and retrieve data.

Telegram API

This back-end is built with FastAPI and is responsible for direct communication with Telegram.

Docker API

Also developed using FastAPI, this backend interacts directly with Docker. Specifically it handles everything related to hermes-reader containers.

Message classification process

This process involves a series of Python scripts designed to manage and classify messages. In order for each script to be an independent micro-service, Redis is used so that the script takes the messages arriving from the previous step, processes them and sends them to the next step. Overall, the process is divided into these steps:

Reader

Fetches messages from selected Telegram groups and places them into Redis for further processing.

Cleaner

Takes the messages and cleans them up by removing emojis, unnecessary spaces, and other irrelevant content.

Translator

Converts messages from any language into English, as the AI models are designed to work with English text.

Classifier

Processes the cleaned and translated messages by feeding them into three different AI models and collects the results.

Specifically the models, which were selected and used thanks to Hugging Face, are:

• facebook/bart-large-mnli
• dima806/news-category-classifier-distilbert
• custom user model (fine-tuned from distilbert/distilbert-base-uncased + user feedback)

The first two models serve to understand what the message (topic) is about, while the third model gives an index of how interested the user is in that type of message.

This is to give, as far as possible, even more accurate results since we do not have a single model that has been trained for this type of task.

Evaluator

Interprets the results from the Classifier to determine whether a message is interesting or not. The final result is calculated with these parameters:

• Topic of the post

• User preference for that topic

• Result of the custom model

Based on these parameters, the message will then be marked as very interesting, interesting or will be deleted.

For the recognition of the topic, a weight of 20% is given to model facebook/bart-large-mnli and 80% to model dima806/news-category-classifier-distilbert.

Depending on the user's preference for the topic, there is a 25% increase (or decrease) on the result of the custom model.

Notifier

Sends messages deemed interesting to the mobile app to alert users of relevant content via Firebase Cloud Messaging (FCM).

Trainer

Collects all the feedback generated by the user and uses it to retrain the user's custom model, improving its accuracy and performance over time.

A cron job is currently scheduled to run at midnight daily, executing the script and training the templates for all users in the system.