The average data growth in the system is about 20 million records per month. How to ensure a proper operation of the platform with such a large information volume, so that it will be possible to draw up reports in real time and group the data? It is necessary to ensure the high-speed performance of the system and its high efficiency. While it is also necessary to consider fall-over protection in case of failures.
The aim of the project: to build an aggregation system that will work with a large number of records from various vendors, who can connect through REST-API and send their data to the system for further analysis.