The client asked us about proposing an abzodan solution that would allow for mapping connections between websites and easy / quick connection between them, eg which addresses connect domains A and B. The amount of data for analysis is over 2 terabytes.
Open source solutions dedicated to graphs have not worked well. Hadoop / batch processing was too slow
We have built our own MongoDB cluster-based solution for several machines
Thanks to the careful scheme of the base schema, queries carried out under 1 second