In my enterprise, our team lead wants to integrate Elastiflow with our Grafana stack (include Grafana and Clickhouse) so I need someone who’ve tried this stack and give me some advice about how to visualize the data and some best practices. Thanks
You probably won’t find find many examples of this at the moment, however we are working on a few things that will make Grafana/ClickHouse much more of a “first choice” data platform for ElastiFlow, similar to our support for Elasticsearch/OpenSearch today. Is there anything more that you can share about how you are using these platforms today, and what you want to achieve by adding flow and other network data?
Luckily ElastiFlow supports Kafka as a way to output data, then from Kafka I sink the data to Clickhouse for storage and then we try to visualize in Grafana by copying the dashboard from ECS stack. So do you have any advice for me? And it’s more perfect if your team can try and create the same template for Grafana stack
I was referring to the use-cases you are looking to achieve. Is your need just about storing everything in one place, or about actually correlating network traffic to applications to compute, etc? If just a question of common storage then sure… ElastiFlow → Kafka → ClickHouse is fine. If looking to better analyze IT issues across all parts of the stack, then schema and other aspects become important.
Our use case is that we try to find a solution for network observabilty and yeah, Elastiflow is a good choice for the idea u mentioned : “about actually correlating network traffic to applications to compute…”. But the problem is that Elastiflow is sticked to ECS Stack which is not our observability stack. Then we try to migrate to our stack and so I ask for how to implement and visualize in Grafana like Kibana