Threats IP Reputation doesn't work

ElastiFlow Community
1

Hello all,
I use Elastiflow 7.9.0 with ELK stack in my LAB environment, and my Threats IP Reputation doesn’t show any data anymore.
Kibana doens’t see any fields sec.threat name and here is my flowcoll.yml input configuration:

EF_PROCESSOR_ENRICH_APP_ID_ENABLE: “false”
EF_PROCESSOR_ENRICH_APP_ID_PATH: “/etc/elastiflow/app/cisco.yml”
#EF_PROCESSOR_ENRICH_APP_ID_TTL: 7200
#EF_PROCESSOR_ENRICH_APP_IPPORT_ENABLE: “false”
#EF_PROCESSOR_ENRICH_APP_IPPORT_PATH: “”
#EF_PROCESSOR_ENRICH_APP_IPPORT_PRIVATE: “true”
#EF_PROCESSOR_ENRICH_APP_IPPORT_PUBLIC: “false”
#EF_PROCESSOR_ENRICH_APP_IPPORT_TTL: 7200
#EF_PROCESSOR_ENRICH_APP_REFRESH_RATE: 15
#EF_PROCESSOR_ENRICH_ASN_PREF: lookup
#EF_PROCESSOR_ENRICH_COMMUNITYID_ENABLE: “true”
#EF_PROCESSOR_ENRICH_COMMUNITYID_SEED: 0
#EF_PROCESSOR_ENRICH_CONVERSATIONID_ENABLE: “true”
#EF_PROCESSOR_ENRICH_CONVERSATIONID_SEED: 0
EF_PROCESSOR_ENRICH_IPADDR_DNS_ENABLE: “true”
#EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_PATH: “”
#EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_REFRESH_RATE: 15
EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_IP: “x.x.x.x”
#EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_TIMEOUT: 3000
EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PRIVATE: “true”
EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PUBLIC: “true”
#EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_PATH: “”
#EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_REFRESH_RATE: 15
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_ENABLE: “true”
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_ASN_PATH: /etc/elastiflow/maxmind/GeoLite2-ASN.mmdb
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_ENABLE: “true”
#EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_PATH: “”
#EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_INCLEXCL_REFRESH_RATE: 15
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_LANG: en
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_PATH: /etc/elastiflow/maxmind/GeoLite2-City.mmdb
EF_PROCESSOR_ENRICH_IPADDR_MAXMIND_GEOIP_VALUES: city,country,country_code,location,timezone
#EF_PROCESSOR_ENRICH_IPADDR_METADATA_ENABLE: “false”
#EF_PROCESSOR_ENRICH_IPADDR_METADATA_REFRESH_RATE: 15
#EF_PROCESSOR_ENRICH_IPADDR_METADATA_USERDEF_PATH: “”
EF_PROCESSOR_ENRICH_IPADDR_NETINTEL_AS_PREFIX_PRECISION: all
#EF_PROCESSOR_ENRICH_IPADDR_TTL: 7200
#EF_PROCESSOR_ENRICH_JOIN_ASN: “true”
#EF_PROCESSOR_ENRICH_JOIN_CLOUD: “true”
#EF_PROCESSOR_ENRICH_JOIN_GEOIP: “true”
#EF_PROCESSOR_ENRICH_JOIN_NETATTR: “true”
#EF_PROCESSOR_ENRICH_JOIN_SEC: “true”
#EF_PROCESSOR_ENRICH_JOIN_SUBNETATTR: “true”
EF_PROCESSOR_ENRICH_NETIF_FLOW_OPTIONS_ENABLE: “true”
#EF_PROCESSOR_ENRICH_NETIF_METADATA_ENABLE: “false”
#EF_PROCESSOR_ENRICH_NETIF_METADATA_REFRESH_RATE: 15
#EF_PROCESSOR_ENRICH_NETIF_METADATA_USERDEF_PATH: “”
#EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_ENABLE: “false”
#EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_PATH: /etc/elastiflow/settings/snmp_access.yml
#EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_REFRESH_RATE: 15
#EF_PROCESSOR_ENRICH_NETIF_SNMP_COMMUNITIES: public
#EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE: “false”
#EF_PROCESSOR_ENRICH_NETIF_SNMP_PORT: 161
#EF_PROCESSOR_ENRICH_NETIF_SNMP_RETRIES: 1
#EF_PROCESSOR_ENRICH_NETIF_SNMP_TIMEOUT: 2
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PASSPHRASE: “”
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PROTOCOL: noauth
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PASSPHRASE: “”
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PROTOCOL: nopriv
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_USERNAME: “”
#EF_PROCESSOR_ENRICH_NETIF_SNMP_VERSION: 2
#EF_PROCESSOR_ENRICH_NETIF_TTL: 7200
#EF_PROCESSOR_ENRICH_OPTION_ENUM_TTL: 7200
#EF_PROCESSOR_ENRICH_SAMPLERATE_CACHE_SIZE: 32768
#EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_ENABLE: “false”
#EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_OVERRIDE: “false”
#EF_PROCESSOR_ENRICH_SAMPLERATE_USERDEF_PATH: /etc/elastiflow/settings/sample_rate.yml
EF_PROCESSOR_ENRICH_TOTALS_IF_NO_DELTAS: “false”
#EF_PROCESSOR_EXPAND_CLISRV: “true”
#EF_PROCESSOR_EXPAND_CLISRV_NO_L4_PORTS: “true”
#EF_PROCESSOR_IFA_ENABLE: “false”
#EF_PROCESSOR_IFA_POOL_SIZE: 0
#EF_PROCESSOR_IFA_QUEUE_SIZE: 64
#EF_PROCESSOR_KEEP_CPU_TICKS: “false”
#EF_PROCESSOR_PERCENT_NORM: 100
#EF_PROCESSOR_POOL_SIZE: 0
#EF_PROCESSOR_TIMESTAMP_PRECISION: ms
#EF_PROCESSOR_TRANSLATE_KEEP_IDS: default

What could be a problem?
It worked couple days ago, even restoring back my VM from snapshot with previously working state doesn’t get any results.

2

I would check the logs for errors, specifically related to ‘netintel.’

Regards,
Dexter

3

In my opinion, logs don’t show anything suspicious.

sudo cat /var/log/elastiflow/flowcoll/flowcoll.log | grep thre

{“level”:“info”,“ts”:“2025-04-14T17:39:14.434Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:285”,“msg”:“initializing threat type collection”}
{“level”:“info”,“ts”:“2025-04-14T17:39:14.435Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/threat_collection.pb”}
{“level”:“info”,“ts”:“2025-04-14T17:39:14.436Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:290”,“msg”:“Threat Type size: 53230 bytes”}
{“level”:“info”,“ts”:“2025-04-14T17:39:14.441Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:264”,“msg”:“initializing ipdb”}
{“level”:“info”,“ts”:“2025-04-14T17:39:14.442Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/ipdb.pb.gz”}
{“level”:“info”,“ts”:“2025-04-14T17:39:23.007Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:269”,“msg”:“ipdb size: 245847242 bytes”}
{“level”:“info”,“ts”:“2025-04-14T17:40:10.077Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:242”,“msg”:“refreshed ipdb & threat collection”}
{“level”:“info”,“ts”:“2025-04-14T18:02:47.901Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:285”,“msg”:“initializing threat type collection”}
{“level”:“info”,“ts”:“2025-04-14T18:02:47.901Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/threat_collection.pb”}
{“level”:“info”,“ts”:“2025-04-14T18:02:47.902Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:290”,“msg”:“Threat Type size: 53230 bytes”}
{“level”:“info”,“ts”:“2025-04-14T18:02:47.904Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:264”,“msg”:“initializing ipdb”}
{“level”:“info”,“ts”:“2025-04-14T18:02:47.905Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/ipdb.pb.gz”}
{“level”:“info”,“ts”:“2025-04-14T18:02:54.390Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:269”,“msg”:“ipdb size: 245847242 bytes”}
{“level”:“info”,“ts”:“2025-04-14T18:03:24.944Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:227”,“msg”:“resources successfully initialized”}
{“level”:“info”,“ts”:“2025-04-14T18:03:24.946Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:247”,“msg”:“running netintel”}
{“level”:“info”,“ts”:“2025-04-14T18:03:45.791Z”,“logger”:“flowcoll”,“caller”:“metrics/queuegauge.go:88”,“msg”:“flow processor to output writer is 90% full. This is normal when the collector is starting. If it persists for hours, it may indicate that you are at your license threshold or your system is under-resourced.”}
{“level”:“info”,“ts”:“2025-04-14T19:03:24.961Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:285”,“msg”:“initializing threat type collection”}
{“level”:“info”,“ts”:“2025-04-14T19:03:24.961Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/threat_collection.pb”}
{“level”:“info”,“ts”:“2025-04-14T19:03:24.964Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:290”,“msg”:“Threat Type size: 53230 bytes”}
{“level”:“info”,“ts”:“2025-04-14T19:03:24.966Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:264”,“msg”:“initializing ipdb”}
{“level”:“info”,“ts”:“2025-04-14T19:03:24.967Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/ipdb.pb.gz”}
{“level”:“info”,“ts”:“2025-04-14T19:03:33.252Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:269”,“msg”:“ipdb size: 245847242 bytes”}
{“level”:“info”,“ts”:“2025-04-14T19:04:10.493Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:242”,“msg”:“refreshed ipdb & threat collection”}
{“level”:“info”,“ts”:“2025-04-14T20:03:24.961Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:285”,“msg”:“initializing threat type collection”}
{“level”:“info”,“ts”:“2025-04-14T20:03:24.963Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/threat_collection.pb”}
{“level”:“info”,“ts”:“2025-04-14T20:03:24.964Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:290”,“msg”:“Threat Type size: 53230 bytes”}
{“level”:“info”,“ts”:“2025-04-14T20:03:24.965Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:264”,“msg”:“initializing ipdb”}
{“level”:“info”,“ts”:“2025-04-14T20:03:24.965Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:367”,“msg”:“fetching resource from file: /var/lib/elastiflow/flowcoll/ipdb.pb.gz”}
{“level”:“info”,“ts”:“2025-04-14T20:03:32.143Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:269”,“msg”:“ipdb size: 245847242 bytes”}
{“level”:“info”,“ts”:“2025-04-14T20:04:09.075Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:242”,“msg”:“refreshed ipdb & threat collection”}
{“level”:“info”,“ts”:“2025-04-14T21:03:24.960Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:285”,“msg”:“initializing threat type collection”}
{“level”:“info”,“ts”:“2025-04-14T21:03:24.962Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:351”,“msg”:“fetching resource from server: RESOURCE_THREAT_COLLECTION”}
{“level”:“info”,“ts”:“2025-04-14T21:03:25.809Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:290”,“msg”:“Threat Type size: 53230 bytes”}
{“level”:“info”,“ts”:“2025-04-14T21:03:25.811Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:264”,“msg”:“initializing ipdb”}
{“level”:“info”,“ts”:“2025-04-14T21:03:25.811Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:351”,“msg”:“fetching resource from server: RESOURCE_IP_DB_COMPRESSED”}
{“level”:“info”,“ts”:“2025-04-14T21:03:51.548Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:269”,“msg”:“ipdb size: 245847242 bytes”}
{“level”:“info”,“ts”:“2025-04-14T21:04:25.890Z”,“logger”:“ipaddr_enricher.netintel_threats”,“caller”:“netintel/enricher.go:242”,“msg”:“refreshed ipdb & threat collection”}

4

You should use “grep -i error” on the log file. If you want to message me the log file I can look through it.

And a screen shot of the missing data/empty dashboard would be helpful, too.

I also don’t see any information in your. configuration file regarding licensing or output. Has anything changed with those settings?

Finally, in my own home lab, with pretty standard home traffic, it will take a day or two for any ‘threats’ to appear simply because I don’t have many.

5

Problem solved. Threats IP reputation was empty because there weren’t any threats. After configuring different Flow Exporter threats were showed immediately.
Thanks Dexter again :smiley:

1 Like