Snmp version 3 doesn't work with EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_*

ElastiFlow Community
1

I’m trying to use netbox_get_device-v3.py from ElastiFlow-Tools, to create the snmp config file for EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_PATH, or wherever flowcoll might read this information from by default.

When configuring snmp version 3 in /etc/elastiflow/flowcoll.yml like so, it works:

#EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_ENABLE: "true"
#EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_PATH: /etc/elastiflow/snmp/defaults/device.yml
#EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_REFRESH_RATE: 15
#EF_PROCESSOR_ENRICH_NETIF_SNMP_COMMUNITIES: public
EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE: "true"
EF_PROCESSOR_ENRICH_NETIF_SNMP_PORT: 161
EF_PROCESSOR_ENRICH_NETIF_SNMP_RETRIES: 2
EF_PROCESSOR_ENRICH_NETIF_SNMP_TIMEOUT: 1000
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PASSPHRASE: "********"
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PROTOCOL: SHA-256
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PASSPHRASE: "********"
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PROTOCOL: AES
EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_USERNAME: "********"
EF_PROCESSOR_ENRICH_NETIF_SNMP_VERSION: 3

But, when trying to pull all details from /etc/elastiflow/snmp/defaults/device.yml, errors are thrown due to snmp_version being included in this YAML file.

Flowcoll config when using this device.yml file:

EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_ENABLE: "true"
EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_PATH: /etc/elastiflow/snmp/defaults/device.yml
EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_REFRESH_RATE: 15
#EF_PROCESSOR_ENRICH_NETIF_SNMP_COMMUNITIES: public
EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE: "true"
#EF_PROCESSOR_ENRICH_NETIF_SNMP_PORT: 161
#EF_PROCESSOR_ENRICH_NETIF_SNMP_RETRIES: 2
#EF_PROCESSOR_ENRICH_NETIF_SNMP_TIMEOUT: 1000
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PASSPHRASE: "********"
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_AUTHENTICATION_PROTOCOL: SHA-256
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PASSPHRASE: "********"
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_PRIVACY_PROTOCOL: AES
#EF_PROCESSOR_ENRICH_NETIF_SNMP_V3_USERNAME: "********"
#EF_PROCESSOR_ENRICH_NETIF_SNMP_VERSION: 3

And including version = 3 in a file referenced by EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_PATH, then flowcoll throws this error:

Oct 29 16:59:10 server.domain.com flowcoll[2810474]: {"level":"error","ts":"2025-10-29T16:59:10.210+0100","logger":"ipaddr_enricher.netintel_threats","caller":"netintel/enricher.go:225","msg":"error initializing ipdb & threat collection","error":"error initializing threat collection: failed to unmarshal threat collection bytes: proto: cannot parse invalid wire-format data","stacktrace":"github.com/elastiflow/go-enrich-ipaddr/netintel.(*NetIntel).Run\n\t/go/pkg/mod/github.com/elastiflow/go-enrich-ipaddr@v1.2.4/netintel/enricher.go:225\ngithub.com/elastiflow/go-enrich-ipaddr/enrichipaddr.New\n\t/go/pkg/mod/github.com/elastiflow/go-enrich-ipaddr@v1.2.4/enrichipaddr/enrichipaddr.go:147\ngithub.com/elastiflow/flowcoll/pkg/processors/flowprocessor/cached.(*Components).generateEnrichIPAddrCache\n\t/tmp/collectors/pkg/processors/flowprocessor/cached/cached.go:227\ngithub.com/elastiflow/flowcoll/pkg/processors/flowprocessor/cached.(*Components).buildCaches\n\t/tmp/collectors/pkg/processors/flowprocessor/cached/cached.go:93\ngithub.com/elastiflow/flowcoll/pkg/processors/flowprocessor/cached.NewCachedComponents\n\t/tmp/collectors/pkg/processors/flowprocessor/cached/cached.go:58\ngithub.com/elastiflow/flowcoll/pkg/processors/flowprocessor.NewInstantiatorRegistration.newCrea
teInstanceFunc.func1\n\t/tmp/collectors/pkg/processors/flowprocessor/instance_registration.go:158\ngithub.com/elastiflow/go-env-conf/instantiator.(*Instantiator).Run\n\t/go/pkg/mod/github.com/elastiflow/go-env-conf@v0.8.11/instantiator/instantiator.go:81\ngithub.com/elastiflow/flowcoll/pkg/apps/unified_flowcoll.(*App).Run\n\t/tmp/collectors/pkg/apps/unified_flowcoll/app.go:170\nmain.main\n\t/tmp/collectors/cmd/flowcoll/main.go:106\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:272"}

Removing version altogether from devices.yml allows flowcoll to start, but it will then fall back to snmp version 2c. And setting snmp version 3 in flowcall config as the default (EF_PROCESSOR_ENRICH_NETIF_SNMP_VERSION: 3) doesn’t work, it will still use 2c.

If device.yml contains version = ‘3’ a different error is thrown as detailed here.

For completeness, this is an example of a device.yml file created by netbox_get_device-v3.py:

fw10.site:
  device_groups:
  - paloalto_firewall
  exponential_timeout: false
  ip: 10.0.0.4
  poll_interval: 60
  port: 161
  retries: 2
  timeout: 1000
  v3_credentials:
    authentication_passphrase: ********
    authentication_protocol: SHA-256
    privacy_passphrase: ********
    privacy_protocol: AES
    username: ********
  version: 3

How can I get flowcoll to behave without falling back to snmp version 2c? I was verifying snmp packets with tcpdump and can reliably say that snmp version 3 is only used when including the SNMP account in the flowcoll config file. Which makes it impossible to use different snmp credentials for different categories of devices.

It would also be great if flowcoll could cope with the snmp-version being parsed as a string '3’, so it doesn’t throw this error:

Oct 29 16:00:29 server.domain.com flowcoll[2740733]: {"level":"panic","ts":"2025-10-29T16:00:29.757+0100","logger":"flowcoll","caller":"flowprocessor/instance_registration.go:160","msg":"failed to initialize cached components","code":"flowprocessor/cached-error","reason":"error building caches: error generating enrich netif cache: error creating enrichNetif: Network Interface SNMP Enricher initialization failed: Could not parse user-defined access values in /etc/elastiflow/snmp/defaults/device.yml: yaml: unmarshal errors:\n  line 16: cannot unmarshal !!str `3` into uint8\n  line 32: cannot unmarshal !!str `3` into uint8\n  line 48: cannot unmarshal !!str `3` into uint8\n  line 64: cannot unmarshal !!str `3` into uint8","stacktrace":"github.com/elastiflow/flowcoll/pkg/processors/flowprocessor.NewInstantiatorRegistration.newCreateInstanceFunc.func1\n\t/tmp/collectors/pkg/processors/flowprocessor/instance_registration.go:160\ngithub.com/elastiflow/go-env-conf/instantiator.(*Instantiator).Run\n\t/go/pkg/mod/github.com/elastiflow/go-env-conf@v0.8.11/instantiator/instantiator.go:81\ngithub.com/elastiflow/flowcoll/pkg/apps/unified_flowcoll.(*App).Run\n\t/tmp/collectors/pkg/apps/unified_flowcoll/app.go:170\nmain.main\n\t/tmp/collectors/cmd/flowcoll/main.go:106\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:272"}

I know this is primarily down to how Python handles variables, but it shows that some data sanitisation wouldn’t go amiss.

2

What I’m seeing looks similar to this post, but I’ve got some logging and further detail to add.

The two-link maximum restriction prevented me from including this line in the post.

3

Trying to use the example file included in flowcoll also fails with the same error:

default:
  enabled: false
  port: 161
  retries: 1
  timeout: 20
  v3_credential:
    username: '********'
    authentication_passphrase: ********
    authentication_protocol: SHA-256
    privacy_passphrase: ********
    privacy_protocol: 'AES'
  version: 3

10.0.0.0/8:
  enabled: true

With this in the config file: EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_PATH: /etc/elastiflow/settings/snmp_access.yml

4

authentication_protocol and privacy_protocol need to be lowercase and according to the expected values detailed in the documentation. But, making these changes does not resolve the issue of flowcoll falling back to snmp version 2c when version 3 is specified.

5

The netbox_get_device-v3.py tool is not a supported or maintained product so I really don’t have any insight to add on its usage or effectiveness. I did not know it existed until today.

The “error initializing ipdb & threat collection” error is unrelated to SNMP enrichment. This is the flow collector attempting to connect to the NetIntel server to download the NetIntel dataset for AS lookup enrichment and security threats.

In my local lab I have the following in /etc/elastiflow/settings/snmp_access.yml:

default:
  enabled: true
  port: 161
  version: 2
  community: 295mgrdTX
  retries: 1
  timeout: 2

127.0.0.1:
  version: 3
  v3_credential:
    username: dxturner
    authentication_protocol: sha
    authentication_passphrase: *********
    privacy_protocol: aes
    privacy_passphrase: *********

The ‘default’ section works for all of the SNMPv2 devices in my local network. The entry for 127.0.0.1 is because I have SNMPv3 enabled on the server running the flow collector and I generate synthetic flow traffic for testing.

The only uncommented SNMP enrichment settings in flowcoll.yml are:

EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_ENABLE: “true”
EF_PROCESSOR_ENRICH_NETIF_SNMP_ACCESS_PATH: /etc/elastiflow/settings/snmp_access.yml
EF_PROCESSOR_ENRICH_NETIF_SNMP_ENABLE: “true”

I have found that if you use ‘snmp_access.yml’ it is best to put a ‘default’ section in that contains the most used values, and then add custom settings for the devices that do not use those values.

6

Thank you, I will give that a go again and see if I can get snmp v3 working.

The Python script is easy to red and update and I’ve already proposed some fixes, so if it’s unmaintained I may well clone it and update/fix it for my needs.

7

Right, it seems I had the wrong end of the stick. The Python script I found and referenced is meant for Elastiflow SNMP collector and not flowcoll. Much simpler configuring network devices manually in /etc/elastiflow/settings/snmp_access.yml and this is working fine now using CDIR notation to specify credentials for different appliances.

@dxturner Thank you!

1 Like