Error creating index, an index exists with the same name as the alias

ElastiFlow Community
1

Hi all,

I am using two Elastiflow instances with Docker, sharing the same configuration, and a Virtual IP (VIP) that migrates between the two instances.

When the VIP migrates, I encounter an error: “An index exists with the same name as the alias.”
I understand that I can use OUTPUT_OPENSEARCH_INDEX_SUFFIX to differentiate the index names, but I would prefer to keep all Netflow data in the same index.

Is there a way to achieve this, or is using a suffix the only solution to this problem?

2

It would be helpful to see the full error or logs, but you may want to only push templates from one of the collectors. See this setting documentation:

3

Hi, thanks for your attention. Here is the full error:
{“level”:“error”,“ts”:“2025-01-21T15:03:39.611Z”,“logger”:“bootstrapper[opensearch]”,“caller”:“opensearch/bootstrap.go:125”,“msg”:“failed to bootstrap opensearch. retrying…”,“code”:“opensearch/bootstrap-failure”,
“reason”:"error while creating write indices: error creating write index: error inserting index ‘elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover-000001’:
error creating index: {"index":"elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover","index_uuid":"W595hKMzSLWd6X-M89AyPA","reason":
"Invalid alias name [elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover], an index exists with the same name as the alias","root_cause":[{"index":
"elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover","index_uuid":"W595hKMzSLWd6X-M89AyPA","reason":
“Invalid alias name [elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover], an index exists with the same name as the alias",
"type":"invalid_alias_name_exception"}],"type":"invalid_alias_name_exception"}”,“stacktrace”:“github.com/elastiflow/flowcoll/pkg/outputs/opensearch.
(*Bootstrap).Run\n\t/app/pkg/outputs/opensearch/bootstrap.go:125\ngithub.com/elastiflow/flowcoll/pkg/outputs/opensearch.NewInstantiatorRegistration.newCreateInstanceFunc.
func1\n\t/app/pkg/outputs/opensearch/instance_registration.go:159\ngithub.com/elastiflow/go-env-conf/instantiator.(*Instantiator).Run\n\t/go/pkg/mod/github.com/elastiflow/go-env-conf@v0.8.6/
instantiator/instantiator.go:78\ngithub.com/elastiflow/flowcoll/pkg/apps/unified_flowcoll.(*App).Run\n\t/app/pkg/apps/unified_flowcoll/app.go:156\nmain.main\n\t/app/cmd/flowcoll/main.go:108
\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:271”}

Here is my elastiflow env:
EF_LICENSE_ACCEPTED: ‘true’
EF_FLOW_SERVER_UDP_IP: ‘0.0.0.0’
EF_FLOW_SERVER_UDP_PORT: 2055
EF_LOGGER_LEVEL: ‘error’
EF_LOGGER_FILE_LOG_MAX_SIZE: 100
EF_LOGGER_FILE_LOG_MAX_BACKUPS: 4
EF_OUTPUT_OPENSEARCH_ENABLE: ‘true’
EF_OUTPUT_OPENSEARCH_ECS_ENABLE: ‘true’
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_SHARDS: 1
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_REPLICAS: 0
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_ISM_POLICY: ‘delete-netflow-24h’
EF_OUTPUT_OPENSEARCH_ADDRESSES: ‘opensearchnode-001:9200,opensearchnode-002:9200,opensearchnode-003:9200,opensearchnode-004:9200’
EF_OUTPUT_OPENSEARCH_USERNAME: ‘some_user’
EF_OUTPUT_OPENSEARCH_PASSWORD: ‘some_pass’
EF_OUTPUT_OPENSEARCH_TLS_ENABLE: ‘true’
EF_OUTPUT_OPENSEARCH_INDEX_SUFFIX: ‘netflow-logs’
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_CODEC: ‘best_compression’
EF_OUTPUT_OPENSEARCH_TLS_CA_CERT_FILEPATH: ‘/etc/elastiflow/opensearch_cacert.pem’
EF_OUTPUT_OPENSEARCH_INDEX_PERIOD: ‘rollover’
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_OVERWRITE: ‘true’

4

Hi, thanks for your attention.

Here is the full logs:

{"level":"error","ts":"2025-01-21T15:03:39.611Z","logger":"bootstrapper[opensearch]","caller":"opensearch/bootstrap.go:125","msg":"failed to bootstrap opensearch. retrying...","code":"opensearch/bootstrap-failure",
"reason":"error while creating write indices: error creating write index: error inserting index 'elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover-000001': 
error creating index: {\"index\":\"elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover\",\"index_uuid\":\"W595hKMzSLWd6X-M89AyPA\",\"reason\":
\"Invalid alias name [elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover], an index exists with the same name as the alias\",\"root_cause\":[{\"index\":
\"elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover\",\"index_uuid\":\"W595hKMzSLWd6X-M89AyPA\",\"reason\":\
"Invalid alias name [elastiflow-flow-ecs-8.0-2.4-netflow-logs-rollover], an index exists with the same name as the alias\",
\"type\":\"invalid_alias_name_exception\"}],\"type\":\"invalid_alias_name_exception\"}","stacktrace":"github.com/elastiflow/flowcoll/pkg/outputs/opensearch.
(*Bootstrap).Run\n\t/app/pkg/outputs/opensearch/bootstrap.go:125\ngithub.com/elastiflow/flowcoll/pkg/outputs/opensearch.NewInstantiatorRegistration.newCreateInstanceFunc.
func1\n\t/app/pkg/outputs/opensearch/instance_registration.go:159\ngithub.com/elastiflow/go-env-conf/instantiator.(*Instantiator).Run\n\t/go/pkg/mod/github.com/elastiflow/go-env-conf@v0.8.6/
instantiator/instantiator.go:78\ngithub.com/elastiflow/flowcoll/pkg/apps/unified_flowcoll.(*App).Run\n\t/app/pkg/apps/unified_flowcoll/app.go:156\nmain.main\n\t/app/cmd/flowcoll/main.go:108
\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:271"}

My elastiflow environment is:

EF_LICENSE_ACCEPTED: 'true'
EF_FLOW_SERVER_UDP_IP: '0.0.0.0'
EF_FLOW_SERVER_UDP_PORT: 2055
EF_LOGGER_LEVEL: 'error'
EF_LOGGER_FILE_LOG_MAX_SIZE: 100
EF_LOGGER_FILE_LOG_MAX_BACKUPS: 4
EF_OUTPUT_OPENSEARCH_ENABLE: 'true'
EF_OUTPUT_OPENSEARCH_ECS_ENABLE: 'true'
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_SHARDS: 1
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_REPLICAS: 0
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_ISM_POLICY: 'delete-netflow-24h'
EF_OUTPUT_OPENSEARCH_ADDRESSES: 'opensearchnode-001:9200,opensearchnode-002:9200,opensearchnode-003:9200,opensearchnode-004:9200'
EF_OUTPUT_OPENSEARCH_USERNAME: 'some_user'
EF_OUTPUT_OPENSEARCH_PASSWORD: 'some_pass'
EF_OUTPUT_OPENSEARCH_TLS_ENABLE: 'true'
EF_OUTPUT_OPENSEARCH_INDEX_SUFFIX: 'netflow-logs'
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_CODEC: 'best_compression'
EF_OUTPUT_OPENSEARCH_TLS_CA_CERT_FILEPATH: '/etc/elastiflow/opensearch_cacert.pem'
EF_OUTPUT_OPENSEARCH_INDEX_PERIOD: 'rollover'
EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_OVERWRITE: 'true'

Is there something i missing?

5

Sorry, Akismet has hidden my answer.I’m not sure how long it will take to be reviewed.

6

The EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_ENABLE option enables the writing of templates for Opensearch templates. You may want to enable that option in only one of your collectors, or set the EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_OVERWRITE to true on one collector and false on the other.

The other option, as you mentioned, is to use the suffix option, which is what many people running more than one collector do. This helps to keep the data separate and allows each collector to manage its own index templates.

Let us know if this is helpful.

Regards,
Dexter

7

Hi,

Thanks very much, it works perfectly now with adding EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_ENABLE and EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_OVERWRIT set to true in one collector and set to false on the other one.