Graylog2-server: Server start fails with `Index with name: compound_0 already exists with a different name`

[mpfz0r]

If you upgrade from graylog 3.1.0 to 3.1.1 and use mongodb >= 4.2
the server startup fails, because it tries to create an already existing index with a different name.

2019-09-04 02:35:21,620 ERROR: org.graylog2.bootstrap.CmdLineTool - Guice error (more detail on log level debug): Error injecting constructor, com.mongodb.MongoCommandException: Command failed with error 85: ‘Index with name: compound_0 already exists with a different name’ on server mongodb1:27017. The full response is { “operationTime” : { “$timestamp” : { “t” : 1567564515, “i” : 1 } }, “ok” : 0.0, “errmsg” : “Index with name: compound_0 already exists with a different name”, “code” : 85, “codeName” : “IndexOptionsConflict”, “$clusterTime” : { “clusterTime” : { “$timestamp” : { “t” : 1567564515, “i” : 1 } }, “signature” : { “hash” : { “$binary” : “cYudYhmDdPbXU1WHfTCcoWaXL7s=”, “$type” : “00” }, “keyId” : { “$numberLong” : “6721582427640168450” } } } }

This is caused by the fix for #6322 which we did not test with mongodb 4.2.

https://docs.mongodb.com/manual/reference/command/createIndexes/#index-names

Starting in version 4.2, the createIndexes command and the mongo shell helpers db.collection.createIndex() and db.collection.createIndexes() report an error if you create an index with one name, and then try to create the same index again but with another name.

In previous versions, MongoDB did not create the index again, but would return a response object with ok value of 1 and a note that implied that the index was not recreated. For example:

Possible Solution

Handle the already existing index.

Your Environment

• Graylog Version: 3.1.1
• MongoDB Version: 4.2

[mpfz0r]

Possible (untested) workarounds:

• Manually delete the old index before starting graylog:
  db.processing_status.dropIndex("updated_at_1_input_journal.uncommitted_entries_1_input_journal.written_messages_1m_rate_1")

• Run mongodb with db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } )

[tillo]

I seem to have hit the same kind of failure (at service startup) in the same condition (same versions) and a similar error:

2019-09-05T13:12:08.780+02:00 ERROR [CmdLineTool] Guice error (more detail on log level debug): Error injecting constructor, com.mongodb.MongoCommandException: Command failed with error 85: 'Index with name: compound_0 already exists with a different name' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "Index with name: compound_0 already exists with a different name", "code" : 85, "codeName" : "IndexOptionsConflict" }
2019-09-05T13:12:08.781+02:00 ERROR [Server]

################################################################################

ERROR: Unable to connect to MongoDB. Is it running and the configuration correct?
Details: Command failed with error 85: 'Index with name: compound_0 already exists with a different name' on server localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "Index with name: compound_0 already exists with a different name", "code" : 85, "codeName" : "IndexOptionsConflict" }

The mentioned workarounds don't seem to have any effect.

[tillo]

The mentioned workarounds don't seem to have any effect.

I have to rectify, I didn't select the Graylog database before removing the offending index.
The first workaround (and subsequently the provided patch) is definitely working.

[Shantira]

This bug still presents in new release 3.1.2

[mpfz0r]

@Shantira are you still seeing this error message constantly?
It can happen that it showed up shortly before the migration was executed.

[Shantira]

@Shantira are you still seeing this error message constantly?
It can happen that it showed up shortly before the migration was executed.

No, it's still there on Mongodb 4.2 with graylog in docker container graylog/graylog:3.1.2. With switching on 3.1.0 all works fine.

[kiv-box]

graylog:3.1.2|
mongo:4.1.13

Error shows after update from graylog:3.1.0|,

db.processing_status.dropIndex("updated_at_1_input_journal.uncommitted_entries_1_input_journal.written_messages_1m_rate_1")

and restart fixed the problem