Elasticsearch: HLClient forces using `include_type_name` from 6.7

Created on 5 Apr 2019 · 2Comments · Source: elastic/elasticsearch

Here is a Java code I tested against an elasticsearch node running a 6.7.1.

My pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>fr.pilato.test.elasticsearch</groupId>
    <artifactId>hlclient</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <java.compiler.version>1.8</java.compiler.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.0</version>
                <configuration>
                    <source>${java.compiler.version}</source>
                    <target>${java.compiler.version}</target>
                    <encoding>UTF-8</encoding>
                    <optimize>true</optimize>
                    <showDeprecation>true</showDeprecation>
                    <showWarnings>true</showWarnings>
                    <compilerArgument>-Xlint:all,-serial,-path,-rawtypes,-unchecked</compilerArgument>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>6.6.2</version>
        </dependency>
    </dependencies>

</project>

My code:

package fr.pilato.test.elasticsearch.hlclient;

import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;

import java.io.IOException;

public class App {
    public static void main(String[] args) throws IOException {
        RestHighLevelClient client = new RestHighLevelClient(
            RestClient.builder(HttpHost.create("http://localhost:9200")));
        try {
            CreateIndexRequest cir = new CreateIndexRequest("foo");
            cir.source("{\n" +
                    "  \"mappings\": {\n" +
                    "    \"_doc\": {\n" +
                    "      \"properties\": {\n" +
                    "        \"content\": {\n" +
                    "          \"type\": \"text\"\n" +
                    "        }\n" +
                    "      }\n" +
                    "    }\n" +
                    "  }\n" +
                    "}\n", XContentType.JSON);
            client.indices().create(cir, RequestOptions.DEFAULT);
        } catch (Exception e) {
            e.printStackTrace(System.err);
        } finally {
            client.close();
        }
    }
}

This is running well with the usual deprecation warnings:

AVERTISSEMENT: request [PUT http://localhost:9200/foo?master_timeout=30s&timeout=30s] returned 2 warnings: [299 Elasticsearch-6.7.1-2f32220 "the default number of shards will change from [5] to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you must manage this on the create index request or with an index template"],[299 Elasticsearch-6.7.1-2f32220 "[types removal] The parameter include_type_name should be explicitly specified in create index requests to prepare for 7.0. In 7.0 include_type_name will default to 'false', and requests are expected to omit the type name in mapping definitions."]

When I switch the exact same code to 6.7.1:

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>6.7.1</version>
</dependency>

It's still working well with a slightly different message though:

AVERTISSEMENT: request [PUT http://localhost:9200/foo?master_timeout=30s&include_type_name=true&timeout=30s] returned 1 warnings: [299 Elasticsearch-6.7.1-2f32220 "the default number of shards will change from [5] to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you must manage this on the create index request or with an index template"]

But this line is now deprecated:

client.indices().create(cir, RequestOptions.DEFAULT);

To remove the compilation warning, I need to change this import:

import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;

To:

import org.elasticsearch.client.indices.CreateIndexRequest;

Running the exact same code now fails with:

ElasticsearchStatusException[Elasticsearch exception [type=illegal_argument_exception, reason=The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true.]]
    at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
    at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:2053)
    at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2030)
    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1777)
    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1749)
    at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1708)
    at org.elasticsearch.client.IndicesClient.create(IndicesClient.java:152)
    at fr.pilato.test.elasticsearch.hlclient.App.main(App.java:50)
    Suppressed: org.elasticsearch.client.ResponseException: method [PUT], host [http://localhost:9200], URI [/foo?master_timeout=30s&include_type_name=false&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."}],"type":"illegal_argument_exception","reason":"The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."},"status":400}
        at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:936)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:233)
        at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764)
        ... 4 more
    Caused by: org.elasticsearch.client.ResponseException: method [PUT], host [http://localhost:9200], URI [/foo?master_timeout=30s&include_type_name=false&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."}],"type":"illegal_argument_exception","reason":"The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true."},"status":400}
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:552)
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:537)
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119)
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177)
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
        at java.base/java.lang.Thread.run(Thread.java:844)

:CorFeatureJava High Level REST Client >bug >regression

Source

dadoonet

Most helpful comment

@dadoonet - thank you for the very detailed bug report !

This is abit confusing, so I had to make my self a table :)

| |6.7|7.0|org.elasticsearch.client.indices.CreateIndexRequest| org.elasticsearch.action.admin.indices.create.CreateIndexRequest*|
|------|----|---|---------------------------------------------------|------------------------------------------------------------------|
|include_type_name |true |false|false|true

* deprectated

The true or false in the table represents the default value of include_type_name. The idea is that in 6.7 to raise visibility if you are using a type (for specific APIs such as this). This is to prepare for 7.0 were types are deprecated and eventually removed in 8.0.

You are correct that the Java API is deprecated, and you can't simply switch to the non-deprecated version AND send a type. This is intentional (though admittedly a bit confusing). The Java API is raising the awareness that you are still using a type in this call. (_doc is still considered a type).

Switching to the non-deprecated version requires removal of the type. Since you are already using _doc (the implicit type) this should be as simple as updating to org.elasticsearch.client.indices.CreateIndexRequest and removing the _doc. Once you do that everything should work, and you are also prepared for 7.x. (If you weren't already using _doc you would need update your queries and ensure that you write a new index or re-index with a typeless template)

```java
import org.apache.http.HttpHost;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.common.xcontent.XContentType;

import java.io.IOException;

public class App {
public static void main(String[] args) throws IOException {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(HttpHost.create("http://localhost:55555")));
try {
CreateIndexRequest cir = new CreateIndexRequest("foo5");
cir.source("{\n" +
" \"mappings\": {\n" +
" \"properties\": {\n" +
" \"content\": {\n" +
" \"type\": \"text\"\n" +
" }\n" +
" }\n" +
" }\n" +
"}\n", XContentType.JSON);
client.indices().create(cir, RequestOptions.DEFAULT);
} catch (Exception e) {
e.printStackTrace(System.err);
} finally {
client.close();
}
}
}