Elasticsearch: vectors/10_dense_vector_basic/Deprecated function signature fails on 7.x

[mayya-sharipova]

Build: https://gradle-enterprise.elastic.co/s/khl2jslp4kl76

REPRODUCE WITH:
./gradlew ':x-pack:plugin:integTestRunner' --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=vectors/10_dense_vector_basic/Deprecated function signature}" -Dtests.seed=F523C89C12CE161 -Dtests.security.manager=true -Dtests.locale=de-GR -Dtests.timezone=Pacific/Ponape -Dcompiler.java=13 -Druntime.java=8 -Dtests.rest.blacklist=getting_started/10_monitor_cluster_health/*

[elasticmachine]

Pinging @elastic/es-search (:Search/Search)

[mayya-sharipova]

closed with pr #50717

[pgomulka]

I just had this falure on my PR with freshly fetched 7.x
https://gradle-enterprise.elastic.co/s/umd5jkgvada6a
both of these failed. Similarly they failed last time together.

org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=vectors/10_dense_vector_basic/Deprecated function signature}
org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=vectors/35_sparse_vector_l1l2/L1 norm}

./gradlew ':x-pack:plugin:integTestRunner' --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=vectors/10_dense_vector_basic/Deprecated function signature}" -Dtests.seed=66BC9C4482030E81 -Dtests.security.manager=true -Dtests.locale=el-CY -Dtests.timezone=America/Cambridge_Bay -Dcompiler.java=13 -Druntime.java=8 -Dtests.rest.blacklist=getting_started/10_monitor_cluster_health/*

./gradlew ':x-pack:plugin:integTestRunner' --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=vectors/10_dense_vector_basic/Deprecated function signature}" -Dtests.seed=66BC9C4482030E81 -Dtests.security.manager=true -Dtests.locale=el-CY -Dtests.timezone=America/Cambridge_Bay -Dcompiler.java=13 -Druntime.java=8 -Dtests.rest.blacklist=getting_started/10_monitor_cluster_health/*
both do not reproduce localy for me

[mayya-sharipova]

The errors about not getting expected warning headers.
Looks like failing tests always involved 2 shards.

"_shards" : {
        "total" : 2,
        "successful" : 2,
        "skipped" : 0,
        "failed" : 0
      },

May be worth to limit indexes to a single shard to ensure to get an expected warning?

[mark-vieira]

I'm seeing a few more of these:

https://gradle-enterprise.elastic.co/s/eegnhjtnxt3ew/tests/fcijb24gqtmnc-76jrmwenri5xm
https://gradle-enterprise.elastic.co/s/dn33qhkbnoxl2/tests/by-project?search=test+%7Bp0%3Dvectors%2F30_sparse_vector_basic%2FCosine+Similarity%7D

[cbuescher]

Most likely related, also failures in 35_sparse_vector_l1l2
https://gradle-enterprise.elastic.co/s/wni2k22lifatu/console-log?task=:x-pack:plugin:integTestRunner

REPRODUCE WITH: ./gradlew ':x-pack:plugin:integTestRunner' --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=vectors/35_sparse_vector_l1l2/L2 norm}" -Dtests.seed=8DEC507297BA3C82 -Dtests.security.manager=true -Dtests.locale=en-CA -Dtests.timezone=America/St_Vincent -Dcompiler.java=13 -Druntime.java=14 -Dtests.rest.blacklist=getting_started/10_monitor_cluster_health/*

org.elasticsearch.xpack.test.rest.XPackRestIT > test {p0=vectors/35_sparse_vector_l1l2/L2 norm} FAILED
    java.lang.AssertionError: Failure at [vectors/35_sparse_vector_l1l2:80]: did not get expected warning header [
        The [sparse_vector] field type is deprecated and will be removed in 8.0.
    ]

[cbuescher]

Also https://gradle-enterprise.elastic.co/s/p6h4e2nuvyjk2 on 7.6

[jtibshirani]

May be worth to limit indexes to a single shard to ensure to get an expected warning?

Thanks @mayya-sharipova for debugging this, it indeed seems deprecation warnings can be dropped when merging results across shards (#33936). Although the default number of shards in 7.x is 1, we randomly use 2 shards instead in REST tests to make sure we have good coverage. I'll submit a PR to hardcode the number of shards to 1 in vector tests.

[jtibshirani]

I merged #51643 to address the issue. I'll close this for now, but please re-open it if the failures come up again.