Elasticsearch: Building ES plugin for 7.5.0 fails

[consulthys]

Elasticsearch version (bin/elasticsearch --version):
7.5.0

Plugins installed:
[]

JVM version (java -version):
OpenJDK Runtime Environment (build 12.0.2+10)

OS version (uname -a if on a Unix-like system):
Darwin iMac.local 17.7.0 Darwin Kernel Version 17.7.0:

Description of the problem including expected versus actual behavior:
When trying to build the ingest-h3 plugin for the 7.5.0 stack the build fails while it worked perfectly well on 7.4.1/2. Trying to build without test cases using -x test fails as well.

Steps to reproduce:

Checking out the plugin repo and buiding it against the 7.4.1/2 stack works perfectly well and produces this output:

Step 1: Build against 7.4.1/2 => WORKS

iMac:elasticsearch-ingest-h3 consulthys$ ./gradlew clean check

> Task :printGlobalBuildInfo UP-TO-DATE
=======================================
Elasticsearch Build Hamster says Hello!
  Gradle Version        : 5.6.2
  OS Info               : Mac OS X 10.13.6 (x86_64)
  JDK Version           : 12 (Oracle Corporation 12.0.2 [OpenJDK 64-Bit Server VM 12.0.2+10])
  JAVA_HOME             : /Users/consulthys/.sdkman/candidates/java/12.0.2-open
  Random Testing Seed   : 6893B67B7CBCDA19
=======================================

BUILD SUCCESSFUL in 1m 9s
36 actionable tasks: 34 executed, 2 up-to-date

Step 2: Build against 7.5.0 => FAILS

iMac:elasticsearch-ingest-h3 consulthys$ ./gradlew clean check 

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':extractElasticsearch7.5.0'.
> Expected configuration ':elasticsearch_7.5.0_integ_test_zip_null' to contain exactly one file, however, it contains no files.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 601ms

Provide logs (if relevant):

Including the stack trace yields the following:

iMac:elasticsearch-ingest-h3 consulthys$ ./gradlew clean check --stacktrace

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':extractElasticsearch7.5.0'.
> Expected configuration ':elasticsearch_7.5.0_integ_test_zip_null' to contain exactly one file, however, it contains no files.

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.internal.tasks.TaskDependencyResolveException: Could not determine the dependencies of task ':extractElasticsearch7.5.0'.
        at org.gradle.api.internal.tasks.CachingTaskDependencyResolveContext.getDependencies(CachingTaskDependencyResolveContext.java:68)
        at org.gradle.execution.plan.TaskDependencyResolver.resolveDependenciesFor(TaskDependencyResolver.java:46)
        at org.gradle.execution.plan.LocalTaskNode.getDependencies(LocalTaskNode.java:106)
        at org.gradle.execution.plan.LocalTaskNode.resolveDependencies(LocalTaskNode.java:79)
        at org.gradle.execution.plan.DefaultExecutionPlan.addEntryTasks(DefaultExecutionPlan.java:176)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph.addEntryTasks(DefaultTaskExecutionGraph.java:139)
        at org.gradle.execution.TaskNameResolvingBuildConfigurationAction.configure(TaskNameResolvingBuildConfigurationAction.java:49)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter.configure(DefaultBuildConfigurationActionExecuter.java:58)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter.access$200(DefaultBuildConfigurationActionExecuter.java:26)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter$2.proceed(DefaultBuildConfigurationActionExecuter.java:66)
        at org.gradle.execution.DefaultTasksBuildExecutionAction.configure(DefaultTasksBuildExecutionAction.java:45)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter.configure(DefaultBuildConfigurationActionExecuter.java:58)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter.access$200(DefaultBuildConfigurationActionExecuter.java:26)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter$2.proceed(DefaultBuildConfigurationActionExecuter.java:66)
        at org.gradle.execution.ExcludedTaskFilteringBuildConfigurationAction.configure(ExcludedTaskFilteringBuildConfigurationAction.java:48)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter.configure(DefaultBuildConfigurationActionExecuter.java:58)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter.access$200(DefaultBuildConfigurationActionExecuter.java:26)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter$1.run(DefaultBuildConfigurationActionExecuter.java:44)
        at org.gradle.internal.Factories$1.create(Factories.java:26)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry.withLenientState(DefaultProjectStateRegistry.java:133)
        at org.gradle.api.internal.project.DefaultProjectStateRegistry.withLenientState(DefaultProjectStateRegistry.java:125)
        at org.gradle.execution.DefaultBuildConfigurationActionExecuter.select(DefaultBuildConfigurationActionExecuter.java:40)
        at org.gradle.initialization.DefaultTaskExecutionPreparer.prepareForTaskExecution(DefaultTaskExecutionPreparer.java:38)
        at org.gradle.initialization.BuildOperatingFiringTaskExecutionPreparer$CalculateTaskGraph.populateTaskGraph(BuildOperatingFiringTaskExecutionPreparer.java:82)
        at org.gradle.initialization.BuildOperatingFiringTaskExecutionPreparer$CalculateTaskGraph.run(BuildOperatingFiringTaskExecutionPreparer.java:57)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.initialization.BuildOperatingFiringTaskExecutionPreparer.prepareForTaskExecution(BuildOperatingFiringTaskExecutionPreparer.java:45)
        at org.gradle.initialization.DefaultGradleLauncher.prepareTaskExecution(DefaultGradleLauncher.java:206)
        at org.gradle.initialization.DefaultGradleLauncher.doClassicBuildStages(DefaultGradleLauncher.java:142)
        at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:126)
        at org.gradle.initialization.DefaultGradleLauncher.executeTasks(DefaultGradleLauncher.java:106)
        at org.gradle.internal.invocation.GradleBuildController$1.execute(GradleBuildController.java:60)
        at org.gradle.internal.invocation.GradleBuildController$1.execute(GradleBuildController.java:57)
        at org.gradle.internal.invocation.GradleBuildController$3.create(GradleBuildController.java:85)
        at org.gradle.internal.invocation.GradleBuildController$3.create(GradleBuildController.java:78)
        at org.gradle.internal.work.DefaultWorkerLeaseService.withLocks(DefaultWorkerLeaseService.java:189)
        at org.gradle.internal.work.StopShieldingWorkerLeaseService.withLocks(StopShieldingWorkerLeaseService.java:40)
        at org.gradle.internal.invocation.GradleBuildController.doBuild(GradleBuildController.java:78)
        at org.gradle.internal.invocation.GradleBuildController.run(GradleBuildController.java:57)
        at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:31)
        at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
        at org.gradle.launcher.exec.BuildOutcomeReportingBuildActionRunner.run(BuildOutcomeReportingBuildActionRunner.java:63)
        at org.gradle.tooling.internal.provider.ValidatingBuildActionRunner.run(ValidatingBuildActionRunner.java:32)
        at org.gradle.launcher.exec.BuildCompletionNotifyingBuildActionRunner.run(BuildCompletionNotifyingBuildActionRunner.java:39)
        at org.gradle.launcher.exec.RunAsBuildOperationBuildActionRunner$3.call(RunAsBuildOperationBuildActionRunner.java:51)
        at org.gradle.launcher.exec.RunAsBuildOperationBuildActionRunner$3.call(RunAsBuildOperationBuildActionRunner.java:45)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
        at org.gradle.launcher.exec.RunAsBuildOperationBuildActionRunner.run(RunAsBuildOperationBuildActionRunner.java:45)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter$1.transform(InProcessBuildActionExecuter.java:50)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter$1.transform(InProcessBuildActionExecuter.java:47)
        at org.gradle.composite.internal.DefaultRootBuildState.run(DefaultRootBuildState.java:78)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:47)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:31)
        at org.gradle.launcher.exec.BuildTreeScopeBuildActionExecuter.execute(BuildTreeScopeBuildActionExecuter.java:42)
        at org.gradle.launcher.exec.BuildTreeScopeBuildActionExecuter.execute(BuildTreeScopeBuildActionExecuter.java:28)
        at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:78)
        at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:52)
        at org.gradle.tooling.internal.provider.SubscribableBuildActionExecuter.execute(SubscribableBuildActionExecuter.java:59)
        at org.gradle.tooling.internal.provider.SubscribableBuildActionExecuter.execute(SubscribableBuildActionExecuter.java:36)
        at org.gradle.tooling.internal.provider.SessionScopeBuildActionExecuter.execute(SessionScopeBuildActionExecuter.java:68)
        at org.gradle.tooling.internal.provider.SessionScopeBuildActionExecuter.execute(SessionScopeBuildActionExecuter.java:38)
        at org.gradle.tooling.internal.provider.GradleThreadBuildActionExecuter.execute(GradleThreadBuildActionExecuter.java:37)
        at org.gradle.tooling.internal.provider.GradleThreadBuildActionExecuter.execute(GradleThreadBuildActionExecuter.java:26)
        at org.gradle.tooling.internal.provider.ParallelismConfigurationBuildActionExecuter.execute(ParallelismConfigurationBuildActionExecuter.java:43)
        at org.gradle.tooling.internal.provider.ParallelismConfigurationBuildActionExecuter.execute(ParallelismConfigurationBuildActionExecuter.java:29)
        at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:60)
        at org.gradle.tooling.internal.provider.StartParamsValidatingActionExecuter.execute(StartParamsValidatingActionExecuter.java:32)
        at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:55)
        at org.gradle.tooling.internal.provider.SessionFailureReportingActionExecuter.execute(SessionFailureReportingActionExecuter.java:41)
        at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:48)
        at org.gradle.tooling.internal.provider.SetupLoggingActionExecuter.execute(SetupLoggingActionExecuter.java:32)
        at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:68)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:39)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:27)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:35)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.create(ForwardClientInput.java:78)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.create(ForwardClientInput.java:75)
        at org.gradle.util.Swapper.swap(Swapper.java:38)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:75)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.LogAndCheckHealth.execute(LogAndCheckHealth.java:55)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:63)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:82)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:37)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:104)
        at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:52)
        at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:297)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Caused by: java.lang.IllegalStateException: Expected configuration ':elasticsearch_7.5.0_integ_test_zip_null' to contain exactly one file, however, it contains no files.
        at org.gradle.api.internal.file.AbstractFileCollection.getSingleFile(AbstractFileCollection.java:64)
        at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration_Decorated.getSingleFile(Unknown Source)
        at org.elasticsearch.gradle.DistributionDownloadPlugin.lambda$setupRootDownload$1(DistributionDownloadPlugin.java:141)
        at org.gradle.util.GUtil.uncheckedCall(GUtil.java:461)
        at org.gradle.util.DeferredUtil.unpackNestableDeferred(DeferredUtil.java:64)
        at org.gradle.api.internal.file.collections.UnpackingVisitor.add(UnpackingVisitor.java:74)
        at org.gradle.api.internal.file.collections.UnpackingVisitor.add(UnpackingVisitor.java:84)
        at org.gradle.api.internal.file.collections.UnpackingVisitor.add(UnpackingVisitor.java:84)
        at org.gradle.api.internal.file.collections.DefaultConfigurableFileCollection.visitContents(DefaultConfigurableFileCollection.java:209)
        at org.gradle.api.internal.file.CompositeFileCollection.visitDependencies(CompositeFileCollection.java:186)
        at org.gradle.api.internal.file.collections.DefaultConfigurableFileCollection.visitDependencies(DefaultConfigurableFileCollection.java:216)
        at org.gradle.api.internal.file.CompositeFileCollection$1.visitDependencies(CompositeFileCollection.java:135)
        at org.gradle.api.internal.file.CompositeFileTree$FilteredFileTree.visitDependencies(CompositeFileTree.java:128)
        at org.gradle.api.internal.tasks.CachingTaskDependencyResolveContext$TaskGraphImpl.getNodeValues(CachingTaskDependencyResolveContext.java:111)
        at org.gradle.internal.graph.CachingDirectedGraphWalker$GraphWithEmptyEdges.getNodeValues(CachingDirectedGraphWalker.java:213)
        at org.gradle.internal.graph.CachingDirectedGraphWalker.doSearch(CachingDirectedGraphWalker.java:121)
        at org.gradle.internal.graph.CachingDirectedGraphWalker.findValues(CachingDirectedGraphWalker.java:73)
        at org.gradle.api.internal.tasks.CachingTaskDependencyResolveContext.getDependencies(CachingTaskDependencyResolveContext.java:66)
        ... 109 more


* Get more help at https://help.gradle.org

BUILD FAILED in 646ms

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Build)

[spinscale]

Is there anything, that can be done in order to be able to build external plugins against 7.5.0 or is it to wait for 7.5.1?

[consulthys]

Everything I tried so far was unsuccessful, so I guess I'll have to wait for 7.5.1.

[rjernst]

Adding this to the build.gradle will workaround this failure:

configurations.whenObjectAdded { c ->
  if (c.name == 'elasticsearch_7.5.0_integ_test_zip_null') {
    project.dependencies.add('elasticsearch_7.5.0_integ_test_zip_null', 'org.elasticsearch.distribution.integ-test-zip:elasticsearch:7.5.0@zip'
  }
}

Note, however, that does not yet make the integ tests runnable. There seems to be another issue related to testclusters which I am looking into:

* What went wrong:
Execution failed for task ':integTestRunner'.
> Can't wait for `node{::integTest-0}` as it's not started. Does the task have `useCluster` ?

I will reopen this until that is fixed.

[rjernst]

Alright, I will open a PR to fix this for 7.5.1, but in the meantime, here is a workaround. In addition to the whenObjectAdded block above, add these imports to the top of build.gradle:

import org.elasticsearch.gradle.testclusters.TestClustersRegistry
import org.elasticsearch.gradle.testclusters.TestClustersPlugin

And then at the bottom of build.gradle add:

TestClustersRegistry registry = project.rootProject.extensions.create("testClustersRegistry", TestClustersRegistry)
TestClustersPlugin.configureClaimClustersHook(project.gradle, registry)
TestClustersPlugin.configureStartClustersHook(project.gradle, registry)
TestClustersPlugin.configureStopClustersHook(project.gradle, registry)

[consulthys]

Thank you so much @rjernst !

[consulthys]

The trick worked out @rjernst
I could build the plugin with 7.5.0
Much appreciated, thanks!

[consulthys]

@rjernst Thanks for fixing all this. I've tried building the plugin against 7.5.1. The issue related to testclusters doesn't seem to be fixed, though, the build only works by adding what you suggested here at the bottom of the build.gradle file, otherwise I still get the following error:

* What went wrong:
Execution failed for task ':integTestRunner'.
> Can't wait for `node{::integTest-0}` as it's not started. Does the task have `useCluster` ?

Just thought you'd like to know.

[Ethan-Zhang]

@rjernst Thanks for fixing all this. I've tried building the plugin against 7.5.1. The issue related to testclusters doesn't seem to be fixed, though, the build only works by adding what you suggested here at the bottom of the build.gradle file, otherwise I still get the following error:

* What went wrong:
Execution failed for task ':integTestRunner'.
> Can't wait for `node{::integTest-0}` as it's not started. Does the task have `useCluster` ?

Just thought you'd like to know.

@rjernst I got this problem either, is it bug being to fix?

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':integTestRunner'.
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$3.accept(ExecuteActionsTaskExecuter.java:166)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$3.accept(ExecuteActionsTaskExecuter.java:163)
    at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:191)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:156)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:62)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:108)
    at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
    at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:94)
    at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:95)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
    at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
    at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
    at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
    at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
    at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
    at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
    at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
    at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Caused by: java.lang.NullPointerException: Can't wait for `node{::integTest-0}` as it's not started. Does the task have `useCluster` ?
    at org.elasticsearch.gradle.testclusters.ElasticsearchNode.isProcessAlive(ElasticsearchNode.java:1200)
    at org.elasticsearch.gradle.testclusters.TestClusterConfiguration.lambda$waitForConditions$0(TestClusterConfiguration.java:123)
    at org.elasticsearch.gradle.testclusters.TestClusterConfiguration.waitForConditions(TestClusterConfiguration.java:116)
    at org.elasticsearch.gradle.testclusters.ElasticsearchNode.waitForAllConditions(ElasticsearchNode.java:1208)
    at org.elasticsearch.gradle.testclusters.ElasticsearchNode.getAllTransportPortURI(ElasticsearchNode.java:739)
    at org.elasticsearch.gradle.testclusters.ElasticsearchCluster.lambda$writeUnicastHostsFiles$30(ElasticsearchCluster.java:382)
    at org.elasticsearch.gradle.testclusters.ElasticsearchCluster.writeUnicastHostsFiles(ElasticsearchCluster.java:382)
    at org.elasticsearch.gradle.testclusters.ElasticsearchCluster.waitForAllConditions(ElasticsearchCluster.java:421)
    at org.elasticsearch.gradle.testclusters.ElasticsearchCluster.getAllHttpSocketURI(ElasticsearchCluster.java:409)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.elasticsearch.gradle.test.RestIntegTestTask$_closure2.doCall(RestIntegTestTask.groovy:63)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.elasticsearch.gradle.SystemPropertyCommandLineArgumentProvider.lambda$asArguments$0(SystemPropertyCommandLineArgumentProvider.java:21)
    at org.elasticsearch.gradle.SystemPropertyCommandLineArgumentProvider.asArguments(SystemPropertyCommandLineArgumentProvider.java:22)
    at org.gradle.process.internal.DefaultJavaForkOptions.copyTo(DefaultJavaForkOptions.java:220)
    at org.gradle.api.tasks.testing.Test.copyTo(Test.java:566)
    at org.gradle.api.tasks.testing.Test.createTestExecutionSpec(Test.java:578)
    at org.gradle.api.tasks.testing.Test.createTestExecutionSpec(Test.java:138)
    at org.gradle.api.tasks.testing.AbstractTestTask.executeTests(AbstractTestTask.java:451)
    at org.gradle.api.tasks.testing.Test.executeTests(Test.java:613)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:103)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:49)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:42)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:28)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:717)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:684)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$5.run(ExecuteActionsTaskExecuter.java:476)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
    at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:461)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:444)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.access$200(ExecuteActionsTaskExecuter.java:93)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.execute(ExecuteActionsTaskExecuter.java:237)
    at org.gradle.internal.execution.steps.ExecuteStep.lambda$execute$1(ExecuteStep.java:33)
    at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:33)
    at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:26)
    at org.gradle.internal.execution.steps.CleanupOutputsStep.execute(CleanupOutputsStep.java:58)
    at org.gradle.internal.execution.steps.CleanupOutputsStep.execute(CleanupOutputsStep.java:35)
    at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:48)
* Get more help at https://help.gradle.org

[dweiss]

I just tried upgrading to 7.5.2 and removed the hack above but then it results in:
````
FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':integTestRunner'.
  > Can't wait for node{::integTest-0} as it's not started. Does the task have useCluster ?
  ````

With the hack the build passes.
````
// >>> Build HACK for 7.5.2

import org.elasticsearch.gradle.testclusters.TestClustersRegistry
import org.elasticsearch.gradle.testclusters.TestClustersPlugin

configure(rootProject) {
TestClustersRegistry registry = rootProject.extensions.create("testClustersRegistry", estClustersRegistry)
TestClustersPlugin.configureClaimClustersHook(project.gradle, registry)
TestClustersPlugin.configureStartClustersHook(project.gradle, registry)
TestClustersPlugin.configureStopClustersHook(project.gradle, registry)
}

// <<< Build HACK for 7.5.2
````

[rjernst]

@dweiss Please try upgrading to 7.6.0. I did try backporting the fixes into 7.5, but I don't remember if I got all of them. At this point if there is still any issue we would fix in 7.6.

[stefanobranco]

@dweiss Is this solved for you now? I just upgraded to 7.6, and now I'm getting

FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':extractElasticsearch7.6.0'.
> Could not resolve all files for configuration ':elasticsearch_7.6.0_integ_test_zip_null'.
   > Could not find org.elasticsearch.distribution.integ-test-zip:elasticsearch:7.6.0.
     Required by:
         project :

@rjernst I tried the fixes, but those did not work for me either (though that might well be user error on my end). The build hack "works" if I access the TestClustersHookPlugin static class, but does not solve the issue for me. The other one does nothing for me as well.

[dweiss]

Thanks @rjernst Yes, 7.6.0 works without any hacks needed (except for logger check, I'll file a separate issue).

[dweiss]

@stefanobranco Works for me in 7.6.0 without any special treatment:

https://github.com/carrot2/elasticsearch-carrot2/blob/master/build.gradle

[dweiss]

Filed logger issue here.
https://github.com/elastic/elasticsearch/issues/52408

[hosuaby]

@stefanobranco I had that problem that just appeared to be misconfiguration of gradle repositories. Ensure that you included followings repositories in your gradle:

repositories {
    mavenCentral()
    mavenLocal()
    jcenter{
        url "https://jcenter.bintray.com/"
        metadataSources{
            artifact()
        }
    }
}

If you put your plugin in submodule of gradle project, ensure that those repositories are listed in build.gradle of parent project.