Spotless: gradletest can't download dependency

[scphantm]

using the very basic example in my enterprise plugin system. whenever i try to activate any of the groovy features, my gradletests spit this error.

Why is this trying to dynamically download this dependency? Why isn't it built into the normal dependencies?

we run a custom version of gradle that has its own init with these repos in it, but it looks like your system doesn't pick those repos up when run inside a gradletest.

project.plugins.apply(SyzygyJavascriptPlugin)
|       |       |
|       |       org.gradle.api.internal.plugins.PluginApplicationException: Failed to apply plugin [class 'com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin']
|       [org.gradle.language.base.plugins.LifecycleBasePlugin@7a83823a, org.gradle.api.plugins.BasePlugin@47dcb632, org.gradle.api.plugins.ReportingBasePlugin@5d4a4daa, org.gradle.platform.base.plugins.ComponentBasePlugin@6ad4ddd1, org.gradle.language.base.plugins.LanguageBasePlugin@11ed620c, org.gradle.platform.base.plugins.BinaryBasePlugin@3227834b, org.gradle.api.plugins.JavaBasePlugin@6f433986, org.gradle.api.plugins.JavaPlugin@4937face, org.gradle.api.plugins.GroovyBasePlugin@36e0d656, org.gradle.api.plugins.GroovyPlugin@69f3b241, org.gradle.plugins.ide.idea.IdeaPlugin@36330d88, org.gradle.plugins.ide.eclipse.EclipsePlugin@1542f688, org.gradle.api.plugins.MavenPlugin@522eb329, org.gradle.api.plugins.ProjectReportsPlugin@4a9c7ca0, com.hmhs.frameworks.syzygy.gradle.core.plugins.SyzygyTasktreePlugin@3ed10838, org.gradle.api.plugins.quality.CodeNarcPlugin@7c7c28f3, org.gradle.api.plugins.quality.CheckstylePlugin@595d49, com.diffplug.gradle.spotless.SpotlessPlugin@69c17f7a]
root project 'test'
    at com.hmhs.frameworks.syzygy.gradle.js.tasks.WebjarTaskTest.can apply javascript plugin(WebjarTaskTest.groovy:13)

    Caused by:
    org.gradle.api.internal.plugins.PluginApplicationException: Failed to apply plugin [class 'com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin']
        at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:150)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:80)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:86)
        at org.gradle.api.internal.plugins.DefaultPluginContainer.apply(DefaultPluginContainer.java:92)
        at com.hmhs.frameworks.syzygy.gradle.lib.ProjectTaskCreaterMethods.addPluginLocal(ProjectTaskCreaterMethods.groovy:58)
        at com.hmhs.frameworks.syzygy.gradle.core.plugins.SyzygyCorePlugin.apply(SyzygyCorePlugin.groovy:60)
        at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:42)
        at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:50)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:164)
        at org.gradle.api.internal.plugins.DefaultPluginManager.access$200(DefaultPluginManager.java:47)
        at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:252)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:80)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:86)
        at org.gradle.api.internal.plugins.DefaultPluginContainer.apply(DefaultPluginContainer.java:92)
        at com.hmhs.frameworks.syzygy.gradle.lib.ProjectTaskCreaterMethods.addPluginLocal(ProjectTaskCreaterMethods.groovy:58)
        at com.hmhs.frameworks.syzygy.gradle.js.plugins.SyzygyJavascriptPlugin.apply(SyzygyJavascriptPlugin.groovy:49)
        at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:42)
        at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:50)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:164)
        at org.gradle.api.internal.plugins.DefaultPluginManager.access$200(DefaultPluginManager.java:47)
        at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:252)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:80)
        at org.gradle.api.internal.plugins.DefaultPluginManager.addImperativePlugin(DefaultPluginManager.java:86)
        at org.gradle.api.internal.plugins.DefaultPluginContainer.apply(DefaultPluginContainer.java:92)
        ... 1 more

        Caused by:
        org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException: Could not resolve all files for configuration ':detachedConfiguration1'.
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.rethrowFailure(DefaultConfiguration.java:995)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$1700(DefaultConfiguration.java:121)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getFiles(DefaultConfiguration.java:969)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.getFiles(DefaultConfiguration.java:428)
            at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolve(DefaultConfiguration.java:424)
            at com.diffplug.gradle.spotless.GradleProvisioner.lambda$fromProject$1(GradleProvisioner.java:42)
            at com.diffplug.spotless.Provisioner.provisionWithDependencies(Provisioner.java:39)
            at com.diffplug.spotless.JarState.from(JarState.java:62)
            at com.diffplug.spotless.extra.groovy.GrEclipseFormatterStep.lambda$create$0(GrEclipseFormatterStep.java:49)
            at com.diffplug.spotless.FormatterStepImpl.calculateState(FormatterStepImpl.java:56)
            at com.diffplug.spotless.LazyForwardingEquality.state(LazyForwardingEquality.java:56)
            at com.diffplug.spotless.LazyForwardingEquality.toBytes(LazyForwardingEquality.java:85)
            at com.diffplug.spotless.LazyForwardingEquality.equals(LazyForwardingEquality.java:94)
            at java.util.ArrayList.indexOf(ArrayList.java:321)
            at com.diffplug.gradle.spotless.FormatExtension.replaceStep(FormatExtension.java:235)
            at com.diffplug.gradle.spotless.GroovyExtension$GrEclipseConfig.configFile(GroovyExtension.java:112)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin$_apply_closure1.closure4$_closure11(SyzygyQualityPlugin.groovy:73)
            at groovy.lang.Closure.call(Closure.java:414)
            at groovy.lang.Closure.call(Closure.java:430)
            at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:71)
            at org.gradle.util.ConfigureUtil.configureTarget(ConfigureUtil.java:155)
            at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:106)
            at org.gradle.util.ConfigureUtil$WrappedConfigureAction.execute(ConfigureUtil.java:167)
            at com.diffplug.gradle.spotless.SpotlessExtension.configure(SpotlessExtension.java:156)
            at com.diffplug.gradle.spotless.SpotlessExtension.groovy(SpotlessExtension.java:108)
            at org.gradle.internal.metaobject.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:479)
            at org.gradle.internal.metaobject.BeanDynamicObject.tryInvokeMethod(BeanDynamicObject.java:191)
            at org.gradle.internal.metaobject.CompositeDynamicObject.tryInvokeMethod(CompositeDynamicObject.java:98)
            at org.gradle.internal.metaobject.MixInClosurePropertiesAsMethodsDynamicObject.tryInvokeMethod(MixInClosurePropertiesAsMethodsDynamicObject.java:30)
            at org.gradle.internal.metaobject.ConfigureDelegate.invokeMethod(ConfigureDelegate.java:57)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin.apply_closure1$_closure4(SyzygyQualityPlugin.groovy:68)
            at groovy.lang.Closure.call(Closure.java:414)
            at groovy.lang.Closure.call(Closure.java:430)
            at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:71)
            at org.gradle.util.ConfigureUtil.configureTarget(ConfigureUtil.java:155)
            at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:106)
            at org.gradle.util.ConfigureUtil$WrappedConfigureAction.execute(ConfigureUtil.java:167)
            at org.gradle.api.internal.plugins.ExtensionsStorage$ExtensionHolder.configure(ExtensionsStorage.java:183)
            at org.gradle.api.internal.plugins.ExtensionsStorage.configureExtension(ExtensionsStorage.java:67)
            at org.gradle.api.internal.plugins.DefaultConvention.configureExtension(DefaultConvention.java:399)
            at org.gradle.api.internal.plugins.DefaultConvention.access$500(DefaultConvention.java:45)
            at org.gradle.api.internal.plugins.DefaultConvention$ExtensionsDynamicObject.tryInvokeMethod(DefaultConvention.java:336)
            at org.gradle.internal.metaobject.CompositeDynamicObject.tryInvokeMethod(CompositeDynamicObject.java:98)
            at org.gradle.internal.metaobject.MixInClosurePropertiesAsMethodsDynamicObject.tryInvokeMethod(MixInClosurePropertiesAsMethodsDynamicObject.java:30)
            at org.gradle.internal.metaobject.AbstractDynamicObject.invokeMethod(AbstractDynamicObject.java:160)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin.apply_closure1(SyzygyQualityPlugin.groovy:63)
            at groovy.lang.Closure.call(Closure.java:414)
            at groovy.lang.Closure.call(Closure.java:430)
            at com.hmhs.frameworks.syzygy.gradle.core.quality.plugins.SyzygyQualityPlugin.apply(SyzygyQualityPlugin.groovy:40)
            at org.gradle.api.internal.plugins.ImperativeOnlyPluginTarget.applyImperative(ImperativeOnlyPluginTarget.java:42)
            at org.gradle.api.internal.plugins.RuleBasedPluginTarget.applyImperative(RuleBasedPluginTarget.java:50)
            at org.gradle.api.internal.plugins.DefaultPluginManager.addPlugin(DefaultPluginManager.java:164)
            at org.gradle.api.internal.plugins.DefaultPluginManager.access$200(DefaultPluginManager.java:47)
            at org.gradle.api.internal.plugins.DefaultPluginManager$AddPluginBuildOperation.run(DefaultPluginManager.java:252)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
            at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
            at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
            ... 36 more

            Caused by:
            org.gradle.internal.resolve.ModuleVersionNotFoundException: Cannot resolve external dependency com.diffplug.spotless:spotless-ext-greclipse:2.3.0 because no repositories are defined.
            Required by:
                project :
                at org.gradle.api.internal.artifacts.ivyservice.ivyresolve.NoRepositoriesResolver.resolve(NoRepositoriesResolver.java:69)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.ComponentResolversChain$DependencyToComponentIdResolverChain.resolve(ComponentResolversChain.java:150)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.SelectorState.resolve(SelectorState.java:125)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.performSelection(DependencyGraphBuilder.java:224)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.performSelectionSerially(DependencyGraphBuilder.java:205)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.resolveEdges(DependencyGraphBuilder.java:194)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.traverseGraph(DependencyGraphBuilder.java:149)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.builder.DependencyGraphBuilder.resolve(DependencyGraphBuilder.java:118)
                at org.gradle.api.internal.artifacts.ivyservice.resolveengine.DefaultArtifactDependencyResolver.resolve(DefaultArtifactDependencyResolver.java:92)
                at org.gradle.api.internal.artifacts.ivyservice.DefaultConfigurationResolver.resolveGraph(DefaultConfigurationResolver.java:150)
                at org.gradle.api.internal.artifacts.ivyservice.ShortCircuitEmptyConfigurationResolver.resolveGraph(ShortCircuitEmptyConfigurationResolver.java:76)
                at org.gradle.api.internal.artifacts.ivyservice.ErrorHandlingConfigurationResolver.resolveGraph(ErrorHandlingConfigurationResolver.java:66)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$4.run(DefaultConfiguration.java:507)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
                at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
                at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveGraphIfRequired(DefaultConfiguration.java:498)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.resolveToStateOrLater(DefaultConfiguration.java:483)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.access$1800(DefaultConfiguration.java:121)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getSelectedArtifacts(DefaultConfiguration.java:978)
                at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$ConfigurationFileCollection.getFiles(DefaultConfiguration.java:966)
                ... 93 more

If you're just submitting a feature request or question, no need for the above.

[jbduncan]

Good question! I've never fully understood this myself.

@scphantm For your information, what the dynamic dependency download is doing is it's trying to download the formatter library used by greclipse to do its Groovy code formatting.

Currently, behind-the-scenes, this dynamic downloading is achieved with a single _detached configuration_ per dependency. I wonder if each of them could be replaced with a named configuration e.g. "spotlessGreclipse", without forcing each dependency to be downloaded earlier than strictly necessary.

@nedtwigg Thoughts?

[nedtwigg]

it looks like your system doesn't pick those repos

Are those repos project repos or buildscript repos? Spotless downloads from the buildscript repos, which can cause some confusion. The history and consequences of this started in #99.

I wonder if each of them could be replaced with a named configuration e.g. "spotlessGreclipse"

They definitely could, but it'd be pretty big change, and probably require a major-version bump and should be discussed in a new issue, iff somebody wants to take that on. I think that approach has some sizable downsides w.r.t. to documentation and portability.

[scphantm]

oh the master init file has both buildscript and normal dependency blocks. im just kinda confused why its not picking it up in the gradletest script. but ive never seen anyone do a dependency like you guys did, so im still a little confused. i will tinker with it today

[scphantm]

i see whats going on now. gradleTest has a lot in it to increase performance of the tests. For example, my plugin has over 300 gradletests in it. So to increase performance, the gradleTest runtime goes out and downloads the dependencies into the .gradle cache at the beginning of the run, then it rewrites a new init.gradle file that it injects into the test run. this init.gradle file eliminates all the repositories and sets everything to file from the cache, that way you don't waste time with the tests searching mvncentral 300 times.

What that does for this plugin is since this plugin uses a rather non-standard way of downloading dependencies, since gradletest nukes the repos, it can't pull down the secondary dependencies you are looking for.

Pretty sure this is going to take a while to fix. I am probably going to have to put a patch into gradletest so that i can add repositories to that init file. But that will be a hard sell.

[nedtwigg]

Spotless works with many formatters that each require conflicting versions of various dependencies. By provisioning jars for each formatter individually, and running them in their own classloader, we can support many versions of many formatters all in one plugin. We're not doing it just to be difficult :)

If you'd like to submit a PR to improve Spotless to use named configurations rather than improve gradleTest, we'd be happy to merge. The improvement to spotless would be a fair amount of work, but if gradletest is hard to patch then maybe it'd be worth your time.

[scphantm]

Oh I get it. One design change I made blew up for trying to load groovy 2.3, gradle runs 2.14. Since it works in its own thing, gradle wasn’t able to resolve the dependency with resolution rules. I get why you did it. Just excercising my right to not be happy about it. ;)

I’m a core dev for gradletest and know that project well. I can knock that change out fairly quickly over the weekend. And I don’t have to learn another code base.

You should check out gradletest. It allows you to write gradle test kit tests as basic build files. It’s pretty awesome.

https://gitlab.com/ysb33rOrg/gradleTest

[scphantm]

in case anyone is wondering, i made the changes to gradletest to do this, we are working on building it now. but i had setup spotless to be disabled when the CI server runs the tests. What happened is with this thing dynamically downloading files, each time, it added over an hour to our unit test run and was blowing up circuit breakers.