Pub: error publishing linter ("Bad state: Unable to find the context")

[pq]

Hi! I'm trying to publish the latest linter package (0.1.79) but it's failing.

[~/src/repos/linter] (master) $ pub --version

Pub 2.2.0-edge.983b77dda72acd73e1616bb63bf98acd8b5fee82

pub_trace.txt

Thanks for any help!

[pq]

Publishing works w/ 2.1.0.

FWIW @bwilkerson suggested this might be the result of changes to ContextManager @stereotype441 made recently in HEAD...

[stereotype441]

@bwilkerson can you clarify what changes you are thinking of, and what you are seeing in the log that suggests the failure might be related to those changes? AFAICT the last change I made to ContextManager (other than comment changes) was in https://github.com/dart-lang/sdk/commit/993b4caf0e0b7f094ef5bb2ce91152e8c0e1468e, in May.

[jonasfj]

I think we have a similar issue in: https://github.com/dart-lang/pub/issues/2087

[pq]

@stereotype441, @bwilkerson : I may have mischaracterized the issue. Sorry for that. It does persist though!

[pq]

FWIW this is still an issue and I've been resorting to an old version of pub for publishing package:linter. Not a big deal at the moment but probably not desirable in the long run!

[passsy]

I run into the same error with dart 2.2 and dart 2.3 when trying to publish ktdart. I always have to switch to dart 2.1 to publish.

$ pub --version
Pub 2.3.0
$ pub publish -n        
Bad state: Unable to find the context to /Users/pascalwelsch/Projects/kt.dart/test/collection/list_test.dart
package:pub/src/dart.dart 146:5                            AnalysisContextManager._getExistingSession
package:pub/src/dart.dart 112:23                           AnalysisContextManager.parse
package:pub/src/dart.dart 127:16                           AnalysisContextManager.parseImportsAndExports
package:pub/src/validator/strict_dependencies.dart 40:36   StrictDependenciesValidator._findPackages
dart:core                                                  _SyncIterator.moveNext
package:pub/src/validator/strict_dependencies.dart 102:12  StrictDependenciesValidator._validateBenchmarkExampleTestTool
package:pub/src/validator/strict_dependencies.dart 77:5    StrictDependenciesValidator.validate
===== asynchronous gap ===========================
package:pub/src/validator.dart 132:64                      Validator.runAll.<fn>
dart:async                                                 Future.wait
package:pub/src/validator.dart 132:19                      Validator.runAll
package:pub/src/command/lish.dart 156:32                   LishCommand._validate
dart:async                                                 _AsyncAwaitCompleter.start
package:pub/src/command/lish.dart 155:25                   LishCommand._validate
package:pub/src/command/lish.dart 136:15                   LishCommand.run
dart:async                                                 _AsyncAwaitCompleter.start
package:pub/src/command/lish.dart 112:13                   LishCommand.run
package:args/command_runner.dart 194:27                    CommandRunner.runCommand
dart:async                                                 _AsyncAwaitCompleter.start
package:args/command_runner.dart 142:23                    CommandRunner.runCommand
package:pub/src/command_runner.dart 171:39                 PubCommandRunner.runCommand.<fn>
dart:async                                                 new Future.sync
package:pub/src/utils.dart 109:12                          captureErrors.<fn>
dart:async                                                 runZoned
package:pub/src/utils.dart 126:5                           captureErrors
package:pub/src/command_runner.dart 171:13                 PubCommandRunner.runCommand

[pq]

For comparison, here's trace from just now for the linter:

Bad state: Unable to find the context to /Users/pquitslund/src/repos/linter/test/rules/experiments/experiments.dart
package:pub/src/dart.dart 146:5                            AnalysisContextManager._getExistingSession
package:pub/src/dart.dart 112:23                           AnalysisContextManager.parse
package:pub/src/dart.dart 127:16                           AnalysisContextManager.parseImportsAndExports
package:pub/src/validator/strict_dependencies.dart 40:36   StrictDependenciesValidator._findPackages
dart:core                                                  _SyncIterator.moveNext
package:pub/src/validator/strict_dependencies.dart 102:12  StrictDependenciesValidator._validateBenchmarkExampleTestTool
package:pub/src/validator/strict_dependencies.dart 77:5    StrictDependenciesValidator.validate
===== asynchronous gap ===========================
package:pub/src/validator.dart 132:64                      Validator.runAll.<fn>
dart:async                                                 Future.wait
package:pub/src/validator.dart 132:19                      Validator.runAll
package:pub/src/command/lish.dart 156:32                   LishCommand._validate
dart:async                                                 _AsyncAwaitCompleter.start
package:pub/src/command/lish.dart 155:25                   LishCommand._validate
package:pub/src/command/lish.dart 136:15                   LishCommand.run
dart:async                                                 _AsyncAwaitCompleter.start
package:pub/src/command/lish.dart 112:13                   LishCommand.run
package:args/command_runner.dart 194:27                    CommandRunner.runCommand
dart:async                                                 _AsyncAwaitCompleter.start
package:args/command_runner.dart 142:23                    CommandRunner.runCommand
package:pub/src/command_runner.dart 171:39                 PubCommandRunner.runCommand.<fn>
dart:async                                                 new Future.sync
package:pub/src/utils.dart 109:12                          captureErrors.<fn>
dart:async                                                 runZoned
package:pub/src/utils.dart 126:5                           captureErrors
package:pub/src/command_runner.dart 171:13                 PubCommandRunner.runCommand

fyi @jonasfj

[jack-r-warren]

I seem to be getting the same error, but only when I try to publish on Linux. When I publish from Windows locally all is fine, but my deployment script running on Linux on Travis remotely throws up this context error for a test file inside the package's example directory. That same deployment script works fine on another package of mine.

The error appeared in this build; the stack trace is available in this one. Moving down to 2.1.0 does fix the issue.

The package in my case is pretty small; the source is here if that's useful.

[pq]

@jonasfj: you suggested you had an idea for a next step? I'm happy to chip in here if you want to share any thoughts.

[jonasfj]

I think just debugging... feel free to dig in... I don't know when I'll get time to fix this. It's definitely on my list of things I don't want to forget again..

[jonasfj]

ideally, if we can create a minimal example of what goes wrong, and create a test case that would be good.

[pulyaevskiy]

The same error happens when I try to publish latest changes in test_coverage:

Bad state: Unable to find the context to /Users/anatoly/Projects/test-coverage/test/stub_package/test/c.dart
package:pub/src/dart.dart 146:5                            AnalysisContextManager._getExistingSession
package:pub/src/dart.dart 112:23                           AnalysisContextManager.parse
package:pub/src/dart.dart 127:16                           AnalysisContextManager.parseImportsAndExports
package:pub/src/validator/strict_dependencies.dart 40:36   StrictDependenciesValidator._findPackages
dart:core                                                  _SyncIterator.moveNext
package:pub/src/validator/strict_dependencies.dart 102:12  StrictDependenciesValidator._validateBenchmarkExampleTestTool
package:pub/src/validator/strict_dependencies.dart 77:5    StrictDependenciesValidator.validate
===== asynchronous gap ===========================
dart:async                                                 _AsyncAwaitCompleter.completeError
package:pub/src/validator/strict_dependencies.dart         StrictDependenciesValidator.validate

trace.txt

dart --version
Dart VM version: 2.3.0 (Fri May 3 10:32:31 2019 +0200) on "macos_x64"

Is there any way to bypass this validation? I tried --force but it doesn't help.

[jack-r-warren]

@pulyaevskiy A few workarounds:

1. Windows doesn't seem to have the issue
2. Dart 2.1 doesn't seem to have the issue
3. Use a multi-package setup akin to built_value. There isn't a top-level pubspec.yaml; instead, each top-level folder has its own, and those folder are published individually as needed (so built_value is published but built_value_test doesn't need to be). Since the issue seems to be nested packages, this strategy would let you have a stub package for tests that's accessible during development without needing to nest it within test_coverage.

As for 1, there's support for Dart on Windows on Travis CI. I've been using Travis to deploy my packages to pub; I'd be happy to rewrite my script for that in Dart to make it cross-platform.

[pulyaevskiy]

@jack-r-warren Thanks!

(3) seems like too much of a workaround for a bug.

I downloaded 2.1.1 and 2.1.0 of Dart SDK. And 2.1.1 is broken, while 2.1.0 gives me a couple of analyzer warnings:

Suggestions:
* line 1, column 1 of test/stub_package/test/a_test.dart: This package doesn't depend on stub_package.
  import 'package:stub_package/stub_package.dart';
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* line 1, column 1 of test/stub_package/test/nested/b_test.dart: This package doesn't depend on stub_package.
  import 'package:stub_package/stub_package.dart';
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Wondering if Pub relies on dartanalyzer for this work? If so it'd make sense to take into account analysis_options.yaml where I do have an exclude item for stub_package in tests and it works well.

[jack-r-warren]

It isn't dartanalyzer, it is actually the same _validateBenchmarkTestTool that is in the chain of errors for this issue (it is just working correctly here). There isn't much of a way around it performing its checks.

Since your tests just need access to the add function in lib/stub_package.dart, import that file directly and tell dartanalyzer to stand down:

// ignore: avoid_relative_lib_imports
import '../lib/stub_package.dart';

[pulyaevskiy]

it is just working correctly here

Does it mean it’s by design? I’d be surprised if it is. The problem is that Pub doesn’t recognize nested packages and interprets everything as part of the root one.

It shouldn’t be that hard to make it recognize a nested package by presence of pubspec.yaml?

If not that then at least there should be a way for a package maintainer to disable or bypass this error as it completely blocks from being able to publish a new version.

[jack-r-warren]

The inner workings of pub and dependencies have proved above my skill level, but I've been able to reproduce this issue without there actually being another pubspec.yaml or any nested packages: I tried moving all my dependencies to the root pubspec and got the same error.

There may be problematic handling of nested packages, but this issue might not be directly related to that. It is within the realm of expectation for _validateBenchmarkTestTool to complain about things not being depended on in the root package, but those checks shouldn't throw stack traces up on screen on Mac/Linux.

As best as I can tell, there is a chance this issue isn't related to pub. It is _getExistingSession that is throwing the error, but that is because the analyzer package isn't saying that the path has been, well, analyzed. That is determined by the Context class in the path package, and Context has to deal with different platforms representing paths differently.

My guess may well be off-course because I don't have a good Linux environment to work with. Context may be working correctly for all I know; there may be something else platform-dependent somewhere that is changing something about what paths are analyzed or how they are represented.

[jonasfj]

If you remove analysis_options.yaml from the package (or simply remove the analyzer.exclude patterns), then this problem will go away.

I've proposed in https://github.com/dart-lang/pub/pull/2143 that we ignore the analysis_options.yaml from the package.

[johnpryan]

I also ran into this - removing the analyzer.exclude section from analysis_options.yaml works. 👍

Another workaround is to temporarily remove the example/ directory before publishing.

[jonasfj]

Note, this is fixed, it should be part of Dart 2.4 when that comes out.

[evanweible-wf]

@jonasfj we just hit this issue on Dart 2.4 and 2.4.1 – did the fix make it out in 2.4 as expected?

[jonasfj]

@evanweible-wf, yes it should be in 2.4.0: https://github.com/dart-lang/sdk/blob/2.4.0/DEPS#L116

Is it possible that this is a different issue? I tried out linter 0.1.79 as originally reported here, pub publish --dry-run fails prior to 2.4.0.

@evanweible-wf, can you create a minimal example (maybe post it as a gist) and file a new issue?
Ideally prove that pub publish --dry-run fails.

[evanweible-wf]

@jonasfj yeah I'll have to get back to you on that, the package we're hitting this in is about as far from a minimal repro as possible :) I'll see if I can narrow it down, just wanted to confirm first that this fix was released.

[jonasfj]

Hehe, yeah, these things rarely happens in minimal packages :)