Bazel: incompatible_depset_is_not_iterable: Make depset not iterable

[c-parsons]

This is a tracking issue for offering a migration solution for
--incompatible_depset_is_not_iterable

This flag would enforce that depset is no longer treatable as an iterable. Thus, len(depset()) would throw an error.
This is useful in it prevents users from using this gravely-inefficient construct (expansion of depsets should be avoidable at all costs, as they are incredibly expensive operations)

[dslomov]

What is missing for migration: migration docs, length of migration window. After these are done, please add "migration-ready" label

[manekinekko]

Hi, how would you coerce a depset to a flat list without using calling .to_list() (not recommended) nor having to iterate?

[c-parsons]

The recommendation is to avoid flattening a depset in Starlark at all costs -- depsets can be passed to action creation directly (via actions.run)

[m3rcuriel]

I'm curious then as to what the recommended syntax would be for checking if a label exists in a depset, i.e what would formerly be if blah in your_depset

[c-parsons]

From https://docs.bazel.build/versions/master/skylark/lib/depset.html :

Depsets are not implemented as hash sets and do not support fast membership tests. If you need a general set datatype, you can simulate one using a dictionary where all keys map to True.

You cannot check if a label exists in a depset without flattening it.

[laurentlb]

Status update. Flag flip still blocked by:

• rules_foreign_cc
• rules_typescript
• rules_webtesting
• Gerrit (cc @davido)

[davido]

@laurentlb My impression was, that it is fixed in all our transitive dependencies and in our own build tool chain. Can you point me to a log, specific to this breakage?

[laurentlb]

@davido See https://buildkite.com/bazel/bazelisk-plus-incompatible-flags/builds/114#e39742ca-35a8-48d6-9ce9-68dfbfdce53e

[davido]

Thanks. I see it now:

File "/var/lib/buildkite-agent/builds/bk-docker-n5kr/bazel-downstream-projects/gerrit/tools/bzl/junit.bzl", line 38, in _AsClassName
[x.path for x in fname.files]
type 'depset' is not iterable. Use the `to_list()` method to get a list. Use --incompatible_depset_is_not_iterable=false to temporarily disable this check.

I filed: [1].

• [1] https://bugs.chromium.org/p/gerrit/issues/detail?id=10888

[lberki]

This broke rules_scala (no Travis link, since I realized this after updating it for two other incompatible flags, which also broke). How was this flag flipped then?

(The story (still developing) is under https://github.com/bazelbuild/rules_scala/pull/793; I hope that this was the last flag I have to fix up...)

[laurentlb]

It was discussed by email, but for the record: we test Bazel on BuildKite on many projects.

It looks like rules_scala is not fully tested on BuildKite (some tests are disabled), which can hide some issues.

[Magsun]

Hello to anyone who sees this issue report, I would be so happy if someone can help.
I met this unrecognized option error while I'm trying to convert local video to yt8m format features.
The command line is listed below:
bazel build -c opt \
--define MEDIAPIPE_DISABLE_GPU=1 --define no_aws_support=true \
mediapipe/examples/desktop/youtube8m:extract_yt8m_features

The whole info of this error is:
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=168
INFO: Reading rc options for 'build' from /workspace/paddlevideo/mediapipe/.bazelrc:
'build' options: --jobs 128 --define=absl=1 --cxxopt=-std=c++14 --copt=-Wno-sign-compare --copt=-Wno-unused-function --copt=-Wno-uninitialized --copt=-Wno-unused-result --copt=-Wno-comment --copt=-Wno-return-type --copt=-Wno-unused-local-typedefs --copt=-Wno-ignored-attributes --incompatible_disable_deprecated_attr_params=false --incompatible_depset_is_not_iterable=false --apple_platform_type=macos --apple_generate_dsym
ERROR: Unrecognized option: --incompatible_depset_is_not_iterable=false

My bazel version is 2.0.0 installed by binary .sh file, and sys version is Ubuntu 16.04.
FFMPEG and OPENCV are installed.
Local video to yt8m:https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/youtube8m

[laurentlb]

The flag incompatible_depset_is_not_iterable has been removed (it was useful only for the transition) and shouldn't be used anymore.

Can you remove it? It looks like it comes from /workspace/paddlevideo/mediapipe/.bazelrc.

If the flag was really needed (i.e. you get errors related to depset iteration), you may have to fix the code before using Bazel 2.0. This should work with Bazel 1.2 though.