vscode 🚀 - workspaceContains starts a search over full workspace, including .git/, node_modules/

I suggest we do the following:

Analyse how many extensions in the marketplace use workspaceContains glob patterns and take a look if anyone of them would be negatively impacted if we would include the search excludes in there.

@sandy081 is looking into some tooling for such use cases.

alexdima on 21 Sep 2017

👍1

@sandy081 can vscode-tools help us analyze this yet?

It would really help if a workspaceContains search would skip files.exclude. But in theory, extensions that activate on SCM files, particular node_modules being present, etc are valid use cases, so I'm curious whether extensions like that actually exist.

We also have https://github.com/Microsoft/vscode/issues/33528 to look at this for findFiles

roblourens on 7 Nov 2017

@roblourens Yes. It supports querying on activationEvent by passing a regEx value. For example

activationEvents:`^workspaceContains.*\\*.csx\`

will give you extensions activating on workspace containing .csx files.

sandy081 on 7 Nov 2017

Thanks! I only see one that depends on **/node_modules. A few others look in node_modules, but with no glob patterns, so it can be done without a search. But I guess it should be consistent whether glob patterns are present or not. One is actually mine, which just activates when a ./node_modules folder is present at all. I don't see any looking at .git/

roblourens on 8 Nov 2017

Is this considered for November 2017? This would be important for me.

jens1o on 21 Nov 2017

I wonder if there is a general way to limit the amount of resources workspaceContains consumes. While useful for medium sized workspaces, it currently is too expensive for large ones.

One option might be to stop the search after some time (e.g., 10s) and treat it as no match (to avoid grabbing more resources).

chrmarti on 27 Feb 2018

@chrmarti I like your suggestion.

I would also add, perhaps there is a depth setting we could use, i.e. search 4 folder levels in, and not deeper, etc.

alexdima on 27 Feb 2018

There is a max depth parameter for Ripgrep. The downside of that would be that it would break scenarios with few files in a deep folder structure. In other cases it might not change much, because already 4 levels can be too many files.

Maybe using file.excludes and search.exclude for filtering and a 10 seconds limit would improve things by using the knowledge in the settings about what is of interest while also having a fallback if that is too slow.

We then need to document that workspaceContains should not be relied on exclusively, which is fair considering how much resources it can take.

chrmarti on 28 Feb 2018

👍2

Yes this is exactly what I want. If you are using the C# extension, and for some reason we can't find a single csproj or .cs file in 10 seconds, then you'll probably open a csharp file at some point (activating the extension by onLanguage) or run a csharp command. If none of this, you probably won't benefit from the extension.

We could also check opened files against the workspaceContains patterns, I don't think we do that currently.

roblourens on 7 Mar 2018

👍1

@chrmarti Maybe opposite of that, I mean let we have search.include instead

Thaina on 6 May 2018

@Thaina search.include wouldn't have sensible defaults, so it would only be of help here if the user configured it which is asking too much from the user for fixing this issue. search.include might make sense on its own, you could open a new issue to track this as a feature request.

chrmarti on 7 May 2018

If you can't limit the scope of the search cache or persist it to disk, please add a setting to disable the cache entirely. #34487 is not an “improvement” in a repo with half a million files after npm install. Rather: to solve a problem I didn't have, it's burning my battery for five minutes while chewing 2GB RAM each time the language server misses a change and forces me to reload.

garthk on 14 Aug 2018

@alexandrudima we can't change the behavior of workspaceContains overnight because it would require extensions to have a fallback activationEvent. Could we track workspaceContains searches that are terminated early, then each time a file is opened, check it against those patterns? Does that make sense?

roblourens on 15 Aug 2018

We could try to limit the search to files that are 1 or 2 levels deep. I intuitively think 2 levels deep should cover most workspaceContains cases.

alexdima on 16 Aug 2018

The minimal change here would be to add a timeout to have a hard limit on the resource consumption and a fallback to matching opened files against the pattern when the timeout hit to avoid regressions.

Limiting the depth of the search and using the file.excludes and search.exclude settings to prune the traversal are improvements on top.

chrmarti on 16 Aug 2018

So this is being tackled for the August release?

jens1o on 16 Aug 2018

I wonder if any extensions use workspaceContains to look for a more specific pattern? foo/bar/**/*.file. That would be defeated by a depth limit. I like the timeout instead of the depth limit because

We don't have to add the depth limit to the extension API
The time that the process is running is the factor that we really care about

I would hesitate to apply files.exclude or search.exclude because that could break extensions. We might want to give extensions more warning on a change like that.

roblourens on 16 Aug 2018

I don't think applying the exclusions is essential. Having the fallback you suggest that matches opened files against the patterns that had their search terminated by the timeout seems more important (and could also be used to buffer other changes like the exclusions).

chrmarti on 17 Aug 2018

What I don't like a lot about the timing suggestion is ... the timing aspect. Imagine you are an end user and half the times you open a specific folder an extension works and half the times it doesn't... But I guess we can live with that for the sake of being fast...

I suggest we use the existing body of extensions to drive this. Thanks to @jrieken here are the extensions that use workspaceContains:

workspaceContains.txt

I've taken the time to select only those that use a glob pattern so we can look over them together. I've also taken the time to group them in "simple file searches" and "problematic":

Simple

766b.go-outliner/package.json
  15,10:         "workspaceContains:**/*.go"

ACharLuk.luna-vscode/package.json
  23,10:         "workspaceContains:*.luna"

axetroy.vscode-auto-schema/package.json
  35,6:     "workspaceContains:appsettings.*.json",
  37,6:     "workspaceContains:*.asmdef",
  49,6:     "workspaceContains:*.cf.json",
  50,6:     "workspaceContains:*.cf.yml",
  51,6:     "workspaceContains:*.cf.yaml",
  55,6:     "workspaceContains:*.sam.json",
  56,6:     "workspaceContains:*.sam.yml",
  57,6:     "workspaceContains:*.sam.yaml",
  83,6:     "workspaceContains:grunt/*.json",
  84,6:     "workspaceContains:*-tasks.json",
  96,6:     "workspaceContains:*.schema.json",
  98,6:     "workspaceContains:*.jsonld",
  99,6:     "workspaceContains:*.patch",
  103,6:     "workspaceContains:*.lsdl.yaml",
  116,6:     "workspaceContains:*.resjson",
  118,6:     "workspaceContains:*.sarif",
  119,6:     "workspaceContains:*.sarif.json",
  122,6:     "workspaceContains:skyuxconfig.*.json",
  125,6:     "workspaceContains:*.map",
  126,6:     "workspaceContains:*.sprite",
  143,6:     "workspaceContains:*.vg",
  144,6:     "workspaceContains:*.vg.json",
  145,6:     "workspaceContains:*.vl",
  146,6:     "workspaceContains:*.vl.json",
  149,6:     "workspaceContains:*.nesting.json",
  150,6:     "workspaceContains:*.vsext",
  152,6:     "workspaceContains:*.webmanifest",
  155,6:     "workspaceContains:*.ckan",
  157,6:     "workspaceContains:*.cryproj"

basys.vscode-basys/package.json
  102,10:         "workspaceContains:**/basys.json",

bmewburn.vscode-intelephense-client/package.json
  23,10:         "workspaceContains:**/*.php"

bung87.rails/package.json
  38,10:         "workspaceContains:**/Gemfile"

camel-tooling.vscode-apache-camel/package.json
  46,6:     "workspaceContains:*.xml"

christianvoigt.argdown-vscode/package.json
  36,10:         "workspaceContains:**/*.{argdown,ad,argdn,adown}"

cyitianyou.autorunscript/package.json
  21,10:         "workspaceContains:*.bat"

cyrilletuzi.angular-schematics/package.json
  41,10:         "workspaceContains:**/angular.json",
  42,10:         "workspaceContains:**/.angular-cli.json"

dail.aconite-devtool/package.json
  14,6:     "workspaceContains:*"

danixeee.textx-ls/package.json
  19,4:         "workspaceContains:**/.txconfig",

DanTup.dart-code/package.json
  36,4:         "workspaceContains:**/pubspec.yaml",
  37,4:         "workspaceContains:**/*.dart"

Dart-Code.dart-code/package.json
  36,4:         "workspaceContains:**/pubspec.yaml",
  37,4:         "workspaceContains:**/*.dart",

docuraio.cm/package.json
  24,10:         "workspaceContains:**/*.cm"

elektronenhirn.bake/package.json
  16,10:         "workspaceContains:**/Project.meta"

felixfbecker.php-intellisense/package.json
  1,696: "workspaceContains:**/*.php"

fmiras.vscode-decentraland-touchbar/package.json
  26,6:     "workspaceContains:**/scene.json"

grays.intersky-icloud-dev-tools/package.json
  15,10:         "workspaceContains:**/iCloud.json",

iakimov.check-npm-updates/package.json
  26,10:         "workspaceContains:**/package.json"

IBM.XMLLanguageSupport/package.json
  12,10:         "workspaceContains:**/*.xml"

igordvlpr.pack-vsix/package.json
  35,4:   "workspaceContains:**/*extension.ts",
  36,4:   "workspaceContains:**/*extension.js",

Ikuyadeu.r/package.json
  33,6:     "workspaceContains:*.r|.rd|.rmd",

ilich8086.ColdFusion/package.json
  31,10:         "workspaceContains:**/*.cfm",
  32,10:         "workspaceContains:**/*.cfml",
  33,10:         "workspaceContains:**/*.cfc"

Ionide.Ionide-fsharp/package.json
  1372,4:       "workspaceContains:**/*.fs",
  1373,4:       "workspaceContains:**/*.fsx",
  1374,4:       "workspaceContains:**/*.fsproj",
  1375,4:       "workspaceContains:**/*.sln",

Ionide.mechanic/package.json
  21,6:     "workspaceContains:**/*.fsproj"

jan-dolejsi.pddl/package.json
  34,6:     "workspaceContains:**/*.pddl",
  35,6:     "workspaceContains:**/*ptest.json",

jasonHzq.swag/package.json
  11,25:   "activationEvents": ["workspaceContains:**/swag-config.json"],

jbelford.angular-components/package.json
  19,10:         "workspaceContains:**/{.angular-cli,angular}.json"

jpahnen.dotnet-cli-wrapper/package.json
  24,10:         "workspaceContains:**/*.csproj",
  25,10:         "workspaceContains:**/*.sln"

KamasamaK.vscode-cflint/package.json
  31,10:         "workspaceContains:**/*.cfm",
  32,10:         "workspaceContains:**/*.cfml",
  33,10:         "workspaceContains:**/*.cfc",

KamasamaK.vscode-cfml/package.json
  30,6:     "workspaceContains:**/*.cfm",
  31,6:     "workspaceContains:**/*.cfml",
  32,6:     "workspaceContains:**/*.cfc"

kdcro101.favorites/package.json
  41,10:         "workspaceContains:**/.*"

kdcro101.typescript-code-navigator/package.json
  27,10:         "workspaceContains:**/.*"

kooiot.iot-editor/package.json
  24,10:         "workspaceContains:**/iot_editor_properties.json"

kuboja.fcs-vscode/package.json
  33,10:         "workspaceContains:**/*.fcs",

LambdaFactory.neptune/package.json
  33,6:     "workspaceContains:**/*.fs",
  34,6:     "workspaceContains:**/*.fsx",
  35,6:     "workspaceContains:**/*.fsproj",
  36,6:     "workspaceContains:**/*.sln",

Levertion.mcjson/package.json
  21,10:         "workspaceContains:*.mcmeta"

manux54.angular-localization-helper/package.json
  36,10:         "workspaceContains:**/*.xlf",

marekkajda.templates-manager/package.json
  14,10:         "workspaceContains:**/*.emailtemplate",
  15,10:         "workspaceContains:**/*.pdftemplate"

markkeaton.netsuitesdf/package.json
  15,10:         "workspaceContains:**/.sdf",
  16,10:         "workspaceContains:**/.sdfcli.json",

milannankov.vscode-ui5/package.json
  32,10:         "workspaceContains:**/manifest.json"

morissonmaciel.typescript-auto-compiler/package.json
  26,10:         "workspaceContains:**/*.ts",
  27,10:         "workspaceContains:**/tsconfig.json"

mrdziuban.good-enough-scala-lsp/package.json
  25,6:     "workspaceContains:**/*.{sc,scala}"

ms-azuretools.vscode-azureeventgrid/package.json
  48,10:         "workspaceContains:**/*.[eE][vV][eE][nN][tT][gG][eE][nN][eE][rR][aA][tT][oO][rR].[jJ][sS][oO]{[nN],[nN][cC]}"

ms-azuretools.vscode-azureterraform/package.json
  28,6:     "workspaceContains:**/*.tf",

ms-vscode.autorest/package.json
  38,6:     "workspaceContains:**/*.json",
  39,6:     "workspaceContains:**/*.yaml",
  40,6:     "workspaceContains:readme.md"

ms-vscode.csharp/package.json
  287,6:     "workspaceContains:project.json",
  288,6:     "workspaceContains:*.csproj",
  289,6:     "workspaceContains:*.sln",
  290,6:     "workspaceContains:*.csx",
  291,6:     "workspaceContains:*.cake",
  292,6:     "workspaceContains:**/*.csproj",
  293,6:     "workspaceContains:**/*.sln",
  294,6:     "workspaceContains:**/*.csx",
  295,6:     "workspaceContains:**/*.cake"

msospo.1es-oss-assistant/package.json
  24,6:     "workspaceContains:**/package.json",
  25,6:     "workspaceContains:**/*.csproj"

nadako.vshaxe/package.json
  54,10:         "workspaceContains:*.hxml"

natemoo-re.vscode-stencil-tools/package.json
  39,6:     "workspaceContains:**/stencil.config.js",
  40,6:     "workspaceContains:**/.stencilTools",

notaus.lively/package.json
  15,10:         "workspaceContains:**/*.html"

olback.bar/package.json
  37,10:         "workspaceContains:**/.vscode/bar.conf.json",

olback.vscode-serial/package.json
  45,10:         "workspaceContains:**/.vscode/serial.json"

openfl.lime-vscode-extension/package.json
  39,10:         "workspaceContains:**/*.hxp",
  40,10:         "workspaceContains:**/*.lime",
  41,10:         "workspaceContains:**/project.xml",
  42,10:         "workspaceContains:**/Project.xml",

Orta.vscode-ios-common-files/package.json
  14,10:         "workspaceContains:Podfile",
  15,10:         "workspaceContains:*.podspec",
  16,10:         "workspaceContains:Fastfile",
  17,10:         "workspaceContains:fastlane/Fastfile"

ortus-solutions.vscode-coldbox/package.json
  18,10:         "workspaceContains:**/*.cfm",
  19,10:         "workspaceContains:**/*.cfml",
  20,10:         "workspaceContains:**/*.cfc"

osyrisrblx.vscode-rofresh/package.json
  23,4:         "workspaceContains:**/rofresh.json",

p1c2u.docker-compose/package.json
  31,10:         "workspaceContains:**/docker-compose.yml",

pca.datapack-helper/package.json
  106,10:         "workspaceContains:**/pack.mcmeta",

PhilAlsford.git-duet-vscode/package.json
  19,10:         "workspaceContains:.git/*/**"

Prisma.vscode-graphql/package.json
  33,6:     "workspaceContains:**/.graphqlconfig",
  34,6:     "workspaceContains:**/.graphqlconfig.yml",
  35,6:     "workspaceContains:**/.graphqlconfig.yaml"

ritwickdey.live-sass/package.json
  32,10:         "workspaceContains:**/*.s[ac]ss",

sasxa-net.angular-gui/package.json
  28,6:     "workspaceContains:**/.angular-cli.json"

semonec.vscode-mbed/package.json
  38,10:         "workspaceContains:**/.mbed",
  39,10:         "workspaceContains:**/.mbed_settings.py",
  40,10:         "workspaceContains:**/mbed_app.json"

SharpNinja.cc65/package.json
  163,10:         "workspaceContains:src/**/*.s",
  164,10:         "workspaceContains:src/**/*.asm",
  165,10:         "workspaceContains:src/**/*.inc",

sjuulwijnia.kx-vscode-angular-context-creator/package.json
  33,10:         "workspaceContains:**/.angular-cli.json"

SqrTT.prophet/package.json
  65,10:         "workspaceContains:**/cartridge/*.properties"

stpn.vscode-graphql/package.json
  37,25:   "activationEvents": ["workspaceContains:**/.graphqlconfig"],

swc.swc-dev/package.json
  21,10:         "workspaceContains:**/Cargo.toml"

thegtproject.hoverhex/package.json
  23,10:         "workspaceContains:*"

tintoy.msbuild-project-tools/package.json
  28,10:         "workspaceContains:NuGet.config",
  29,10:         "workspaceContains:global.json",
  30,10:         "workspaceContains:**/*.*proj",
  31,10:         "workspaceContains:**/*.props",
  32,10:         "workspaceContains:**/*.targets",

TomasHubelbauer.vscode-markdown-link-suggestions/package.json
  26,10:         "workspaceContains:**/*.md",

TomasHubelbauer.vscode-markdown-todo/package.json
  25,10:         "workspaceContains:**/*.md",

tomhodder.much-assembly-required-upload-on-save/package.json
  25,10:         "workspaceContains:**/*.mar"

tsufeki.tenkawa-php/package.json
  36,10:         "workspaceContains:**/*.php"

TysonAndre.php-phan/package.json
  42,6:     "workspaceContains:**/*.php"

vsciot-vscode.azure-iot-edge/package.json
  44,6:     "workspaceContains:**/deployment.template.json"

vsciot-vscode.vscode-iot-workbench/package.json
  25,6:     "workspaceContains:**/.iotworkbenchproject",

xulion.codescope/package.json
  25,10:         "workspaceContains:**/.vscode"

Problematic:

arcsine.travetto-test-plugin/package.json
  47,6:     "workspaceContains:**/node_modules/@travetto/test"

carlowenig.angular-explorer/package.json
  20,10:         "workspaceContains:**/.angular-cli.json",
  21,10:         "workspaceContains:**/node_modules/@angular/core/package.json"

Orta.vscode-react-native-storybooks/package.json
  18,6:     "workspaceContains:**/node_modules/@storybook/react-native/package.json"

I would like to suggest that if we touch the behaviour of workspaceContains we do it in one go and not change it every few months. i.e. if we are tackling its performance and changing its semantics I think we should do it entirely in one milestone. I suggest we do both ignoring of exclude patterns and the time limit in one go. This should make it less likely that the time limit is hit.

The download counts for the problematic ones are 166 + 271 + 22K. Their activation events are:

arcsine.travetto-test-plugin/package.json
  "activationEvents": [
    "onLanguage:typescript",
    "onCommand:extension.triggerDebug",
    "onCommand:extension.triggerDebugKey",
    "workspaceContains:**/node_modules/@travetto/test"
  ]

carlowenig.angular-explorer/package.json
    "activationEvents": [
        "onView:angularExplorer"
    ]

Orta.vscode-react-native-storybooks/package.json
  "activationEvents": [
    "onCommand:extension.showStorybookPreview",
    "workspaceContains:**/node_modules/@storybook/react-native/package.json"
  ],

The first one looks OK, it will get activated quite quickly if you open a TS file. The second one has since changed the version of package.json on master and gave up on workspaceContains. I think the only problematic one is the third one. I suggest we quickly reach out to the third extension and let them know that they will be broken by our optimizations.

TL;DR

we run the search and use ignore patterns (i.e. .git, node_modules, whatever is configured)
we add a time limit to the search (a constant, we can start with 5 seconds or something?)
we activate an extension if a file that matches the pattern is opened
we reach out to Orta.vscode-react-native-storybooks and let them know they will be impacted

@roblourens @chrmarti Is that OK with you?

alexdima on 17 Aug 2018

What about aborting the search as soon as all lookups are resolved?

jens1o on 17 Aug 2018

What about aborting the search as soon as all lookups are resolved?

That is already the case today. As soon as a single result is found, the search is stopped.

alexdima on 17 Aug 2018

👍1

Other extensions could be implicitly depending on files only found under an excluded path, and they will no longer be activated.

This one will break too:

PhilAlsford.git-duet-vscode/package.json
  19,10:         "workspaceContains:.git/*/**"

I wonder if we should skip files.exclude but not search.exclude? I think of the first as "these aren't really part of my project" and the second as "these are not source files that I'm constantly working in". Someone might have important project config files under search.exclude.

Activating when a certain node module is installed seems like a valid scenario. The two above would have to activate on * instead, or just onCommand.

roblourens on 17 Aug 2018

Other extensions could be implicitly depending on files only found under an excluded path, and they will no longer be activated.

But that is the whole point of my previous comment. We searched through extensions that use workspaceContains and looked at the ones using globs and picked up the ones that will probably be impacted. The data might not be 100% up-to-date (@jrieken's download was not 100% up-to-date), so please try to grep for workspaceContains on your data and find more extensions that will be impacted.

I agree about PhilAlsford.git-duet-vscode, we missed that, @jrieken's download was not super up-to-date, we should reach out to this author too. But the impact is small (83 installs).

Activating when a certain node module is installed seems like a valid scenario

This can be done via a file probe, that goes on a different code path than disk search. i.e. workspaceContains:node_modules/myModule/package.json. This file will be probed directly using fs and not with DiskSearch.

I don't know which one of the excludes to choose. I would say search.exclude because that seems to be the configuration for searching and we are technically launching a search.

If nothing works, then yes, extensions should activate on *. The whole point of activation events is that we save CPU time by not activating extensions eagerly, but if the workspaceContains activation event ends up costing 1 minute of CPU, then we might as well have activated that extension on start-up, and it would have probably consumed 10-100ms of CPU by activating... So IMHO we need to be aggressive here and exclude as much as possible, and then institute a 5second limit, and if the time limit is hit, then we should just give up and activate the extension...

alexdima on 20 Aug 2018

The default for search is to follow symlinks, that doesn't combine well with ignoring the exclude patterns when package managers like cnpm extensively use symlinks to minimize disk usage. Ignoring symlinks just for workspaceContains might help, but would also be confusing, ignoring symlinks in general might break too many users.

Should we consider deprecating glob patterns for workspaceContains and just activate these extensions as if they activate on *?

chrmarti on 20 Aug 2018

@chrmarti

Should we consider deprecating glob patterns for workspaceContains and just activate these extensions as if they activate on *?

My opinion is to not give up, and continue to have workspaceContains with glob patterns. But the search will be limited to whatever the search UI searches. If you can't find it in the UI, then an extension trying to activate will not find it. Extensions that are ok with this limitation (probably 95%) can continue to use workspaceContains. Extensions which are not ok with this limitation can move to use *.

alexdima on 20 Aug 2018

Sounds good. We could in addition serialize the glob searches, currently all (on for each extension with workspaceContains) are run in parallel.

chrmarti on 20 Aug 2018

We could in addition serialize the glob searches, currently all (on for each extension with workspaceContains) are run in parallel.

Should we make that into a separate issue. I am not quite sure how to tackle that. Today, the logic is the following:

for each extension E that uses workspaceContains with at least a glob pattern
gather all the glob patterns used within that extension
launch a DiskSearch query and ask it to stop at the first match with any of the glob patterns

If we were to launch a single DiskSearch query for multiple extensions we would not know which of the extensions to activate.

alexdima on 20 Aug 2018

Not sure how to combine the searches, my suggestion is to simply run one glob search after the other instead of all at the same time. That might take some pressure from already slow machines (we might need to lower parallelization in Ripgrep to truly achieve that, @roblourens?). With that we could start the timer once and when it finishes just activate all extensions that haven't had their search run yet.

chrmarti on 20 Aug 2018

I think that one rg process will use about all the CPU that it can, so I don't know if running just one at a time is that much better than running a few at a time.

With that we could start the timer once and when it finishes just activate all extensions that haven't had their search run yet.

This is possibly a less risky way to fall back than checking opened files, although you could end up activating a lot of extensions that you didn't mean to.

roblourens on 20 Aug 2018

👍1

Telling rg to use fewer threads is also something we can do. I don't know how that fits into the extension api though.

roblourens on 20 Aug 2018

Simple example of the above: https://github.com/Microsoft/vscode/pull/57047

Applying the exclude patterns is easier with https://github.com/Microsoft/vscode/issues/57046

I don't know whether we still want to do this for August this late? We could add some telemetry around how long these searches are taking and use that to help decide what the timeout value should actually be.

roblourens on 23 Aug 2018

I think with #57046 we'll get that telemetry anyway

roblourens on 23 Aug 2018

Just to add to the overall picture: @reggiew2, the reporter of #55649, saw the issue when opening the home folder and the reported hardware looks similar to mine, so I ran the following measurements with (the default) and without following symlinks on my home folder (hoping that would have a somewhat similar structure to the reported one):

time (rg --files --follow 2>/dev/null | wc -l)
14974573 files in 81.09 seconds

time (rg --files 2>/dev/null | wc -l)
1474396 files in 10.538 seconds

Not following symlinks ("search.followSymlinks": false) might well help in this case.

chrmarti on 23 Aug 2018

I forgot: Ripgrep's default is to use .gitignore files, so here are the same measurements with that turned off which is what workspaceContains does:

time (rg --files --no-ignore --follow 2>/dev/null | wc -l)
17349358 files in 83.72 seconds

time (rg --files --no-ignore 2>/dev/null | wc -l)
2225053 files in 28.880 seconds

chrmarti on 23 Aug 2018

Yeah we can use the configured values of those settings to follow what someone said above, if normal search can find it, workspaceContains will find it.

roblourens on 23 Aug 2018

I think we should put this in the first week of September so it has plenty of time in Insiders, unless anyone has any objection.

roblourens on 24 Aug 2018

👍2

Fixed in https://github.com/Microsoft/vscode/pull/57047

Should use the user's excludes, and configured "use ignore files" and "follow symlinks" settings, and searches are limited to 5s, after which the extension is activated anyway.

The possible downside is that you activate extensions which actually shouldn't be activated. Hopefully this will be ok.

roblourens on 6 Sep 2018

I've noticed extensions being activated from timeout when my macbook is doing a lot. If I set "window.restoreWindows": "all" and reopen vscode with several windows, then sometimes one will have an extension activated from timeout.

I added telemetry so we can see how much this happens in the wild

roblourens on 15 Sep 2018

Besides what we can see in telemetry, I want to get an idea of when this will happen. I published an extension that does nothing but show a notification when it's activated, and is activated on a workspaceContains pattern that doesn't exist. If anyone wants to help out, you can install this extension https://marketplace.visualstudio.com/items?itemName=roblourens.vscode-workspacecontains-canary and take note when you see the notification.

If you open a very large workspace, or many windows all at once, or something like that, then you will see the notification and that is expected and by design. But if you see it when it's not expected, then you can take note of what else your computer was doing at the time that might have slowed search down. Then let me know if you see it happening at random times, and if so, we may want to raise the limit by a couple seconds, or tweak it somehow.

For example, I was thinking that instead of measuring 5s in extensionHostMain where we look at the workspaceContains pattern, we might want to include a timeout as a search option and measure closer to where the search is actually performed. Not across the extension API boundary but if we measure at the point where a search provider is invoked, that will avoid a couple IPC hops that could be blocked when the computer is very busy.

roblourens on 17 Sep 2018

From my understanding, this is how it's supposed to work:

I create an extension that uses this activation pattern "workspaceContains:**/node_modules/express/package.json"
When I have **/node_modules excluded in files.exclude, the extension should not activate. However, in testing, this is not the case.
When I have node_modules specified in .gitignore and "search.useIgnoreFiles": true, the extension should not activate. However, in testing, this is not the case.

octref on 26 Sep 2018

When I use Ripgrep, I get roughly 4 - 5 individual rg.exe in Task Manager,each roughly taking 5 - 15% CPU (Making my usage go to a constant 77% and slowing my PC heavily) and between 5 - 30 MB RAM each

VSCode 1.29.0-exploration (observed this in insiders)
Windows 10 1803 17134.345

beastdestroyer on 21 Oct 2018

Can you open a new issue with more details, because it's not clear what you mean and probably isn't relevant to this issue

roblourens on 22 Oct 2018

Vscode: workspaceContains starts a search over full workspace, including .git/, node_modules/

Most helpful comment

All 42 comments

Related issues