Nx: [Feature Request] affected:projects

[dereklin]

_Please make sure you have read the submission guidelines before posting an issue_

Prerequisites

Please answer the following questions for yourself before submitting an issue.
YOU MAY DELETE THE PREREQUISITES SECTION.

• [x] I am running the latest version
• [x] I checked the documentation and found no answer
• [x] I checked to make sure that this issue has not already been filed
• [x] I'm reporting the issue to the correct repository (not related to Angular, AngularCLI or any dependency)

Expected Behavior

affected:projects to combine affected:apps and affected:libs

So I can just do:

projects=$(./node_modules/.bin/nx affected:projects "$(git merge-base refs/remotes/origin/master HEAD)" HEAD)

Current Behavior

Currently, when I want to find out what projects are affected, I need to do two checks:

apps=$(./node_modules/.bin/nx affected:apps "$(git merge-base refs/remotes/origin/master HEAD)" HEAD)
libs=$(./node_modules/.bin/nx affected:libs "$(git merge-base refs/remotes/origin/master HEAD)" HEAD)
projects="$apps $libs"

[FrozenPandaz]

First, I think adding a feature to provide a full list of projects which were affected is a sensible feature, mostly for display/ information.

However, I am curious why E2E projects aren't a part of the Expected Behavior

affected:projects to combine affected:apps and affected:libs

Why is it necessary to _only_ have apps and libs? What do you intend to use this for? There might be a better solution.

[dereklin]

@FrozenPandaz Thanks for responding. I actually realized that problem after opening the issue. However, I wanted to gauge the community interest before further defining it.

Because we do custom stuff like using the high memory build instead of the regular build...

We want to find out the affected project and then decide what other stuff to do.

For example, we want to lint all projects, but we only want to build apps. We want to run e2e when the apps are affected or when the e2e are affected. We want to test the apps, but we don't want to test the e2e.

In the current way, we get back a space delimited string when running

apps=$(./node_modules/.bin/nx affected:apps "$(git merge-base refs/remotes/origin/master HEAD)" HEAD)

with affected:project, in order for it to be useful, we would need to return back a list of objects, like:

[{type: 'app', name: 'abc'}, {type: 'e2e', name: 'abc-e2e}, {type: 'lib', name: 'fancy-lib'}]

I think that might be difficult with shell script, right?

If that's too difficult, we can close this for now. I'll just continue to use the separate scripts (with the overhead to checking multiple times)

[FrozenPandaz]

Thank you for the details!

Because we do custom stuff like using the high memory build instead of the regular build...

I guess this is in an environment with limited resources? The way I imagine it, skipping parts of the build means that things that are _usually_ checked and _expected_ to work is not verified and therefore can fail _unexpectedly_. This is assuming all parts of the build are necessary and not "nice to haves".

We want to find out the affected project and then decide what other stuff to do.

I think this could be solved of affected could decide which stuff to do. 😉We have an exclude flag which allows you to pass project names to exclude. But I can see how it can be annoying to update this list.

Solutions

Spoiler: I like Solution 2 more 🤷‍♀️

Solution 1

This solution is closer to what you are imagining. We make a change to allow --exclude to accept something along the lines of Type.app, Type.lib, Type.e2e which would exclude projects of type. Then you can execute nx affected:build --exclude Type.lib.

Solution 2

This solution is _almost_ possible today and I believe _should_ be possible. We currently allow nx affected --target extract-i18n to run that target on any project which has that defined in angular.json. We could make a change to extend that functionality to accept target and configuration and only run projects with the target and configuration. This would mean the following would be possible:

"app1": {
  "architect": {
    "build": {
      "configurations": {
        "production": { "optimization": true }
      }
    }
  },
},
"demo-app": {
  "architect": {
    "build": {
      "configurations": {} // This app never gets built to prod, there's no reason to build it with affected
    }
  }
}

nx affected --target build:production

In your case, you could do nx affected --target build:low-memory

Hindsight (aka me thinking out loud)

In hindsight, nx affected:apps has a clear purpose. These are likely products that your users see, or some other application sees. It's important to get a list of only them for display or deployment purposes to tell an individual not familiar with the source code, "These are the projects (by business definition) affected by these commit(s)"

nx affected:libs.. just felt right at the time. We have apps.. so why not libs. But in hindsight, it isn't as important. libs never affect an end user on their own (unless it's publishable). And thus, the importance is only for libs that an organization knows for sure is critical to the business such as Authentication, Ads, .etc. The same would be possible with an affected:projects command which showed all projects.

[dereklin]

@FrozenPandaz
The spirit or purpose of this feature request is so that we don't need to run multiple git remote comparisons.

Currently in my shell script, I do things like:

apps=$(./node_modules/.bin/nx affected:apps "$(git merge-base ${remoteMaster} HEAD)" HEAD)
read -r -a appsArray <<< "$apps"
e2es=$(echo "$apps" | sed -E 's/([^ ]+)/\1-e2e/g')
read -r -a e2esArray <<< "$e2es"
libs=$(./node_modules/.bin/nx affected:libs "$(git merge-base ${remoteMaster} HEAD)" HEAD --exclude some-lib)
read -r -a libsArray <<< "$libs"
read -r -a projects <<< "$apps $e2es $libs"

So I need to do two git remote comparisons to figure out what are the affected apps, e2e's, and libs. And then everything together are projects.

Ideally, I should only need to do one git remote check. That will save a little bit of time.

In terms of custom build. It's a known issue that when building large apps, you need to increase the node memory allocation.

Using affected:build, affected:test, affected:lint separately would be different discussion. But using them separately like that will actually incur more git remote checks, right?

[vsavkin]

The newly added print-affected command is a much more generic version of this. We are working on making it more robust so it handles all scenarios where you need to "introspect" the state of the workspace.

So I'm going to closer this issue.