Angular-cli: MergeStrategy seems to be completely broken (mergeWith failure)

[william-lohan]

Re-post of https://github.com/angular/devkit/issues/745

Bug Report or Feature Request (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request

Area

- [ ] devkit
- [x] schematics

Versions

Node v8.11.1
NPM 5.8.0
Mac OS High Sierra

Repro steps

Use the following schematic

import { Rule, url, apply, mergeWith, chain, MergeStrategy } from '@angular-devkit/schematics';

export function index(): Rule {
  return chain([
    simpleMerge(),
  ]);
}

function simpleMerge(): Rule {

  const source = url('./files');

  return mergeWith(source, MergeStrategy.Overwrite);
}

The log given by the failure

schematics .:boilerplate

ERROR! /src/app/app.component.html already exists.
ERROR! /src/app/app.component.spec.ts already exists.
ERROR! /src/app/app.component.ts already exists.
ERROR! /src/app/app.module.ts already exists.
ERROR! /src/assets/.gitkeep already exists.
ERROR! /src/environments/environment.prod.ts already exists.
ERROR! /src/environments/environment.ts already exists.
ERROR! /src/index.html already exists.

Desired functionality

All files should be overwritten.

Mention any other details that might be useful

I tried every MergeStrategy possible (AllowOverwriteConflict, etc.), same result.

https://github.com/angular/devkit/issues/745
https://stackoverflow.com/questions/48957132/how-to-overwrite-file-with-angular-schematics

[clydin]

Could you provide the versions used?

I cannot answer with certainty without knowing the versions but this PR (https://github.com/angular/angular-cli/pull/11274) will most likely correct the behavior.

[william-lohan]

version 0.6.8

dup repo https://github.com/gatimus/schematics-merge-strategy

[hansl]

Could you try with the RC of 0.7? I fixed a lot of issues by reworking the whole Tree implementation, should have fixed a few of those merging issues (amongst others, e.g. branching).

[hansl]

Dupe of https://github.com/angular/angular-cli/issues/11246. Let's keep this one for tracking though as it has more information.

[william-lohan]

Still duplicating on 0.7.0-rc.3 https://github.com/gatimus/schematics-merge-strategy/tree/RC-0.7

[cgatian]

This is making me cry 😢

[joeson1]

Is this already fixed?
I have following workaround:

const templateSource = apply(url('./project-files'), [
      template({ ...strings, ...options }),
      move(options.path),
      forEach((fileEntry: FileEntry) => {
        if (host.exists(fileEntry.path)) {
          host.overwrite(fileEntry.path, fileEntry.content);
        }
        return fileEntry;
      }),
    ]);

[tinesoft]

Hi,

For me, the issue appears after upgrading my schematics "dependencies" from:

 {
    "@angular-devkit/core": "^0.6.8",
    "@angular-devkit/schematics": "^0.6.8",
    "typescript": "^2.5.1"
  },

to:

 {
    "@angular-devkit/core": "^0.7.3",
    "@angular-devkit/schematics": "^0.7.3",
    "typescript": "^2.9.0"
  },

Version v0.6.8's mergeWith was working fine...

@joeson1 : i tried your workaround, but it didn't help 😞 ...

[william-lohan]

@joeson1 where do you assign host?

[william-lohan]

@tinesoft I get the exact same behavior with v0.6.8 as with v0.7.3

[Bored-Bohr]

I believe this is the same issue as #12088.

I originally commented this on #12088:
This issue still exists in 0.8.4.

The issue occurs on HostTree merge. I've detailed the simplified propagation below:

1. The url function generates actions with kind 'c'
2. Since there exists no prior actions this._willCreate(path) || this._willOverwrite(path) returns false.
3. this._record.create triggers.
4. This throws the fileDoesNotExistException on execute.

[egervari]

I'm still having this issue, in case someone needs to know it's still not fixed. Using version 0.8.6

[bautistaaa]

I have the same issue qith 0.8.6, however if I run schematics my-schematic:my-schematic it will work BUT if I run ng g my-schematic:my-schematic it will give me the already exists error message. Can anyone explain why?

[cgatian]

@egervari and @bautistaaa could you let me know if this workaround works for you?

https://stackoverflow.com/a/53230034/684869

[mattezell]

As others have stated, this still seems to be an issue... I've reproduced with 0.6.8 and 0.7.2... Based on one of the suggestions in the thread linked by @cgatian, I was able to formulate a workaround for my specific situation...

In my method, I was attempting to call move() within the rules array of apply()... Essentially the source url() was a subdirectory of my 'files' directory in my schematic - and the destination in move() was a subdirectory of the project tree.

So, I had the schematics source assets in: schematics\ng-add\files\help\my-help\index.html

And I wanted my schematics source assets to be dropped in the projects source directory: \src\help\my-help\index.html.

When calling url('./files/help') and move('.', '/src/'), I would get "src/help/my-help/index.html already exists. The Schematic workflow failed."

To work around this, I changed my folder structure in my schematics assets to match the project folder structure - so instead of 'schematics\ng-add\files\help....' I used 'schematics\ng-add\files\src\help....'. I then updated url to be "url('./files')" and move to be "move('.')"... So now instead of explicitly setting the URL source as a subfolder containing the assets, it's just using the whole 'files' directory to overlay on top of the root of the project (move('.')).

I ran across this workaround when I by chance notice that I only got the 'already exists' error when operating on subfolders of the project - but in my troubleshooting, I realized that things behaved as expected when operating against the root of the project...

Here's a sample of my updated method which now works as desired in case it assists:

/** Add help assets to application */
// NOTE: work around due to busted MergeStrategy - https://github.com/angular/angular-cli/issues/11337
function addHelpToApplicationRoot(options: Schema) {
  return (tree: Tree, context: SchematicContext) => {
    context.logger.info(`Adding help assets to application help.`);
    const rule = mergeWith(
      apply(url('./files'), [
        forEach((fileEntry) => {
          if (tree.exists(fileEntry.path)) {
            tree.overwrite(fileEntry.path, fileEntry.content);
            return null;
          }
          return fileEntry;
        }),
        move('.')
      ]),
    );
    return rule(tree, context);
  };
}

Seems that the important detail, at least in my case, was to make my 'files' assets be structured in a way that matches the project's folder structure rather than relying on url() and move() to work on specific subfolders.

[denisoby]

@mattezell Thank you for workaround example. However, I'd assume that you don't need this "move('.')" - everything should work even w/o it.

[matheo]

I have the same problem not using a Source but trying to create/override custom stuff in the Tree. There's no simple way via MergeStrategy to avoid the exists check?

My hacky intuition thought that using tree.overwrite with MergeStrategy.AllowOverwriteConflict would default to create the files I'm trying to overwrite. Or create with AllowCreationConflict would overwrite, something like that would be useful.

[shusson]

There's no simple way via MergeStrategy to avoid the exists check?

I believe this is exactly what MergeStrategy.Overwrite is suppose to do.

[GregOnNet]

Hi,

I experience the same issue.
I setup a minimal reproduction: https://github.com/GregOnNet/merge-strategie-issue

Setup

• The repository contains a file .prettierrc
• My schematic overwrite loads another .prettierrc from ./templates
• MergeStrategy.Overwrite is configured as 2nd parameter of apply.
• The repository contains a unit test to check if the template overwrites the origin file.

Observed behaviour

• The unit test succeeds
• The integration test using schematics fails
  > Please refer to section Repro Steps to see the details.

Versions

Node v11.11.0
NPM 6.9.0
Mac MacOS Mojave 10.14.3

@angular-devkit/core: 7.3.6
@angular-devkit/schematics: 7.3.6

Repro Steps

git clone https://github.com/GregOnNet/merge-strategie-issue.git
cd merge-strategie-issue

npm install
npm test # Everything is fine
npm run build

schematics .:overwrite
# ERROR
ERROR! /.prettierrc already exists.
The Schematic workflow failed. See above.

If I can provide any further information, please let me know. 👍

[tscislo]

Is there any plan to fix this? Any current workarounds?

[william-lohan]

@tscislo this is my current workaround:

export function overwriteIfExists(host: Tree): Rule {
  return forEach(fileEntry => {
    if (host.exists(fileEntry.path)) {
      host.overwrite(fileEntry.path, fileEntry.content);
      return null;
    }
    return fileEntry;
  });
}

export function mySchematic(options: MyOptions): Rule {
  return (host: Tree, context: SchematicContext) => {
    /* ...logic... */
    const templateSource = apply(
      url('./files'),
      [
        template({
          ...strings,
          ...options
        }),
        move(options.path),
        options.overwrite ? overwriteIfExists(host) : noop()
      ]
    );

    return mergeWith(templateSource);
  };
}

derived from other variations in answers here https://stackoverflow.com/questions/48957132/how-to-overwrite-file-with-angular-schematics

[gund]

For me even workarounds did not work.
When I apply them - changes are simply ignored and I get message Nothing to be done....

[myin142]

A note about the workaround using manual forEach and overwrite.
Make sure that you put the forEach before the move since the file paths that you get are relative to the root project. And also make sure that you create the files first before trying to overwrite them. (e.g calling externalSchematic first).Took me a while to notice that.