Aws-cdk: [cli] - New progress bar method of giving feedback while deploying the stacks hides useful details

[farzad-xgen]

:question: General Issue

The Question

The new feedback method, showing the progress bar plus the few last lines of the CloudFormation events, hides useful information from the user:
1- It doesn't give you the proper sense of progress by hiding what is really going on.
2- You cannot review the changes while it is deploying, finding possible flaws.
3- If the update fails, you have no way to know what was the cause unless you open the CloudFormation console and review the events from there.

I really appreciate if you make using this new feedback method optional, or at least provide a flag so I can fall back to the old one (progressive report of all the CloudFormation events).

Environment

• CDK CLI Version: 1.49.1 (build 7d6321f)
• Module Version: 1.49.1
• Node.js Version: v10.21.0
• OS: Ubuntu 18.04.4 LTS
• Language (Version): Python 3.6.9

[pgollucci]

To be fair you can use the CLI et al too, but yes, this has been driving me
bonkers. You routinely even just see a blank screen on a stack with 50+
resources. Esp if you are running in a VSCode Integrated terminal.

+1 cli option or flag please. If someone links me to the original change,
I'll implement the flag.

On Fri, Jul 3, 2020 at 11:00 AM farzad-xgen [e-mail]
wrote:

❓ General Issue The Question

The new feedback method, showing the progress bar plus the few last lines
of the CloudFormation events, hides useful information from the user:
1- It doesn't give you the proper sense of progress by hiding what is
really going on.
2- You cannot review the changes while it is deploying, finding possible
flaws.
3- If the update fails, you have no way to know what was the cause unless
you open the CloudFormation console and review the events from there.

I really appreciate if you provide using this new feedback method
optional, or at least provide a flag so I can fall back to the old one
(progressive report of all the CloudFormation events)
Environment

• CDK CLI Version:
• Module Version: 1.49.1 (build 7d6321f
  https://github.com/aws/aws-cdk/commit/7d6321f2ce44f3c7ff6b669a9a48b19b10921a27
  )
• Node.js Version: v10.21.0
• OS: Ubuntu 18.04.4 LTS
• Language (Version): Python 3.6.9

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-cdk/issues/8893, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAIL57SLJ3QJHN6VH5TQCLRZXXBNANCNFSM4OP4EGPQ
.

--
Philip M. Gollucci
IT Executive and Engineering Leader
http://www.linkedin.com/in/pgollucci/
301.818.0719

[rix0rrr]

-v will also switch back to the old progress method, as a temporary fix.

I'd like to understand the objections in a little more detail though.

1- It doesn't give you the proper sense of progress by hiding what is really going on.

I specifically changed this to show you "what is going on", right now. What information are you missing?

2- You cannot review the changes while it is deploying, finding possible flaws.

What kind of flaws do you normally detect in this way, and how do you detect them?

3- If the update fails, you have no way to know what was the cause unless you open the CloudFormation console and review the events from there.

It should definitely be showing you the failures front and center, all the time, instead of burying it in a scrollback log. Another one of the purposeful changes I made. If you aren't seeing the failures, I'd consider that a bug. Can you share an example?

You routinely even just see a blank screen on a stack with 50+ resources.

I assume this is because of a small vertical terminal height, right? Or does the display malfunction in another way inside an embedded VSCode terminal?

[crash7]

3- If the update fails, you have no way to know what was the cause unless you open the CloudFormation console and review the events from there.

It should definitely be showing you the failures front and center, all the time, instead of burying it in a scrollback log. Another one of the purposeful changes I made. If you aren't seeing the failures, I'd consider that a bug. Can you share an example?

I have this issue, and the error I see a lot is:

```
mystack-integration: creating CloudFormation changeset...

Error occurred while monitoring stack: RangeError: Invalid count value

at String.repeat (<anonymous>)

at CurrentActivityPrinter.progressBar (/home/circleci/project/node_modules/aws-cdk/lib/api/util/cloudformation/stack-activity-monitor.ts:558:24)

at CurrentActivityPrinter.print (/home/circleci/project/node_modules/aws-cdk/lib/api/util/cloudformation/stack-activity-monitor.ts:494:23)

at StackActivityMonitor.flushEvents (/home/circleci/project/node_modules/aws-cdk/lib/api/util/cloudformation/stack-activity-monitor.ts:151:18)

at StackActivityMonitor.tick (/home/circleci/project/node_modules/aws-cdk/lib/api/util/cloudformation/stack-activity-monitor.ts:130:12)

at processTicksAndRejections (internal/process/task_queues.js:97:5)

```
Lots of this... and then the message (with the error) in vertical:

So the error is there (if I read the vertical line I can see the error) but hard to read.

[rix0rrr]

Wooaahhh. Thought that was already fixed :(

[rix0rrr]

Can you share CDK CLI version if you get the cut-off column?

[crash7]

Can you share CDK CLI version if you get the cut-off column?
I found that I was using 1.46, so I upgraded the CI build to use 1.49.1 and the "progress bar error" is gone but the cut-off column is still here.

OP also says it's in 1.49.1

[jackm-ts]

Can you share CDK CLI version if you get the cut-off column?
I found that I was using 1.46, so I upgraded the CI build to use 1.49.1 and the "progress bar error" is gone but the cut-off column is still here.

OP also says it's in 1.49.1

I am on cdk cli 1.50.0 and still getting the cut-off column

[farzad-xgen]

This happens when there's a failed update mostly because some of the outputs can't be deleted or other dependency related issues. I've also seen that when something very resource-related goes wrong.

As shown above, the actual error message is gone and only the final failed message is shown. I'm using CLI 1.49.1

[farzad-xgen]

-v will also switch back to the old progress method, as a temporary fix.

I'd like to understand the objections in a little more detail though.

1- It doesn't give you the proper sense of progress by hiding what is really going on.

I specifically changed this to show you "what is going on", right now. What information are you missing?

2- You cannot review the changes while it is deploying, finding possible flaws.

What kind of flaws do you normally detect in this way, and how do you detect them?

3- If the update fails, you have no way to know what was the cause unless you open the CloudFormation console and review the events from there.

It should definitely be showing you the failures front and center, all the time, instead of burying it in a scrollback log. Another one of the purposeful changes I made. If you aren't seeing the failures, I'd consider that a bug. Can you share an example?

You routinely even just see a blank screen on a stack with 50+ resources.

I assume this is because of a small vertical terminal height, right? Or does the display malfunction in another way inside an embedded VSCode terminal?

-v doesn't work for me, it throws out too much irrelevant and debug information. I really prefer to have the option to see the progress like before.
I usually traceback all the changes that are happening, scrolling up and down many times while the stack is updating. I usually look for deletion, or updates that I think are not the ones I meant to have. This was very helpful. I really need to see what's going on.

[udondan]

Things getting mixed here. This ain't no bug @SomayaB. This is a request to implement a flag to use the old output. Fixing the output issues does not resolve this issue.

I first liked the new progress output but it has become the single most annoying thing in CDK for me. If my stacks were not using the latest @aws-cdk's I would downgrade the cdk program.

I spent a full day yesterday trying to trick cdk believing, it runs in a non-interactive shell. I failed. Please, implement a flag for this or check for a CI env var.

-v is no solution. Output is useless with all the debug info.

I do not want to permanently stare on the screen to see what's going on. I want a summary. Not only in CI.

[udondan]

Success on day 2. I experimented with wrapping cdk in small ruby and perl scripts. No success. I finally came up with this workaround:

#!/bin/bash

CDK_LOG=$(mktemp)
tail -f "${CDK_LOG}" &
PID=$!
/usr/local/bin/cdk "$@" &> "${CDK_LOG}"
kill "${PID}" &> /dev/null

Redirecting stderr is the only way to change the behavior. Unfortunately this is the same trigger that disabled color output.

If you want to use this, save it in your $PATH as cdk. The location where you store it needs to be listed before the location of the original cdk program. mine is under ~/bin/cdk

[elgordino]

I would just like to add to the request to have an option to use the CI behaviour when running with a TTY.

Usually I just kick off a build, wander off, make a tea, whatever. When I come back it is immensely more useful to see a log of what happened, rather than just a summary of whether it worked. CF already doesn't report on resources that were not changed, so all of the CREATE / UPDATE / DELETE messages are very useful, particularly I want to keep an eye out for any DELETE messages that I didn't expect! With just the progress bar I might see 'all good' when in fact a bunch of stuff was accidentally deleted.

[farzad-xgen]

Any updates? This progress bar is really annoying, and it's useless. I'm wondering why it was introduced at the first place, I mean, who did request the feature? I think it wasn't anything out of UI, just some nice-looking feature. I really need the old one, now I need to keep the Cloudformation console open to trace any failed stack deployment. The error messages are simply gone in the aws-cdk deploy task and can't be traced without referring to CloudFormation.

[farzad-xgen]

You see: Again, no error details, just a final "failed" message.

[dboitnot]

Agreed. I'd like to get rid of the progress bar entirely and go back to the previous "log-style" output.

[elgordino]

The code that determines whether 'fancy' output is available is here: https://github.com/aws/aws-cdk/blob/16d2a48a4d0e049d0ae6cd8c9165eccc816ebd4b/packages/aws-cdk/lib/api/util/cloudformation/stack-activity-monitor.ts#L82

It seems like it's be pretty trivial to add an additional command line option. Maybe no-fancy-output or something, to match the existing no-color option? Or show-all-events or something like that?

Or I see the 'verbose' flag can now be specified multiple times, https://github.com/aws/aws-cdk/commit/fa4196b11a4b843af1401cbcfd3fe075986ec5c0 once for debug and twice for trace. Maybe once could be 'old style regular log', then twice for debug and three times for trace?

[vik-singh]

The output of this in CircleCI is useless as its all just a vertical line of characters - would prefer the old output in CI scenarios at least so it can be useful when reviewing builds.

[elgordino]

Here's an example of where it is much more difficult to provide support in a forum because of the new style output:

https://www.reddit.com/r/aws/comments/i0tof3/aws_cdk_api_gateway_endpoints_arent_being_created/

If the old style output was there it would be obvious whether CF was creating a new 'Deployment' for API Gateway or not.

[andreialecu]

I opened a PR at https://github.com/aws/aws-cdk/pull/9516 that detects whether the CI environment variable is set and reverts the output to previous behavior.

It's a minor change, so to anyone that is affected by this:

• you can install aws-cdk locally in the project and use https://www.npmjs.com/package/patch-package to patch it. see https://github.com/aws/aws-cdk/pull/9516/files

I have confirmed that this fixes the problem with garbled output on CircleCI.

[cynicaljoy]

I'm still seeing the issue in 1.60.0 -- here's some debug logging from my CI server:

Executing command: $ cdk version
1.60.0 (build 8e3f53a)
------------------------------
Executing command: $ echo $CI
1
------------------------------
Executing command: $ cat package.json
{
  "name": "cdk-sandbox",
  "version": "0.1.0",
  "bin": {
    "cdk-sandbox": "bin/cdk-sandbox.js"
  },
  "scripts": {
    "build": "tsc",
    "watch": "tsc -w",
    "test": "jest",
    "cdk": "cdk"
  },
  "devDependencies": {
    "@aws-cdk/assert": "^1.60.0",
    "@types/jest": "^25.2.1",
    "@types/node": "10.17.5",
    "aws-cdk": "^1.60.0",
    "jest": "^25.5.0",
    "ts-jest": "^25.3.1",
    "ts-node": "^8.1.0",
    "typescript": "~3.7.2"
  },
  "dependencies": {
    "@aws-cdk/aws-s3": "^1.60.0",
    "@aws-cdk/core": "^1.60.0",
    "source-map-support": "^0.5.16"
  },
  "private": true
}
------------------------------
Executing command: $ cdk deploy --ci
CdkSandboxStack: deploying...
CdkSandboxStack: creating CloudFormation changeset...
K
K
K
K[
K·
K·
K·
K·
K·
K·
K·
K·
K]
K
K(
K0
K/
K3
K)
K
...

 ✅  CdkSandboxStack

[robertd]

(comment copied from https://github.com/aws/aws-cdk/pull/9516#issuecomment-676852954)

@andreialecu @rix0rrr I'm still not retaining the cfn events. Also, using CI env variable I've lost color output.
I'm using iTerm2 and zsh (oh-my-zsh).

Using export CI="":

export CI=""; cdk deploy
^[IsaidCloudFrontDistribution: deploying...
IsaidCloudFrontDistribution: creating CloudFormation changeset...
[██████████████████████████████████████████████████████████] (2/2)




 ✅  IsaidCloudFrontDistribution

Stack ARN:
arn:aws:cloudformation:us-west-2:123456789:stack/IsaidCloudFrontDistribution/c12044d0-e22b-11ea-858b-06675510bae4

Using --ci:

cdk deploy --ci
IsaidCloudFrontDistribution: deploying...
IsaidCloudFrontDistribution: creating CloudFormation changeset...
[██████████████████████████████████████████████████████████] (2/2)




 ✅  IsaidCloudFrontDistribution

Stack ARN:
arn:aws:cloudformation:us-west-2:123456789:stack/IsaidCloudFrontDistribution/c12044d0-e22b-11ea-858b-06675510bae4

[robertd]

@shivlaks This issue should've been closed by https://github.com/aws/aws-cdk/pull/9623, right?

[udondan]

I hope this stays open. Unless there is an option to disable the bad progress bar without disabling the colors (=Original behavior) there is an issue. The ci flag disabled colors.

[robertd]

@udondan Have you seen the https://github.com/aws/aws-cdk/pull/9623 that just landed?

[udondan]

Oh, no, perfect! Thanks @robertd

[shivlaks]

@robertd yes that was the intention! @udondan - feel free to give it a spin as well

[udondan]

Finally was able to test this with 1.62.0. Works nicely! Thanks so much @shivlaks ❤️

[udondan]

Ping @shivlaks @SomayaB I think this could be closed.

[SomayaB]

Thanks @udondan, glad it's working! Closing the issue.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.