packer build -debug not pausing after each step

[rram]

This appears to have broken in Packer v1.3.0
Packer version: 1.3.4
Host platform: Ubuntu bionic (running in virtualbox) amd64
Debug log: https://gist.github.com/rram/ec1dfe5de1ae5a89df2062562783730e

[SwampDragons]

Thanks for opening; I can't reproduce on OSX, so I'll spin up an ubuntu machine on Monday to see if I can figure out what's going on.

[SwampDragons]

This is very strange. I can't reproduce on ubuntu inside a virtualbox vm, either. Looking at your logs, it's printing the expected lines:

==> null: Pausing after run of step 'StepProvision'. Press enter to continue. ==> null: Pausing before cleanup of step 'StepProvision'. Press enter to continue. Build 'null' finished.

Can you give me some more detail on the specific behavior you're seeing? And when you say it appears to have broken in 1.3.0, did you check with the intervening versions?

[rram]

The output just appears instantly; there is no pause between the prompts. Note that the prompts appear on the same line and have no timestamp printed between the statements. In 1.2.5 I see a timestamp and "ui: ask:" prefix on the prompt.

I redid the test on a clean install and made a screen capture:

I tested all of 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, and 1.3.5. Every version in the 1.3.x line behaved the same: no pauses at the prompts. 1.2.5 behaved as expected.

[SwampDragons]

Okay interesting I can repro on Ubuntu 18, but not on lower versions. I'm going to do a bisect and see if I can find the breaking commit

[SwampDragons]

Hm. I can't get it to work with 1.2.5, either.

[SwampDragons]

🤦‍♀️ didn't call --debug with v1.2.5; nevermind, it works.

[SwampDragons]

Okay so in my earlier comment I was wrong because I had forgotten to use the --debug flag; I can't reproduce this on ubuntu 18.04, 16.04, or 18.10. I'm running ubuntu inside of a virtualbox vm in all those situations. I reached out to a friend of mine who runs linux natively; couldn't reproduce there either.

I'm incredibly stumped here.

Is there any chance you'd be willing to build from source so that you can run a git bisect against your machine so we can find where this was introduced?

[rram]

I'll see what I can find and get back to you in a day or two.

[SwampDragons]

Thank you!

[Neki]

I can reproduce this on Ubuntu 16.04. I've run the git bisect (kudos for your excellent documentation on building from source), but I've had to skip a few commits that didn't build. The result is

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
16ecb3ad9a4b235a29adc4e8b3eb7ec7e35f5d47
ac27e54c959c457d55a5785d670fd5c647f17e20
fd2fef87386228aa9b6541c99dd6332a2d5511e7
We cannot bisect more!

I'm happy to provide more info if needed, or to test a patch.
Not sure if it's of any importance, but I'm running ZSH on gnome-terminal. (edit: I also tried Bash and urxvt and could reproduce the issue).

The same issue appears with -on-error=ask.

[SwampDragons]

Thank you so much for doing that. It definitely narrows things down; all of these are related to implementing the progress bar, so we know generally what change is causing it. I'll take a closer look at the progress bar code and see if I get any bright ideas.

[SwampDragons]

Meanwhile, could the breakpoint provisioner will provide a workaround for you? https://www.packer.io/docs/provisioners/breakpoint.html

[Neki]

I don't think the breakpoint provisioner can replace on-error=ask (but it should be OK for -debug) when debugging VM provisioning issues. I need to inspect the VM before it's torn down when a provisioner (Ansible for me) exits with an error. Even with the breakpoint provisioner, I guess Packer will fail and clean up the VM before I get a chance to inspect it (I haven't tested that).

It doesn't matter for me however, I can simply use 1.2.5.

[SwampDragons]

You're right; it'll only help for -debug.

Glad you aren't blocked.

[azr]

Hello there, I'll try to see if I can fix this, can you please tell me wether 4e55278 fixes it for you ? Thanks. Here's a linux_amd64.zip binary. This is to build only with vendor folder, not go modules.

Edit: this should totally break the progress bar on unix systems, but should be fine with the small test file provided up there ^

[Neki]

I can't unzip your archive (error message below), but the binary I get by building this commit with make dev does work with -debug.

---

% unzip linux_amd64.zip 
Archive:  linux_amd64.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of linux_amd64.zip or
        linux_amd64.zip.zip, and cannot find linux_amd64.zip.ZIP, period.

[azr]

Ah, weird, it works on my mac, sorry ! Here's a fresh build from recent go 1.12 🙂
packer_linux_amd64.zip

[Neki]

I can confirm that this build also works for me :slightly_smiling_face:

[azr]

Awesome, thanks for confirming ( twice ) !!! So we found the cause, it's an init() in a 3rd party vendor >.<

So that means we either have to fix the 3rd party, find a better one or make our own progress bar, anyways, a fix is coming !

[azr]

I have a fix in #7352, (could reproduce and my pr fixed it), could someone please confirm that it also works for them ? Thanks !

[edulop91]

built branch research_7299
make dev
packer version
Packer v1.3.5-dev (7f5c794e5)

packer build -debug -on-error=ask -color=false template.json works for me!

[azr]

Awesome, thanks, on ubuntu bionic, right ?

Okay, I need to fix the tests now. 🙂

[Neki]

I had this issue on Ubuntu 16.04 (Xenial), but I can confirm that 22df491c67e51f9086dc91a9dc5c57c1d18be00e (branch research_7299) does fix this on Xenial too.

You mentioned in your PR that we need to press enter twice for a readline to trigger, but I'm not encountering any issue on my Ubuntu system.

Thanks to everyone involved here, it's impressive how quickly this was identified and fixed!

[azr]

Hey @Neki, thanks for the update, I didn't realise about this ! I'll dig/explain more in that PR.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.