Packer: "azure-arm: EOF" with ansible remote provisioner

[urbanchef]

$ packer --version
1.4.0

• Host platform: CentOS Linux release 7.6.1810
  Look for the following here: https://gist.github.com/urbanchef/23c598f8d4dcfed29fa06460246e8f40
• debug log output from PACKER_LOG=1 packer build template.json
• test_playbook.yml
• json packer template

Steps to reproduce:

1. Spin up CentOS 7.6 VM in azure
2. install packer 1.4.0
3. install ansible 2.7.10
4. run packer build template.json
5. observe: "azure-arm: EOF" error message. The rest of the execution is hung. At least for me nothing happened for more than an hour.

Expected outcome:

• successfully finished image build, provisioned by ansible remote

[azr]

Hello there @urbanchef, thanks for opening, I think this is issue was already solved, can you please confirm that the nightly build solves this for you?

https://github.com/hashicorp/packer/releases

[urbanchef]

@azr Hi! Appreciate such a prompt reply! But unfortunately, I am facing the same behavior here. The nightly build 1.4.1 did not fix the issue.

[azr]

Okay, thanks for the nice steps to reproduce ! I'm looking into it 🙂

[azr]

Okay, I'm waiting for azure credentials to test right now but the provided buildfile just worked on centos using the GCP builder.

[SwampDragons]

Just to clarify whether this is a 1.4.0 regression or not -- did this happen for you with Packer v. 1.3.5?

[urbanchef]

@SwampDragons Haven't tested it on 1.3.5. Let me quickly check and get back to you.

[urbanchef]

@SwampDragons Hitting same error along with the other ones. Seems more meaningful...
Please, review: https://gist.github.com/urbanchef/dce861efc2dd942a883082431ef9c700

[SwampDragons]

Thanks for checking for me!

[urbanchef]

@azr @SwampDragons Hello guys! I was just wondering if you had any plans of fixing it soon? Would be glad to help out, despite of being more of an ops/infra guy, than dev. But my GoLand does not seem to respect breakpoints with remote dlv debug. Though it works with a simple "Hello World" program, it does not when I enable dlv remote debug for packer.

[azr]

Hey @urbanchef I can repro and I'm on it 🙂

[azr]

Howdy @urbanchef, I have this PR #7654 that is really simple but seems to fix the issue. At least it dit for me but I can't believe it. Can you please confirm it was also fixed for you ?

fresh binaries

[urbanchef]

@azr Hi there! Unfortunately, for me it does not... Here's the log: https://gist.github.com/urbanchef/070e84fe613f5cef21d843a5047a5f94

[azr]

Ah sorry my bad, my test case was wrong ! I'm still on it then, thanks !

[azr]

Hey @urbanchef, I was debugging a bit more and your issue seems to be caused by that command:
/usr/sbin/waagent -force -deprovision+user.

waagent docs:

-deprovision: Attempt to clean the system and make it suitable for re-provisioning, by deleting the following:

All SSH host keys (if Provisioning.RegenerateSshHostKeyPair is 'y' in the configuration file)

Nameserver configuration in /etc/resolv.conf

Root password from /etc/shadow (if Provisioning.DeleteRootPassword is 'y' in the configuration file)

Cached DHCP client leases

Resets host name to localhost.localdomain

WARNING! Deprovision does not guarantee that the image is cleared of all sensitive information and suitable for redistribution.

-deprovision+user: Performs everything under deprovision (above) and also deletes the last provisioned user account and associated data.

It seems like to me that you are deleting the packer user :

azure-arm: WARNING! packer account and entire home directory will be deleted.

packer then should not be able to run anything anymore. This feel like a bug because the following command will stop in the middle of the execution but in fact, I think it's just that the change takes a little time to propagate ? Don't you agree ?

[urbanchef]

I see, the correct way for waagent to run would be after all provisioners execution have been finished (somehow deferred). As a workaround, make changes to template.json to invoke waagent as the last provisioner command. Right?

[azr]

I think this would solve your issue, yes ! But I'm not sure this is acceptable for your project. I'm going to close this issue for now. Please comment here if you think we could do something code wise in packer to fix this. 👍

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.