Terraform: File provisioning failed

Created on 21 Jun 2018  路  11Comments  路  Source: hashicorp/terraform

Terraform Version

Terraform v0.11.7
+ provider.aws v1.23.0

Terraform Configuration Files

resource "aws_instance" "dev-server" {
  ami                    = "${var.ami}"
  instance_type          = "t2.small"
  key_name               = "my-key-name"
  vpc_security_group_ids = ["${aws_security_group.sg-office-only.id}"]
  subnet_id              = "${aws_subnet.dev-server-subnet.id}"

  connection {
    type        = "ssh"
    user        = "ubuntu"
    private_key = "${file("~/.ssh/my-key-name.pem")}"
  }

  provisioner "remote-exec" {
    inline = "export GITHUB_TOKEN=${var.github_token}"
  }

  provisioner "remote-exec" {
    script = "scripts/install_setup.sh"
  }

  provisioner "file" {
    source      = "files/"
    destination = "/tmp"
  }

  provisioner "file" {
    source      = "data/dump.sql"
    destination = "/tmp"
  }

  provisioner "remote-exec" {
    scripts = [
      "scripts/config_setup.sh",
      "scripts/mysql_setup.sh",
      "scripts/project_setup.sh",
      "scripts/run.sh",
    ]
  }
}

Debug Output

aws_instance.dev-server: Provisioning with 'file'...
2018-06-21T11:39:16.750+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] connecting to TCP connection for SSH
2018-06-21T11:39:16.755+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] handshaking with SSH
2018-06-21T11:39:16.821+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Telling SSH config to forward to agent
2018-06-21T11:39:16.821+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Setting up a session to request agent forwarding
2018-06-21T11:39:16.821+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] opening new ssh session
2018-06-21T11:39:16.898+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [INFO] agent forwarding enabled
2018-06-21T11:39:16.898+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Uploading dir 'files/' to '/tmp'
2018-06-21T11:39:16.898+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] opening new ssh session
2018-06-21T11:39:16.909+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Starting remote scp process:  scp -rvt /tmp
2018-06-21T11:39:16.915+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Started SCP session, beginning transfers...
2018-06-21T11:39:16.915+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] SCP: starting directory upload: config
2018-06-21T11:39:16.917+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Beginning file upload...
2018-06-21T11:39:16.923+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] SCP: starting directory upload: nginx
2018-06-21T11:39:16.927+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Beginning file upload...
2018-06-21T11:39:16.933+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Beginning file upload...
2018-06-21T11:39:16.939+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] SCP: starting directory upload: php-fpm
2018-06-21T11:39:16.941+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Beginning file upload...
2018-06-21T11:39:16.945+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] SCP session complete, closing stdin pipe.
2018-06-21T11:39:16.945+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] Waiting for SSH session to complete.
2018-06-21T11:39:16.956+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [ERROR] scp stderr: "Sink: D0755 0 config\nSink: C0644 341 hosts\nSink: D0755 0 nginx\nSink: C0644 1011 dev-server-fpm.conf\nSink: C0644 766 nginx.conf\nSink: E\nSink: D0755 0 php-fpm\nSink: C0644 12941 php-fpm.conf\nSink: E\nSink: E\n"
aws_instance.dev-server: Provisioning with 'file'...
2018-06-21T11:39:16.957+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] connecting to TCP connection for SSH
2018-06-21T11:39:16.962+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:16 [DEBUG] handshaking with SSH
2018-06-21T11:39:17.010+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] Telling SSH config to forward to agent
2018-06-21T11:39:17.010+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] Setting up a session to request agent forwarding
2018-06-21T11:39:17.010+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] opening new ssh session
2018-06-21T11:39:17.084+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [INFO] agent forwarding enabled
2018-06-21T11:39:17.084+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] opening new ssh session
2018-06-21T11:39:17.093+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] Starting remote scp process:  scp -vt /
2018-06-21T11:39:17.099+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] Started SCP session, beginning transfers...
2018-06-21T11:39:17.099+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] Copying input data into temporary file so we can read the length
2018-06-21T11:39:17.151+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] Beginning file upload...
2018-06-21T11:39:17.171+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] SCP session complete, closing stdin pipe.
2018/06/21 11:39:17 [ERROR] root: eval: *terraform.EvalApplyProvisioners, err: Upload failed: Process exited with status 1
2018-06-21T11:39:17.171+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [DEBUG] Waiting for SSH session to complete.
2018/06/21 11:39:17 [ERROR] root: eval: *terraform.EvalSequence, err: Upload failed: Process exited with status 1
2018/06/21 11:39:17 [TRACE] [walkApply] Exiting eval tree: aws_instance.dev-server
2018/06/21 11:39:17 [TRACE] dag/walk: upstream errored, not walking "provisioner.file (close)"
2018/06/21 11:39:17 [TRACE] dag/walk: upstream errored, not walking "provisioner.remote-exec (close)"
2018-06-21T11:39:17.171+0900 [DEBUG] plugin.terraform: file-provisioner (internal) 2018/06/21 11:39:17 [ERROR] Process exited with status 1
2018/06/21 11:39:17 [TRACE] dag/walk: upstream errored, not walking "aws_eip.ip"
2018/06/21 11:39:17 [TRACE] dag/walk: upstream errored, not walking "provider.aws (close)"
2018/06/21 11:39:17 [TRACE] dag/walk: upstream errored, not walking "meta.count-boundary (count boundary fixup)"
2018/06/21 11:39:17 [TRACE] dag/walk: upstream errored, not walking "root"
2018/06/21 11:39:17 [TRACE] Preserving existing state lineage "f6356c09-e953-7086-c9c4-9c312bc6148b"
2018/06/21 11:39:17 [TRACE] Preserving existing state lineage "f6356c09-e953-7086-c9c4-9c312bc6148b"
2018/06/21 11:39:17 [TRACE] Preserving existing state lineage "f6356c09-e953-7086-c9c4-9c312bc6148b"

2018/06/21 11:39:17 [DEBUG] plugin: waiting for all plugin processes to complete...
Error: Error applying plan:

1 error(s) occurred:

* aws_instance.dev-server: Upload failed: Process exited with status 1

Expected Behavior

The second file also should be uploaded without fail.

Actual Behavior

First file provisioning works well, but second file provisioning failed with following error message.

2018/06/21 11:39:17 [DEBUG] plugin: waiting for all plugin processes to complete...
Error: Error applying plan:

1 error(s) occurred:

* aws_instance.dev-server: Upload failed: Process exited with status 1

Steps to Reproduce

  1. terraform init
  2. terraform apply <-

Additional Context

First files are just small configuration files. But data/dump.sql has about 11MB (It is just SQL file which was dumped from my own mysql).

Is there file size limit?

bug provisionefile v0.11
Source
picture mingrammer
👍1

All 11 comments

This issue has been automatically migrated to terraform-providers/terraform-provider-aws#4912 because it looks like an issue with that provider. If you believe this is _not_ an issue with the provider, please reply to terraform-providers/terraform-provider-aws#4912.

hashibot[bot] picture hashibot[bot] on 21 Jun 2018

When I've copied the data/dump.sql to ec2 instance manually with scp, it succeeded.

mingrammer picture mingrammer on 21 Jun 2018

hey @mingrammer

Sorry I noticed I've mis-tagged this - this should be opened on this repository and not the AWS repo - apologies for the confusion, I'll re-open this.

tombuildsstuff picture tombuildsstuff on 21 Jun 2018

This issue has been automatically migrated to terraform-providers/terraform-provider-aws#4918 because it looks like an issue with that provider. If you believe this is _not_ an issue with the provider, please reply to terraform-providers/terraform-provider-aws#4918.

hashibot[bot] picture hashibot[bot] on 21 Jun 2018

@tombuildsstuff I'm ok haha.

Today, I changed the source from data/dump.sql to data/, then it worked well without fails.

What's wrong with data/dump.sql?

mingrammer picture mingrammer on 22 Jun 2018
👍1

@mingrammer both specifying a file and a directory using the file provisioner as you're doing above should be fine (at least according to the docs). Whilst I can't see any notes/anything in the code about using multiple file provisioners to the same directory - does this work for you if you upload to different directories?

Thanks!

tombuildsstuff picture tombuildsstuff on 22 Jun 2018

I'll try that on working day!

mingrammer picture mingrammer on 22 Jun 2018

@tombuildsstuff Hi tom. I tried uploading the data/dump.sql to a different directory, but it was failed.

I think it is not a problem about same directory, because I put the file provisioning of data/dump.sql to first order, but failed. (So following file provisioning never ran)

mingrammer picture mingrammer on 25 Jun 2018

I should debug it

mingrammer picture mingrammer on 25 Jun 2018

Ran into this same issue with GCP today. Any reasons why this started happening?

mbukosky picture mbukosky on 25 Apr 2019
👍1

It seems resolved at least to me.

terraform --version

Terraform v0.11.7
+ provider.aws v1.28.0

Your version of Terraform is out of date! The latest version
is 0.12.0. You can update by downloading from www.terraform.io/downloads.html

My AWS provider was updated to provider.aws v1.28.0 from provider.aws v1.23.0.

@mbukosky Is your GCP provider latest version?

mingrammer picture mingrammer on 3 Jun 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

darron picture darron  路  3Comments
ketzacoatl picture ketzacoatl  路  3Comments
Seraf picture Seraf  路  3Comments
thebenwaters picture thebenwaters  路  3Comments
pawelsawicz picture pawelsawicz  路  3Comments