Aws-parallelcluster: Unable to create cluster with snapshot

Created on 20 Sep 2019 · 2Comments · Source: aws/aws-parallelcluster

Environment:

AWS ParallelCluster / CfnCluster version: aws-parallelcluster-2.4.1
OS: Ubuntu 16.04
Scheduler: Slurm
Master instance type: c5.large
Compute instance type: c5.12xlarge

Bug description and how to reproduce:
I'm attempting to create a cluster utilizing a snapshot of another cluster. When I create the cluster, it hangs on Status: ComputeFleet - CREATE_COMPLETE until it times out and reports the following error...

~$ pcluster create dart-cluster2 -t dart-cluster --norollback
Beginning cluster creation for cluster: dart-cluster2
Creating stack named: parallelcluster-dart-cluster2
Status: parallelcluster-dart-cluster2 - CREATE_FAILED                                
Cluster creation failed.  Failed events:
  - AWS::CloudFormation::Stack parallelcluster-dart-cluster2 The following resource(s) failed to create: [MasterServerWaitCondition]. 
  - AWS::CloudFormation::WaitCondition MasterServerWaitCondition Failed to receive 1 resource signal(s) within the specified duration

I am able to create clusters without the snapshot (using an identical configuration and commenting out just ebs_snapshot_id) and I can see in my EC2 Dashboard that it successfully creates a volume from the snapshot and then attaches it to an c5.large instance, as it should.

Additional context:
Any other context about the problem. E.g.:

configuration file without any credentials or personal data.

[aws]
aws_region_name = us-east-1
[global]
update_check = true
sanity_check = true

[aliases]
ssh = ssh {CFN_USER}@{MASTER_IP} {ARGS}

[cluster dart-cluster]
key_name = dart-cluster
vpc_settings = public
master_instance_type = c5.large
compute_instance_type = c5.12xlarge
initial_queue_size = 0
max_queue_size = 5
scheduler = slurm
cluster_type = ondemand
spot_price = 0.4
placement_group = DYNAMIC
placement = compute
master_root_volume_size = 200
compute_root_volume_size = 200
base_os = ubuntu1604
scaling_settings = 10min
ebs_settings = dart

[scaling 10min]
scaledown_idletime = 10

[ebs dart]
shared_dir = /shared/
volume_size = 200
volume_type = gp2
ebs_snapshot_id = snap-06ec2e7141c4ffc99

pre/post-install scripts, if any
None.
screenshots, if useful
None.
if the cluster fails creation, please re-execute create action using --norollback option and attach /var/log/cfn-init.log, /var/log/cloud-init.log and /var/log/cloud-init-output.log files from the Master node

cloud-init-output.log
cloud-init.log
cfn-init.log

if a compute node was terminated due to failure, there will be a directory /home/logs/compute. Attach one of the instance-id.tar.gz from that directory

[Didn't get this far]

if you encounter scaling problems please attach /var/log/nodewatcher from the Compute node and /var/log/jobwatcher and /var/log/sqswatcher from the Master node

[Didn't get this far]

Thanks!

-Joe

bug

Source