Aws-parallelcluster: Slurm commands all are missing perl Switch.pm - Perl path seems corrupted?

[anders-lgi]

Environment:

• AWS ParallelCluster / CfnCluster version [e.g. aws-parallelcluster-2.5.1]: using 2.5.1
• Config (i.e. ~/.parallelcluster/config) with personal data removed
  [aws]
  aws_region_name = us-east-1

[global]
cluster_template = default
update_check = true
sanity_check = true

[aliases]
ssh = ssh {CFN_USER}@{MASTER_IP} {ARGS}

[cluster default]
key_name = us-east-1
base_os = centos7
scheduler = slurm
maintain_initial_size = true
vpc_settings = default
master_instance_type = m5.xlarge
compute_instance_type = c5d.9xlarge
ebs_settings = default
custom_ami = ami-04cb48b8721aa368c

[vpc default]
vpc_id = vpc-0cf0a227aca63aeb1
master_subnet_id = subnet-00d97f8a25ad37b10
compute_subnet_id = subnet-0c02cb73efee58759
use_public_ips = false

[ebs default]
volume_size = 200

Bug description and how to reproduce:

Tried the command:

qsub test.batch

Got this result:

Can't locate Switch.pm in @INC (@INC contains: /opt/slurm/lib64/perl5 /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /opt/slurm/bin/qsub line 51, <DATA> line 604.
BEGIN failed--compilation aborted at /opt/slurm/bin/qsub line 51, <DATA> line 604.

Have replicated the issue in Canada region and using default centos7 AMI.

Other commands (e.g., qstat) return the same issue.

Thank you.

Anders.

[sean-smith]

Have you tried using our pre-built Centos 7 AMI? Just leave out the custom_ami line.

[demartinofra]

qsub, qstat, etc are not Slurm native commands. Slurm offers a wrapper of PBS scheduling commands as described here which requires some Perl dependencies that are missing in ParallelCluster AMI.

On one side I would recommend you to use Slurm native commands and this doc can help in the transition. In case you still need to use pbs/torque wrappers you could do that by installing libswitch-perl package on the cluster master node.

[anders-lgi]

Thank you @sean-smith. I did try it with a pre-built AMI and got the same result. But, as @demartinofra points out (and which I had completely forgotten), I should be using sbatch and not qsub.

Thank you both for the reply. Sorry for causing trouble! Hopefully, this issue will help others.

I am very impressed at how easy it is to launch an HPC cluster on AWS! Excellent work.