Skip to content

This tutorial covers from the cluster creation using PClusterManager to the configuration and compilation of WRF Version 4.3.3 (Weather Research and Forecasting) and WPS version 4.3.1 (WRF Preprocessing System) using Spack.

Create Cluster

We have to create a computer cluster on AWS to install and run WRF. The links to AWS and PCluster Managers are below. Go to the PCluster link to create the cluster after having a account on AWS.

  • Link to AWS
  • Link to PCluster Manager

Alt Text

Upload YML file

PCluster Manager makes it easy for us to create and manage the clusters through interface. This is an example configuration. For your first time, you may choose to upload the yml file we have prepared for you and configure to your prefered headnode and shared storage later by updating the yaml file further. In order for AWS to verify your identity, you will need to create your own EC2 Key Pair using Amazon EC2.

YML file for EBS

YAML File
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
HeadNode:
  InstanceType: c5a.xlarge
  Ssh:
    KeyName: PClusterManager
  Networking:
    SubnetId: subnet-0412e7315562aa1da
  LocalStorage:
    RootVolume:
      VolumeType: gp3
      Size: 50
  Iam:
    AdditionalIamPolicies:
      - Policy: arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
      - Policy: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
  Dcv:
    Enabled: true
Scheduling:
  Scheduler: slurm
  SlurmQueues:
    - Name: queue0
      ComputeResources:
        - Name: queue0-hpc6a48xlarge
          MinCount: 0
          MaxCount: 10
          InstanceType: hpc6a.48xlarge
          Efa:
            Enabled: true
            GdrSupport: true
          DisableSimultaneousMultithreading: true
      Networking:
        SubnetIds:
          - subnet-01a39ae1f7194644a
        PlacementGroup:
          Enabled: true
      ComputeSettings:
        LocalStorage:
          RootVolume:
            VolumeType: gp3
            Size: 50
      Iam:
        AdditionalIamPolicies:
          - Policy: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
Region: us-east-2
Image:
  Os: alinux2
SharedStorage:
  - Name: Ebs0
    StorageType: Ebs
    MountDir: /shared
    EbsSettings:
      VolumeType: gp3
      DeletionPolicy: Delete
      Size: '200'
      Encrypted: false

YML file for EFS

YAML File
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
HeadNode:
  InstanceType: c5a.xlarge
  Ssh:
    KeyName: PClusterManager
  Networking:
    SubnetId: subnet-0412e7315562aa1da
  LocalStorage:
    RootVolume:
      VolumeType: gp3
      Size: 50
  Iam:
    AdditionalIamPolicies:
      - Policy: arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
      - Policy: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
  Dcv:
    Enabled: true
  Imds:
    Secured: true
Scheduling:
  Scheduler: slurm
  SlurmQueues:
    - Name: queue0
      ComputeResources:
        - Name: queue0-hpc6a48xlarge
          MinCount: 0
          MaxCount: 10
          Efa:
            Enabled: true
            GdrSupport: true
          Instances:
            - InstanceType: hpc6a.48xlarge
      Networking:
        SubnetIds:
          - subnet-01a39ae1f7194644a
        PlacementGroup:
          Enabled: true
      ComputeSettings:
        LocalStorage:
          RootVolume:
            VolumeType: gp3
            Size: 50
      Iam:
        AdditionalIamPolicies:
          - Policy: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
  SlurmSettings: {}
Region: us-east-2
Image:
  Os: alinux2
SharedStorage:
  - Name: Efs0
    StorageType: Efs
    MountDir: /shared
    EfsSettings:
      ThroughputMode: bursting
      DeletionPolicy: Retain
Imds:
  ImdsSupport: v2.0
Tags:
  - Key: parallelcluster-ui
    Value: 'true'

Create Cluster

Alt Text

  • Dry Run

You can opt to dry run your cluster before creating it. After dry run, you will see this message printed: Request would have succeeded, but DryRun flag is set.

  • Create

Creating cluster will take around 15-20 minutes. You may check if you are interested in the steps or current progress of stack events.

Create in Progress

Alt Text

Create Complete

Now you should be able to see the cluster WRF in CREATE_COMPLETE status from your PCluster Manager interface. Click on Shell and login to your own AWS account to access the terminal.

Alt Text