Skip to content

Introduction to Using WRFExpress for Running WRF on AWS

WRFExpress is designed to streamline the process of running Weather Research and Forecasting (WRF) models on AWS clusters. This tool simplifies complex configurations and automates various steps, making it accessible even to those without deep technical expertise. Here’s how you can utilize WRFExpress to get started with your WRF simulations.

Website Interface

The web interface of WRFExpress allows you to easily define your simulation domain by selecting an area on an interactive map. Here’s a step-by-step guide on how to use it:

  1. Access the Website: Navigate to the WRFExpress website and log in with your credentials.
  2. Select a Domain: Use the interactive map to draw rectangles defining your simulation domains. You can create multiple nested domains by drawing new rectangles within existing ones.
  3. Configuration Settings: On the right side of the screen, fill in the necessary configuration details:
    • Orcid ID: Enter your ORCID ID.
    • API Token: Provide your API token.
    • Start Date and End Date: Specify the start and end dates for your simulation period.
    • Time Interval: Set the time interval for data points (e.g., 6 hours).
    • DX and DY: Specify the grid spacing for your simulation.
  4. Update Configuration: After filling in the details, click on "Update Configuration" to save your settings.
  5. Generate Namelist and Download Data Code: Click on "Generate Namelist" to create the namelist.wps file and get the Python script for downloading necessary data files. Use the "Copy to Clipboard" button to copy the generated code and namelist.

Running WRF on AWS Cluster

After configuring your simulation domain and generating necessary files, follow these steps to run WRF on the AWS cluster:

  1. Go to WRFExpress.com and log in.
  2. Select your desired simulation area using the interactive map drawing tool.
  3. After entering the necessary info and clicking on the "Update Configuration" button, copy the namelist text by clicking on the "Generate Namelist" and "Copy to Clipboard" buttons.
  4. On AWS Parallel Cluster WRF-P2, paste the copied namelist in the terminal and press Enter.
  5. After pressing Enter, the python3 main.py command will execute automatically.
Image title
Directory structure of the automation files utilized in the project.
  1. Select the step to start from (1-5):
    • 1. Geogrid: Prepares the geographical data for the simulation. If selected, the process will continue through steps 2-4.
    • 2. UNGRIB: Processes the meteorological data for the model. If selected, the process will continue through steps 3-4.
    • 3. METGRID: Interpolates the meteorological data onto the model grid. If selected, the process will continue through step 4.
    • 4. WRF: Runs the Weather Research and Forecasting model.
    • 5. Delete: Cleans up temporary and intermediate files from the WRF run directory. This step is optional and can be selected independently.
  2. To initiate the process, you can start by pressing 5 to perform a cleanup operation. After the cleanup, select 1 to begin with the Geogrid step, which will continue automatically through steps 2 (UNGRIB), 3 (METGRID), and 4 (WRF).
  3. Choose 1 for the node number.
  4. Choose 2 for the tasks per node.
  5. Choose ”Hourly” for the NC file output resolution.
  6. Select the parameterization scheme. You will be prompted to choose from available options (1) and type "yes" to confirm your selection.
  7. Wait for the run to complete.
  8. You can check the progress by opening another shell and typing squeue in the terminal. You can see the progress when the model has started running.
  9. Note that if you need to rerun the model using the same config, you can call main.py again and choose step 4.
  10. When the run is complete, you can see the data on the WRFExpress website after refreshing the screen.
  11. Download the data by selecting the file, files are named with the time (e.g., 2019-07-23 _16-05-18) when the run was complete, clicking the "Copy AWS Commands" and pasting the code in the terminal.

Key Features of the Python Script

  • Shell Command Execution: Automates the execution of multiple WRF-related shell scripts.
  • S3 Integration: Uploads simulation outputs to an S3 bucket and generates a presigned URL for easy download.
  • Progress Monitoring: Uses the rich library to display real-time progress of script execution and job queue status.
  • Error Handling: Provides informative logs for troubleshooting any issues that arise during execution.

By following these steps, WRFExpress simplifies the setup and execution of WRF simulations on AWS, allowing users to focus more on their research and less on the technical complexities of the modeling process.