Skip to content

Optimize ACC port of atm_advance_scalars_work:4760#1492

Draft
abishekg7 wants to merge 6 commits into
MPAS-Dev:developfrom
abishekg7:atmosphere/opt_adv_scalars_4760
Draft

Optimize ACC port of atm_advance_scalars_work:4760#1492
abishekg7 wants to merge 6 commits into
MPAS-Dev:developfrom
abishekg7:atmosphere/opt_adv_scalars_4760

Conversation

@abishekg7

Copy link
Copy Markdown
Collaborator

This PR introduces optimizations for the OpenACC port of atm_advance_scalars_work:4760.

The table below lists the timings for a real, global 30km experiment on A100 GPU with nvhpc, and Derecho CPUs with several compiler suites.

Version GPU kernel time (ms) CPU timer - nvhpc CPU timer - gnu   CPU timer - intel25
base 22.569 0.02903 0.03156   0.03324
1 In case default, swap the seq and vector collapse(2) loops and fuse horiz_flux_arr 18 0.02908 0.03318   0.03389
2 Remove ica loop, directly subst advCellsForEdge 16.832 0.03088 0.03248   0.03351
3 case 10, remove separate scalar_weight2 loop and inline it in next loop 12.023 0.03459 0.03204   0.0324
4 local_sign 13.83 0.03459 0.032   0.03239

For nvhpc gpu runs, -gpu=math_uniform is introduced as a build flag to ensure optimizations are bit-identical, and the we report the numbers using NV_ACC_TIME=1. The GPU runs are on 1 Derecho GPU node, using 1 A100 via 1 MPI task.

The numbers reported for the CPU runs with gnu and intel compilers use the newly-added timers local to this region, and are averaged across three runs. The CPU runs use a single derecho CPU node each, fully subscribed to 128 MPI tasks.

This work was completed in part at the NCAR/NLR/NOAA Open Hackathon, part of the Open Hackathons program. The authors would like to acknowledge OpenACC-Standard.org for their support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant