High Performance Scientific Computing   Coursera Edition

OpenMP

Makefiles

# MPI¶

MPI stands for Message Passing Interface and is a standard approach for programming distributed memory machines such as clusters, supercomputers, or heterogeneous networks of computers. It can also be used on a single shared memory computer, although it is often more cumbersome to program in MPI than in OpenMP.

## MPI implementations¶

A number of different implementations are available (open source and vendor supplied for specific machines). See this list, for example.

## MPI on the class VM¶

The VM has open-mpi partially installed.

You will need to do the following:

$sudo apt-get update$ sudo apt-get install openmpi-dev

On other Ubuntu installations you will also have to do:

$sudo apt-get install openmpi-bin # Already on the VM You should then be able to do the following: $ cd $UWHPSC/codes/mpi$ mpif90 test1.f90

## Reduction example¶

The next example uses MPI_REDUCE to add up partial sums computed by independent processes.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 ! $UWHPSC/codes/mpi/pisum1.f90 ! Computes pi using MPI. ! Compare to$UWHPSC/codes/openmp/pisum2.f90 program pisum1 use mpi implicit none integer :: ierr, numprocs, proc_num, points_per_proc, n, & i, istart, iend real (kind=8) :: x, dx, pisum, pisum_proc, pi call mpi_init(ierr) call mpi_comm_size(MPI_COMM_WORLD, numprocs, ierr) call mpi_comm_rank(MPI_COMM_WORLD, proc_num, ierr) ! Ask the user for the number of points if (proc_num == 0) then print *, "Using ",numprocs," processors" print *, "Input n ... " read *, n end if ! Broadcast to all procs; everybody gets the value of n from proc 0 call mpi_bcast(n, 1, MPI_INTEGER, 0, MPI_COMM_WORLD, ierr) dx = 1.d0/n ! Determine how many points to handle with each proc points_per_proc = (n + numprocs - 1)/numprocs if (proc_num == 0) then ! Only one proc should print to avoid clutter print *, "points_per_proc = ", points_per_proc end if ! Determine start and end index for this proc's points istart = proc_num * points_per_proc + 1 iend = min((proc_num + 1)*points_per_proc, n) ! Diagnostic: tell the user which points will be handled by which proc print '("Process ",i2," will take i = ",i6," through i = ",i6)', & proc_num, istart, iend pisum_proc = 0.d0 do i=istart,iend x = (i-0.5d0)*dx pisum_proc = pisum_proc + 1.d0 / (1.d0 + x**2) enddo call MPI_REDUCE(pisum_proc,pisum,1,MPI_DOUBLE_PRECISION,MPI_SUM,0, & MPI_COMM_WORLD,ierr) if (proc_num == 0) then pi = 4.d0 * dx * pisum print *, "The approximation to pi is ",pi endif call mpi_finalize(ierr) end program pisum1