Parallel computing is discussed in Slides from lectures starting with Lecture 12.
The recent book of [Rauber-Ruenger] gives a good introduction to many topics, including OpenMP (starting in Section 6.3) and MPI Note that there is an on-line version available from UW computers here.
See [Lin-Snyder] for a good general introduction to some concepts of parallel computing.
The book [Scott-Clark-Bagheri] also has a good general introduction and discusses in more detail some algorithmic issues of importance in scientific computing.
See the Bibliography and further reading for more references.