Parallel and Distributed Computing Related Links
Posted on Sunday, January 22, 2006 at 12:20 AM by Malcolm
Parallel Computing Books
- Introduction to Parallel Computing (2nd Edition) by Ananth Grama, George Karypis, Vipin Kumar and Anshul Gupta
- Patterns for Parallel Programming by Timothy G. Mattson, Beverly A. Sanders, and Berna L. Massingill
- Parallel Programming in C with MPI and OpenMP by Michael J. Quinn
Parallel and Distributed Computing Related Sites/Blogs
- xlmpp XtremeLargeMPP: A Real World Supercomputing Blog
- Computing at Scale: New directions in massively parallel processing for business and the web by Bill McColl
- Research at Intel
- InsideHPC.com
- Amazon Web Services Blog
Others
Gustafson Law: Reevaluating Amdahl's Law
(Link) Parallelizing an algorithm, however, will soon run into Amdahl's Law, which states that the speedup from parallelizing a process is an inverse function of the portion of the process that is necessarily sequential. Thus, not only do we need good algorithms to minimize the sequential portion of a process, but there is also an upper bound to the speedup we can achieve, even with an infinite number of parallel processors.
While that may be a depressing result, parallel processing has been rescued by Gustafson's Law, which states that while the speedup for a fixed-size process is indeed limited by Amdahl's Law, we can achieve nearly linear speedups if we consider the amount of processing that can be done in a fixed period.
