Introduction to Parallel and Distributed Computing (SS 2010)

Introduction to Parallel and Distributed Computing (326.603, SS 2010)

---

Time: Monday, 8:30-10:00, Room: T 642, Start: March 1, 2010

---

The efficient application of parallel and distributed systems (multi-processors and computer networks) is nowadays an important task for computer scientists and mathematicians. The goal of this course is to provide an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modeling and analysis, experimenting and measuring.

Class presentation will be accompanied by hands-on experience on an SGI Altix 4700 distributed shared memory multiprocessor using the following programming models:

• Auto-parallelization and OpenMP (Intel C/C++ or Fortran),
  
• Java networking and multi-threading API,
• Message Programming Interface (MPI-C/C++ or Fortran).

Some background in programming in C/C++ and/or Fortran and/or Java is assumed. Course grades will be based on a couple of programming exercises.

---

To take part in the course, you have to enrol in the KUSSS system. Since the exercises will be submitted via Moodle, you also have to login in Moodle and register as a course participant. You will then also receive per email all messages posted in the News forum.

Topics

• Introduction
• The SGI Altix System
• Semi-Automatic Parallelization
• Performance of Parallel Programs
• Multi-Threaded Client/Server Programming
  
• Parallel Program Design
  
• The Message Passing Interface MPI
• Distributed Memory Algorithms

Literature

Restricted Material
The password to this section is handed out in class.

Introduction

The SGI Altix System
Architecture and operating system.

• SGI Altix 4700 (datasheet)
• The SGI Altix 3000 Global Shared Memory Architecture
  
• SGI Altix Hardware Architecture
• SGI Altix Linux and Compiling Environment

Auto-Parallelization and OpenMP
Basic concepts and their application on the Altix; material partially taken from the SGI developer documentation.

• Automatic Parallelism (Compiler Output)
• Automatic Parallelization Example
• OpenMP Example
  
• Linux Application Tuning Guide
• SGI Altix Applications Development and Optimization (restricted)
  
• Programming Documentation
  
• Intel C++ Compiler User and Reference Guide
• Intel Fortran Compiler User and Reference Guide
• Using OpenMP to Parallelize a Program
• Pragma Directives for Parallel Processing
  
• OpenMP Tutorial
  

Performance of Parallel Programs
Modeling and analyzing the performance of parallel programs, some material from Chapter 3 of Foster's Designing and Building Parallel Programs.

• Presentation
• Figures (restricted)

Multi-Threaded Client/Server Programming
Developing Internet-based parallel programs by simple language/operating system mechanisms.

• Alle Fäden in der Hand (in German)
• Sockets - The GNU C Library
  
• Thread-Level Parallelism (Section from "Topics in IRIX Programming")
  
• Java networking and multi-threading API
• Numerische Aufgaben mit Java lösen (restricted)

The Message Passing Interface MPI
Programming in MPI based on Chapter 8 of Foster's Designing and Building Parallel Programs.

• Presentation (restricted)
• Message Passing Interface (restricted)
  
• MPI Home Page
• SGI Altix MPI - Programming and Tuning
• Message Passing Toolkit: MPI Programmer's Manual

Distributed Memory Algorithms

Exercises
Four programming/benchmarking exercises will be handed out which will be used for grading.