NCSI - Parallel/Cluster Workshop - HPC Overview

What is High Performance Computing?

• Fastest technology available at any given point in time.

• A (rough) hardware taxonomy:
  1. SMP and CC-NUMA machines (MIMD)
  2. Beowulf clusters built with COTS components (MIMD)
  3. Grid resources (MIMDish)
  4. Others - vector machines (SIMD), clusters of SMP machines, massively parallel processors

What's in a Computer?

• CPU
• RAM
• Disk
• Network interface

• Relative performance and quantity of each
• Ratio of memory to CPU is changing, forcing a change in algorithms

What's in a CPU?

• Microcode - hardware is just petrified software
• Fetch, decode, execute
• Floating Point Unit
• Integer Unit
• Memory Management Unit

• CISC and RISC
• Scaler
• Super Scaler
• AltiVec and SSE - small scale SIMD

• Out of order execution - how the compiler and scheduler can change your results

Memory Hierarchy

• Registers
• Cache - L1, L2, split data and instruction
• RAM
• Disk

• Data locality
• Cache thrashing

Cluster Specific Topics

• Network fabric and communications protocols
  1. Latency and bandwidth review
  2. Ethernet, Fast ethernet, Gigabit ethernet, channel bonding
  3. Switches - un-managed, managed, backplane capacity
  4. Myrinet and other high performance interconnects
  5. VIA

• Clusters have a very good memory to CPU ratio, particularly good for problems that are memory intensive.

• Distributed shared memory, one way to port code to a cluster environment.

• Reliability of the cluster vs an individual machine.

Benchmarking and Tuning

• There are many benchmarks available, generally the best one to use is your own application or a benchmarking kernel derived from your own application.

• Using the 80/20 rule to identify where the heavy lifting is within your application. From this a benchmarking kernel can be developed.

• Profiling, statement and statistical

• Tuning is best done from the outside in, that is:
  1. the application
  2. the compiler
  3. the operating system
  4. the network

• When purchasing new hardware it is often possible to obtain loaner gear from vendors for evaluation. This is the perfect time to have a benchmarking kernel available since they tend to be much easier to port and run.

Other

• Monitoring service availability; Nagios, Big Sister

• Measuring and recording resource consumption; Ganglia, Cricket

• Lots of data to manage - PVFS