NOC:Introduction to Parallel Programming in OpenMP (USB)

Media Storage Type : 32 GB USB Stick

NPTEL Subject Matter Expert : Dr. Yogish Sabharwal

NPTEL Co-ordinating Institute : IIT Delhi

NPTEL Lecture Count : 84

NPTEL Course Size : 5.0 GB

NPTEL PDF Text Transcription : Available and Included

NPTEL Subtitle Transcription : Available and Included (SRT)

Lecture Titles:

Lecture 1 - Introduction to Parallel Programming
Lecture 2 - Parallel Architectures and Programming Models
Lecture 3 - Pipelining
Lecture 4 - Superpipelining and VLIW
Lecture 5 - Memory Latency
Lecture 6 - Cache and Temporal Locality
Lecture 7 - Cache, Memory bandwidth and Spatial Locality
Lecture 8 - Intuition for Shared and Distributed Memory architectures
Lecture 9 - Shared and Distributed Memory architectures
Lecture 10 - Interconnection networks in Distributed Memory architectures
Lecture 11 - OpenMP: A parallel Hello World Program
Lecture 12 - Program with Single thread
Lecture 13 - Program Memory with Multiple threads and Multi-tasking
Lecture 14 - Context Switching
Lecture 15 - OpenMP: Basic thread functions
Lecture 16 - OpenMP: About OpenMP
Lecture 17 - Shared Memory Consistency Models and the Sequential Consistency Model
Lecture 18 - Race Conditions
Lecture 19 - OpenMP: Scoping variables and some race conditions
Lecture 20 - OpenMP: thread private variables and more constructs
Lecture 21 - Computing sum: first attempt at parallelization
Lecture 22 - Manual distribution of work and critical sections
Lecture 23 - Distributing for loops and reduction
Lecture 24 - Vector-Vector operations (Dot product)
Lecture 25 - Matrix-Vector operations (Matrix-Vector Multiply)
Lecture 26 - Matrix-Matrix operations (Matrix-Matrix Multiply)
Lecture 27 - Introduction to tasks
Lecture 28 - Task queues and task execution
Lecture 29 - Accessing variables in tasks
Lecture 30 - Completion of tasks and scoping variables in tasks
Lecture 31 - Recursive task spawning and pitfalls
Lecture 32 - Understanding LU Factorization
Lecture 33 - Parallel LU Factorization
Lecture 34 - Locks
Lecture 35 - Advanced Task handling
Lecture 36 - Matrix Multiplication using tasks
Lecture 37 - The OpenMP Shared Memory Consistency Model
Lecture 38 - Applications finite element method
Lecture 39 - Applications deep learning
Lecture 40 - Introduction to MPI and basic calls
Lecture 41 - MPI calls to send and receive data
Lecture 42 - MPI calls for broadcasting data
Lecture 43 - MPI non blocking calls
Lecture 44 - Application distributed histogram updation
Lecture 45 - MPI collectives and MPI broadcast
Lecture 46 - MPI gathering and scattering collectives
Lecture 47 - MPI reduction and alltoall collectives
Lecture 48 - Discussion on MPI collectives design
Lecture 49 - Characteriziation of interconnects
Lecture 50 - Linear arrays 2D mesh and torus
Lecture 51 - d dimensional torus
Lecture 52 - Hypercube
Lecture 53 - Trees and cliques
Lecture 54 - Hockney model
Lecture 55 - Broadcast and Reduce with recursive doubling
Lecture 56 - Scatter and Gather with recursive doubling
Lecture 57 - Reduce scatter and All gather with recursive doubling
Lecture 58 - Discussion of message sizes in analysis
Lecture 59 - Revisiting Reduce scatter on 2D mesh
Lecture 60 - Reduce scatter and Allreduce on the Hypercube
Lecture 61 - Alltoall on the Hypercube
Lecture 62 - Lower bounds
Lecture 63 - Pipeline based algorithm for Allreduce
Lecture 64 - An improved algorithm for Alltoall on the Hypercube using E-cube routing
Lecture 65 - Pipeline based algorithm for Broadcast
Lecture 66 - Introduction to parallel graph algorithms
Lecture 67 - Breadth First Search BFS using matrix algebra
Lecture 68 - BFS Shared memory parallelization using OpenMP
Lecture 69 - Distributed memory settings and data distribution
Lecture 70 - Distributed BFS algorithm
Lecture 71 - Performance considerations
Lecture 72 - Prims Algorithm
Lecture 73 - OpenMP based shared memory parallelization for MST
Lecture 74 - MPI based distributed memory parallelization for MST
Lecture 75 - Sequential Algorithm Adaptation from Prims
Lecture 76 - Parallelization Strategy for Prims algorithm
Lecture 77 - Dry run with the parallel strategy
Lecture 78 - Johnsons algorithm with 1D data distribution
Lecture 79 - Speedup analysis on a grid graph
Lecture 80 - Floyds algorithm for all pair shortest paths
Lecture 81 - Floyds algorithm with 2D data distribution
Lecture 82 - Adaptation to transitive closures
Lecture 83 - Parallelization strategy for connected components
Lecture 84 - Analysis for parallel connected components