OpenMP


OpenMP is an application programming interface that supports multi-platform shared-memory multiprocessing programming in C, C++, and Fortran, on many platforms, instruction-set architectures and operating systems, including Solaris, AIX, FreeBSD, HP-UX, Linux, macOS, Windows and OpenHarmony. It consists of a set of compiler directives, library routines, and environment variables that influence run-time behavior.
OpenMP is managed by the nonprofit technology consortium OpenMP Architecture Review Board, jointly defined by a broad swath of leading computer hardware and software vendors, including Arm, AMD, IBM, Intel, Cray, HP, Fujitsu, Nvidia, NEC, Red Hat, Texas Instruments, and Oracle Corporation.
OpenMP uses a portable, scalable model that gives programmers a simple and flexible interface for developing parallel applications for platforms ranging from the standard desktop computer to the supercomputer.
An application built with the hybrid model of parallel programming can run on a computer cluster using both OpenMP and Message Passing Interface, such that OpenMP is used for parallelism within a node while MPI is used for parallelism between nodes. There have also been efforts to run OpenMP on software distributed shared memory systems, to translate OpenMP into MPI
and to extend OpenMP for non-shared memory systems.

Design

OpenMP is an implementation of multithreading, a method of parallelizing whereby a primary thread forks a specified number of sub-threads and the system divides a task among them. The threads then run concurrently, with the runtime environment allocating threads to different processors.
The section of code that is meant to run in parallel is marked accordingly, with a compiler directive that will cause the threads to form before the section is executed. Each thread has an ID attached to it which can be obtained using a function. The thread ID is an integer, and the primary thread has an ID of 0. After the execution of the parallelized code, the threads join back into the primary thread, which continues onward to the end of the program.
By default, each thread executes the parallelized section of code independently. Work-sharing constructs can be used to divide a task among the threads so that each thread executes its allocated part of the code. Both task parallelism and data parallelism can be achieved using OpenMP in this way.
The runtime environment allocates threads to processors depending on usage, machine load and other factors. The runtime environment can assign the number of threads based on environment variables, or the code can do so using functions. The OpenMP functions are included in a header file labelled in C/C++.

History

The OpenMP Architecture Review Board published its first API specifications, OpenMP for Fortran 1.0, in October 1997. In October the following year they released the C/C++ standard. 2000 saw version 2.0 of the Fortran specifications with version 2.0 of the C/C++ specifications being released in 2002. Version 2.5 is a combined C/C++/Fortran specification that was released in 2005.
Up to version 2.0, OpenMP primarily specified ways to parallelize highly regular loops, as they occur in matrix-oriented numerical programming, where the number of iterations of the loop is known at entry time. This was recognized as a limitation, and various task parallel extensions were added to implementations. In 2005, an effort to standardize task parallelism was formed, which published a proposal in 2007, taking inspiration from task parallelism features in Cilk, X10 and Chapel.
Version 3.0 was released in May 2008. Included in the new features in 3.0 is the concept of tasks and the task construct, significantly broadening the scope of OpenMP beyond the parallel loop constructs that made up most of OpenMP 2.0. OpenMP 3.1 was released in September 2011, enhancing tasking with features like task finalization and task dependencies. It also improved support for Fortran 2003 and added atomic construct enhancements.
Version 4.0 of the specification was released in July 2013. It adds or improves the following features: support for accelerators; atomics; error handling; thread affinity; tasking extensions; user defined reduction; SIMD support; Fortran 2003 support. OpenMP 4.0 also introduced support for accelerators via the target directive, enabling offloading computations to devices like GPUs or FPGAs. It also added SIMD directives for vectorization and user-defined reductions. Version 4.5 enhanced device support, improved tasking with dependencies, and introduced loop transformations for better optimization.
Version 5.0 added significant features like memory management for accelerators, support for deep copy in target directives, and loop transformation constructs. It also improved interoperability with other programming models like MPI. OpenMP 5.1 focused on usability, adding features like loop fusion, improved task reduction, and better support for C++11/14/17 constructs. Version 5.2 refined existing features, improved compatibility with modern hardware, and added support for descriptive loop directives..
Version 6.0 was released in November 2024. This version focuses on enhancing OpenMP for heterogeneous computing, integrating with AI/ML workloads, and improving performance portability across diverse architectures.
Note that not all compilers support the full set of features for the latest version/s.

Core elements

The core elements of OpenMP are the constructs for thread creation, workload distribution, data-environment management, thread synchronization, user-level runtime routines and environment variables.
In C/C++, OpenMP uses s. The OpenMP specific pragmas are listed below.

Thread creation

The pragma omp parallel is used to fork additional threads to carry out the work enclosed in the construct in parallel. The original thread will be denoted as master thread with thread ID 0.
Example : Display "Hello, world." using multiple threads.

  1. include
  2. include
int main

Use flag -fopenmp to compile using GCC:

$ gcc -fopenmp hello.c -o hello -ldl

Output on a computer with two cores, and thus two threads:

Hello, world.
Hello, world.

However, the output may also be garbled because of the race condition caused from the two threads sharing the standard output.

Hello, wHello, woorld.
rld.

Whether printf is atomic depends on the underlying implementation unlike C++11's std::cout, which is thread-safe by default.

Work-sharing constructs

Used to specify how to assign independent work to one or all of the threads.
  • omp for or omp do: used to split up loop iterations among the threads, also called loop constructs.
  • sections: assigning consecutive but independent code blocks to different threads
  • single: specifying a code block that is executed by only one thread, a barrier is implied in the end
  • master: similar to single, but the code block will be executed by the master thread only and no barrier implied in the end.
Example: initialize the value of a large array in parallel, using each thread to do part of the work

// define N as some constant for array length
  1. define N 100000
int main

This example is embarrassingly parallel, and depends only on the value of i. The OpenMP parallel for flag tells the OpenMP system to split this task among its working threads. The threads will each receive a unique and private version of the variable. For instance, with two worker threads, one thread might be handed a version of i that runs from 0 to 49999 while the second gets a version running from 50000 to 99999.

Variant directives

Variant directives are one of the major features introduced in OpenMP 5.0 specification to facilitate programmers to improve performance portability. They enable adaptation of OpenMP pragmas and user code at compile time. The specification defines traits to describe active OpenMP constructs, execution devices, and functionality provided by an implementation, context selectors based on the traits and user-defined conditions, and metadirective and declare directive directives for users to program the same code region with variant directives.
  • The metadirective is an executable directive that conditionally resolves to another directive at compile time by selecting from multiple directive variants based on traits that define an OpenMP condition or context.
  • The declare variant directive has similar functionality as metadirective but selects a function variant at the call-site based on context or user-defined conditions.
The mechanism provided by the two variant directives for selecting variants is more convenient to use than the C/C++ preprocessing since it directly supports variant selection in OpenMP and allows an OpenMP compiler to analyze and determine the final directive from variants and context.

// define array size as some constant for array length:
  1. define N 100000
// code adaptation using preprocessing directives
int v1;
int v2;
int v3;
  1. if defined
  2. pragma omp target teams distribute parallel for map map
for
  1. else
  2. pragma omp target parallel for map map
for
  1. endif
// code adaptation using metadirective in OpenMP 5.0
int v1;
int v2;
int v3;
  1. pragma omp target map map
  2. pragma omp metadirective \
when \
default
for