The boostmpi module contains Python wrappers for Boost.MPI. Boost.MPI is a C++ interface to the Message Passing Interface 1.1, a high-performance message passing library for parallel programming.
This module supports the most commonly used subset of MPI 1.1. All communication operations can transmit any Python object that can be pickled and unpickled, along with C++-serialized data types and separation of the structure of a data type from its content. Collectives that have a user-supplied functions, such as reduce() or scan(), accept arbitrary Python functions, and all collectives can operate on any serializable or picklable data type.
Boost.MPI can transmit user-defined data in several different ways. Most importantly, it can transmit arbitrary Python objects by pickling them at the sender and unpickling them at the receiver, allowing arbitrarily complex Python data structures to interoperate with MPI.
Boost.MPI also supports efficient serialization and transmission of C++ objects (that have been exposed to Python) through its C++ interface. Any C++ type that provides (de-)serialization routines that meet the requirements of the Boost.Serialization library is eligible for this optimization, but the type must be registered in advance. To register a C++ type, invoke the C++ function: boostmpi::register_serialized
Finally, Boost.MPI supports separation of the structure of an object from the data it stores, allowing the two pieces to be transmitted separately. This “skeleton/content” mechanism, described in more detail in a later section, is a communication optimization suitable for problems with fixed data structures whose internal data changes frequently.
Boost.MPI supports all of the MPI collectives (scatter, reduce, scan, broadcast, etc.) for any type of data that can be transmitted with the point-to-point communication operations. For the MPI collectives that require a user-specified operation (e.g., reduce and scan), the operation can be an arbitrary Python function. For instance, one could concatenate strings with all_reduce:
mpi.all_reduce(my_string, lambda x,y: x + y)
The following module-level functions implement MPI collectives:
- all_gather(): Gather the values from all processes.
- all_reduce(): Combine the results from all processes.
- all_to_all(): Every process sends data to every other process.
- broadcast(): Broadcast data from one process to all other processes.
- gather(): Gather the values from all processes to the root.
- reduce(): Combine the results from all processes to the root.
- scan(): Prefix reduction of the values from all processes.
- scatter(): Scatter the values stored at the root to all processes.
Boost.MPI provides a skeleton/content mechanism that allows the transfer of large data structures to be split into two separate stages, with the ‘skeleton’ (or, ‘shape’) of the data structure sent first and the content (or, ‘data’) of the data structure sent later, potentially several times, so long as the structure has not changed since the skeleton was transferred. The skeleton/content mechanism can improve performance when the data structure is large and its shape is fixed, because while the skeleton requires serialization (it has an unknown size), the content transfer is fixed-size and can be done without extra copies.
To use the skeleton/content mechanism from Python, you must first register the type of your data structure with the skeleton/content mechanism from C++. The registration function is:
boostmpi::register_skeleton_and_content
and resides in the <boostmpi.hpp> header.
Once you have registered your C++ data structures, you can extract the skeleton for an instance of that data structure with skeleton(). The resulting SkeletonProxy can be transmitted via the normal send routine, e.g.:
mpi.world.send(1, 0, skeleton(my_data_structure))
SkeletonProxy objects can be received on the other end via recv(), which stores a newly-created instance of your data structure with the same ‘shape’ as the sender in its ‘object’ attribute:
shape = mpi.world.recv(0, 0)
my_data_structure = shape.object
Once the skeleton has been transmitted, the content (accessed via get_content) can be transmitted in much the same way. Note, however, that the receiver also specifies get_content(my_data_structure) in its call to receive:
if mpi.rank == 0:
mpi.world.send(1, 0, get_content(my_data_structure))
else:
mpi.world.recv(0, 0, get_content(my_data_structure))
Of course, this transmission of content can occur repeatedly, if the values in the data structure–but not its shape–changes.
The skeleton/content mechanism is a structured way to exploit the interaction between custom-built MPI datatypes and MPI_BOTTOM, to eliminate extra buffer copies.
Boost.MPI is a C++ library whose facilities have been exposed to Python via the Boost.Python library. Since the Boost.MPI Python bindings are build directly on top of the C++ library, and nearly every feature of C++ library is available in Python, hybrid C++/Python programs using Boost.MPI can interact, e.g., sending a value from Python but receiving that value in C++ (or vice versa). However, doing so requires some care. Because Python objects are dynamically typed, Boost.MPI transfers type information along with the serialized form of the object, so that the object can be received even when its type is not known. This mechanism differs from its C++ counterpart, where the static types of transmitted values are always known.
The only way to communicate between the C++ and Python views on Boost.MPI is to traffic entirely in Python objects. For Python, this is the normal state of affairs, so nothing will change. For C++, this means sending and receiving values of type boost::python::object, from the Boost.Python library. For instance, say we want to transmit an integer value from Python:
comm.send(1, 0, 17)
In C++, we would receive that value into a Python object and then ‘extract’ an integer value:
boost::python::object value;
comm.recv(0, 0, value);
int int_value = boost::python::extract<int>(value);
In the future, Boost.MPI will be extended to allow improved interoperability with the C++ Boost.MPI and the C MPI bindings.
The Communicator class abstracts a set of communicating processes in MPI. All of the processes that belong to a certain communicator can determine the size of the communicator, their rank within the communicator, and communicate with any other processes in the communicator.
abort( (Communicator)arg1, (int)errcode) -> None
Makes a “best attempt” to abort all of the tasks in the group of this communicator. Depending on the underlying MPI implementation, this may either abort the entire program (and possibly return errcode to the environment) or only abort some processes, allowing the others to continue. Consult the documentation for your MPI implementation. This is equivalent to a call to MPI_Abort.
errcode is the error code to return from aborted processes.
barrier( (Communicator)arg1) -> None
Wait for all processes within a communicator to reach the barrier.
iprobe( (Communicator)arg1 [, (int)source=-1 [, (int)tag=-1]]) -> object
This operation determines if a message matching (source, tag) is available to be received. If so, it returns information about that message; otherwise, it returns None. If source is omitted, a message from any process will match. If tag is omitted, a message with any tag will match. The actual source and tag can be retrieved from the returned Status object. To wait for a message to become available, use probe().
irecv( (Communicator)arg1 [, (int)source=-1 [, (int)tag=-1]]) -> RequestWithValue
This routine initiates a non-blocking receive from the process source with the given tag. If the source parameter is not specified, the message can be received from any process. Likewise, if the tag parameter is not specified, a message with any tag can be received. This routine returns a Request object, which can be used to query when the transmission has completed, wait for its completion, or cancel the transmission. The received value be accessible through the value attribute of the Request object once transmission has completed.
As with the recv() routine, when receiving the content of a data type that has been sent separately from its skeleton, user code must provide a value for the buffer argument. This value should be the Content object returned from get_content().
isend( (Communicator)arg1, (int)dest [, (int)tag=0 [, (object)value=None]]) -> Request
This routine executes a nonblocking send with the given tag to the process with rank dest. It can be received by the destination process with a matching recv call. The value will be transmitted in the same way as with send(). This routine returns a Request object, which can be used to query when the transmission has completed, wait for its completion, or cancel the transmission.
probe( (Communicator)arg1 [, (int)source=-1 [, (int)tag=-1]]) -> Status
This operation waits until a message matching (source, tag) is available to be received. It then returns information about that message. If source is omitted, a message from any process will match. If tag is omitted, a message with any tag will match. The actual source and tag can be retrieved from the returned Status object. To check if a message is available without blocking, use iprobe().
recv( (Communicator)arg1 [, (int)source=-1 [, (int)tag=-1 [, (bool)return_status=False]]]) -> object
This routine blocks until it receives a message from the process source with the given tag. If the source parameter is not specified, the message can be received from any process. Likewise, if the tag parameter is not specified, a message with any tag can be received. If return_status is True, returns a tuple containing the received object followed by a Status object describing the communication. Otherwise, recv() returns just the received object.
When receiving the content of a data type that has been sent separately from its skeleton, user code must provide a value for the buffer’ argument. This value should be the :class:`Content object returned from get_content().
send( (Communicator)arg1, (int)dest [, (int)tag=0 [, (object)value=None]]) -> None
This routine executes a potentially blocking send with the given tag to the process with rank dest. It can be received by the destination process with a matching recv() call. The value will be transmitted in one of several ways:
- For C++ objects registered via register_serialized(), the value will be serialized and transmitted.
- For SkeletonProxy objects, the skeleton of the object will be serialized and transmitted.
- For Content objects, the content will be transmitted directly. This content can be received by a matching recv()/irecv() call that provides a suitable buffer argument.
- For all other Python objects, the value will be pickled and transmitted.
split( (Communicator)arg1, (int)color) -> Communicator
Split the communicator into multiple, disjoint communicators each of which is based on a particular color. This is a collective operation that returns a new communicator that is a subgroup of this. This routine is functionally equivalent to MPI_Comm_split.
color is the color of this process. All processes with the same color value will be placed into the same group.
If provided, key is a key value that will be used to determine the ordering of processes with the same color in the resulting communicator. If omitted, the key will default to the rank of the process in the current communicator.
Returns a new Communicator instance containing all of the processes in this communicator that have the same color.
init( (list)argv [, (bool)abort_on_exception=True]) -> bool
Initialize the MPI environment. Users should not need to call this function directly, because the MPI environment will be automatically initialized when the Boost.MPI module is loaded.
initialized() -> bool
Determine if the MPI environment has already been initialized.
abort( (int)errcode) -> None
Aborts all MPI processes and returns to the environment. The precise behavior will be defined by the underlying MPI implementation. This is equivalent to a call to MPI_Abort with MPI_COMM_WORLD. errcode is the error code to return from aborted processes.
finalize() -> None
Finalize (shut down) the MPI environment. Users only need to invoke this function if MPI should be shut down before program termination. Boost.MPI will automatically finalize the MPI environment when the program exits.
finalized() -> bool
Determine if the MPI environment has already been finalized.
all_gather([ (Communicator)comm=world [, (object)value=None]]) -> object
all_gather is a collective algorithm that collects the values stored at each process into a tuple of values indexed by the process number they came from. all_gather is (semantically) a gather followed by a broadcast. The same tuple of values is returned to all processes.
all_reduce( (Communicator)comm=world, (object)value, (object)op) -> object
all_reduce is a collective algorithm that combines the values stored by each process into a single value. The values can be combined arbitrarily, specified via any function. The values a1, a2, .., ap provided by p processors will be combined by the binary function op into the result:
op(a1, op(a2, ... op(ap-1,ap)))
that will be returned to all processes. This function is the equivalent of calling all_gather() and then applying the built-in reduce() function to the returned sequence. op is assumed to be associative.
all_to_all([ (Communicator)comm=world [, (object)values=None]]) -> object
all_to_all is a collective algorithm that transmits values from every process to every other process. On process i, the jth value of the values sequence is sent to process j and placed in the ith position of the tuple that will be returned from all_to_all.
broadcast( (Communicator)comm=world, (object)value=None, (int)root) -> object
broadcast is a collective algorithm that transfers a value from an arbitrary root process to every other process that is part of the given communicator (comm). The root parameter must be the same for every process. The value parameter need only be specified at the root root. broadcast() returns the same broadcasted value to every process.
gather( (Communicator)comm=world, (object)value=None, (int)root) -> object
gather is a collective algorithm that collects the values stored at each process into a tuple of values at the root process. This tuple is indexed by the process number that the value came from, and will be returned only by the root process. All other processes return None.
reduce( (Communicator)comm=world, (object)value, (object)op, (int)root) -> object
reduce is a collective algorithm that combines the values stored by each process into a single value at the root. The values can be combined arbitrarily, specified via any function. The values a1, a2, .., ap provided by p processors will be combined by the binary function op into the result:
op(a1, op(a2, ... op(ap-1,ap)))
that will be returned on the root process. This function is the equivalent of calling gather() to the root and then applying the built-in reduce() function to the returned sequence. All non-root processes return None. op is assumed to be associative.
scan( (Communicator)comm=world, (object)value, (object)op) -> object
scan computes a prefix reduction of values from all processes. It is a collective algorithm that combines the values stored by each process with the values of all processes with a smaller rank. The values can be arbitrarily combined, specified via a binary function op. If each process i provides the value ai, then scan returns:
op(a1, op(a2, ... op(ai-1, ai)))
to the ith process. op is assumed to be associative. This routine is the equivalent of an all_gather(), followed by a built-in reduce() on the first i+1 values in the resulting sequence on processor i. op is assumed to be associative.
scatter( (Communicator)comm=world, (object)values=None, (int)root) -> object
scatter is a collective algorithm that scatters the values stored in the root process (as a container with comm.size elements) to all of the processes in the communicator. The values parameter (only significant at the root) is indexed by the process number to which the corresponding value will be sent. The value received by each process is returned from scatter.
The Request class contains information about a non-blocking send or receive and will be returned from isend or irecv, respectively. When a Request object represents a completed irecv, the value attribute will contain the received value.
cancel( (Request)arg1) -> None
Cancel a pending communication, assuming it has not already been completed.
test( (Request)arg1) -> object
Determine whether the communication associated with this request has completed successfully. If so, returns the Status object describing the communication (for an isend request) or a tuple containing the received value and a Status object (for an irecv request). Note that once test() returns a Status object, the request has completed and wait() should not be called.
wait( (Request)arg1) -> Status
Wait until the communication associated with this request has completed. For a request that is associated with an Communicator.isend(), returns a Status object describing the communication. For an Communicator.irecv() operation, returns the received value by default. However, when return_status=True, a (value, status) pair is returned by a completed Communicator.irecv() request.
A list of Request objects.
The Status class stores information about a given message, including its source, tag, and whether the message transmission was cancelled or resulted in an error.
test_all( (RequestList)requests [, (object)callable=None]) -> bool
Tests if all of the given requests have been completed.
Returns True if all requests have been completed.
If the second parameter callable is provided, it is called with each completed request’s received value (or None) and it s Status object as its arguments. The calls occur in the order given by the requests list.
test_any( (RequestList)requests) -> object
Tests if any of the given requests have been completed, but does not wait for completion.
Returns a triple (value, status, index) like wait_any() or None if no request is complete.
test_some( (RequestList)requests [, (object)callable=None]) -> int
Tests to see if any of the given requests has completed. It completes all of the requests it can, partitioning the input sequence into pending requests followed by completed requests. This routine is similar to wait_some, but does not wait until any requests have completed.
Returns the index of the first completed request. If the second parameter callable is provided, it is called with each completed request’s received value (or None) and it s Status object as its arguments. The calls occur in the order given by the requests list.
wait_all( (RequestList)requests [, (object)callable=None]) -> None
Waits until all of the given requests have been completed.
If the second parameter callable is provided, it is called with each completed request’s received value (or None) and its Status object as its arguments. The calls occur in the order given by the requests list.
wait_any( (RequestList)requests) -> object
Waits until any of the given requests has been completed.
Returns a triple (value, status, index) consisting of received value (or None), the Status object for the completed request, and its index in the RequestList.
wait_some( (RequestList)requests [, (object)callable=None]) -> int
Waits until at least one of the given requests has completed. It then completes all of the requests it can, partitioning the input sequence into pending requests followed by completed requests.
Returns the index of the first completed request. If the second parameter callable is provided, it is called with each completed request’s received value (or None) and its Status object as its arguments. The calls occur in the order given by the requests list.
The SkeletonProxy class is used to represent the skeleton of an object. The SkeletonProxy can be used as the value parameter of send() or isend() operations, but instead of transmitting the entire object, only its skeleton (“shape”) will be sent, without the actual data. Its content can then be transmitted, separately.
User code cannot generate SkeletonProxy instances directly. To refer to the skeleton of an object, use skeleton(object). Skeletons can also be received with the Communicator.recv() and Communicator.irecv() methods.
Note that the skeleton/content mechanism can only be used with C++ types that have been explicitly registered.
The content is a proxy class that represents the content of an object, which can be separately sent or received from its skeleton.
User code cannot generate content instances directly. Call the get_content() routine to retrieve the content proxy for a particular object. The content instance can be used with any of the Communicator.send() or Communicator.recv() variants. Note that get_content() can only be used with C++ data types that have been explicitly registered with the Python skeleton/content mechanism.
get_content( (object)object) -> Content
The get_content function retrieves the content for its object parameter, allowing the transmission of the data in a data structure separately from its skeleton (or “shape”). The skeleton/content mechanism is useful when a large data structure remains structurally the same throughout a computation, but its content (i.e., the values in the structure) changes several times. Tranmission of the content part does not require any serialization or unnecessary buffer copies, so it is very efficient for large data structures.
Only C++ types that have been explicitly registered with the Boost.MPI Python library can be used with the skeleton/content mechanism. Use boostmpi::register_skeleton_and_content.
skeleton( (object)object) -> object
The skeleton function retrieves the SkeletonProxy for its object parameter, allowing the transmission of the skeleton (or “shape”) of the object separately from its data. The skeleton/content mechanism is useful when a large data structure remains structurally the same throughout a computation, but its content (i.e., the values in the structure) changes several times. Tranmission of the content part does not require any serialization or unnecessary buffer copies, so it is very efficient for large data structures.
Only C++ types that have been explicitly registered with the Boost.MPI Python library can be used with the skeleton/content mechanism. Use boostmpi::register_skeleton_and_content.
The Timer class is a simple wrapper around the MPI timing facilities.
restart( (Timer)arg1) -> None
Restart the timer, after which elapsed == 0.