Basic Concepts
==============

The basic idea is that all interaction with external process and
resource managers, as well as the exchange of any information required
to contact other processes in the same parallel job, takes place
through the process management interface or PMI.

There are four separate sets of functionalities:

   1. Creating, connecting with, and exiting parallel jobs

   2. Accessing information about the parallel job or the node on
   which a process is running

   3. Exchanging information used to connect processes together

   4. Exchanging information related to the MPI Name publishing
   interface

While these can be combined within a single, full-featured process
manager, in many cases, each set of services may be provided by a
different actor. For example, creating processes may be managed by a
system such as PBS or LoadLeveler. The Name publishing service may be
accomplished by reading and writing files in a shared
directory. Information about the parallel job and the node may be
provided by mpiexec, and the connection information may be handled
with a scalable, distributed tuple-space system.

There are three groupings of processes that are important in
understanding the process manager interface.

Process: An MPI process; this is usually an OS process (but need not
be; an example would be threads in a language that keep named globals
thread-private by default).

Job: This is a collection of processes managed together by a process
manager that understands parallel applications. A job contains all of
the processes in a single MPI_COMM_WORLD and no more. That is, two
processes are in the same job if and only if they are in the same
MPI_COMM_WORLD

Connected Jobs: This is a collection of jobs that have established a
connection through the use of PMI_Job_Spawn or PMI_Job_Connect. If any
process in a job establishes a connection with any process in another
job, then all processes in both jobs are connected. That is,
connections are established between jobs, not processes. This is
necessary to implement the MPI notion of connected processes.

In addition, it is desirable to allow the PMI client interface to be
implemented with a dynamically loadable library. This allows an
executable to load a version of PMI that is compatible with whatever
process management system will be running the application, without
requiring the process management systems to implement the same
communication (or wire) protocol. The consequence of this is that the
pmi.h header file is standardized across all PMI client
implementations (in MPICH2's PMI v1, each PMI client implementation
could, like MPI, define its own header file).


Sample API
----------

int PMI_Init( int *spawned, int *size, int *rank, int *appnum )

int PMI_Finalize(void)

int PMI_Abort(int flag, const char msg[] )

PMI_KVS_Put( const char key[], const char value[] )

PMI_KVS_Get( const char jobId[], const char key[], char value [], int maxValue, int *valLen )

PMI_KVS_Fence(void)