[Mpi-forum] Question about the semantics of MPI_Comm_disconnect

[Jeff Squyres (jsquyres)]

On Nov 13, 2013, at 4:25 PM, George Bosilca <bosilca at icl.utk.edu> wrote:

> Disconnect doesn’t work at the physical connection level, it is supposed to mark two previously connected world as disconnected in the MPI sense.

Mmm... not quite.  It disconnects two *communicators* -- but the *processes* may still be connected (in the MPI sense).  See the example in my previous mail -- you have to call COMM_DISCONNECT twice and FILE_CLOSE to actually disconnect.  

This is supported by MPI-3 10.5.4 (p399:45-47).

> In MPI there is this transitive notion of connected processes, which can only be reversed by the usage of disconnect. Basically MPI_Finalize is collective over all connected processes, which is a burden in some cases. Disconnect is here to allow the disruption of the connected notion.

Yeah, I know what disconnect is for.  :-)

> Thus, Disconnect only has a special meaning when called on a communicator returned by SPAWN, ACCEPT/CONNECT and JOIN. Any other usage is similar to MPI_Comm_Free. This in well explained in 10.5.4 once at the beginning of the section and then again in the first advice to users.

I'm not debating what COMM_DISCONNECT actually does -- I'm just saying that it's actually a poor name choice: per 10.5.4, it *doesn't* always disconnect you.  Per the example in my previous mail, the processes involved are connected via 3 things:

1. the communicator created by COMM_SPAWN
2. the communicator created by INTERCOMM_MERGE
3. the communicator created by FILE_OPEN

You have to COMM_DISCONNECT the first 2, and FILE_CLOSE then 3rd, and *then* you're actually disconnected.

Hence, all I'm saying is that "COMM_DISCONNECT" is actually a bit of a misnomer -- the action it's really performing is something more like MPI_COMM_FREE_WAIT (i.e., do a COMM_FREE, but actually wait for all communication to complete on that communicator before returning).

For reference, this was my example:

>> -----
>> MPI_Comm_spawn(..., &child);
>> MPI_Intercomm_merge(child, 0, &child_intra);
>> MPI_File_open(child_intra, ..., &file);
>> MPI_Comm_disconnect(&child_intra);
>> MPI_Comm_disconnect(&child);
>> -----

After giving that example, I mused:

>> I wonder if this is worth an AtoU...


-- 
Jeff Squyres
jsquyres at cisco.com
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/