[mpiwg-persistence] Completion call before all partitions marked ready
Joachim Jenke
protze at itc.rwth-aachen.de
Wed Oct 12 11:17:53 CDT 2022
I think it is worth clarifying that it is valid behavior, since the
sentence in MPI4,p104,l15 seems to state that the completion call
follows all the pready calls:
> Once an MPI_PSEND_INIT call has been made, the user may start the
> operation with a call to a starting procedure and complete the
> operation with a number of MPI_PREADY calls equal to the requested
> number of send partitions followed by a call to a completing
> procedure.
The responsibility to avoid deadlocks is clear. From the DL analysis
point of view, such ordering requirement would make the analysis
actually easier :) Without such ordering requirement, a runtime analysis
tool like MUST cannot clearly identify a spin-loop on test as
deadlock/livelock.
Searching for above text, I found an inconsistency in the restrictions
on the partition argument. MPI_Pready has (MPI4,p109,l11):
> Specifying a partition number that is equal to or larger than the
> number of partitions is erroneous.
First, I was surprised that there is no restriction on negative values,
but this restriction is buried in the argument list comments. Having an
unsigned int argument would make it more clear.
MPI_Parrived does not have such upper-bound restriction.
- Joachim
Am 12.10.22 um 17:09 schrieb Ryan Grant:
> This is correct, you’re allowed to call test/wait whenever you want as
> long as you abide by the rules that all pready calls must be made in
> order for completion to happen. So don’t MPI_Wait on an operation to
> complete when you haven’t called all of the pready calls (that the
> process/thread is responsible for) from that process/thread yet (similar
> to your responsibilities to make sure Send/Recv don’t deadlock, but
> obviously a different semantic for completion).
>
> -Ryan
>
> *From: *James Dinan <james.dinan at gmail.com>
> *Date: *Wednesday, October 12, 2022 at 6:43 AM
> *To: *Joachim Jenke <protze at itc.rwth-aachen.de>, Ryan Grant
> <ryan.grant at queensu.ca>
> *Cc: *"mpiwg-persistence at lists.mpi-forum.org"
> <mpiwg-persistence at lists.mpi-forum.org>
> *Subject: *Re: [mpiwg-persistence] Completion call before all partitions
> marked ready
>
> This came up recently and I believe it is allowed to wait/test a send
> request before all partitions are marked as ready. It's not explicitly
> stated, rather there is no restriction disallowing this usage. It may be
> good to clarify this semantic (e.g. in Section 4.2.2) for MPI 4.1 since
> this is a topic that the WG debated at length.
>
> @Ryan Grant <mailto:ryan.grant at queensu.ca> Please fact check.
>
> ~Jim.
>
> On Mon, Oct 10, 2022 at 7:22 AM Joachim Jenke via mpiwg-persistence
> <mpiwg-persistence at lists.mpi-forum.org
> <mailto:mpiwg-persistence at lists.mpi-forum.org>> wrote:
>
> Hello wg-persistence,
>
> Looking at the MPI 4.0 document, it is not clear to us, whether it is
> allowed to call a completion call for a partitioned communication
> request before all partitions are marked ready. A simple
> single-threaded
> example would be:
>
> ```C
> MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag,
> MPI_COMM_WORLD, MPI_INFO_NULL, &request);
> MPI_Start(&request);
> for(i = 0; i < partitions-1; ++i)
> {
> MPI_Pready(i, request);
> }
> MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0
> MPI_Pready(partitions-1, request);
> MPI_Wait(&request, MPI_STATUS_IGNORE);
> MPI_Request_free(&request);
> ```
>
> The question becomes more relevant in a multi-threaded context. One
> thread could finish the work early and call MPI_Wait to detect when all
> partitions were sent.
>
> From my understanding, the only requirement is that all partitions
> must
> be marked ready with explicit ready calls before the operation can
> complete. Replacing the test in above example with a wait call would
> result in deadlock.
>
> Best
> Joachim
>
> --
> Dr. rer. nat. Joachim Jenke
>
> IT Center
> Group: High Performance Computing
> Division: Computational Science and Engineering
> RWTH Aachen University
> Seffenter Weg 23
> D 52074 Aachen (Germany)
> Tel: +49 241 80- 24765
> Fax: +49 241 80-624765
> protze at itc.rwth-aachen.de <mailto:protze at itc.rwth-aachen.de>
> www.itc.rwth-aachen.de
> <https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.itc.rwth-aachen.de%2F&data=05%7C01%7Cryan.grant%40queensu.ca%7C6464a2e4284646255bf008daac3e940c%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C638011682013760361%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n%2B4uUA704BGsh4zIbm7w0Fzs0S8rzlWv13LQX%2BzXF0I%3D&reserved=0>
> _______________________________________________
> mpiwg-persistence mailing list
> mpiwg-persistence at lists.mpi-forum.org
> <mailto:mpiwg-persistence at lists.mpi-forum.org>
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-persistence
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi-forum.org%2Fmailman%2Flistinfo%2Fmpiwg-persistence&data=05%7C01%7Cryan.grant%40queensu.ca%7C6464a2e4284646255bf008daac3e940c%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C638011682013760361%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iOVtiUL9aSL7Bu%2BmeC6dRWfAARYfpi5pxGbGFjTf4uM%3D&reserved=0>
>
--
Dr. rer. nat. Joachim Jenke
IT Center
Group: High Performance Computing
Division: Computational Science and Engineering
RWTH Aachen University
Seffenter Weg 23
D 52074 Aachen (Germany)
Tel: +49 241 80- 24765
Fax: +49 241 80-624765
protze at itc.rwth-aachen.de
www.itc.rwth-aachen.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5906 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-persistence/attachments/20221012/473f4952/attachment.p7s>
More information about the mpiwg-persistence
mailing list