[Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

Erez Haba erezh at [hidden]
Fri Mar 13 14:39:50 CDT 2009 


I've implemented this complete change during the last MPI-forum meeting. It took me about 2.5h hours to implement while participating in the discussion. So I would argue that it's not a big implementation item for any MPI.
(look at the diff last change, out of the 10 lines in the diff, only 1 line is changed)

I would argue that casting away the const to adapt your buffer to your lower level interconnect interface is perfectly fine. The semantic for your lower level interconnect interface is simple, clear and does not modify the send buffer. (for the cases where actually you need to cast the const away)

Thanks,
.Erez

-----Original Message-----
From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W
Sent: Friday, March 13, 2009 11:36 AM
To: MPI 2.2
Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

All -

I spent some time looking at what it would take to implement this proposal in Open MPI as well as the changes made in the MPICH2 implementation.  I think both raise some concerns for this proposal, particularly for MPI 2.2.

The diff for the proposal was almost 10,000 lines long, which is a little frightening for 2.2 change.  However, the places the constness is cast away for the send buffer raises some issues.  In particular, for eager, contiguous sends, the send buffer constness is cast away in MPID_Send, which is the second call in the send call stack.  Since there has been much discussion about how the constness of a buffer shouldn't be cast away at the upper layers of the MPI implementation, it seems problematic that the reference implementation does just that.  It seems problematic not because of the choice made by the implementors of the patch, but because it is not clear that the is really a better implementation option.  Of course, the choice of how to handle non-contiguous data raises some issues for Open MPI's internals.

Open MPI's low layer communication interface is based on a iovec-like interface.  Like most interfaces, the iovec-like interface is shared between the send and receive calls.  The iovec structure can not be const, meaning that we are faced with two choices.  We can either cast away the const interface before this low-level interface or we can create two data structures, one for send and one fore receive.  The const cast option obviously has its problems.  The two data structure option is certainly not appealing for us from a code maintainability stand-point.  This is further complicated by the fact that almost none of the transports Open MPI currently supports can be used without casting away the constness.

I understand that it would be good to promote the use of const for send buffers in lower-level middleware, as well as the fact that not all MPI implementations have such middleware.  However, many interfaces have a similar iovec issue to Open MPI, and they'll be unlike to change.  The fact that we will permanently be casting away const is slightly bothersome to me.  I think given these issues and the large size of the change, pushing this proposal to 3.0 would be appropriate.

Brian

________________________________________
From: mpi-22-bounces_at_[hidden] [mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba [erezh_at_[hidden]]
Sent: Friday, March 13, 2009 10:40 AM
To: MPI 2.2
Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

Hi Jeff,

The point of this ticket is to present a consistent const correct interface to the programmer and hide all the idiosyncrasies of the down-level interfaces.

That said, to answer your questions:

No casting is required for:
* shared memory
* sockets (TCP) when using 'send()' (single buffer passed in)

Casting is required for interfaces that did not define their io vector buffer member as const. (an interface would need to define two io vector structures, one with const buffer for send and one with non-const for receive; most of them don't).

Thus, casting is required when using the scatter/gather io vector with
* sockets (TC)
* Network Direct (IB and iWarp)

With MPICH2 implementation, casting out the const for the send buffer when using these interfaces was there all along as (most of) MPICH2 internal functions already used const to pass the send buffer. No new cast was required in this full const implementation.

Again, casting away the const to use the lower level interface does not violate any const rules as long as the semantic is preserved.

Thanks,
.Erez

-----Original Message-----
From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres
Sent: Friday, March 13, 2009 6:05 AM
To: MPI 2.2
Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

Do you end up casting away the const when invoking the Windows network
interfaces?

On Linux, I think the following interfaces would require casting the
const away:

- shared memory (even using Linux vmsplice())
- MX
- TCP
- OpenFabrics (both verbs and udapl)

These interfaces do not require casting away const:

- SCTP (I *think* -- I'm not an SCTP expert, but a quick look in
sctp.h showed a "const" argument for the sctp_sendmsg() function)

I can't check these offhand:

- Portals
- Elan

On Mar 12, 2009, at 6:06 PM, Erez Haba wrote:

> I urge you, please read the ticket proposal again. It discusses the
> merit of this proposal.
>
> As for IB; yes you'd copy small messages (which const is fine with)
> and some implement RDMA read or write for large messages; you might
> need to cast the const away if your IB interface does not expose
> const SGE's.
>
> As for the compiler; no you are not violating the contact;
> (otherwise we wouldn't have casts); as long as you guarantee that
> you are not changing the buffer. It's a way for the programmer to
> use old, non const correct, interfaces.
>
> .Erez
>
> -----Original Message-----
> From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]
> ] On Behalf Of Underwood, Keith D
> Sent: Thursday, March 12, 2009 2:03 PM
> To: MPI 2.2
> Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C
> bindings- implementation
>
> I didn't think that typical MPI implementations over IB copied the
> data for large messages (as opposed to the bounce buffers for small
> messages).
>
> If you are going to have to cast away the const, what is the point
> of putting it in?  In fact, if you have to cast away the const,
> aren't you violating your contract with the compiler?  Should this
> change be rolled out from the bottom up?  i.e. common low-level APIs
> first and then MPI?
>
> Keith
>
> >-----Original Message-----
> >From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi-
> >forum.org] On Behalf Of Erez Haba
> >Sent: Thursday, March 12, 2009 2:10 PM
> >To: MPI 2.2
> >Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C
> bindings-
> >implementation
> >
> >Possibly you'll have to cast.
> >There are few interfaces that take const for send buffer (like
> sockets).
> >In other cases you copy the buffer (like with IB) which keeps the
> const
> >property.
> >
> >Thanks,
> >.Erez
> >
> >-----Original Message-----
> >From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi-
> >forum.org] On Behalf Of Underwood, Keith D
> >Sent: Thursday, March 12, 2009 12:05 PM
> >To: MPI 2.2
> >Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C
> bindings-
> >implementation
> >
> >It would seem like const would eventually have to be cast away for
> most
> >lower level interfaces.  Portals, Elan, IB Verbs, and as best I can
> tell MX
> >don't have const on their send buffers...
> >
> >Keith
> >
> >>-----Original Message-----
> >>From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi-
> >>forum.org] On Behalf Of Erez Haba
> >>Sent: Thursday, March 12, 2009 12:44 PM
> >>To: MPI 2.2
> >>Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C
> bindings-
> >>implementation
> >>
> >>No, it's not being cast away (although it is a quick way for you to
> >>implement it).
> >>In MPICH2 implementation it trickles down to the lower functions.
> There
> >are
> >>very few places where the const is being cast away because ticket
> #46 does
> >>not to break backward compatibility.
> >>For example, when the user function is called (const was not added
> to the
> >>MPI_User_function) the implementation must cast away const from
> the send
> >>buffer before calling the user function.
> >>
> >>Thanks,
> >>.Erez
> >>
> >>-----Original Message-----
> >>From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi-
> >>forum.org] On Behalf Of Jeff Squyres
> >>Sent: Thursday, March 12, 2009 11:34 AM
> >>To: MPI 2.2
> >>Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C
> bindings-
> >>implementation
> >>
> >>More importantly, how is const handled?  Is it just cast away inside
> >>the MPI library?
> >>
> >>
> >>On Mar 12, 2009, at 2:34 PM, Underwood, Keith D wrote:
> >>
> >>> Since the main purpose of an implementation for something like
> >>> adding const is to see how invasive the change is, would it be
> >>> possible to get a diff against the trees that these were created
> from?
> >>>
> >>> Thanks,
> >>> Keith
> >>>
> >>> From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-
> >>bounces_at_[hidden]
> >>> ] On Behalf Of Erez Haba
> >>> Sent: Thursday, March 12, 2009 12:24 PM
> >>> To: MPI 2.2
> >>> Subject: [Mpi-22] Ticket #46: Add const Keyword to the C
> bindings -
> >>> implementation
> >>>
> >>>
> >>> Implementation for ticket #46 is now available from ANL (thanks
> Pavan)
> >>>
> >>>
> >>> tarballs
> >>>
> >>http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/nightly
> >/
> >>const
> >>>
> >>>
> >>> thanks,
> >>> .Erez
> >>>
> >>> _______________________________________________
> >>> mpi-22 mailing list
> >>> mpi-22_at_[hidden]
> >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
> >>
> >>
> >>--
> >>Jeff Squyres
> >>Cisco Systems
> >>
> >>_______________________________________________
> >>mpi-22 mailing list
> >>mpi-22_at_[hidden]
> >>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
> >>
> >>
> >>_______________________________________________
> >>mpi-22 mailing list
> >>mpi-22_at_[hidden]
> >>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
> >
> >_______________________________________________
> >mpi-22 mailing list
> >mpi-22_at_[hidden]
> >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
> >
> >
> >_______________________________________________
> >mpi-22 mailing list
> >mpi-22_at_[hidden]
> >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
>
> _______________________________________________
> mpi-22 mailing list
> mpi-22_at_[hidden]
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
>
>
> _______________________________________________
> mpi-22 mailing list
> mpi-22_at_[hidden]
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22


--
Jeff Squyres
Cisco Systems
_______________________________________________
mpi-22 mailing list
mpi-22_at_[hidden]
http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
_______________________________________________
mpi-22 mailing list
mpi-22_at_[hidden]
http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22
_______________________________________________
mpi-22 mailing list
mpi-22_at_[hidden]
http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

More information about the Mpi-22 mailing list