[MPI3-IO] I/O working group Notes

[Quincey Koziol]

Hi Mohamad,

On Mar 12, 2013, at 8:28 PM, Mohamad Chaarawi <chaarawi at hdfgroup.org> wrote:

> Hi all,
> 
> I attached the slides that I presented at the Forum, and here are some notes I took when discussing the proposals:

	Good notes, thanks!

> Collective I/O ticket:
> Need a more compelling reason to add them to the standard. Maybe a more in-depth study and demand from users with use cases. 
> Implementation of split collectives hard as is, so to add a harder interface for implementers needs to be justified.
> Will not attempt to deprecate the split collective in this ticket.

	OK, we can provide use cases, drawing from our experiences with HDF5.

> Asynchronous MPI Model:
> discuss whether it makes sense to introduce such a model in MPI
> Can something be done with generalized request?

	Someone always brings this up.  :-)  We should get up to speed on the GRQ routines and know what they can do...

> a more thorough study should be conducted on what internal MPI features get exposed 
> Should consider an implementation path for MPI that does not have threads.
> No overhead should be introduced, which might not be the case with RMA

	What did people think of the idea in general?

> Independent File View changes:
> The file_set_view operation should be used to do optimizations (collectively), so adding an independent version would urge developers to not "do the right thing" with the original routine.
> what happens in some cases when all processes set the file view collectively, then 1 process changes its file view independently? How would the other processes know that? The MPI library might need to communicate that change to other processes if the original set file_view did certain optimizations.
> For our use case, we can introduce a flag on File open, that says set_view is always independent or collective. We can introduce also another collective routine to allow the user to change set_view from independent to collective and vice versa.

	I like some of the latter ideas (file open option, etc).  They would help take the backward compatibility risk out of the equation.

> Grouped I/O operations:
> A file_reopen with a sub group is a feasible solution, but more though needs to be taken into consideration..
> What are the consistency semantics for the new file handle that is returned with open_grouped. this needs to be examine more.
> What happens if you opened the file twice, but with a different communicator? Can MPI optimize that?
> This also affects the file view that is set for the original file handle. How would that play out now? allow grouped file view changes on the new handle, or just allow independent file changes?
> do some benchmarking with existing MPI implementations to see performance of Opening file multiple times.

	OK, sounds positive, we should work out answers to these questions...

> Shared FP bug:
> emphasize the offset returned should be for where access started.
> go over all routines for shared file pointers and indicate why only those three were chosen. We can add a position returned to the ordered operation, it just seemed easy for the user to be able to retrieve it.
> generally seemed ok, but needed to emphasize use cases more in proposal.

	I would _really_like_ the "write_and_get" routine - that would make space allocation in HDF5 _so_ much easier!  Getting a use case for this one won't be a problem. :-)

	Sounds like a good meeting, with some things we can move forward.  :-)

		Quincey


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-io/attachments/20130313/a7352ff4/attachment-0001.html>