[Mpi-comments] MPI_Create_directory

[Axel Huebl]

Just something that can be structured into a tree on which paths or 
inodes we declare the already existing MPI-IO files.

Not necessarily with all the constrains of POSIX.

Basically introducing simple inodes, implemented with supported 
protocols / transports.


Axel

On 07.01.19 16:02, William Gropp wrote:
> Are we assuming a POSIX filesystem (a bad idea in my opinion)? Are we 
> assuming a more restricted set of file system structure operations, 
> along with weaker consistency semantics (matching the MPI-IO approach 
> rather than POSIX)?
> 
> Bill
> 
> William Gropp
> Director and Chief Scientist, NCSA
> Thomas M. Siebel Chair in Computer Science
> University of Illinois Urbana-Champaign
> 
> 
> 
> 
> 
> 
>> On Jan 7, 2019, at 8:28 AM, Axel Huebl via mpi-comments 
>> <mpi-comments at lists.mpi-forum.org 
>> <mailto:mpi-comments at lists.mpi-forum.org>> wrote:
>>
>> Dear MPI-Forum,
>>
>>
>> should we extend the MPI-I/O interfaces for directory operations? 
>> Especially a "mkdir -p" equivalent could be useful.
>>
>> In many scientific apps that create output at scale, an output 
>> structure needs to be created on demand.
>>
>> One can of course decide to do so with e.g. Linux `mkdir` but this can 
>> be tricky:
>>
>> If all ranks do it, one will race the others and it's unnecessary 
>> expensive in terms of operations.
>>
>> If a "root" rank does it, there is no guarantee that e.g. a 
>> MPI_Barrier which syncs the "MPI-World" will also cause the 
>> "filesystem-world" to be up-to-date. For example a mkdir on a slow NFS 
>> local node has bot yet propagated to the other compute nodes.
>>
>> Comments, feedback and overlooked already existing solutions welcome!
>>
>>
>> Best regards,
>> Axel Huebl
>> -- 
>>
>> Axel Huebl
>> Phone: +49 351 260 3582
>> Institute of Radiation Physics
>> http://www.hzdr.de/crp
>> Helmholtz-Zentrum Dresden - Rossendorf (HZDR)
>> Bautzner Landstr. 400 | 01328 Dresden | Germany
>> Board of Directors:
>> Prof. Dr. Dr. h. c. Roland Sauerbrey, Dr. Ulrich Breuer
>> Company Registration Number VR 1693, Amtsgericht Dresden
>> _______________________________________________
>> mpi-comments mailing list
>> mpi-comments at lists.mpi-forum.org
>> https://lists.mpi-forum.org/mailman/listinfo/mpi-comments
> 

-- 

Axel Huebl
Phone: +49 351 260 3582
Institute of Radiation Physics
http://www.hzdr.de/crp
Helmholtz-Zentrum Dresden - Rossendorf (HZDR)
Bautzner Landstr. 400 | 01328 Dresden | Germany
Board of Directors:
Prof. Dr. Dr. h. c. Roland Sauerbrey, Dr. Ulrich Breuer
Company Registration Number VR 1693, Amtsgericht Dresden