[Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements

[Rolf Rabenseifner]

I was reading web pages about segmented address space.
And they show two possibilities (which may go only into the past):

- flattened address space
- segmented with some high order bits refering to the Segment
  and low order bits to address within the segment.

If I understood correctly, then the major problem behind MPI_Aint_diff
and MPI_Aint_add was the following:

(my example is with 8 bit MPI_Aint)

  addr1 01111111 = 127 (signed int) = 127 (unsigned int)  
  addr2 10000001 = -127  (signed int) = 129 (unsigned int)
       
Internally the addreses are viewed by the hardware and OS as unsigned.
MPI_Aint is interpreting the same bits as signed int.

addr2-addr1 = 129 -127 = 2 (as unsigned int)
but in a real application code with "-" operator:
            = -127 -127 = -254
  --> signed int Overflow because 8 bit can express only -128 .. +127
  --> detected or automatically corrected with +256 --> -254+256 = 2 

The segmented problem was solved very early through the wording
on "sequential storage" (unchanged since MPI-1.1 !!! :-) 
in the section "Correct use of addresses".

What I see, that before changing anything, the mpiwg-large-counts WG
should really understand MPI-3.1.

For me, it looked like that there was some misunderstanding
of the concept that absolute and relative addresses
and number of bytes that can be stored in MPI_Aint.  

Best regards
Rolf

----- Original Message -----
> From: "Jeff Squyres" <jsquyres at cisco.com>
> To: "Rolf Rabenseifner" <rabenseifner at hlrs.de>
> Cc: "mpiwg-large-counts" <mpiwg-large-counts at lists.mpi-forum.org>
> Sent: Thursday, October 24, 2019 3:28:47 PM
> Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements

> On Oct 24, 2019, at 2:19 AM, Rolf Rabenseifner <rabenseifner at hlrs.de> wrote:
>> 
>>> Does MPI support segmented address spaces or not?
>> 
>> When  I understand correctly then MPI is and was always prepared for segmented
>> address spaces as long as all integer types are signed integers and the
>> segmentation is done through some higher order bits.
> 
> 
> As I was pondering your replies from yesterday and thinking about how to
> respond, this is exactly the same point that I got to.
> 
> I think what I have been failing to elucidate is that this is not how modern
> segmented address spaces work.  It's not sufficient to have a few bits to
> specify which segment to use.  Instead, you typically need three things:
> 
> 1. The type of memory segment being described (which may be just a few bits).
> In reality, this "type" may correlate to which API you need to use to access
> this memory (e.g., main RAM, RDMA, accelerator memory, persistent memory, other
> IO memory, ...etc.).
> 2. A context of some kind (usually more than a small number of bits -- perhaps
> 32 or 64 bits).
> 3. A pointer (usually 32 or 64 bits).
> 
> For example, if I'm accessing IO memory of some flavor, I probably need some
> kind of context/handle that points to a descriptor with information about my
> data segment on the IO peripheral in question (e.g., a NIC), and then a pointer
> for the specific place that I want in that specific memory.  I use this
> context/handle + pointer with some underlying API to get/set that data.
> 
> I think that's the disconnect here: the prior thinking about MPI_Aint may have
> assumed "it's either main RAM or it's XYZ memory, and the rest of the bits are
> enough to uniquely describe the address."  But that doesn't seem sufficient any
> more to describe the evolution of complex, segmented memory systems (and be
> able to use their supporting APIs).
> 
> Meaning: perhaps a pure integer MPI_Aint may have been sufficient in the past.
> But I'm not sure that's useful any more.
> 
> --
> Jeff Squyres
> jsquyres at cisco.com

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .