How Domain maps map indexes to target locales arra

Question

I didn't find how the domain map maps the indices in the multi-dimensional domains to the multi-dimensional target locales.

1.) How the target locales (one dimension) is arranged in multi-dimension fashion which equals the distribution dimension to map the indexes?

2.) In documentation it states that for multi-dimension case, the computation should be done in every dimension. For the domain {1..8, 1..8} ==> dom assume dom is block-distributed over 6 target locales.

Steps in mapping

1 for 1st dimension (1..8) do the computation
if idx is low<=idx<=high then locid is
floor (idx-low)*N / (high-low+1) gives me an index say i.

Repeat the same for 2nd dimension which gives me an index say j.
Now I have a tuple ( i, j ) how this is mapped to the target locales array of dimension 2?

What the domain map do for changing the 1D target locales array to distribution dimension?

Is something like reshape function ?

Please let me know if this lacks sufficient information.

Answer 1

The specific details about how a domain's indices are mapped to a program's locales are not defined by the Chapel language itself, but rather by the implementation of the domain map used to declare the domain. In the comments under your question, you mention that you're referring to the Block distribution, so I'll focus on that in my answer (documented here), but note that any other domain map could take a different approach.

The Block distribution takes an optional targetLocales argument which permits you to specify the set of locales to be targeted, as well as their virtual topology. For instance, if I declare and populate a few arrays of locales:

var grid1: [1..3, 1..2] locale,   // a 3 x 2 array of locales
    grid2: [1..2, 1..3] locale;   // a 2 x 3 array of locales

for i in 1..3 {
  for j in 1..2 {
    grid1[i,j] = Locales[(2*(i-1) + j-1)%numLocales];
    grid2[j,i] = Locales[(3*(j-1) + i-1)%numLocales];
  }
}

I can then pass them in as the targetLocales arguments to a few instances of a Block-distributed domain:

use BlockDist;

config const n = 8;

const D = {1..n, 1..n},
      D1 = D dmapped Block(D, targetLocales=grid1),
      D2 = D dmapped Block(D, targetLocales=grid2);

Each domain will distribute its n rows to the first dimension of its targetLocales grid and its n columns to the second dimension. We can see the results of this distribution by declaring arrays of integers over these domains and assigning them in parallel to make each element store its owning locale's ID, as follows:

var A1: [D1] int,
    A2: [D2] int;

forall a in A1 do
  a = here.id;

forall a in A2 do
  a = here.id;

writeln(A1, "\n");
writeln(A2, "\n");

When running on six or more locales (./a.out -nl 6), the output is as follows, revealing the underlying grid structure:

0 0 0 0 1 1 1 1
0 0 0 0 1 1 1 1
0 0 0 0 1 1 1 1
2 2 2 2 3 3 3 3
2 2 2 2 3 3 3 3
2 2 2 2 3 3 3 3
4 4 4 4 5 5 5 5
4 4 4 4 5 5 5 5

0 0 0 1 1 1 2 2
0 0 0 1 1 1 2 2
0 0 0 1 1 1 2 2
0 0 0 1 1 1 2 2
3 3 3 4 4 4 5 5
3 3 3 4 4 4 5 5
3 3 3 4 4 4 5 5
3 3 3 4 4 4 5 5

For a 1-dimensional targetLocales array, the documentation says:

If the rank of targetLocales is 1, a greedy heuristic is used to reshape the array of target locales so that it matches the rank of the distribution and each dimension contains an approximately equal number of indices.

For example, if we distribute to a 1-dimensional 4-element array of locales:

var grid3: [1..4] locale;

for i in 1..4 do
  grid3[i] = Locales[(i-1)%numLocales];

var D3 = D dmapped Block(D, targetLocales=grid3);

var A3: [D3] int;

forall a in A3 do
  a = here.id;

writeln(A3);

we can see that the target locales form a square, as expected:

0 0 0 0 1 1 1 1
0 0 0 0 1 1 1 1
0 0 0 0 1 1 1 1
0 0 0 0 1 1 1 1
2 2 2 2 3 3 3 3
2 2 2 2 3 3 3 3
2 2 2 2 3 3 3 3
2 2 2 2 3 3 3 3

The documentation is intentionally vague about how a 1D targetLocales argument will be reshaped if it's not a perfect square, but we can find out what's done in practice by using the targetLocales() query on the domain. Also, note that if no targetLocales array is supplied, the entire Locales array (which is 1D) is used by default. As an illustration of both these things, if the following code is run on six locales:

var D0 = D dmapped Block(D);

writeln(D0.targetLocales());

we get:

LOCALE0 LOCALE1
LOCALE2 LOCALE3
LOCALE4 LOCALE5

illustrating that the current heuristic matches our explicit grid1 declaration above.