Using ScatterV to split an array to multiple proce

问题:

I am working with MPI and I have to send parts of an array to different processes. As an example, consider 3 processes. Then I need to send the red elements to the first process, the greed to the second and the black to the third process.

I know I could use Scatterv twice, but I want to minimize the communication between processes and the real array that I'm splitting apart is huge. Does anyone have a suggestion on how I can accomplish this?

Here is my attempt with a derived data type:

#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>

void print_array(int *array,int n){
    int i;
    printf("\t[");
    for (i=0; i<n; i++) {
        printf(" %d",array[i]);
    }
    printf("]\n");
}

int main(int argc, char **argv){

int rank,world_size,i,n = 16, block_count = 2;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &world_size);

int *array = malloc(n * sizeof(int));
for (i=0; i<n; i++) { array[i]=i;}
if (rank==0) { print_array(array,n);}

int *sendcounts = malloc(world_size * sizeof(int));
int *reccounts = malloc(world_size * sizeof(int));
int *displs = malloc(world_size * sizeof(int));

sendcounts[0]=3; sendcounts[1]=3; sendcounts[2]=2;
displs[0]=0; displs[1]=3; displs[2]=6;

for (i=0; i<world_size; i++) {
    reccounts[i] = sendcounts[i]*block_count;
}

int root = 0;
int *recvbuf = malloc(reccounts[rank] * sizeof(int));
MPI_Datatype newtype;
MPI_Type_contiguous(block_count, MPI_INT, &newtype);
MPI_Type_commit(&newtype);

if (rank==0) {
    MPI_Scatterv(array, sendcounts, displs,
                 newtype, recvbuf, sendcounts[rank],
                 newtype, root, MPI_COMM_WORLD);
}
else {
    MPI_Scatterv(NULL, sendcounts, displs,
                 newtype, recvbuf, reccounts[rank],
                 newtype, root, MPI_COMM_WORLD);
}

MPI_Type_free (&newtype);

print_array(recvbuf,reccounts[rank]);

free(array);array = NULL;
free(sendcounts);sendcounts = NULL;
free(displs);displs = NULL;
free(recvbuf);recvbuf = NULL;
MPI_Finalize();
return 0;
}

回答1:

There is a way, but it is a bit convoluted.

The idea is you create a derived datatype with two elements at offset 0 and 8, and then resize this datatype so the upper bound is the size of one element. Then you can MPI_Scatterv() once with counts={3,3,2} and displs={0,3,6}. Note you also need to create a derived datatype on the receive side, otherwise MPI task 1 would receive {3, 11, 4, 12, 5, 13} when i guess you expect {3, 4, 5, 11, 12, 13}