How to MPI_Gatherv with displacements on the sendi

Question

I'm trying to recombine sub-arrays without the dark-grey rows with MPI_Gatherv. Picture's worth a thousand words:

graphical overview of the ghost/halo dark-grey cells http://img535.imageshack.us/img535/9118/ghostcells.jpg

How would you send only parts of *sendbuf (the first parameter in MPI_Gatherv manual) to the root process (without a wasteful rewriting in another structure, this time without the dark-grey rows)? The *displacements (the 4th parameter) is only relevant to the *recvbuf of the root process.

Thank you.

Update (or, being more precise)

I wanted to also send the "boundary" (light-grey) cells ... not just the "interior" (white) cells. As osgx correctly pointed out: in this case the MPI_Gatherv suffices ... some conditional array indexing will do it.

Answer 1

What about constructing a datatype, which will allow you to send only white (Interior) cells?

The combined (derived) datatype can be a MPI_Type_indexed.

The only problem will be with very first line and very last line in processes P0 and PN, because P1 and PN should send more data then P2....PN-1

For Interior+Boundary you can construct datatype of single "line" with

MPI_Datatype LineType;
MPI_Type_vector (1, row_number, 0 , MPI_DOUBLE, &LineType)
MPI_Type_commit ( &LineType);

Then (for ARRAY sized [I][J] splitted for stripecount stripes )

for (i=0; i< processes_number; ++i) { 
    displs[i] = i*(I/stripecount)+1; // point to second line in each stripe
    rcounts[i] = (I/stripecount) -2 ;
}
rcounts[0] ++; // first and last processes must send one line more
rcounts[processes_number-1] ++;
displs[0] -= 1; // first process should send first line of stripe
// lastprocess displacement is ok, because it should send last line  of stripe

source_ptr = ARRAY[displs[rank]];

lines_to_send = rcounts[rank];

MPI_Gatherv( source_ptr, lines_to_send, LineType, recv_buf, rcounts, displs, LineType, root, comm);