Assign specific RGB colours to 3d mesh/surface/poi

Question

Face and feature landmarks

I have a face image that has labelled face features. The image is stored in standard JPEG format and the landmarks are stored in [x y] format (x,y of point corresponds to its coordinates on the image as shown below)

---

Interpolated 3d face mesh

I have generated depth information (a 3d mesh) for each of the labelled points, and have a matrix in [x y z] format, where the coordinates x and y are the same as that of the points.

The sparse mesh looks like this:

I then interpolated over xrange, yrange and zrange to get a better mesh. Using mesh(xrange,yrange,zrange) gives me the following

The colours for face image pixels can be obtained using imread(face_image.jpg). Given that the (x,y) value of each of the interpolated point corresponds to (x,y) in the image, is it possible to make the colour of the pixel at (x,y,z)[3dmesh] the same as colour of (x,y)[face image]?

This would effectively superimpose/warp the face on the3d mesh, giving me a 3d face model.

Answer 1

I would suggest this:

n=50000; % chose something appropriate
[C,map] = rgb2ind(FaceImageRGB,n);

To map the color in your RGB image into a linear index. Make sure the mesh and the RGB image have the same x-y dimensions.

Then use surf to plot the surface with the indexed values for color (should be in the form surf(X,Y,Z,C)) and the map as color map.

surf(3dmesh, C), shading flat;
colormap(map);

Edit: a working example (with a colorful image this time...):

rgbim=imread('http://upload.wikimedia.org/wikipedia/commons/0/0d/Loriculus_vernalis_-Ganeshgudi,_Karnataka,_India_-male-8-1c.jpg');

n=50000; % chose something apropriate
[C,map] = rgb2ind(rgbim,n);    

% Creation of mesh with the same dimensions as the image:
[X,Y] = meshgrid(-floor(size(rgbim, 2)/2):floor(size(rgbim, 2)/2), -floor(size(rgbim, 1)/2):floor(size(rgbim, 1)/2));

% An arbitrary function for Z:
Z=-(X.^2+Y.^2);

% Display the surface with the image as color value:
surf(X, Y, Z, double(C)), shading flat
colormap(map);

Result: