I am trying to render a spaceship directly in front of a camera, with the model rotating with the camera's view. This works, as long as the player only yaws or rolls. If he does both, it goes into a tumble. A video of the problem is here: http://youtu.be/voKTsdy5TFY
The relevant code follows, edited for clarity:
D3DXMatrixRotationAxis(&playeryaw, &up, yawangle);
D3DXMatrixRotationAxis(&playerpitch, &right, pitchangle);
playerrot = playeryaw * playerpitch;
D3DXMatrixTranslation(&playertrans, pos.x, pos.y, pos.z);
D3DXMatrixScaling(&playerscale, 0.05 / ((float)i + 1), 0.05 / ((float)i + 1), 0.05 / ((float)i + 1));
playercom = playerscale * playerrot * playertrans;
device->SetTransform(D3DTS_WORLD, &playercom);
playermesh.render();
EDIT: Expanded code at the behest of Optillect Team.
That looks like Gimbol lock, which is a known limitation of using Euler angles for representing three-dimensional rotations. You might want to use matrices or quaternions instead to represent your spaceship's rotation.
Na7coldwater may be right on the gimbal lock. The 2 angles you use, yawangle and pitchangle are euler angles and hence susceptible to gimble lock.
You could avoid gimbal lock by either using quaternions or by storing the current rotation and then build a delta rotation matrix and multiply to the previous rotation matrix.
ie where you build your rotation matrix you will be storing the delta from the previous rotation rather than the absoloute rotation from the model's origin state.
e.g
where playerrot is stored from frame to frame.
It seems you apply the same transform to the spaceship. You should:
Something like this: