We have multiple products here that shared some common libraries. These libraries are part of a separate solution (so they can be built by TFS independently), but the problem is during development, one has to modify the common library, compile it to binary, copy it to the common location, compile the product solution.
In order to avoid this actually I am wondering if its possible to have conditional references, so for a debug configuration, I would reference them as project references, while in release configuration they would be binary references.
You might want to have a look at NuGet:
(where you would be the third party yourself in this case)
Note: This would not give you conditional references, but it would ease updating the common components.
You should be able to do this with conditional constructs by editing the project file directly (VS IDE won't do this for you).
For example, you might do something like this using the "Choose" element:
MSDN has more information about using conditional constructs.