Visual Studio 2010 & 2008 can't handle source

Question

Direct Question: If I have two files with the same name (but in different directories), it appears that only Visual Studio 2005 can handle this transparently?? VS 2008 & 2010 require a bunch of tweaking? Aside from my naming convention, am I doing something wrong?

Background:

I'm developing C++ statistical libraries... I have two folders:

/ Univariate

Normal.cpp
Normal.h
Beta.cpp
Beta.h
Adaptive.cpp
Adaptive.h

/ Multivariate

Normal.cpp
Normal.h
Beta.cpp
Beta.h
Adaptive.cpp
Adaptive.h

I need to support cross compilation -- I'm using g++/make to compile these same files into a library in Linux. They work just fine.

I had been using Visual Studio 2005 without issue, but I need to upgrade to Visual Studio 2008 or 2010 (currently drooling over nVidia's nsight tool). However, I'm having trouble if I add files to a project with the same name (even if they're in a different directory). I'm willing to change my naming convention, but I'm curious if others have encountered this problem and have found any well documented solutions??

I'm further boggled by the fact that if I upgrade from 2005 projects to 2010 projects, it appears that VS 2010 is able to correctly handle two source files with the same name in different directories; however, if I remove one of the duplicate files and then add it back to the project I am greeted by the following warning:

Distributions\Release\Adaptive.obj : warning LNK4042: object specified more than once; extras ignored

Now I have the intermediate directory specified as $(ProjectName)\$(Configuration) -- I need to have my object files in a different location from my source tree. So I can see why it's copying the object files on top of each other, but when the projects are converted from 2005 to 2008 or 2010, a bunch of conditional compiles are added:

<ObjectFileName Condition="'$(Configuration)|$(Platform)'=='Release|x64'">$(IntDir)%(Filename)1.obj</ObjectFileName>
<XMLDocumentationFileName Condition="'$(Configuration)|$(Platform)'=='Release|x64'">$(IntDir)%(Filename)1.xdc</XMLDocumentationFileName>

These are accessible from the Source file Properties page in C/C++ -> Output Files -> "Object File Name" & "XML Documentation File Name". But if I simply add the file directly (or remove and re-add them), VS doesn't complain until I try to compile, but also never adds the conditional directives -- So in order for things to work correctly, I have to add the conditional directives myself for every single configuration. Am I making a mistake / poor assumption or have I uncovered a valid bug in VS 2008 / 2010?

Answer 1

You're right, VS can't handle that, and never could. The root problem is that it generates a .obj file for each .cpp file in the project, and they're all placed in the same folder. So you end up with multiple .cpp files compiling to Adaptive.obj in your case, for example.

At least the linker generates a warning for it now. That wasn't always the case.

You should be able to work around this by ensuring the files use different Intermediate Directory paths, but it is a bit of a hack around something that ought to be possible.

Of course, you could always file a bug report or feature request on it on Microsoft Connect

Answer 2

The other solutions in this thread suffer from the RelativeDir-based ../.. problems and having to set things manually on each source file.

Not to mention, they wreck /MP. Any solution which specifies an exact .obj for %(ObjectFileName) will result in a different /Fo for each .cpp file (to map it to a specific .obj file) passed to CL.exe and thus Visual Studio can't batch them. Without batching several .cpp files with identical commandlines (including /Fo) the /MP can't work.

Here's a new approach. This works on vs2010 through to vs2015 at least. Add this to your vcxproj in the <project>

<!-- ================ UNDUPOBJ ================ -->
<!-- relevant topics -->
<!-- https://stackoverflow.com/questions/3729515/visual-studio-2010-2008-cant-handle-source-files-with-identical-names-in-diff/26935613 -->
<!-- https://stackoverflow.com/questions/7033855/msvc10-mp-builds-not-multicore-across-folders-in-a-project -->
<!-- https://stackoverflow.com/questions/18304911/how-can-one-modify-an-itemdefinitiongroup-from-an-msbuild-target -->
<!-- other maybe related info -->
<!-- https://stackoverflow.com/questions/841913/modify-msbuild-itemgroup-metadata -->
<UsingTask TaskName="UNDUPOBJ_TASK" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
  <ParameterGroup>
    <OutputDir ParameterType="System.String" Required="true" />
    <ItemList ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
    <OutputItemList ParameterType="Microsoft.Build.Framework.ITaskItem[]" Output="true" />
  </ParameterGroup>
  <Task>
    <Code><![CDATA[
            //general outline: for each item (in ClCompile) assign it to a subdirectory of $(IntDir) by allocating subdirectories 0,1,2, etc., as needed to prevent duplicate filenames from clobbering each other
            //this minimizes the number of batches that need to be run, since each subdirectory will necessarily be in a distinct batch due to /Fo specifying that output subdirectory

            var assignmentMap = new Dictionary<string,int>();
            HashSet<string> neededDirectories = new HashSet<string>();
            foreach( var item in ItemList )
            {
              //solve bug e.g. Checkbox.cpp vs CheckBox.cpp
              var filename = item.GetMetadata("Filename").ToUpperInvariant(); 

              //assign reused filenames to increasing numbers
              //assign previously unused filenames to 0
              int assignment = 0;
              if(assignmentMap.TryGetValue(filename, out assignment))
                assignmentMap[filename] = ++assignment;
              else
                assignmentMap[filename] = 0;

              var thisFileOutdir = Path.Combine(OutputDir,assignment.ToString()) + "/"; //take care it ends in / so /Fo knows it's a directory and not a filename
              item.SetMetadata( "ObjectFileName", thisFileOutdir );
            }

            foreach(var needed in neededDirectories)
              System.IO.Directory.CreateDirectory(needed);

            OutputItemList = ItemList;
            ItemList = new Microsoft.Build.Framework.ITaskItem[0];

        ]]></Code>
  </Task>
</UsingTask>

<Target Name="UNDUPOBJ">
  <!-- see stackoverflow topics for discussion on why we need to do some loopy copying stuff here -->
  <ItemGroup>
    <ClCompileCopy Include="@(ClCompile)"/>
    <ClCompile Remove="@(ClCompile)"/>
  </ItemGroup>
  <UNDUPOBJ_TASK OutputDir="$(IntDir)" ItemList="@(ClCompileCopy)" OutputItemList="@(ClCompile)">
    <Output ItemName="ClCompile" TaskParameter="OutputItemList"/>
  </UNDUPOBJ_TASK>
</Target>
<!-- ================ UNDUPOBJ ================ -->

And then modify <project> so that it reads:

<Project InitialTargets="UNDUPOBJ" ...

The result will be something like myproj/src/a/x.cpp and myproj/src/b/x.cpp compiling to Debug/0/x.obj and Debug/1/x.obj. RelativeDirs arent employed and so aren't a problem.

Additionally, in this case, there will be only two different /Fo passed to CL.exe: Debug/0/ and Debug/1/. Consequently, no more than two batches will get issued to CL.exe, allowing the /MP to work more efficiently.

Other approaches would be basing the .obj subdirectories on the .cpp subdirectories, or making the .obj filename contain some memento of the original .cpp directory so that you can readily see a .cpp->.obj mapping, but those result in more /Fo and therefore less batching. Future work could dump a mapping file for quick reference, perhaps.

See this for more details on /MP and batching : MSVC10 /MP builds not multicore across folders in a project

I've tested this in production for quite a while on vs2010 and vs2015 on a variety of toolchains. It seems bulletproof, but there's always a chance it may interact badly with other msbuild customizations or exotic toolchains.

Starting in vs2015, if you get a warning "warning MSB8027: Two or more files with the name of X.cpp will produce outputs to the same location" then you can add this to your project or msbuild files:

<PropertyGroup Label="Globals"><IgnoreWarnCompileDuplicatedFilename>true</IgnoreWarnCompileDuplicatedFilename></PropertyGroup>

See more at https://connect.microsoft.com/VisualStudio/feedback/details/797460/incorrect-warning-msb8027-reported-for-files-excluded-from-build and How to suppress specific MSBuild warning