(Environment: VSTS2010 using IronPython and the TFS SDK)
Workflow #1 The following workflow behaves as I expect:
- Create a new testrun
- Add testpoint(s)
- Save testrun
My expected behavior: The testrun is saved with the testpoints.
Observed behavior: Matches my expected behavior - verified with a testrun.QueryTestResults() call.
Workflow #2 The following workflow does not behave as I expect:
- Create a new testrun
- Add testpoint
- Save testrun
- Add another testpoint
- Save testrun
My expected behavior: The testrun should be saved with the test points.
Observed behavior: The first testpoint is saved. All other testpoints after the initial save do not get saved. There are no errors thrown or any feedback from the SDK indicating failures.
Workflow #3 Similarly, the following workflow does not behave as I expect:
- Get an existing testrun by id testManagementService.QueryTestRuns("SELECT * FROM TestRun WHERE TestRunId=%s" % testrunId)
- Add testpoint
- Save testrun
My expected behavior: The testrun should be saved with the test points.
Observed behavior: The added testpoint is not added. There are no errors thrown or any feedback from the SDK indicating failures.
Can anyone explain why the observed behavior differs from my expected behavior on workflows #2 and #3?
UPDATE (2012-11-16 12:00 CST)
Answering (editing) my own question since digging for this was not straight forward.
I found the following paragraph at http://blogs.msdn.com/b/nidhithakur/archive/2011/04/08/importing-testcase-results-to-mtm.aspx after doing a msdn.microsoft.com serach (search terms: itestrun add testpoint failure).
To be able to add results to this test plan, create a new test run in the test plan. map the test points which are present in the dictionary and add to this test run. Ideally, you should be able to add the results while adding the test point to the run but the run.Save() API is only working for single Save right now. So, you will need to add all the testpoints, save the test run and then iterate over the run collection to add results individually. For better performance, save the result collection once after all the results have been added/updated.
So this appears to be a limitation with VSTS2010. You cannot add test points after the first testrun save.
