Testing: How to test that view contains desired da

问题:

Say a Chef can make Recipes, and Sous-Chefs can create Recipes that must be approved by a Head Chef.

You want to test that, when a Head Chef views her homepage, she sees Recipes that she herself created. You also want to test that she sees there are Recipes awaiting her approval.

I can think of two ways to do this:

1. Test that the view contains certain words, like "Your recipes" and "Recipes awaiting your approval"
2. Add unnecessary attributes to the html elements you're using so that you can check for an element with "id=recipe_1" or "data-for-the-sake-of-testing=1"

I very much dislike both of these approaches.

Why approach #1 sucks

1. Incredibly brittle tests. Every time you want to make minor updates to the copy, tests will break.
2. i18n? How will that work with this approach?

There are probably more reasons, but those two are pretty huge.

Why approach #2 sucks

How annoying to have superfluous markup just for the sake of testing! The user should not have an increased download size for the sake of tests.

---

What is a good approach to this? I'm interested to hear any alternatives at all, in whatever language you think in. I mostly think in Ruby, Test::Unit, Minitest, RSpec, and Cucumber (though my Cuke skills are stale), but if other languages/frameworks have this figured out, I'd love to see what they're doing, too.

回答1:

Use a page paradigm.

Phrase the steps in as human a way as you can, at the level of capabilities (high level) wherever possible, and use specific examples. For instance, if I'm using Cucumber I might say:

Given the sous-chef has created a recipe for Frog Pie
When the chef looks for recipes to approve
Then the recipe for Frog Pie should be in the list.

Inside the code for these steps, instantiate or find the particular page you're looking for, where the page is an object that represents the capabilities of the page. That page can then have all the things that the user can do with the page - look for recipes, approve recipes, move to another page, etc.

This way, if you need to change the underlying code for the step, you only have to change it in one place, and all the changes for a particular page will be together. Because you've phrased the scenario in terms of capabilities you're delivering, it's unlikely that the scenario will need to change much (unless you discover that your business need different capabilities to the ones you're delivering).

This also works pretty well for window-based apps too, with each widget or module being a particular page.

It's also fine to have extra ids just for testing. Sometimes designers like to use them too.

回答2:

I see at least two options:

1. Avoid testing business logic through the UI. Write a "service" or "use case controller" object that returns plain data structures. In other words, you build an API to your system. Your unit test accesses the system through the API. Your UI accesses the system through the same API, but then there should be almost no logic in the view. See http://www.confreaks.com/videos/759-rubymidwest2011-keynote-architecture-the-lost-years or http://www.cleancoders.com/codecast/clean-code-episode-7/show.

2. Use the "page object" pattern. Write an object that reads in the HTML code that your app produces, parses it, and makes interesting data available through getters. This will do wonders to make your test code clear. Your objection could be that you still have problem #2. In fact I don't think it's really a problem. If you use structural HTML markup, it should be fairly easy to extract the information you need. It will be much easier if you attach an ID to a key element of the page; in your example I would have a div with id="my-recipes" and another div with id="to-be-approved". That should be enough; anything else should be easy to find with xpath or css selectors. Why do you find this objectionable? These IDs will probably be useful for other purposes, such as attaching behaviour with unobtrusive JavaScript or attaching styles with a CSS stylesheet.

回答3:

Live with #2, perhaps using brief comments (no i18n issues and not visble to the end user):

<!-- APPROVAL -->

The documentation of simpletest has a nice take on it:

Next chance you get, look at a circuit board, perhaps the motherboard of the computer you are looking at right now. On most boards you will find the odd empty hole, or solder joint with nothing attached or perhaps a pin or socket that has no obvious function. Chances are that some of these are for expansion and variations, but most of the remainder will be for testing.

If a small amount of superfluous markup makes your product more testable and reliable then just live with it!

回答4:

I personally try to do not test views at all. I mean generated markup, since those tests appearing to be much fragile.

Instead, I focus on "data-provider" side, in case of MVC web framework is Controller. As soon as controller is covered by unit tests that check what kind of data controller prepares, you are pretty much safe. The view you create is easy to test by just running the application and see that it looks OK.

Nethertheless, there some approaches of view testing. First one is based on "end-to-end" testing simulation with Selenium Driver. It runs the browsers and initialize requests to you application. Tests are checking the output HTML. Tests logon to "known" edition, it means tests know that current localization is EN, for instance.

You should basically combine the approaches, where it works use HTML markup values ("Recipies") otherwise use HTML elements id's or classes. I would not add any additional markup for the testing.

Another approach you can try is Approval Testing. I believe there is a Ruby driver for that - http://approvaltests.sourceforge.net/. With approvals you render the view and save the HTML as golden master. The test will fail in case of View had changed. It much more easier to implement than Selenium tests.