Design considerations for internationalization

Question

I've read Joel's article on Unicode and I feel that I have at least a basic grasp of internationalization from a character set perspective. In addition to reading this question, I've also done some of my own research on internationalization in regards to design considerations, but I can't help but suspect that there is a lot more out there that I just don't know or don't know to ask.

Some of the things I've learned:

• Some languages read right-to-left instead of left-to-right.
• Calendar, dates, times, currency, and numbers are displayed differently from language to language.
• Design should be flexible enough to accommodate a lot more text because some languages are far more verbose than others.
• Don't take icons or colors for granted when it comes to their semantic meaning as this can vary from culture to culture.
• Geographical nomenclature varies from language to language.

Where I'm at:

• My design is flexible enough to accommodate a lot more text.
• I automatically translate each string, including error messages and help dialogs.
• I haven't come to a point yet where I've needed to display units of time, currency or numbers, but I'll be there shortly and will need to develop a solution.
• I'm using the UTF-8 character set across the board.
• My menus and various lists in the application are sorted alphabetically for each language for easier reading.
• I have a tag parser that extracts tags by filtering out stop words. The stop words list is language specific and can be swapped out.

What I'd like to know more about:

• I'm developing a downloadable PHP web application, so any specific advice in regards to PHP would be greatly appreciated. I've developed my own framework and am not interested in using other frameworks at this time.
• I know very little about non-western languages. Are there specific considerations that need to be taken into account that I haven't mentioned above? Also, how do PHP's array sorting functions handle non-western characters?
• Are there any specific gotchas that you've experienced in practice? I'm looking in terms of both the GUI and the application code itself.
• Any specific advice for working with date and time displays? Is there a breakdown according to region or language?
• I've seen a lot of projects and sites let their communities provide translation for their applications and content. Do you recommend this and what are some good strategies for ensuring that you have a good translation?
• This question is basically the extent of what I know about internationalization. What don't I know that I don't know that I should look into further?

Edit: I added the bounty because I would like to have more real-world examples from experience.

Answer 1

A thing about numbers: in English, as I understand, you just use a singular with 1 and plural with 2 or more. Like: “You have 1 message”; “2 messages”; “3... messages”. In Russian, these things get more complicated. You use singular for 1, 21, 31, 41... 101, 121 (so, for everything ending with 1 except when it ends with 11). Then you use singular genitive case for 2, 3, 4; 22, 23, 24; 32, 33, 34... 102, 103, 104; 122, 123, 124. And in all other cases you use plural genitive case.

It’s not really hard to implement. What is hard though is to implement something that will know how to deal with any a priori unknown language with all its weirdness :-)

And that’s just numbers :-)

Answer 2

• Collation/sorting rules can differ wildly between languages: ä is sorted differently in German than it is in Swedish. So sorting needs to be culture-specific.
• Upper/lowercasing can hold suprises: The German "sharp S" character ß does not have an uppercase version, and is either transformed to "SS", or stays lowercase if exactness is important. Turkish has a dotless lowercase i and an uppercase dotted I.
• For multilingual web apps, think carefully about how to decide what version to show and how to work it into the URL. The user should always be able to manually choose the language, and you want search engines to find different language versions under different URLs.
• Some East Asian languages (namely Japanese and Chinese, maybe others) don't have spaces between words
• Japanese (maybe others too) has separate versions ("full width") of arabic digits and space, and even two versions of some of its own characters (half-width and full-width katakana).

Answer 3

I would like to make the following comments - these are from some company guidelines where the class-1 products are translated in 31 different locales. Following these guidelines has given us (our development team rather than the whole company) the greatest productivity in translation.

• Don't try to reuse snippets of error messages. For example, don't think that because you have the two errors "You selected the wrong menu item" and "That menu item is not yet available", you can extract "menu item" into a separate item and use it in both places. All messages should be self contained as their translations may change based on context.

• Use a professional translator knowledgeable about technology. If you go near a service like BabelFish, you're going to get everything you deserve. For example, "Microsoft Windows" is "Microsoft Windows" everywhere on the planet, it doesn't become "Microsoft Fenster" in Germany.

• Try not to embed variables within your messages (such as "The %1 has failed" where %1 changes dynamically) since positions and, indeed, gender may change: "La table est rubbish" vs. "L'Homme est drunk", or "The red table" vs "La table rouge". Better to use a generic noun with appended parameters: "The item has failed [%1]".

• Only translate things which the user is expected to see. Log messages in a log file (that only you will use) should be in English (or your native language), not translated to something like Swahili that you couldn't read anyway.

• Menus should be sorted by functionality, not collating order.

• Translatable units should be stored external to the code and loaded in at runtime. This makes translation an issue of just shipping off the external file, not trying to shoehorn changes into the middle of code. It also makes adding other languages easier in future.

That's enough for now. Better to stop before you all fall asleep :-)

Answer 4

My first answer in StackOverflow, so pardon if some stupid was said.

From my experience:

• PHP: gettext has been extremely helpful;
  
• non-western languages: UTF-8 everywhere (code, DB) and so far we're doing well;
  
• Are there any specific gotchas that you've experienced in practice? Breaking long paragraphs for i18n into different sentences can be less expensive to translate, if the string is repeated more than once in the site you only need to have it translated once. But, be careful, if you fragment the text too much translators will lose context;
  
• I've seen a lot of projects and sites let their communities provide translation for their applications and content. Do you recommend this and what are some good strategies for ensuring that you have a good translation? If you have a very large number of volunteers go for it, but depending on how much text you have, you might really need a ton of volunteers. Always make sure also that you have someone you trust being the leader of a language project to be the proof-reader controlling the accuracy of the translation.
  

Answer 5

Yes, this is a massive subject. Getting it right is an awful lot of work.

In my program I use an integer key for every piece of text and look it up in a file as needed depending on the language. There's no literal strings anywhere in the code, only keys. I define them with an "enum" in C++ so I'm not actually typing numbers. I wrote a utility to synchronize the various language files when I add more enums and the translators fill in the blanks.

Each key also has an associated tooltip, image, keyboard shortcut, etc.

As for times and dates ... again, this is much more complex than you might think but doesn't PHP handle this for you? (I don't know, I'm a C++ guy...)

Answer 6

Our game Gemsweeper has been translated to 8 different languages. Some things I have learned during that process:

• If the translator is given single sentences to translate, make sure that he knows about the context that each sentence is used in. Otherwise he might provide one possible translation, but not the one you meant. Tools such as Babelfish translate without understanding the context, which is why the result is usually so bad. Just try translating any non-trivial text from English to German and back and you'll see what I mean.

• Sentences that should be translated must not be broken into different parts for the same reason. That's because you need to maintain the context (see previous point) and because some languages might have the variables at the beginning or end of the sentence. Use placeholders instead of breaking up the sentence. For example, instead of

"This is step" "of our 15-step tutorial"

Write something like:

"This is step %1 of our 15-step tutorial"

and replace the placeholder programmatically.

• Don't expect the translator to be funny or creative. He usually isn't motivated enough to do it unless you name the particular text passages and pay him extra. For example, if you have and word jokes in your language assets, tell the translator in a side note not to try to translate them, but to leave them out or replace them with a more somber sentence instead. Otherwise the translator will probably translate the joke word by word, which usually results in complete nonsense. In our case we had one translator and one joke writer for the most critical translation (English).

• Try to find a translator who's first language is the language he is going to translate your software to, not the other way round. Otherwise he is likely to write a text that might be correct, but sounds odd or old-fashioned to native speakers. Also, he should be living in the country you are targeting with your translation. For example a German-speaking guy from Switzerland would not be a good choice for a German translation.

• If any possible, have one of your public beta test users who understands the particular translation verify translated assets and the completed software. We've had some very good and very bad translations, depending on the person who provided it. According to some of our users, the Swedish translation was total gibberish, but it was too late to do anything about it.

• Be aware that, for every updated version with new features, you will have to have your languages assets translated. This can create some serious overhead.

• Be aware that end users will expect tech support to speak their language if your software is translated. Once again, Babelfish will most probably not do.

Edit - Some more points

• Make switching between localizations as easy as possible. In Gemsweeper, we have a hotkey to switch between different languages. It makes testing much easier.

• If you are going to use exotic fonts, make sure these include special characters. The fonts we chose for Gemsweeper were fine for English text, but we had to add quite a few characters by hand which only exist in German, French, Portughese, Swedish,...

• Don't code your own localization framework. You're probably much better off with an open source framework like Gettext. Gettext supports features like variables within sentences or pluralization and is rock-solid. Localized resources are compiled, so nobody can tamper with them. Plus, you can use tools like Poedit for translating your files / checking someone else's translation and making sure that all strings are properly translated and still up to date in case you change the underlying source code. I've tried both rolling my own and using Gettext instead and I have to say that Gettext plus PoEdit were way superior.

Edits - Even More Points

• Understand that different cultures have different styles of number and date formats. Numbering schemes are not only different per culture, but also per purpose within that culture. In EN-US you might format a number '-1234'; '-1,234' or (1,234) depending on what the purpose of the number is. Understand other cultures do the same thing.

• Know where you're getting your globalization information from. E.g. Windows has settings for CurrentCulture, UICulture, and InvariantCulture. Understand what each one means and how it interacts with your system (they're not as obvious as you might think).

• If you're going to do east Asian translating, really do your homework. East-Asian languages have quite a few differences from languages here. In addition to having multiple alphabets that are used simultaneously, they can use different layout systems (top-down) or grid-based. Also numbers in east Asian languages can be very different. In the en-US you only change systems for limited conditions (e.g. 1 versus 1st), there are additional numeric considerations besides just comma and period.