FoxPro, Google Translate, and Unicode

Question

I have recently been given the task of writing a piece of FoxPro to interface with Google Translate so we can translate text in our software to the language of the current user/machine.

The code I have found/modified/written works perfectly for Latin-based character sets, but if you try something like Chinese it comes back all question marks.

I have already tried using the VFP function STRCONV() with every option combination possible, but no success. I also tried setting the LocaleID before the text is manipulated in any way--still no luck.

At this point, I am out of ideas. Being an old DOS programmer, I have little to no experience in dealing with unicode.

I have not included any code, because as I said, the code works fine unless you try to use it with Chinese (or Japanese).

Please help!

Edit: This is the function which does the communication with google. There are other support functions, but they dont have to do with the encoding.

*   MODIFIED BY: MICHAEL COOLEY - 11/19/2012
*   PURPOSE: TRANSLATE TEXT FROM ONE LANGUAGE TO ANOTHER
*   EXPECTS: STRING (SOURCE LANGUAGE CODE)
*            STRING (DESTINATION LANGUAGE CODE)
*            STRING (THE TEXT TO TRANSLATE)
*   RETURNS: 
FUNCTION Translate(lcFrom,lcTo,lcText)
    LOCAL lcHttp AS MSXML2.XMLHTTP
    LOCAL lcRequest AS String

    lcRequest = "http://translate.google.com/translate_a/t?client=j" + ;
                "&"+"text="+this.EncodeURL(lcText)+"&"+"hl="+lcTo+"&"+"sl="+lcFrom+"&"+"tl="+lcTo

    lcHttp = CREATEOBJECT("MSXML2.XMLHTTP")
    lcHttp.open("GET",lcRequest,.f.)

    IF lcHttp.status == 200
        lcText = this.GetTranslationString("trans", lcHttp.responseText) + CHR(10)
    ELSE
        lcText = ""
    ENDIF

    RETURN lcText
ENDFUNC

Answer 1

omg, I can't believe people still using FoxPro. The problem with Chineese is that it's probably using 2 bytes to encode characters whereas latin based langs are most likely single byte and thus are backward compatible with latin1 encoding which probably is the default character encoding that foxpro is using.

Where does your app actually break? As in, is it just a rendering issue? Or do the characters you get from whatever you are using to communicate to the Google Translate service are coming back garbled ?

E.g. if you are using some kind of stream representation (sorry I am not a VBA programmer, don't have the details), this stream object should have a facility to set the character encoding.

EDIT: try setting request headers like so

lcHttp.setRequestHeader "Content-Type", "text/html; charset=utf-8"

Answer 2

The Unicode issue is not new in VFP, I suggest read this article from Rick Strahl:

--- Using Unicode in Visual FoxPro Web and Desktop Applications ---

http://www.west-wind.com/presentations/foxunicode/foxunicode.asp

In this article, I describe how you can use Unicode with your application in the context of supporting multiple languages simultaneously. Although Visual FoxPro can’t display Unicode directly, it can display different character set through use of CodePages – a locale specific character mapping - which VFP readily supports. This works well for applications that display content only from a single language/locale. But this approach has serious limitations if you need to display strings from multiple languages simultaneously. I start with an overview of the issues and how to work with Unicode in general, then show you how to retrieve and update Unicode data, and finally, show you how to get the Unicode content to display both in your Web and desktop user interfaces