CCSID on MQ Managers on different platforms

Question

If a Solaris based MQ Manager is sending messages to an intermediate Windows MQ Manager that then sends on messages to a Linux MQ manager, does the CCSID need to be changed on them all to be the same? I don't think so but I am being pushed to do so...

I have an issue where a client is sending messages from an application on Solaris via an intermediate MQ Manager on Windows but the Linux MQ Manager final destination based application receives the message with CR/LF line ending characters it can't deal with. Should the receiver end group write a conversion exit program? Or an MCA?

Answer 1

IBM MQ does not deal with line feeds. It is not like transferring a file with ftp and using ascii mode where ftp will convert from Unix LF end of line to Window CR and LF end of line. For MQ it is just another character in the character set to translate. If the sending app is sending data that includes CR/LF characters, MQ will treat them as any other character in the data. If the receiving application expecting a file with end of lines to be sent as a MQ message it would be required to deal with the end of lines.

---

MQ Classes for Java applications running on a Solaris server default to CCSID 819, and a IBM MQ queue manager running on Solaris also will default to CCSID 819. CCSID 819 is described as ISO 8859-1 ASCII.

I created a file called test.txt containing the single word "test" and ran unix2dos against the file.

The output below shows the ASCII characters as HEX values:

$cat test.txt | hexdump -C
00000000  74 65 73 74 0d 0a                                 |test..|
00000006

Note that in the above output the ASCII hex values 0d is the CR and 0a is the LF, these are the common Windows end of line.

If we assume that the default CCSID 819 is used for both the Solaris MQ Classes for Java application and the Solaris queue manager, then we can start out with the assumption that the above two hex values represent CR/LF at the end of each line.

You stated that your Windows queue manager has CCSID 437 which is typical for a US based Windows server. CCSID 437 is described as USA PC-DATA and is also ASCII.

Linux queue managers typically default to CCSID 1208. CCSID 1208 is described as UTF-8 with IBM PUA and is a Variable Byte character set it can have from 1 to four bytes per character. This can represent over a million characters including all ASCII characters. All 127 ASCII characters are represented in UTF-8 as the same single byte HEX values as in ASCII.

Going from ASCII to UTF-8 is loss less, going from UTF-8 to ASCII can have loss if non-ASCII characters are used since there is not a equivalent character in ASCII and MQ converts it to the default substitution character with HEX value 1A. I have seen this for example with a Euro symbol. Most if not all of the first 255 characters of UTF-8 are the same as CCSID 819.

Given the above CCSID assumptions the conversion would look like this:

Original IBM MQ Classes for Java app running on Solaris sending data with CR/LF end of line characters:

$cat test.txt | hexdump -C
00000000  74 65 73 74 0d 0a                                 |test..|
00000006

Solaris Queue Manager Sender channel with CONVERT(YES) sending to Windows Queue Manager with CCSID 437:

cat test.txt | iconv -f IBM819 -t IBM437 | hexdump -C
00000000  74 65 73 74 0d 0a                                 |test..|
00000006

As expected the output is the same since both 819 and 437 are ASCII character sets and the data was not representing any thing above the first 127 ASCII characters.

Solaris Queue Manager Sender channel with CONVERT(YES) sending to Windows Queue Manager with CCSID 437 Sender channel with CONVERT(YES) sending to Linux Queue Manager with CCSID 1208:

cat test.txt | iconv -f IBM819 -t IBM437 | iconv -f IBM437 -t UTF-8 | hexdump -C
00000000  74 65 73 74 0d 0a                                 |test..|
00000006

As expected the output is the same since both 819 and 437 are ASCII character sets and the first 127 characters of UTF-8 (1208) are normal ASCII characters.

---

Summary: If the sending application is sending CR/LF in the data, MQ message conversion will not change this and if the CCSIDs in use are the above listed CCSIDs it does not even change the actual HEX character value. The sending application would need to change what they are sending or the receiving application would need to accommodate these characters.

---

A good reference on ASCII, UNICODE, UTF-8 and more can be found in the article "Unicode, UTF8 & Character Sets: The Ultimate Guide".

Answer 2

I smell a bad setup in your MQ environment and anyone who says to make them all the same doesn't know or doesn't understand MQ (& should have zero opinion about the configuration).

Also, setting the sender channel CONVERT attribute to YES is a VERY bad idea and ONLY should be used in extreme cases. see IBM Best Practices for more information. The ONLY time data conversion should be done is when the application issues an MQGET with Convert.

Since, Solaris has no clue about CR/LF, I going to guess at what is going wrong:

• The message's Format field of MQMD has the value 'MQSTR' set.
• The sender channel between the Solaris queue manager has the attribute CONVERT set to YES
• The receiving application on Linux did NOT issue and MQGET with Convert.

Questions:

1. Did the Solaris application actually put the message with CR/lF?
2. Did the Solaris application set the MQMD Format field to 'MQSTR'?
3. Why are the messages hoping through the Windows queue manager? Why not make a direct connection between Solaris queue manager and the Linux queue manager?
4. What value does the CONVERT attribute of the Solaris sender channel have? i.e. Is it Yes or No?

The simplest solution is to have the Linux application issue an MQGET with Convert assuming the MQMD Format field is set to 'MQSTR'. MQ will automatically convert the message data to the correct platform.

If the MQMD Formet field is not set to 'MQSTR' then MQ will NOT convert the message data between platforms. That would imply that the Solaris application put the CR/LF in the message. I find this hard to believe. If a developer did this then they truly do not know MQ (and should not be programming it).