COMP-3 data unpacking in Java (Embedded in Pentaho

问题:

We are facing a challenge in reading the COMP-3 data in Java embedded inside Pentaho ETL. There are few Float values stored as packed decimals in a flat file along with other plain text. While the plain texts are getting read properly, we tried using Charset.forName("CP500");, but it never worked. We still get junk characters.

Since Pentaho scripts doesn't support COMP-3, in their forums they suggested to go with User Defined Java class. Could anyone help us if you have come across and solved such?

回答1:

Is it a Cobol File ???, Do you have a Cobol Copybook ???. Possible options include

1. As Bill said Convert the Comp-3 to Text on the source machine
2. Write your own Conversion Code
3. Use a library like JRecord. Note: I am the author of JRecord

---

Converting Comp-3

in Comp-3,

Value    Comp-3 (signed)   Comp-3 (Unsigned)   Zoned-Decimal
 123     x'123c'           x'123f' ??            "12C"
-123     x'123d'                                 "12L" 

There is more than one way to convert a comp-3 to a decimal integer. One way is to

1. Connvert x'123c' ->> String "123c"
2. Drop the last character and test for the sign

Java Code to convert comp3 (from a byte array:

        public static String getMainframePackedDecimal(final byte[] record,
                                               final int start,
                                               final int len) {  

            String hex  = getDecimal(record, start, start + len);
                //Long.toHexString(toBigInt(start, len).longValue());
            String ret  = "";
            String sign = "";

            if (! "".equals(hex)) {
                switch (hex.substring(hex.length() - 1).toLowerCase().charAt(0)) {
                    case 'd' : sign = "-";
                        case 'a' :
                        case 'b' :
                        case 'c' :
                        case 'e' :
                        case 'f' :
                            ret = sign + hex.substring(0, hex.length() - 1);
                        break;
                        default:
                            ret = hex;
                }
            }

            if ("".equals(ret)) {
                ret = "0";
            }
        }

        public static String getDecimal(final byte[] record, final int start, final int fin) {
            int i;
            String s;
            StringBuffer ret = new StringBuffer("");
            int b;

            for (i = start; i < fin; i++) {
                b = toPostiveByte(record[i]);
                s = Integer.toHexString(b);
                if (s.length() == 1) {
                    ret.append('0');
                }
                ret.append(s);

            }

            return ret.toString();
        }

---

JRecord

In JRecord, if you have a Cobol Copybook, there is

• Cobol2Csv a program to convert a Cobol-Data file to CSV using a Cobol Copybook
• Data2Xml convert a Cobol Data file to Xml using a Cobol Copybook.
• Read Cobol-Data File with a Cobol Copybook.
• Read a Fixed width file with a Xml Description
• Define the Fields in Java

Reading with Cobol Copybook in JRecord

        ICobolIOBuilder ioBldr = JRecordInterface1.COBOL
                .newIOBuilder(copybookName)
                    .setDialect( ICopybookDialects.FMT_MAINFRAME)
                    .setFont("cp037")
                    .setFileOrganization(Constants.IO_FIXED_LENGTH)
                .setDropCopybookNameFromFields(true);
        AbstractLine saleRecord;

        AbstractLineReader reader  = ioBldr.newReader(salesFile);
        while ((saleRecord = reader.read()) != null) {
            ....
        }

        reader.close();

Defining the File in Java with JRecord

        AbstractLineReader reader = JRecordInterface1.FIXED_WIDTH.newIOBuilder()
                                .defineFieldsByLength()
                                    .addFieldByLength("Sku"  , Type.ftChar,   8, 0)
                                    .addFieldByLength("Store", Type.ftNumRightJustified, 3, 0)
                                    .addFieldByLength("Date" , Type.ftNumRightJustified, 6, 0)
                                    .addFieldByLength("Dept" , Type.ftNumRightJustified, 3, 0)
                                    .addFieldByLength("Qty"  , Type.ftNumRightJustified, 2, 0)
                                    .addFieldByLength("Price", Type.ftNumRightJustified, 6, 2)
                                .endOfRecord()
                                .newReader(this.getClass().getResource("DTAR020_tst1.bin.txt").getFile());
        AbstractLine saleRecord;

        while ((saleRecord = reader.read()) != null) {
        }

---

Zoned Decimal

Another Mainframe-Cobol numeric format is Zoned-Decimal. It is a text format where the sign is Over-typed on the last digit. In zoned-decimal 123 is "12C" while -123 is "12L".