java.io.IOException: Not in GZIP format with class

Question

I am trying to load some GZIP-ed data from a resource in my .jar, but I get a java.io.IOException: Not in GZIP format message.

When I load the same data from a file, I don't get any error. Why? (It is a maven project I compile with NetBeans)

Here is the test code generating the issue:

public static void main(String[] args) throws IOException {

    byte[] dummy = new byte[10];

    // Reading data from file
    File f = new File("C:\\Temp\\422\\convert1900.data");
    DataInputStream is = new DataInputStream(
            new GZIPInputStream(new FileInputStream(f)));

    while ( is.read(dummy) != -1 );

    // Reading data from resource
    InputStream ins = CompareTest2.class.getResourceAsStream(
            "/net/cv/convert1900.data");

    is = new DataInputStream(
        new GZIPInputStream(ins)); // Issue happens here

    while ( is.read(dummy) != -1 );

}

EDIT

Both 'files' have the same content.

EDIT 2

I just tried to count the number of bytes I get with both methods using the following code:

public static void main(String[] args) throws IOException {

    int total = 0;
    int retr = 0;

    byte[] dummy = new byte[10];

    // Reading data from file
    File f = new File("C:\\Temp\\422\\convert1900.data");
    InputStream is = new FileInputStream(f);

    do {
        retr = is.read(dummy);
        if ( retr != -1 )
            total += retr;
    } while ( retr != -1 );

    System.out.println("Total file: " + total);

    // Reading data from resource
    InputStream ins = CompareTest2.class.getResourceAsStream(
            "/net/cv/convert1900.data");

    total = 0;
    retr = 0;

    do {
        retr = ins.read(dummy);
        if ( retr != -1 )
            total += retr;
    } while ( retr != -1 );

    System.out.println("Total resource: " + total);

}

and I get:

Total file: 9132744
Total resource: 16399085

Very strange !!

EDIT 3

I have just noticed the size of the resource in the .jar is 16399085 (uncompressed) instead of 9132744. It seems that the issue is in the compilation process.

May be I have an issue with the following in my pom.xml:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-resources-plugin</artifactId>
            <version>2.5</version>
            <configuration>
                <encoding>${project.build.sourceEncoding}</encoding> // UTF8
            </configuration>
        </plugin>

Answer 1

I compiled your code and can't reproduce your situation. My code snippet follows.

Try to assert that you get proper data thru getResourceAsStream - for example dump it to a file without un-gzip and reload it using the File approach. Might be you have the /net/cv/convert... visible twice in your classpath and the runtime gets the wrong one?

java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03-384-10M3425)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02-384, mixed mode)

javac A.java 
java -cp mega.jar:. A

import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.IllegalStateException;
import java.util.zip.GZIPInputStream;

public class A {
public static void main(String[] args) throws IOException {
    final byte[] dummy = new byte[10];

    // Reading data from file
    final File f = new File("/tmp/mine.data");
    DataInputStream is = new DataInputStream(new GZIPInputStream(new FileInputStream(f)));

    int count = 0;
    while (is.read(dummy) != -1) count++;
    System.out.println("count = " + count);

    // Reading data from resource
    InputStream ins = A.class.getResourceAsStream("/do/do/mine.data");
    if (ins == null)
        throw new IllegalStateException("Failed to locate data.");

    is = new DataInputStream(new GZIPInputStream(ins)); // Issue happens here
    count = 0;
    while (is.read(dummy) != -1) count++;

    System.out.println("count = " + count);
}
}

Answer 2

I am pretty sure that line InputStream ins = CompareTest2.class.getResourceAsStream("/net/cv/convert1900.data"); returns null. Check this. If so check your pass /net/cv/convert1900.data, check your jar, verify the classpath when running from maven: probably the resource is not there.

BTW it is possible! All resources in maven must be under directory resources. Is it correct for your project? If for example resource files are under main/java they will not be copied to target directory.

Answer 3

My issue lied in the filtering of my resources. A solution is available here.