Duke Fast Deduplication: java.lang.UnsupportedOper

I'm trying to use the Duke Fast Deduplication Engine to search for some duplicate records in the database at the company where I work.

I run it from the command line like this:

java -cp "C:\utils\duke-0.6\duke-0.6.jar;C:\utils\duke-0.6\lucene-core-3.6.1.jar" no.priv.garshol.duke.Duke --showmatches --verbose .\config.xml

But I get an error:

Exception in thread "main" java.lang.UnsupportedOperationException: Operation no
t yet supported
        at sun.jdbc.odbc.JdbcOdbcResultSet.isClosed(Unknown Source)
        at no.priv.garshol.duke.datasources.JDBCDataSource$JDBCIterator.close(JD
BCDataSource.java:115)
        at no.priv.garshol.duke.Processor.deduplicate(Processor.java:152)
        at no.priv.garshol.duke.Duke.main_(Duke.java:135)
        at no.priv.garshol.duke.Duke.main(Duke.java:38)

My configuration file looks like this:

<duke>
   <schema>
         <threshold>0.82</threshold>
         <maybe-threshold>0.80</maybe-threshold>
         <path>test</path>

         <property type="id">
              <name>ID</name>
         </property>

         <property>
             <name>LNAME</name>
             <comparator>no.priv.garshol.duke.comparators.ExactComparator</comparator>
             <low>0.6</low>
             <high>0.8</high>
         </property>

         <property>
             <name>FNAME</name>
             <comparator>no.priv.garshol.duke.comparators.ExactComparator</comparator>
             <low>0.6</low>
             <high>0.8</high>
         </property>

         <property>
             <name>MNAME</name>
             <comparator>no.priv.garshol.duke.comparators.ExactComparator</comparator>
             <low>0.3</low>
             <high>0.5</high>
         </property>

         <property>
             <name>SSN</name>
             <comparator>no.priv.garshol.duke.comparators.ExactComparator</comparator>
             <low>0.0</low>
             <high>1.0</high>
         </property>

   </schema>
   <jdbc>
          <param name="driver-class" value="sun.jdbc.odbc.JdbcOdbcDriver" />
          <param name="connection-string" value="jdbc:odbc:VT_DeDupe" />
          <param name="user-name" value="aleer" />
          <param name="password" value="**" />
          <param name="query" value="select SocialSecurityNumber, LastName, FirstName, MiddleName, empssn from T_Employees" />

          <column name="SocialSecurityNumber" property="ID" />
          <column name="LastName" property="LNAME" />
          <column name="FirstName" property="FNAME" />
          <column name="MiddleName" property="MNAME" />
          <column name="empssn" property="SSN" />
   </jdbc>
</duke>

It doesn't really tell me what is unsupported...I'm just trying it out, nothing serious with the configuration yet.

回答1:

As mbonaci says, the problem is that the JDBC driver's isClosed() method is not implemented. This even though implementing it would be no harder than simply writing "return closed".

I added an ugly workaround for this issue now. Please do an "hg pull" and try again.

回答2:

Which Java version are you using?
sun.jdbc.odbc.JdbcOdbcResultSet.isClosed first appeared in Java 1.6. and it still looks like this in v1.7 (I haven't checked in Java 8):

public boolean isClosed() throws SQLException {
    throw new UnsupportedOperationException("Operation not yet supported");
}

So simply don't call that method. Use some other way of checking whether resultset is closed.

Or if you cannot change the code ask the project's authors for help (I see there was an effort to solve exception when closing RS).