Why is Gradle excluding required dependencies?

2019-06-12 17:06发布

问题:

Environment: Linux, Artifactory, Java web apps, Gradle 2.12 with Spring dependency management plug-in and Flyway plug-in.

We are NOT using the Artifactory plug-in. We are just hitting it as if it were a generic maven repository.

Project is massive with something between 80 and 90 sub-projects. We have had several people working on migrating it over from Maven but there have been many headaches. We've beaten most of them but this particular series of knots is very close to sinking us.

Problems:

  1. The build is not faster than Maven using --daemon and not --parallel. This seems pretty weird. Profiling shows that dependency resolution is taking a very long time.
  2. Some aspect of the build is not stable. This is to say that we get various compiler issues when we try to build. This happens regardless of whether we use --parallel or not, although it happens to a greater degree with --parallel. It also happens regardless of local or remote dependencies. We suspect that it is related to some aspect of dependency resolution or perhaps as a result of unexpected exclusion behavior on the part of Gradle. Specifically, we get "Class def not found" and / or "missing symbol". Usually this seems to be related to a transitive dependency. If I put it in explicitly, the error goes away and I get a new one pertaining to a different transitive. I've experimented with putting them in as both transitive and as transitive = false.
  3. Complicating matters, we have many BOMs. Perhaps 10? Some are ours and others belong to 3rd parties. Two or three are more than a page in length.

Without --parallel we can't seem to get the performance we need to justify the migration but we could, perhaps, survive that and wait for --parallel to finish incubating IF we could get it to build reliably. We've been cycling on --debug for about 2 weeks now but our shop is, naturally, inexperienced with Groovy and Gradle.

Clarification: It works about 30% of the time with --parallel and about 90% of the time without. If you build the specific project in question, using the full project path or changing to that sub-directory, there is no problem building.

From the --debug output, for example, I see:

...
-classpath 
/development/.../build/classes/main
/development/.../build/resources/main
/home/someUser/.gradle/caches/modules-2/files-2.1/org.springframework/spring-context/4.1.7.RELEASE/8c6c02bcccfa23a74db59f7b7725e69e1af38f04/spring-context-4.1.7.RELEASE.jar
/development/.../build/libs/data-6.7.0-SNAPSHOT.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.hibernate/hibernate-core/4.2.18.Final/2b4c72104fe1910d368d766243f0d0ac2608e134/hibernate-core-4.2.18.Final.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/joda-time/joda-time/2.8.1/f5bfc718c95a7b1d3c371bb02a188a4df18361a9/joda-time-2.8.1.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/commons-lang/commons-lang/2.6/ce1edb914c94ebc388f086c6827e8bdeec71ac2/commons-lang-2.6.jar
/development/.../build/libs/pnq-orgmanagement-6.7.0-SNAPSHOT.jar
/development/.../build/libs/pnq-stuffmanagement-6.7.0-SNAPSHOT.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/com.pnq/pnq-someutils/6.7.0-SNAPSHOT/642a90ae7cc4f3f9b281184cacbfa1294f176df8/pnq-someutils-6.7.0-SNAPSHOT.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/com.eaio.stringsearch/stringsearch/2/44618501b2fb6b2d1bb851d00b264a6eaf1fc0b0/stringsearch-2.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.bouncycastle/bcprov-jdk15on/1.52/88a941faf9819d371e3174b5ed56a3f3f7d73269/bcprov-jdk15on-1.52.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/commons-collections/commons-collections/3.2.1/761ea405b9b37ced573d2df0d1e3a4e0f9edc668/commons-collections-3.2.1.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.10/4b95f4897fa13f2cd904aee711aeafc0c5295cd8/commons-codec-1.10.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.springframework/spring-beans/4.1.7.RELEASE/e52148e9671e2918a2172c9cf56b77bede2042ce/spring-beans-4.1.7.RELEASE.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/commons-validator/commons-validator/1.4.1/2231238e391057a53f92bde5bbc588622c1956c3/commons-validator-1.4.1.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.springframework/spring-orm/4.1.7.RELEASE/d8923bdffe631904899875e719d8b363bc3deaea/spring-orm-4.1.7.RELEASE.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.javolution/javolution-core-java/6.2.0rc3/5bae18008a0182d80155dbbe2c91d8907d2496d/javolution-core-java-6.2.0rc3.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.springframework/spring-tx/4.1.7.RELEASE/3465a5d3b24006d51482cf45abd13e347f64582a/spring-tx-4.1.7.RELEASE.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.6.1/6f3b8a24bf970f17289b234284c94f43eb42f0e4/slf4j-api-1.6.1.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/com.pnq/pnq-batchutils/6.7.0.2/9c025552239645478a759267748c73f3e1e43a47/pnq-batchutils-6.7.0.2.jar
/development/.../build/libs/pnq-mailutils-6.7.0-SNAPSHOT.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/com.pnq/pnq-utilities/6.7.0.2/d1e8ad707ebd2eddbb29eb0052c05b34acd98b46/pnq-utilities-6.7.0.2.jar
/development/.../build/libs/pnq-database-6.7.0-SNAPSHOT.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/com.twilio.sdk/twilio-java-sdk/3.3.9/133f7bde99aefae1df78ecd9787e5e60bc521ad2/twilio-java-sdk-3.3.9.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.apache.velocity/velocity/1.7/2ceb567b8f3f21118ecdec129fe1271dbc09aa7a/velocity-1.7.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/javax.mail/mail/1.4.1/8b7bc69010655425dabf091b51d1e90b4de36715/mail-1.4.1.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5/a1e6cbb3cc2c5f210dd1310ff9fcb2c09c0d1438/httpclient-4.5.jar
/development/.../build/libs/pnq-infoassembler-6.7.0-SNAPSHOT.jar
/development/.../build/libs/pnq-stufflookup-6.7.0-SNAPSHOT.jar
/home/someUser/.gradle/caches/modules-2/files-2.1/com.pnq/pnq-rcrtrlss/6.7.0-SNAPSHOT/58d6caa20da9035ce7b311f54915448c5bb903dc/pnq-wierdorules-6.7.0-SNAPSHOT.jar
/development/.../build/libs/pnq-cmanagement-6.7.0-SNAPSHOT.jar
/development/.../build/libs/pnq-translation-6.7.0-SNAPSHOT.jar
/development/.../build/libs/pnq-eutilities-6.7.0-SNAPSHOT.jar
/development/.../build/libs/pnq-control-6.7.0-SNAPSHOT.jar
/development/../database/build/libs/test-pnq-database-6.7.0-SNAPSHOT.jar 
...
    14:50:06.439 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.graph.DependencyGraphBuilder] junit:junit is excluded from com.project:project-enterprise-test:6.7.0.2(default).
    ...
        st.java:3: error: package org.junit does not exist
        14:50:06.481 [ERROR] [system.err] import static org.junit.Assert.assertEquals;
        14:50:06.481 [ERROR] [system.err]                        ^

Note: A global search shows there are NO excludes for junit anywhere in the Gradle build. Also, what DON'T I see in the -classpath? Junit. ...but in the relevant build.gradle file I see:

testCompile ("com.project:project-enterprise-test")

The above is a BOM containing Maven dependencies:

<dependencies>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-test</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context</artifactId>
    </dependency>
    <dependency>
      <groupId>com.oracle</groupId>
      <artifactId>ojdbc7</artifactId>
    </dependency>
    <dependency>
      <groupId>org.dbunit</groupId>
      <artifactId>dbunit</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-pool2</artifactId>
    </dependency>
    <dependency>
      <groupId>org.easymock</groupId>
      <artifactId>easymock</artifactId>
      <exclusions>
        <exclusion>
          <groupId>cglib</groupId>
          <artifactId>cglib-nodep</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.mockito</groupId>
      <artifactId>mockito-core</artifactId>
      <exclusions>
        <exclusion>
          <groupId>org.hamcrest</groupId>
          <artifactId>hamcrest-core</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.hamcrest</groupId>
      <artifactId>hamcrest-integration</artifactId>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>jul-to-slf4j</artifactId>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>jcl-over-slf4j</artifactId>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-log4j12</artifactId>
    </dependency>
    <dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
    </dependency>
    <dependency>
      <groupId>joda-time</groupId>
      <artifactId>joda-time</artifactId>
    </dependency>
    <dependency>
      <groupId>org.hamcrest</groupId>
      <artifactId>hamcrest-all</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-dbcp2</artifactId>
    </dependency>
    <dependency>
      <groupId>org.fitnesse</groupId>
      <artifactId>fitlibrary</artifactId>
    </dependency>
    <dependency>
      <groupId>javax.transaction</groupId>
      <artifactId>jta</artifactId>
    </dependency>
    <dependency>
      <groupId>javax.ws.rs</groupId>
      <artifactId>jsr311-api</artifactId>
    </dependency>
    <dependency>
      <groupId>com.sun.jersey</groupId>
      <artifactId>jersey-bundle</artifactId>
    </dependency>
    <dependency>
      <groupId>net.sf.dbfit</groupId>
      <artifactId>dbfit</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-jexl</artifactId>
    </dependency>
    <dependency>
      <groupId>commons-beanutils</groupId>
      <artifactId>commons-beanutils</artifactId>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>com.springsource.org.apache.commons.io</artifactId>
    </dependency>
    <dependency>
      <groupId>javax.jms</groupId>
      <artifactId>jms</artifactId>
    </dependency>
    <dependency>
      <groupId>commons-logging</groupId>
      <artifactId>commons-logging</artifactId>
    </dependency>
    <dependency>
      <groupId>net.servicefixture</groupId>
      <artifactId>servicefixture</artifactId>
    </dependency>
  </dependencies>

In the root build.gradle:

dependencyManagement {
    ...
    imports {
      mavenBom "com.project:project-enterprise-test:$bomversion"
    }
    ...
  }

回答1:

It's really hard to help you if you don't include

  • related build.gradle snippets
  • the actual errors

For the dependency version mismatches, I suggest that you compare mvn dependency:tree against gradle dependencyInsight to see what's different.

Maven and Gradle have different ways of dealing with multiple versions of the same dependency within the dependency hierarchy.

  • Maven uses a "nearest definition wins" strategy.
  • By default, Gradle will choose the latest version (or you can also define a custom ResolutionStrateegy)