I add a dependency(let's name it as A) to ivy.xml which has a pom file in maven central. Ivy uses ibiblio for resolving the maven dependencies. The dependency(A) which is added to ivy.xml has a transitive dependency(B). So far so good till here. The dependency(C) of transitive dependency(B) can not be resolved by ivy.

I defined A in ivy.xml like this:

<dependency org="Z" name="A" rev="0.6-SNAPSHOT" conf="*->default"/>

In pom file of B, C is defined both in compile and test scopes like below:

<dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
    </dependency>
    <dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
      <type>test-jar</type>
      <scope>test</scope>
</dependency>

When I look the xml file of B which is resolved by ivy in ivy's cache file(~/.ivy2/cache/X/C/ivy-0.98.8-hadoop2.xml), it looks like this:

<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)"/>
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)">
  <artifact name="C" type="test-jar" ext="jar" conf="" m:classifier="tests"/>
</dependency>

For this reason, ivy can not define C scopes correctly. For the record, I don't have permissions to modify the pom files as they are third party projects. How can I fix it ?

I reviewed the ivy usage of the nutch project and apologies but my conclusion is that it's overly complex for the following reasons:

"compile" and "test" targets are issuing separate calls to the resolve task
Each plugin is also calling an ivy resolve task
Complex logic for maintaining classpaths. Could be simplified using the cachepath task and ivy configurations.
Build plugins are not managed by ivy (Sonar, eclipse, rat)

I started to refactor the build, but had to stop when I realised that I didn't understand the relationship between the main nutch artifact and the plugins... (I discovered NUTCH-1515 the hard way... big time-waster The feed plugin has missing dependencies).

I also noticed issue NUTCH-1371 calling for the removal of ivy. This would be a tricky refactoring without significant change to the current codebase. I suspect it would have to be a multi-module build with each plugin listing its own dependencies.

In conclusion, this work does not answer your question, but thought I needed to at least document the result of a few hours analysis :-) In light of NUTCH-1371 I don't know if your project will tolerant major ivy refactoring?

Refactoring ivy

Here follows what I achieved so far:

Private "development" fork of the nutch project
Diff with trunk

Benefits:

Single ivy report showing all configurations (New ivy-resolve target)
New mechanism for installing ivy (New ivy-install target)
Classpaths are managed using ivy configurations (See use of ivy cachepath task and configurations in ivy file)
Eclipse, sonar and rat ANT tasks automatically installed using ivy (The Eclipse plugin is noteworthy as it uses a packager resolver to download and extract jar from a tar archive).

Impacts the following Nutch issues

NUTCH-1881 : This new approach removes resolve-test and resolve-default targets and manages the classpaths using ivy instead of the ${build.lib.dir}
NUTCH-1805 : Can easily setup a separate configuration for the job target with it's own dependencies.
NUTCH-1755 : I think this one is fixed by assigning a name to the the build.xml (see: diff)