Multiple deployments in maven

Question

We have a internal artifactory repository. At the moment all snapshots will be deployed there. We also want to have a different server with a web interface, and want to copy the to it the created artifacts.

For our builds we use Hudson, but the post-build action "Deploy artifacts to Maven repository" together with scp doesn't work. So there is the question of doing it in some other elegant way. Why isn't maven able to have several distribution repositories? Any ideas?

The nicest thing would be if artifactory would support an (automatic!) incremental export to a standard maven repository after each new deployment.

Answer 1

Artifactory does have an automatic export feature. From the documentation:

You can automatically and periodically back up the whole Artifactory system. The backup process creates a timestamped directory (or zip file) in the target backup dir, and is basically identical to running full system export with metadata. [...] Each backup can have its own schedule and excluded certain repositories [...]

The content of the backup (when extracted) is in standard Maven format and can be loaded into any external Maven repository [...]

Artifactory supports backing up incrementally to the same target directory (named "current") in the target backup dir. This kind of backup is only writing deltas to the output dir, resulting in extremely fast backups.

Isn't that exactly what you need? To transfer the files, you can either mount a shared directory to the remote server and do the backup there, or do the backup locally and then rsync it.

Answer 2

If you are willing to use a custom plugin, you can configure Maven to deploy to a list of "mirror" locations at the same time as the standard deployment. I'd recommend defining this in a profile so you can control what deployments are mirrored (it might not be appropriate to do this on every build).

To define a new plugin you need to create a new Maven project and specify the POM has packaging maven-plugin:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>name.seller.rich</groupId>
  <artifactId>maven-mirror-plugin</artifactId>
  <packaging>maven-plugin</packaging>
  <version>0.0.1</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven</groupId>
      <artifactId>maven-plugin-api</artifactId>
      <version>2.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-deploy-plugin</artifactId>
      <version>2.4</version>
    </dependency>
  </dependencies>
</project>

In src/main/java define a Mojo. The code below declares a "mirror" goal, it takes a list of mirrorRepository items (containing a repositoryId and url) to mirror the artifact deployment to. The plugin uses the same approach to deployment as the maven-deploy-plugin, and takes most of the same parameters.

Note that you still need to define a server in your settings.xml for each repository with appropriate permissions to do the deployment or the build will fail!

package name.seller.rich;

import java.io.File;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

import org.apache.maven.artifact.Artifact;
import org.apache.maven.artifact.deployer.ArtifactDeployer;
import org.apache.maven.artifact.deployer.ArtifactDeploymentException;
import org.apache.maven.artifact.metadata.ArtifactMetadata;
import org.apache.maven.artifact.repository.ArtifactRepository;
import org.apache.maven.artifact.repository.ArtifactRepositoryFactory;
import org.apache.maven.artifact.repository.layout.ArtifactRepositoryLayout;
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugin.MojoExecutionException;
import org.apache.maven.plugin.MojoFailureException;
import org.apache.maven.project.MavenProject;
import org.apache.maven.project.artifact.ProjectArtifactMetadata;

/**
 * @goal mirror
 * @phase deploy
 */
public class MirrorMojo extends AbstractMojo {
    /**
     * @parameter expression=
     *            "${component.org.apache.maven.artifact.deployer.ArtifactDeployer}"
     * @required
     * @readonly
     */
    private ArtifactDeployer deployer;

    /**
     * Map that contains the layouts
     * 
     * @component role=
     *            "org.apache.maven.artifact.repository.layout.ArtifactRepositoryLayout"
     */
    private Map repositoryLayouts;

    /**
     * Component used to create a repository
     * 
     * @component
     */
    private ArtifactRepositoryFactory repositoryFactory;

    /**
     * The type of remote repository layout to deploy to. Try <i>legacy</i> for
     * a Maven 1.x-style repository layout.
     * 
     * @parameter expression="${repositoryLayout}" default-value="default"
     * @required
     */
    private String repositoryLayout;

    /**
     * Parameter used to update the metadata to make the artifact as release.
     * 
     * @parameter expression="${updateReleaseInfo}" default-value="false"
     */
    private boolean updateReleaseInfo;

    /**
     * Whether to deploy snapshots with a unique version or not.
     * 
     * @parameter expression="${uniqueVersion}" default-value="true"
     */
    private boolean uniqueVersion;

    /**
     * @parameter expression="${mirrorRepositories}"
     * @required
     */
    private MirrorRepository[] mirrorRepositories;

    /**
     * @parameter expression="${localRepository}"
     * @required
     * @readonly
     */
    private ArtifactRepository localRepository;

    /**
     * @parameter expression="${project}"
     * @required
     * @readonly
     */
    private MavenProject project;

    /**
     * Deploy all artifacts for the project to each mirror repository.
     */
    public void execute() throws MojoExecutionException, MojoFailureException {
        ArtifactRepositoryLayout layout;

        layout = (ArtifactRepositoryLayout) repositoryLayouts
                .get(repositoryLayout);

        for (int i = 0; i < mirrorRepositories.length; i++) {
            MirrorRepository mirrorRepository = mirrorRepositories[i];

            ArtifactRepository deploymentRepository = repositoryFactory
                    .createDeploymentArtifactRepository(mirrorRepository
                            .getRepositoryId(), mirrorRepository.getUrl(),
                            layout, uniqueVersion);

            String protocol = deploymentRepository.getProtocol();

            if ("".equals(protocol) || protocol == null) {
                throw new MojoExecutionException("No transfer protocol found.");
            }

            deployToRepository(deploymentRepository);
        }

    }

    /**
     * Deploy all artifacts to the passed repository.
     */
    private void deployToRepository(ArtifactRepository repo)
            throws MojoExecutionException {
        String protocol = repo.getProtocol();

        if (protocol.equalsIgnoreCase("scp")) {
            File sshFile = new File(System.getProperty("user.home"), ".ssh");

            if (!sshFile.exists()) {
                sshFile.mkdirs();
            }
        }

        File pomFile = project.getFile();
        Artifact artifact = project.getArtifact();
        // Deploy the POM
        boolean isPomArtifact = "pom".equals(project.getPackaging());
        if (!isPomArtifact) {
            ArtifactMetadata metadata = new ProjectArtifactMetadata(artifact,
                    pomFile);
            artifact.addMetadata(metadata);
        }

        if (updateReleaseInfo) {
            artifact.setRelease(true);
        }

        try {
            List attachedArtifacts = project.getAttachedArtifacts();

            if (isPomArtifact) {
                deployer.deploy(pomFile, artifact, repo, localRepository);
            } else {
                File file = artifact.getFile();

                if (file != null && !file.isDirectory()) {
                    deployer.deploy(file, artifact, repo, localRepository);
                } else if (!attachedArtifacts.isEmpty()) {
                    getLog()
                            .info(
                                    "No primary artifact to deploy, deploy attached artifacts instead.");
                } else {
                    String message = "The packaging for this project did not assign a file to the build artifact";
                    throw new MojoExecutionException(message);
                }
            }

            for (Iterator i = attachedArtifacts.iterator(); i.hasNext();) {
                Artifact attached = (Artifact) i.next();

                deployer.deploy(attached.getFile(), attached, repo,
                        localRepository);
            }
        } catch (ArtifactDeploymentException e) {
            throw new MojoExecutionException(e.getMessage(), e);
        }
    }
}

The mojo references a MirrorRepository type to encapsulate the repositoryId and url, it is a simple bean:

package name.seller.rich;

public class MirrorRepository {
    private String repositoryId;
    private String url;

    public String getRepositoryId() {
        return repositoryId;
    }

    public void setRepositoryId(String repositoryId) {
        this.repositoryId = repositoryId;
    }

    public String getUrl() {
        return url;
    }

    public void setUrl(String url) {
        this.url = url;
    }
}

Here's an example configuration using the plugin. Note all the deploy formats are supported (http, scp, ftp):

<plugin>
  <groupId>name.seller.rich</groupId>
  <artifactId>maven-mirror-plugin</artifactId>
  <executions>
    <execution>
      <id>mirror</id>
      <phase>deploy</phase>
      <goals>
        <goal>mirror</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <mirrorRepositories>
      <mirrorRepository>
        <repositoryId>mirror</repositoryId>
        <url>http://path/to/mirror</url>
      </mirrorRepository>
    </mirrorRepositories>
    <!--any other deploy configuration needed-->
  </configuration>
</plugin>

Answer 3

I think in Artifactory, by default, it maintains different logical repositories for uploading snapshots and non-snapshots. Using permissions, you can make the snapshot repository visible only to some.

If that is not sufficient, another solution that works with Artifactory 2.0 is to have Artifactory use a MySQL database that does asynchronous replication to another MySQL database, which in turn is being read by a separate installation of Artifactory. If that's too real time, you can simply have two different installations that do updates based on business rules.

Answer 4

I don't think maven supports deploying to multiple repositories for a single profile, but perhaps profiles could change the id and urls of the repository.

  <distributionManagement>
    <repository>
      <id>${repo-id}</id>
      <name>${repo-name}</name>
      <url>${repo-url}</url>
    </repository>
  </distributionManagement>

Maven Deployment

Then use profiles to pick which repo to deploy to:

<profiles>
  <profile>
    <id>repo1</id>
    <activation>
      <activeByDefault>true</activeByDefault>
    </activation>
    <properties>
      <repo-id>repo1</repo-id>
      <repo-name>Repo1 Name </repo-name>
      <repo-url>http://url.com/maven2</repo-url>
    </properties>
  </profile>
  <profile>
    <id>repo2</id>
    <properties>
      <repo-id>repo2</repo-id>
      <repo-name>Repo2 Name </repo-name>
      <repo-url>http://url2.com/maven2</repo-url>
    </properties>
  </profile>
</profiles>

Maven profiles