How to download the latest artifact from Artifacto

问题:

I need the latest artifact (for example, a snapshot) from a repository in Artifactory. This artifact needs to be copied to a server (Linux) via a script.

What are my options? Something like Wget / SCP? And how do I get the path of the artifact?

I found some solutions which require Artifactory Pro. But I just have Artifactory, not Artifactory Pro.

Is it possible at all to download from Artifactory without the UI and not having the Pro-Version? What is the experience?

I'm on OpenSUSE 12.1 (x86_64) if that matters.

回答1:

Artifactory has a good extensive REST-API and almost anything that can be done in the UI (perhaps even more) can also be done using simple HTTP requests.

The feature that you mention - retrieving the latest artifact, does indeed require the Pro edition; but it can also be achieved with a bit of work on your side and a few basic scripts.

Option 1 - Search:

Perform a GAVC search on a set of group ID and artifact ID coordinates to retrieve all existing versions of that set; then you can use any version string comparison algorithm to determine the latest version.

Option 2 - the Maven way:

Artifactory generates a standard XML metadata that is to be consumed by Maven, because Maven is faced with the same problem - determining the latest version; The metadata lists all available versions of an artifact and is generated for every artifact level folder; with a simple GET request and some XML parsing, you can discover the latest version.

回答2:

Something like the following bash script will retrieve the lastest com.company:artifact snapshot from the snapshot repo:

# Artifactory location
server=http://artifactory.company.com/artifactory
repo=snapshot

# Maven artifact location
name=artifact
artifact=com/company/$name
path=$server/$repo/$artifact
version=$(curl -s $path/maven-metadata.xml | grep latest | sed "s/.*<latest>\([^<]*\)<\/latest>.*/\1/")
build=$(curl -s $path/$version/maven-metadata.xml | grep '<value>' | head -1 | sed "s/.*<value>\([^<]*\)<\/value>.*/\1/")
jar=$name-$build.jar
url=$path/$version/$jar

# Download
echo $url
wget -q -N $url

It feels a bit dirty, yes, but it gets the job done.

回答3:

Using shell/unix tools

1. curl 'http://$artiserver/artifactory/api/storage/$repokey/$path/$version/?lastModified'

The above command responds with a JSON with two elements - "uri" and "lastModified"

2. Fetching the link in the uri returns another JSON which has the "downloadUri" of the artifact.

3. Fetch the link in the "downloadUri" and you have the latest artefact.

Using Jenkins Artifactory plugin

(Requires Pro) to resolve and download latest artifact, if Jenkins Artifactory plugin was used to publish to artifactory in another job:

1. Select Generic Artifactory Integration
2. Use Resolved Artifacts as ${repokey}:**/${component}*.jar;status=${STATUS}@${PUBLISH_BUILDJOB}#LATEST=>${targetDir}

回答4:

You can use the wget --user=USER --password=PASSWORD .. command, but before you can do that, you must allow artifactory to force authentication, which can be done by unchecking the "Hide Existence of Unauthorized Resources" box at Security/General tab in artifactory admin panel. Otherwise artifactory sends a 404 page and wget can not authenticate to artifactory.

回答5:

The role of Artifactory is to provide files for Maven (as well as other build tools such as Ivy, Gradle or sbt). You can just use Maven together with the maven-dependency-plugin to copy the artifacts out. Here's a pom outline to start you off...

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>A group id</groupId>
    <artifactId>An artifact id</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>pom</packaging>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.3</version>
                <executions>
                    <execution>
                        <id>copy</id>
                        <phase>package</phase>
                        <goals>
                            <goal>copy</goal>
                        </goals>
                        <configuration>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>The group id of your artifact</groupId>
                                    <artifactId>The artifact id</artifactId>
                                    <version>The snapshot version</version>
                                    <type>Whatever the type is, for example, JAR</type>
                                    <outputDirectory>Where you want the file to go</outputDirectory>
                                </artifactItem>
                            </artifactItems>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Just run mvn install to do the copy.

回答6:

You can use the REST-API's "Item last modified". From the docs, it retuns something like this:

GET /api/storage/libs-release-local/org/acme?lastModified
{
"uri": "http://localhost:8081/artifactory/api/storage/libs-release-local/org/acme/foo/1.0-SNAPSHOT/foo-1.0-SNAPSHOT.pom",
"lastModified": ISO8601
}

Example:

# Figure out the URL of the last item modified in a given folder/repo combination
url=$(curl \
    -H 'X-JFrog-Art-Api: XXXXXXXXXXXXXXXXXXXX' \
    'http://<artifactory-base-url>/api/storage/<repo>/<folder>?lastModified'  | jq -r '.uri')
# Figure out the name of the downloaded file
downloaded_filename=$(echo "${url}" | sed -e 's|[^/]*/||g')
# Download the file
curl -L -O "${url}"

回答7:

You could also use Artifactory Query Language to get the latest artifact.

The following shell script is just an example. It uses 'items.find()' (which is available in the non-Pro version), e.g. items.find({ "repo": {"$eq":"my-repo"}, "name": {"$match" : "my-file*"}}) that searches for files that have a repository name equal to "my-repo" and match all files that start with "my-file". Then it uses the shell JSON parser ./jq to extract the latest file by sorting by the date field 'updated'. Finally it uses wget to download the artifact.

#!/bin/bash

# Artifactory settings
host="127.0.0.1"
username="downloader"
password="my-artifactory-token"

# Use Artifactory Query Language to get the latest scraper script (https://www.jfrog.com/confluence/display/RTF/Artifactory+Query+Language)
resultAsJson=$(curl -u$username:"$password" -X POST  http://$host/artifactory/api/search/aql -H "content-type: text/plain" -d 'items.find({ "repo": {"$eq":"my-repo"}, "name": {"$match" : "my-file*"}})')

# Use ./jq to pars JSON
latestFile=$(echo $resultAsJson | jq -r '.results | sort_by(.updated) [-1].name')

# Download the latest scraper script
wget -N -P ./libs/ --user $username --password $password http://$host/artifactory/my-repo/$latestFile

回答8:

With recent versions of artifactory, you can query this through the api.

https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-RetrieveLatestArtifact

If you have a maven artifact with 2 snapshots

name => 'com.acme.derp'
version => 0.1.0
repo name => 'foo'
snapshot 1 => derp-0.1.0-20161121.183847-3.jar
snapshot 2 => derp-0.1.0-20161122.00000-0.jar

Then the full paths would be

https://artifactory.example.com/artifactory/foo/com/acme/derp/0.1.0-SNAPSHOT/derp-0.1.0-20161121.183847-3.jar

and

https://artifactory.example.com/artifactory/foo/com/acme/derp/0.1.0-SNAPSHOT/derp-0.1.0-20161122.00000-0.jar

You would fetch the latest like so:

curl https://artifactory.example.com/artifactory/foo/com/acme/derp/0.1.0-SNAPSHOT/derp-0.1.0-SNAPSHOT.jar

回答9:

This may be new:

https://artifactory.example.com/artifactory/repo/com/example/foo/1.0.[RELEASE]/foo-1.0.[RELEASE].tgz

For loading module foo from example.com . Keep the [RELEASE] parts verbatim. This is mentioned in the docs but it's not made abundantly clear that you can actually put [RELEASE] into the URL (as opposed to a substitution pattern for the developer).

回答10:

I use Nexus and this code works for me—can retrive both release and last snaphsot, depending on repository type:

server="http://example.com/nexus/content/repositories"
repo="snapshots"
name="com.exmple.server"
artifact="com/example/$name"
path=$server/$repo/$artifact
mvnMetadata=$(curl -s "$path/maven-metadata.xml")
echo "Metadata: $mvnMetadata"
jar=""
version=$( echo "$mvnMetadata" | xpath -e "//versioning/release/text()" 2> /dev/null)
if [[ $version = *[!\ ]* ]]; then
  jar=$name-$version.jar
else
  version=$(echo "$mvnMetadata" | xpath -e "//versioning/versions/version[last()]/text()")
  snapshotMetadata=$(curl -s "$path/$version/maven-metadata.xml")
  timestamp=$(echo "$snapshotMetadata" | xpath -e "//snapshot/timestamp/text()")
  buildNumber=$(echo "$snapshotMetadata" | xpath -e "//snapshot/buildNumber/text()")
  snapshotVersion=$(echo "$version" | sed 's/\(-SNAPSHOT\)*$//g')
  jar=$name-$snapshotVersion-$timestamp-$buildNumber.jar
fi
jarUrl=$path/$version/$jar
echo $jarUrl
mkdir -p /opt/server/
wget -O /opt/server/server.jar -q -N $jarUrl