Purge old release from Nexus 3

问题:

I use Nexus 3 and I want to delete my old releases. In Nexus 2 there is a scheduled task called Remove Releases From Repository. It seems that this tasks is missing in Nexus 3.

How can we delete old release from Nexus 3 ?

Thanks

回答1:

Since the release of nexus 3 you can deploy groovy scripts to the nexus manager. Simply create a new execute script task and use the following script:

import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.common.app.GlobalComponentLookupHelper
import org.sonatype.nexus.repository.maintenance.MaintenanceService
import org.sonatype.nexus.repository.storage.ComponentMaintenance
import org.sonatype.nexus.repository.storage.Query;
import org.sonatype.nexus.script.plugin.RepositoryApi
import org.sonatype.nexus.script.plugin.internal.provisioning.RepositoryApiImpl
import com.google.common.collect.ImmutableList
import org.joda.time.DateTime;
import org.slf4j.Logger

// ----------------------------------------------------
// delete these rows when this script is added to nexus
RepositoryApiImpl repository = null;
Logger log = null;
GlobalComponentLookupHelper container = null;
// ----------------------------------------------------

def retentionDays = 30;
def retentionCount = 10;
def repositoryName = 'maven-releases';
def whitelist = ["org.javaee7.sample/javaee7-simple-sample", "org.javaee7.next/javaee7-another-sample"].toArray();


log.info(":::Cleanup script started!");
MaintenanceService service = container.lookup("org.sonatype.nexus.repository.maintenance.MaintenanceService");
def repo = repository.repositoryManager.get(repositoryName);
def tx = repo.facet(StorageFacet.class).txSupplier().get();
def components = null;
try {
    tx.begin();
    components = tx.browseComponents(tx.findBucket(repo));
}catch(Exception e){
    log.info("Error: "+e);
}finally{
    if(tx!=null)
        tx.close();
}

if(components != null) {
    def retentionDate = DateTime.now().minusDays(retentionDays).dayOfMonth().roundFloorCopy();
    int deletedComponentCount = 0;
    int compCount = 0;
    def listOfComponents = ImmutableList.copyOf(components);
    def previousComp = listOfComponents.head().group() + listOfComponents.head().name();
    listOfComponents.reverseEach{comp ->
        log.info("Processing Component - group: ${comp.group()}, ${comp.name()}, version: ${comp.version()}");
        if(!whitelist.contains(comp.group()+"/"+comp.name())){
            log.info("previous: ${previousComp}");
            if(previousComp.equals(comp.group() + comp.name())) {
                compCount++;
                log.info("ComCount: ${compCount}, ReteCount: ${retentionCount}");
                if (compCount > retentionCount) {
                    log.info("CompDate: ${comp.lastUpdated()} RetDate: ${retentionDate}");
                    if(comp.lastUpdated().isBefore(retentionDate)) {
                        log.info("compDate after retentionDate: ${comp.lastUpdated()} isAfter ${retentionDate}");
                        log.info("deleting ${comp.group()}, ${comp.name()}, version: ${comp.version()}");

                        // ------------------------------------------------
                        // uncomment to delete components and their assets
                        // service.deleteComponent(repo, comp);
                        // ------------------------------------------------

                        log.info("component deleted");
                        deletedComponentCount++;
                    }
                }
            } else {
                compCount = 1;
                previousComp = comp.group() + comp.name();
            }
        }else{
            log.info("Component skipped: ${comp.group()} ${comp.name()}");
        }
    }

    log.info("Deleted Component count: ${deletedComponentCount}");
}

https://github.com/xninjaxelitex/nexus3-cleanup-release-artifact

This script will clean your nexus repository by the specified parameters at the top.

回答2:

See this script as a basis to work with:

https://gist.github.com/emexelem/bcf6b504d81ea9019ad4ab2369006e66 by emexelem

import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.repository.storage.Query;
import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormat;

def fmt = DateTimeFormat.forPattern('yyyy-MM-dd HH:mm:ss');
// Get a repository
def repo = repository.repositoryManager.get('nuget-releases');
// Get a database transaction
def tx = repo.facet(StorageFacet).txSupplier().get();
// Begin the transaction
tx.begin();
// Search assets that havn't been downloaded for more than three months
tx.findAssets(Query.builder().where('last_downloaded <').param(DateTime.now().minusMonths(3).toString(fmt)).build(), [repo]).each { asset ->
    def component = tx.findComponent(asset.componentId());
    // Check if there is newer components of the same name
    def count = tx.countComponents(Query.builder().where('name').eq(component.name()).and('version >').param(component.version()).build(), [repo]);
    if (count > 0) {
        log.info("Delete asset ${asset.name()} as it has not been downloaded since 3 months and has a newer version")
        tx.deleteAsset(asset);
        tx.deleteComponent(component);
    }
}
// End the transaction
tx.commit();

回答3:

We do not have this scheduled task built yet, in the meantime you can use the Delete Component functionality in the UI if you need to manually remove a release.

回答4:

So, I hit a problem with running out of diskspace as our releases/builds were stacking up, so I had a bit of a dig around at creating a script to remove old builds, and got to this:

import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.repository.storage.Query;

def repositoryName = 'Integration';
def maxArtifactCount = 20;

// Get a repository
def repo = repository.repositoryManager.get(repositoryName);
// Get a database transaction
def tx = repo.facet(StorageFacet).txSupplier().get();
try {
    // Begin the transaction
    tx.begin();

    def previousComponent = null;
    def uniqueComponents = [];
    tx.findComponents(Query.builder().suffix(' ORDER BY group, name').build(), [repo]).each{component -> 
        if (previousComponent == null || (!component.group().equals(previousComponent.group()) || !component.name().equals(previousComponent.name()))) {
            uniqueComponents.add(component);
        }
        previousComponent = component;
    }

    uniqueComponents.each {uniqueComponent ->
        def componentVersions = tx.findComponents(Query.builder().where('group = ').param(uniqueComponent.group()).and('name = ').param(uniqueComponent.name()).suffix(' ORDER BY last_updated DESC').build(), [repo]);
        log.info(uniqueComponent.group() + ", " + uniqueComponent.name() + " size " + componentVersions.size());
        if (componentVersions.size() > maxArtifactCount) {
            componentVersions.eachWithIndex { component, index ->
                if (index > maxArtifactCount) {
                    log.info("Deleting Component ${component.group()} ${component.name()} ${component.version()}")
                    tx.deleteComponent(component);
                }
            }
        }
    }
} finally {
    // End the transaction
    tx.commit();
}

This works through a repository, looking for all the components. It then works through all the versions (ordered by last updated - I couldn't figure out ordering by version number, but I think this should be ok), and then removes any over the maxArtifactCount number.

Hopefully this may be of use - and if you see any issues let me know.

回答5:

Although this post is quite old I had a similar requirement but wanted to remove old artifacts/releases based on the versions only, so needed to find a way to sort them and keep only the latest.

This is the script I came up with that is based on some of those examples already given here.

Note this was specifically for cleaning YUM repositories but should work for other types with little modification.

import org.sonatype.nexus.repository.storage.StorageFacet;
import org.sonatype.nexus.repository.storage.Query;
import com.google.common.collect.ImmutableList
import org.joda.time.format.DateTimeFormat;
import org.joda.time.DateTime;
import org.slf4j.Logger

///////////////////////////////
def retentionCount = 3;
def repositoryName = 'repo-name';
def whitelist = [].toArray();
///////////////////////////////

log.info("==================================================");
log.info(":::Cleanup script started...");
log.info("==================================================");
log.info(":::Operating on Repository: ${repositoryName}");
log.info("==================================================");

def repo = repository.repositoryManager.get(repositoryName);
def tx = repo.facet(StorageFacet.class).txSupplier().get();
def components = null;

try {
    // Begin the transaction
    tx.begin();
    components = tx.browseComponents(tx.findBucket(repo));

    if(components != null) {
        int compCount = 0;
        int delCompCount = 0;
        int totalDelCompCount = 0;
        def listOfComponents = ImmutableList.copyOf(components);
        def previousComponent = null;
        def uniqueComponents = [];

        ////////////////////////////////////////////////////////
        final Long MAX_NUMBER = 10000L;

        listOfComponents.each { component ->
            if(!whitelist.contains(component.name())) {
                if (previousComponent == null || !component.name().equals(previousComponent.name())) {
                uniqueComponents.add(component);
            }
            previousComponent = component;
        }
    }

    uniqueComponents.each { uniqueComponent ->

        log.info("Starting Processing on Component: ${uniqueComponent.name()}");

        def artifactVersions = [];
        def numberArray = new ArrayList<Long>();

        tx.findComponents(Query.builder().where('name = ').param(uniqueComponent.name()).build(), [repo]).each { component ->
            def artifactVersion = component.version().replaceAll('-', '.');
            artifactVersions.add(artifactVersion);
        }

        log.info("Found Versions: ${artifactVersions}");

        for(ver in artifactVersions) {

            String[] vers = ver.split('\\.');

            Long num = 0;
            for (int i = 0; i < vers.length; ++i) {
                num = num + Long.valueOf(vers[i]) * (long) Math.pow(MAX_NUMBER, vers.length - i - 1);
            }
            numberArray.add(num);
        }

        numberArray = numberArray.sort(); //.reverse();

        //log.info("Sorting the Versions: ${numberArray}");

        def sortedArtifactVersions = [];

        for (num in numberArray) {
            List<Long> parts = new ArrayList<>();

            while (num > 0) {
                parts.add((long) (num % MAX_NUMBER));
                num = Math.floorDiv(num, MAX_NUMBER);
            }
            String artifact = parts.reverse().join('.');
            sortedArtifactVersions.add(artifact);
        }

        log.info("Sorted Versions: ${sortedArtifactVersions}");

        compCount = sortedArtifactVersions.size();

        def toRemoveArtifactVersions = [];

        if (compCount > retentionCount) {
            toRemoveArtifactVersions = sortedArtifactVersions.dropRight(retentionCount);
        }

        delCompCount = toRemoveArtifactVersions.size();

        totalDelCompCount = totalDelCompCount + delCompCount;

        log.info("Remove Versions: ${toRemoveArtifactVersions}");

        log.info("Component Total Count: ${compCount}");

        log.info("Component Remove Count: ${delCompCount}");

        for (ver in toRemoveArtifactVersions) {

            StringBuilder b = new StringBuilder(ver);
            b.replace(ver.lastIndexOf("."), ver.lastIndexOf(".") + 1, "-" );
            ver = b.toString();

            tx.findComponents(Query.builder().where('name = ').param(uniqueComponent.name()).and('version = ').param(ver).build(), [repo]).each { component ->

                log.info("Deleting Component: ${uniqueComponent.name()} ${ver}");

                // ------------------------------------------------
                // uncomment to delete components and their assets
                // tx.deleteComponent(component);
                // ------------------------------------------------
            }
        }
        log.info("==================================================");
    }
    log.info(" *** Total Deleted Component Count: ${totalDelCompCount} *** ");
    log.info("==================================================");
}

// End the transaction
tx.commit();

} catch(Exception e) {
    log.info("Error: "+e);
    tx.rollback();
} finally {
    tx.close();
}