Get a list of resources from classpath directory

问题:

I am looking for a way to get a list of all resource names from a given classpath directory, something like a method List<String> getResourceNames (String directoryName).

For example, given a classpath directory x/y/z containing files a.html, b.html, c.html and a subdirectory d, getResourceNames(\"x/y/z\") should return a List<String> containing the following strings:[\'a.html\', \'b.html\', \'c.html\', \'d\'].

It should work both for resources in filesystem and jars.

I know that I can write a quick snippet with Files, JarFiles and URLs, but I do not want to reinvent the wheel. My question is, given existing publicly available libraries, what is the quickest way to implement getResourceNames? Spring and Apache Commons stacks are both feasible.

回答1:

Custom Scanner

Implement your own scanner. For example:

private List<String> getResourceFiles(String path) throws IOException {
    List<String> filenames = new ArrayList<>();

    try (
            InputStream in = getResourceAsStream(path);
            BufferedReader br = new BufferedReader(new InputStreamReader(in))) {
        String resource;

        while ((resource = br.readLine()) != null) {
            filenames.add(resource);
        }
    }

    return filenames;
}

private InputStream getResourceAsStream(String resource) {
    final InputStream in
            = getContextClassLoader().getResourceAsStream(resource);

    return in == null ? getClass().getResourceAsStream(resource) : in;
}

private ClassLoader getContextClassLoader() {
    return Thread.currentThread().getContextClassLoader();
}

Spring Framework

Use PathMatchingResourcePatternResolver from Spring Framework.

Ronmamo Reflections

The other techniques might be slow at runtime for huge CLASSPATH values. A faster solution is to use ronmamo\'s Reflections API, which precompiles the search at compile time.

回答2:

Here is the code
Source: forums.devx.com/showthread.php?t=153784

import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Enumeration;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
import java.util.zip.ZipException;
import java.util.zip.ZipFile;

/**
 * list resources available from the classpath @ *
 */
public class ResourceList{

    /**
     * for all elements of java.class.path get a Collection of resources Pattern
     * pattern = Pattern.compile(\".*\"); gets all resources
     * 
     * @param pattern
     *            the pattern to match
     * @return the resources in the order they are found
     */
    public static Collection<String> getResources(
        final Pattern pattern){
        final ArrayList<String> retval = new ArrayList<String>();
        final String classPath = System.getProperty(\"java.class.path\", \".\");
        final String[] classPathElements = classPath.split(System.getProperty(\"path.separator\"));
        for(final String element : classPathElements){
            retval.addAll(getResources(element, pattern));
        }
        return retval;
    }

    private static Collection<String> getResources(
        final String element,
        final Pattern pattern){
        final ArrayList<String> retval = new ArrayList<String>();
        final File file = new File(element);
        if(file.isDirectory()){
            retval.addAll(getResourcesFromDirectory(file, pattern));
        } else{
            retval.addAll(getResourcesFromJarFile(file, pattern));
        }
        return retval;
    }

    private static Collection<String> getResourcesFromJarFile(
        final File file,
        final Pattern pattern){
        final ArrayList<String> retval = new ArrayList<String>();
        ZipFile zf;
        try{
            zf = new ZipFile(file);
        } catch(final ZipException e){
            throw new Error(e);
        } catch(final IOException e){
            throw new Error(e);
        }
        final Enumeration e = zf.entries();
        while(e.hasMoreElements()){
            final ZipEntry ze = (ZipEntry) e.nextElement();
            final String fileName = ze.getName();
            final boolean accept = pattern.matcher(fileName).matches();
            if(accept){
                retval.add(fileName);
            }
        }
        try{
            zf.close();
        } catch(final IOException e1){
            throw new Error(e1);
        }
        return retval;
    }

    private static Collection<String> getResourcesFromDirectory(
        final File directory,
        final Pattern pattern){
        final ArrayList<String> retval = new ArrayList<String>();
        final File[] fileList = directory.listFiles();
        for(final File file : fileList){
            if(file.isDirectory()){
                retval.addAll(getResourcesFromDirectory(file, pattern));
            } else{
                try{
                    final String fileName = file.getCanonicalPath();
                    final boolean accept = pattern.matcher(fileName).matches();
                    if(accept){
                        retval.add(fileName);
                    }
                } catch(final IOException e){
                    throw new Error(e);
                }
            }
        }
        return retval;
    }

    /**
     * list the resources that match args[0]
     * 
     * @param args
     *            args[0] is the pattern to match, or list all resources if
     *            there are no args
     */
    public static void main(final String[] args){
        Pattern pattern;
        if(args.length < 1){
            pattern = Pattern.compile(\".*\");
        } else{
            pattern = Pattern.compile(args[0]);
        }
        final Collection<String> list = ResourceList.getResources(pattern);
        for(final String name : list){
            System.out.println(name);
        }
    }
}  

If you are using Spring Have a look at PathMatchingResourcePatternResolver

回答3:

If you use apache commonsIO you can use for the filesystem (optionally with extension filter):

Collection<File> files = FileUtils.listFiles(new File(\"directory/\"), null, false);

and for resources/classpath:

List<String> files = IOUtils.readLines(MyClass.class.getClassLoader().getResourceAsStream(\"directory/\"), Charsets.UTF_8);

If you don\'t know if \"directoy/\" is in the filesystem or in resources you may add a

if (new File(\"directory/\").isDirectory())

or

if (MyClass.class.getClassLoader().getResource(\"directory/\") != null)

before the calls and use both in combination...

回答4:

Using Google Reflections:

Reflections reflections = new Reflections(null, new ResourcesScanner());
Set<String> resourceList = reflections.getResources(x -> true);

Another sample: get all files with .csv extension from some.package:

Reflections reflections = new Reflections(\"some.package\", new ResourcesScanner());
Set<String> fileNames = reflections.getResources(Pattern.compile(\".*\\\\.csv\"));

回答5:

So in terms of the PathMatchingResourcePatternResolver this is what is needed in the code:

@Autowired
ResourcePatternResolver resourceResolver;

public void getResources() {
  resourceResolver.getResources(\"classpath:config/*.xml\");
}

回答6:

Used a combination of Rob\'s response.

final String resourceDir = \"resourceDirectory/\";
List<String> files = IOUtils.readLines(Thread.currentThread.getClass().getClassLoader().getResourceAsStream(resourceDir), Charsets.UTF_8);

for(String f : files){
  String data= IOUtils.toString(Thread.currentThread.getClass().getClassLoader().getResourceAsStream(resourceDir + f));
  ....process data
}

回答7:

The Spring framework\'s PathMatchingResourcePatternResolver is really awesome for these things:

private Resource[] getXMLResources() throws IOException
{
    ClassLoader classLoader = MethodHandles.lookup().getClass().getClassLoader();
    PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(classLoader);

    return resolver.getResources(\"classpath:x/y/z/*.xml\");
}

Maven dependency:

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-core</artifactId>
    <version>LATEST</version>
</dependency>

回答8:

This should work (if spring is not an option):

public static List<String> getFilenamesForDirnameFromCP(String directoryName) throws URISyntaxException, UnsupportedEncodingException, IOException {
    List<String> filenames = new ArrayList<String>();

    URL url = Thread.currentThread().getContextClassLoader().getResource(directoryName);
    if (url != null) {
        if (url.getProtocol().equals(\"file\")) {
            File file = Paths.get(url.toURI()).toFile();
            if (file != null) {
                File[] files = file.listFiles();
                if (files != null) {
                    for (File filename : files) {
                        filenames.add(filename.toString());
                    }
                }
            }
        } else if (url.getProtocol().equals(\"jar\")) {
            String dirname = directoryName + \"/\";
            String path = url.getPath();
            String jarPath = path.substring(5, path.indexOf(\"!\"));
            try (JarFile jar = new JarFile(URLDecoder.decode(jarPath, StandardCharsets.UTF_8.name()))) {
                Enumeration<JarEntry> entries = jar.entries();
                while (entries.hasMoreElements()) {
                    JarEntry entry = entries.nextElement();
                    String name = entry.getName();
                    if (name.startsWith(dirname) && !dirname.equals(name)) {
                        URL resource = Thread.currentThread().getContextClassLoader().getResource(name);
                        filenames.add(resource.toString());
                    }
                }
            }
        }
    }
    return filenames;
}

回答9:

With Spring it\'s easy. Be it a file, or folder, or even multiple files, there are chances, you can do it via injection.

This example demonstrates the injection of multiple files located in x/y/z folder.

import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;

@Service
public class StackoverflowService {
    @Value(\"classpath:x/y/z/*\")
    private Resource[] resources;

    public List<String> getResourceNames() {
        return Arrays.stream(resources)
                .map(Resource::getFilename)
                .collect(Collectors.toList());
    }
}

It does work for resources in the filesystem as well as in JARs.

回答10:

Based on @rob \'s information above, I created the implementation which I am releasing to the public domain:

private static List<String> getClasspathEntriesByPath(String path) throws IOException {
    InputStream is = Main.class.getClassLoader().getResourceAsStream(path);

    StringBuilder sb = new StringBuilder();
    while (is.available()>0) {
        byte[] buffer = new byte[1024];
        sb.append(new String(buffer, Charset.defaultCharset()));
    }

    return Arrays
            .asList(sb.toString().split(\"\\n\"))          // Convert StringBuilder to individual lines
            .stream()                                   // Stream the list
            .filter(line -> line.trim().length()>0)     // Filter out empty lines
            .collect(Collectors.toList());              // Collect remaining lines into a List again
}

While I would not have expected getResourcesAsStream to work like that on a directory, it really does and it works well.