Maven: how to filter the same resource multiple ti

Question

Our project uses Log4J, configured via log4j.properties file. We have multiple production servers, which log to different log files, so that the logs can be differentiated. So log4j.properties for node 1 looks like this:

...
log4j.appender.Application.File=D:/logs/application_1.log
...
log4j.appender.tx_info.File=D:/logs/tx_info_1.log
...

while the log4j.properties for node 2 looks like

...
log4j.appender.Application.File=D:/logs/application_2.log
...
log4j.appender.tx_info.File=D:/logs/tx_info_2.log
...

We already use Maven profiles for generating our server configuration. Up to now it contained several distinct log4j.properties files which differed only in the log file names as shown above. I would like to generate these files with Maven using a resource template file like this:

...
log4j.appender.Application.File=${log.location}/application${log.file.postfix}.log
...
log4j.appender.tx_info.File=${log.location}/tx_info${log.file.postfix}.log
...

It is easy to run Maven multiple times with different ${log.file.postfix} values to generate a single different log property file each time. However, what I would like is to generate a distinct property file (with the name/path different) for each server in one build. I am fairly sure this can be done, e.g. via the antrun plugin, but I am not familiar with that. What is the simplest way to achieve this?

Answer 1

Here are some approaches that you can try:

1. use the antrun plugin, and the copy task to make duplicates of your resource file in the generate-resources phase. An example of using the antrun plugin is given in the answer to this SO question on copying with maven. You could even use ant's property expansion to expand each ${log.file.postfix} to a distinct value, either the literal values, 1,2,3 etc. or unique placeholders, ${log.file.postfix1}, ${log.file.postfix2} which are finally replaced when maven does the resource filtering.

2. Rather than using antrun, use your version control system to set up multiple copies of the same file. You can then run multiple instances of the resources:copy-resources goal, each with different property values configured, and a different target file name.

Answer 2

Although this is a bit old, I came across the thread recently and would like to propose an updated solution using the iterator-maven-plugin. An overview is found here: http://khmarbaise.github.io/iterator-maven-plugin/

A specific example of how you can accomplish your goal would be to combine the iterator plugin with a copy resource and filter enabled. You could even add custom properties file to use as a filter in case you have other attributes that are unique per node with this approach.

<plugin>

    <groupId>com.soebes.maven.plugins</groupId>
    <artifactId>iterator-maven-plugin</artifactId>
    <version>0.3</version>

    <executions>
        <execution>
            <id>configure-log-properties</id>
            <phase>validate</phase>
            <goals>
                <goal>iterator</goal>
            </goals>
            <configuration>
                <iteratorName>log.file.postfix</iteratorName>
                <content>1,2,3,4,5</content>
                <pluginExecutors>

                    <pluginExecutor>
                        <plugin>
                            <artifactId>maven-resources-plugin</artifactId>
                            <version>2.7</version>
                        </plugin>

                        <goal>copy-resources</goal>

                        <configuration>
                            <outputDirectory>${project.build.directory}/nodes/${log.file.postfix}</outputDirectory>

                            <resources>
                                <resource>
                                    <directory>src/main/resources</directory>
                                    <includes>
                                        <include>log4j.properties</include>
                                    </includes>
                                    <filtering>true</filtering>
                                </resource>
                            </resources>
                        </configuration>

                    </pluginExecutor>
                </pluginExecutors>
            </configuration>
        </execution>
    </executions>
</plugin>

Answer 3

If there are a lot of target configurations that need to be copied, you could use the maven-antrun-plugin together with ant macrodef.

  <plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.3</version>
    <executions>
      <execution>
        <id>copy-resources-all-nodes</id>
        <phase>process-resources</phase>
        <configuration>
          <tasks>
             <macrodef name="copyConfigFile">
                <attribute name="node"/>

                <sequential>
                   <copy file="src/main/resources/log4j.properties"
                         toFile="target/antrun/log4j-@{node}.properties">
                      <filterset>
                         <!-- put the node-specific config in property files node1.properties etc -->
                         <filtersfile file="config/@{node}.properties"/>
                      </filterset>
                   </copy>
                </sequential>
             </macrodef>

             <copyConfigFile node="node1"/>
             <copyConfigFile node="node2"/>
             <copyConfigFile node="node3"/>
             ...

          </tasks>
        </configuration>
        <goals>
          <goal>run</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

If you really have a lot of target configurations, you could also use ant-contrib to iterate over a list of target configurations.

There's an example how to do this here

Answer 4

(...) I am fairly sure this can be done, e.g. via the antrun plugin, but I am not familiar with that. What is the simplest way to achieve this?

You could indeed use resources:copy-resources and several <execution> in your POM (note that resources:copy-resources doesn't allow to change the name of the target file though).

Let's assume you have the following structure:

$ tree .
.
├── pom.xml
└── src
    ├── main
    │   ├── filters
    │   │   ├── filter-node1.properties
    │   │   └── filter-node2.properties
    │   ├── java
    │   └── resources
    │       ├── log4j.properties
    │       └── another.xml
    └── test
        └── java

Where log4j.properties is using place holders and the filter-nodeN.properties files contain the values. For example:

# filter-node1.properties

log.location=D:/logs
log.file.postfix=_1

Then, in your pom.xml, configure the resources plugin and define one <execution> per node to call copy-resources with a specific output directory and a specific filter to use:

<project>
  ...
  <build>
    <resources>
      <!-- this is for "normal" resources processing -->
      <resource>
        <directory>src/main/resources</directory>
        <filtering>true</filtering><!-- you might still want to filter them -->
        <excludes>
          <!-- we exclude the file from "normal" resource processing -->
          <exclude>**/log4j.properties</exclude>
        </excludes>
      </resource>
    </resources>
    <plugins>
      <plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <version>2.4.3</version>
        <executions>
          <execution>
            <id>copy-resources-node1</id>
            <phase>process-resources</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <outputDirectory>${basedir}/target/node1</outputDirectory>
              <resources>
                <resource>
                  <directory>src/main/resources</directory>
                  <filtering>true</filtering>
                  <includes>
                    <include>**/log4j.properties</include>
                  </includes>
                </resource>
              </resources>
              <filters>
                <filter>src/main/filters/filter-node1.properties</filter>
              </filters>
            </configuration>
          </execution>
          <execution>
            <id>copy-resources-node2</id>
            <phase>process-resources</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <outputDirectory>${basedir}/target/node2</outputDirectory>
              <resources>
                <resource>
                  <directory>src/main/resources</directory>
                  <filtering>true</filtering>
                  <includes>
                    <include>**/log4j.properties</include>
                  </includes>
                </resource>
              </resources>
              <filters>
                <filter>src/main/filters/filter-node2.properties</filter>
              </filters>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Running mvn process-resources would produce the following result:

$ tree .
.
├── pom.xml
├── src
│   ├── main
│   │   ├── filters
│   │   │   ├── filter-node1.properties
│   │   │   └── filter-node2.properties
│   │   ├── java
│   │   └── resources
│   │       ├── log4j.properties
│   │       └── another.xml
│   └── test
│       └── java
└── target
    ├── classes
    │   └── another.xml
    ├── node1
    │   └── log4j.properties
    └── node2
        └── log4j.properties

With the appropriate values in each log4j.properties.

$ cat target/node1/log4j.properties 
log4j.appender.Application.File=D:/logs/application_1.log
log4j.appender.tx_info.File=D:/logs/tx_info_1.log

This kinda works, but is verbose and this might be a problem if you have a decent amount of nodes.

---

I tried to write something more concise and maintainable using the Maven AntRun Plugin but I couldn't get the for task from ant-contrib to work under Maven (for an unknown reason, the for task isn't recognized) and I gave up.

Here is an alternative using the Maven AntRun Plugin. Nothing complicated, no loop, I'm just copying the source file to another location, changing its name on the fly and filtering the content:

  <plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.3</version>
    <executions>
      <execution>
        <id>copy-resources-all-nodes</id>
        <phase>process-resources</phase>
        <configuration>
          <tasks>
            <copy file="src/main/resources/log4j.properties" toFile="target/antrun/log4j-node1.properties">
              <filterset>
                <filter token="log.location" value="D:/logs"/>
                <filter token="log.file.postfix" value="_1"/>
              </filterset>
            </copy>
            <copy file="src/main/resources/log4j.properties" toFile="target/antrun/log4j-node2.properties">
              <filterset>
                <filter token="log.location" value="D:/logs"/>
                <filter token="log.file.postfix" value="_2"/>
              </filterset>
            </copy>
          </tasks>
        </configuration>
        <goals>
          <goal>run</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

Note that Ant uses @ by default as delimiters for token (couldn't get it to use maven style delimiters) so the log4j.properties became:

log4j.appender.Application.File=@log.location@/[e-mail]@.log
log4j.appender.tx_info.File=@log.location@/[e-mail]@.log

---

But, since these values seem to be node specific, did you consider using system properties instead (that you could place in the startup scripts)? This is something I've already done (with a log4j.xml), it works well and it would highly simplify things.