Wednesday 13 July 2011

Maven Offline Repository

Interesting new topics often get buried in the bulk of the FuseSource doc library. Every so often, I'll be blogging about these topics, so you can get an idea of what's new in the FuseSource library.

A recent example is the description of how to create your own custom Maven offline repository for Apache ServiceMix. If you've ever used Maven, you could hardly fail to notice the way it downloads a ton of dependencies from the Internet before it starts to build your project. But what if you don't want Maven to download anything from the Internet? What is the alternative?

The solution is to create a customized Maven offline repository, which you can then distribute with your Apache ServiceMix deployment. Although it's easy enough to identify your own application bundles, keeping track of the dependencies is not so easy. You typically have hundreds of transitive dependencies and identifying them by hand would be a nightmareafter all, that's what Maven is for.

This is where the features-maven-plugin comes in. This slightly obscure Maven plug-in was originally developed as a utility for Apache developers, for generating distributions of Apache Karaf and Apache ServiceMix. For example, at FuseSource we use it to generate Fuse ESB, our own distribution of Apache ServiceMix. It turns out that the features-maven-plugin plug-in has a goal, add-features-to-repo, that is ideally suited to creating a custom offline repository. Now, the plug-in does have one trivial limitation: the bundles that you want to include in the offline repository must be packaged in a Karaf 'feature'. This is easily taken care of (for example, see Creating a Feature).

To see how to use the features-maven-plugin, lets take a concrete example of an application, deployed into Fuse ESB 4.3.1, that uses the camel-jms feature and the camel-quartz feature. Let's generate a custom Maven repository containing just those two features and all of their dependencies.

First of all create a Maven pom.xml file with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>org.acme.offline-repo</groupId>
    <artifactId>features-offline</artifactId>
    <version>1.0.0</version>
    <name>Generate offline features repository</name>
    <packaging>pom</packaging>

    <repositories>
      <!-- In general, might need to add the repos
           listed in org.ops4j.pax.url.mvn.repositories property -->
      <!--
    http://repo1.maven.org/maven2, \
    http://repo.fusesource.com/maven2, \
    http://repo.fusesource.com/maven2-snapshot@snapshots@noreleases, \
    http://repo.fusesource.com/nexus/content/repositories/releases, \
    http://repo.fusesource.com/nexus/content/repositories/snapshots@snapshots@noreleases, \
    http://repository.apache.org/content/groups/snapshots-group@snapshots@noreleases, \
    http://repository.ops4j.org/maven2, \
    http://svn.apache.org/repos/asf/servicemix/m2-repo, \
    http://repository.springsource.com/maven/bundles/release, \
    http://repository.springsource.com/maven/bundles/external
      -->
    <repository>
      <id>esb.system.repo</id>
      <name>Fuse ESB internal system repo</name>
      <url>file:///E:/Programs/FUSE/apache-servicemix-4.3.1-fuse-00-00/system</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <repository>
      <id>repo1.maven.org</id>
      <name>Maven central</name>
      <url>http://repo1.maven.org/maven2</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <repository>
      <id>repo.fusesource.com</id>
      <name>FuseSource repo</name>
      <url>http://repo.fusesource.com/maven2</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <repository>
      <id>repo.fusesource.com.snapshot</id>
      <name>FuseSource snapshot repo</name>
      <url>http://repo.fusesource.com/maven2-snapshot</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
      <releases>
        <enabled>false</enabled>
      </releases>
    </repository>
    <repository>
      <id>repo.fusesource.com.nexus</id>
      <name>FuseSource Nexus repo</name>
      <url>http://repo.fusesource.com/nexus/content/repositories/releases</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <repository>
      <id>repo.fusesource.com.nexus.snapshot</id>
      <name>FuseSource Nexus snapshot repo</name>
      <url>http://repo.fusesource.com/nexus/content/repositories/snapshots</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
      <releases>
        <enabled>false</enabled>
      </releases>
    </repository>
    <repository>
      <id>repository.apache.org.snapshot</id>
      <name>Apache snapshot repo</name>
      <url>http://repository.apache.org/content/groups/snapshots-group@snapshots@noreleases</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
      <releases>
        <enabled>false</enabled>
      </releases>
    </repository>
    <repository>
      <id>repository.ops4j.org</id>
      <name>OPS4J repo</name>
      <url>http://repository.ops4j.org/maven2</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <repository>
      <id>svn.apache.org</id>
      <name>Apache SVN repos</name>
      <url>http://svn.apache.org/repos/asf/servicemix/m2-repo</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <repository>
      <id>repository.springsource.com</id>
      <name>Spring repo</name>
      <url>http://repository.springsource.com/maven/bundles/release</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    <repository>
      <id>repository.springsource.com.external</id>
      <name>Spring external repo</name>
      <url>http://repository.springsource.com/maven/bundles/external</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
    </repositories>

    <build>
        <plugins>
          <plugin>
            <groupId>org.apache.karaf.tooling</groupId>
            <artifactId>features-maven-plugin</artifactId>
            <version>2.2.1</version>

            <executions>
              <execution>
                <id>add-features-to-repo</id>
                <phase>generate-resources</phase>
                <goals>
                  <goal>add-features-to-repo</goal>
                </goals>
                <configuration>
                  <descriptors>
                    <!-- List taken from featuresRepositories in etc/org.apache.karaf.features.cfg -->
                    <descriptor>mvn:org.apache.karaf/apache-karaf/2.1.3-fuse-00-00/xml/features</descriptor>
                    <descriptor>mvn:org.apache.servicemix.nmr/apache-servicemix-nmr/1.4.0-fuse-00-00/xml/features</descriptor>
                    <descriptor>mvn:org.apache.servicemix/apache-servicemix/4.3.1-fuse-00-00/xml/features</descriptor>
                    <descriptor>mvn:org.apache.camel.karaf/apache-camel/2.6.0-fuse-00-00/xml/features</descriptor>
                    <descriptor>mvn:org.apache.servicemix/ode-jbi-karaf/1.3.4/xml/features</descriptor>
                    <descriptor>mvn:org.apache.activemq/activemq-karaf/5.4.2-fuse-01-00/xml/features</descriptor>
                  </descriptors>
                  <features>
                    <feature>camel-jms</feature>
                    <feature>camel-quartz</feature>
                  </features>
                  <repository>target/features-repo</repository>
                </configuration>
              </execution>
            </executions>
          </plugin>
        </plugins>
    </build>
    
</project>

There's quite a lot of boilerplate in this POM file. Here is a summary of what you need to include:
  • A reference to the ServiceMix system repository, which is a local repository that contains all of the artifacts shipped with Apache ServiceMix. You really do need to include this, because it might contain artifacts not available from the other repositories.
  • List all of the repositories, local and remote, which might contain artifacts needed by your features. Better to err on the generous side here. In particular, you should list all of the remote repositories used by ServiceMix itself. You can find this list in the etc/org.ops4j.pax.url.mvn.cfg configuration file, in the org.ops4j.pax.url.mvn.repositories property setting.
  • In the configuration/description element of the features-maven-plugin configuration, add the list of relevant feature repositories. At a minimum, you will need to add all of the features repositories used by ServiceMix. You can find this list in the etc/org.apache.karaf.features.cfg file, in the featuresRepository property setting.
  • Finally, in the configuration/features element of the features-maven-plugin, you need to list the features for which you are generating the offline repository.

Now you are ready to generate the custom offline repo by entering the following Maven command:

mvn generate-resources

If this builds successfully, you should find the generated custom repository under the target/features-repo directory of the Maven project.

For a more detailed description of how to set up the POM correctly, see Generating a Custom Offline Repository.