optimizing maven build performance
jenkins, maven, maven-3
Solution
Optimizing the Maven build of a Java project comes down to 3 factors:
- Compilation and Tests
- Complexity of the Maven build
- Optimizing the Maven (binary) performance
Compilation and Tests
If, for example, you have GWT in your application and you're compiling for 20 different permutations then this will be a problem if you're doing this the whole time.
There are ways to get around this by maybe only doing the full build when you need to, and otherwise limiting the build by using the `draftCompile` or `optimizationLevel` options.
The same thing applies to your tests: if you have hundreds of tests, the length of time that each test takes starts to become significant. Common ways to increase performance in tests are:
- Don't use `Thread.sleep(..)`, use mutex's and locks.
- Share configuration (persistence units, mocks) across multiple tests instead of doing the setup per test.
- Make it a unit test instead of an integration test, unless you really need to. Too many integration tests is a bad code smell, in any case.
Complexity of the Maven Build
If you have 10's or hundreds of modules, with a complex dependency tree, maven will need time to calculate the dependency tree. Use `dependency:analyze' to figure out whether or not there are unused dependencies and remove them.
Also, the number of plugins used in each module's build will increase the build time.
Optimizing Maven (binary) performance
I use the following options:
`export MAVEN_OPTS="-Dmaven.wagon.provider.http=httpclient -Dmaven.artifact.threads=12 -Dhttp.tcp.nodelay=false -Xmx2048M -Xss256M -XX:+CMSClassUnloadingEnabled -XX:-UseGCOverheadLimit -XX:MaxPermSize=256M -T2C"`
- `-Dmaven.wagon.provider.http=httpclient`: Changes the http connection mechanism, my theory is that the Apache HttpClient library is faster and better than the default Java `HttpURLConnection` class
- `-Dmaven.artifact.threads=12`: The number of threads used to download artifacts.
- `-Dhttp.tcp.nodelay=false`: Turns off Nagle's algorithm increasing performance (but also bandwidth used). This is a Wagon property.
- `-T2C`: This was an expiremental build introduced in maven 3: the number of threads per build. See Parallel builds in Maven 3
Please feel to edit this post if you have other tips!
Problem
I have a Maven web application. This is taking too much time to build(30-40 mins).I would like to reduce it to less than 10 mins. My pom.xml is as below. ``` <?xml version="1.0" encoding="utf-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>se.mysite.etc.settings</groupId> <artifactId>projects</artifactId> <version>1.5</version> </parent> <groupId>se.myweb.portal</groupId> <artifactId>myweb-se-main</artifactId> <name>myweb-se-main</name> <version>3.1.81-B2_forv-SNAPSHOT</version> <packaging>pom</packaging> <inceptionYear>2009</inceptionYear> <properties> <release.version>${project.version}</release.version> <acc.version>2.3.42-TEST-MAINT-SNAPSHOT</acc.version> <cxf.version>2.5.3</cxf.version> <spring.version>3.0.3.RELEASE</spring.version> <spring.security.version>2.0.4</spring.security.version> <spring.webflow.version>2.1.1.RELEASE</spring.webflow.version> <commonportal.version>1.9.9-SPRING3</commonportal.version> <junit.version>4.5</junit.version> <java.source.version>1.6</java.source.version> <cobertura.maxmem>1024M</cobertura.maxmem> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <scm> <connection>scm:hg:http://test1/myweb</connection> <developerConnection>scm:hg:http://test1/myweb</developerConnection> <tag/> <url>http://test1/myweb</url> </scm> <profiles> <profile> <id>dev</id> <dependencies> <!--some dependencies--> </dependencies> <modules> <module>../project1</module> <module>../project2</module> <module>../project3</module> <module>../project4</module> </modules> </profile> <profile> <id>dist</id> <activation> <activeByDefault>true</activeByDefault> </activation> <dependencies> <!--some dependencies--> </dependencies> <modules> <module>../project1</module> <module>../project2</module> <module>../project3</module> <module>../project4</module> <module>../project5</module> <module>../project6</module> </modules> </profile> <profile> <id>backend</id> <dependencies> <!--some dependencies--> <modules> <module>../project3</module> <module>../project1</module> <module>../project2</module> <module>../project4</module> <module>../project9</module> </modules> </profile> <profile> <id>frontend</id> <!--some dependencies--> <modules> <module>../project10</module> <module>../project11</module> </modules> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.12.4</version> <configuration> <forkMode>always</forkMode> <argLine>-Xms512m -Xmx2048m -XX:MaxPermSize=2048m</argLine> </configuration> </plugin> </plugins> </build> </profile> <profile> <id>webtest</id> <build> <finalName>myweb-web-test</finalName> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <configuration> <argLine>-XX:MaxPermSize=512m</argLine> </configuration> </plugin> </plugins> </build> <!--some dependencies--> <modules> <module>../myweb-web-test</module> </modules> </profile> <profile> <id>redeploy_web_app</id> <activation> <property> <name>redeployWebApp</name> </property> </activation> <properties> <user.tomcat.home>${env.DEST_DIR}</user.tomcat.home> </properties> <build> <pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-antrun-plugin</artifactId> <version>1.6</version> <dependencies> <!--some dependencies--> </dependencies> <executions> <execution> <id>delete_project_artifact</id> <phase>prepare-package</phase> <configuration> <tasks> <taskdef classpathref="maven.plugin.classpath" resource="net/sf/antcontrib/antcontrib.properties"/> <if> <equals arg1="${project.packaging}" arg2="war"/> <then> <echo message="Removing ${project.artifactId}.${project.packaging} from ${user.tomcat.home}/webapps"/> <delete dir="${user.tomcat.home}/webapps/${project.artifactId}"/> <delete file="${user.tomcat.home}/webapps/${project.artifactId}.${project.packaging}"/> </then> </if> </tasks> </configuration> <goals> <goal>run</goal> </goals> </execution> <execution> <id>copy_project_artifact</id> <phase>package</phase> <configuration> <tasks> <taskdef classpathref="maven.plugin.classpath" resource="net/sf/antcontrib/antcontrib.properties"/> <if> <equals arg1="${project.packaging}" arg2="war"/> <then> <echo message="Copying ${project.build.finalName}.${project.packaging}"/> <copy file="${project.build.directory}/${project.build.finalName}.${project.packaging}" overwrite="true" tofile="${user.tomcat.home}/webapps/${project.artifactId}.${project.packaging}"/> </then> </if> </tasks> </configuration> <goals> <goal>run</goal> </goals> </execution> </executions> </plugin> </plugins> </pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-antrun-plugin</artifactId> </plugin> </plugins> </build> </profile> </profiles> <!-- General dependencies --> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>cobertura-maven-plugin</artifactId> <version>2.2</version> <configuration> <instrumentation> <excludes> <exclude>se.mysite/**/Test*.class</exclude> </excludes> </instrumentation> </configuration> <executions> <execution> <goals> <goal>clean</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <!-- TODO: Use UTF-8 here or not??? --> <encoding>UTF-8</encoding> <!-- encoding>ISO-8859-1</encoding--> <source>${java.source.version}</source> <target>${java.source.version}</target> <showDeprecation>true</showDeprecation> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-release-plugin</artifactId> <version>2.1</version> <configuration> <scmCommentPrefix>II Release:</scmCommentPrefix> </configuration> </plugin> </plugins> </build> <organization> <name>mysite AB</name> <url>http://www.mysite.se</url> </organization> </project> ``` My compile command is below. ``` mvn -U -DartifactsTarget=$DEST_DIR/lib/tpp -P dev -Dmaven.test.failure.ignore=true -Dtesting.testsToExclude=**/Test*.java -f pom.xml $* -DredeployWebApp install ``` It is taking almost 30 mins to build. I am using maven 3.0.4. I explored that using the following options we can optimize the build time. - Using maven parallel processing - Using plexus compiler plugin - Using maven power shell Please let me know if I need to any other changes to my pom.xml or any other techniques that would optimize the performance.