Pauses Café

If you ever wanted to use both Jira and TestDirector on a projet, or migrate from TD to Jira, have a look at td2jira. Subtitled “Quick’n'Dirty Way to Share The Issues With Your QA Team” it works OK for us !

Vladimir graciously gave me commit access and I added some new features. Interested, don’t let you get fooled by the front page, and go and grab the latest from subversion.

Talking about Jira and continuous integration, I have a patch waiting for publishing build failures from CruiseControl to Jira. Hopefully it will make it into the upcoming 2.7.2 !

Was searching for a way to mirror a remote subversion repository through http, for backup purposes.

After struggling with various tools, I found this blog entry. Thanks Thomas!

In the past months, that blog was not showing nicely under IE, while it worked well under firefox and opera. I finally took the time to fix the issue. I didn’t want to tackle the issue until I knew what was going on.

Today I’ve found out that IE finally as a good DOM explorer. Using it helped me to quickly identify the issue (related to width and auto-wrapping of text in pre blocks.

Pretty cool move by Thoughtworks, bundling svn, trac and CruiseControl together into buildix, a Debian derived distro.

Now what is only missing for me here is a way to handle mailing lists and archive them. But that is easy to add to the distro once installed.

And of course a maven2 repository manager with appropriate permission management for deployment

Great news for maven users. Finally! Jars (licensed under the CDDL) start entering the maven repositories. No more manual upload to your local/company repository. According to Koshuke, more may be announced before JavaOne.

It’s funny. I keep getting the same questions: “should I use cvs or subversion?”

So I’ll state my answer once and for all. The main thing you may not like about subversion is the way it handles branches and tags. You may also not like the fact that the repository is not stored in a text file format (be sure to have good backups) and that the checked out project takes much more space.

On the plus side for svn, I will remind:

• atomic commit, your whole repository version is identified by a single id (you want that for software tracability reasons, more on that later)
• directory and meta-data versionning
• diff and revert without network access
• plays nice with firewalls
• designed to be a drop in replacement for cvs for daily activities (similar semantics)

So now that the support is good with the tools (IDE integration, Java SVN library, etc…), you should probably switch. Especially as migrating could be as simple as freezing your project for the time it takes to run a conversion script.

As for those afraid of performance, sourceforge with its 130000 projects supports svn site wide. And they are happy of the performance. That should say it all.

So yes, I recomment svn, in most cases. And I am eagerly waiting for CruiseControl 2.5 to be released so that we migrate to subversion.

Introduction

I started using m2 extensively in the past 3 months in various projects. During that time, the m2 core came to be released, then the plugins. In the course of this experimentation/work I encountered various successes and failures. On a particular project with a client (migration of a 150000 lines of code project built from maven/ant to maven2) we had a particular hard time.

First m2 was missing some features for our build, in particular to build webstart applications. The functionality to do that (sign the jars, package the JNLP application, …) was missing in m2, and m2/ant integration wasn’t an option. So I’ve had to implement it. Then I’ve had to slightly tweak my maven build, which was relying too much on various local tweaks in maven.xml files. Finally I’ve had to change somehow the way we produce our final artifact for the various customers. Customization now happens at a different phase of the overall building process, which matches better our work model. m2 profiles were used to make this work.

Doing the migration I lost many hours due to various m2 issues. All in all I think I encountered around 15 issues that directly affected my productivity. I’ve fixed many of those, some are still unpatched and working with m2 today still requires to build my own patched version of m2, and upgrade a couple of times per week. In terms of time, my estimate of the time it would take to write the Webstart plugin was roughtly correct, but I spent much more time trying to work-around m2 bugs and/or fixing it.

Potential Improvements

I know that some people are going to not be particularly happy by this section. But someone has to say it

Here follows a list of issues and potential recommendations:

• bugs, regressions and maturity. Today that’s the main issue to me. As soon as you start using maven/pluggins in ways people haven’t used them, you walk on eggs. In particular, I’ve had many plugins fail when used within the reactor. To prevent these issues, a developer methodology to test plugins should probably be defined to help plugin writers avoid common mistakes.
• debugging the core can be a daunting task. Writing unit tests for it is even harder. Even Jason says it.
• in case of a failure, m2 tells you to rerun the build with the -e option to get a stack trace. That’s completely counter productive. Do not ask me to rerun my full build with -e enabled. Just keep track of the failure somehow, and display it for me.
• XML verbosity. I am sort of tired to write very verbose XML files. First spring now maven2. Yack. Example: try to manually change the version of your multi-project. If you don’t use the maven release plugin (which is known to have some issues), you will have to change all your poms. Why can’t this be in one place? This is linked to the maven way of defining poms, but you wonder if KISS just went away. I don’t have a better solution to propose for the general XML verbosity issue though, apart from designing very carefully the maven’s core. Maybe that will be partially addressed in 2.1.
• better installation. When you rebuild and reinstall some of the core m2 modules, you will sometimes need to copy them into your M2_HOME/lib directory. That was maybe fixed since 2.0?
• documentation. Although very good, some cases are not covered enough. In particular, the model document is sometimes not easy to browse. I wish it was formatted like the CruiseControl config file was. The documentation (guides) is also perhaps not well structured to my taste, and lacking in some areas, probably because it is automatically generated.
• similarly m2 introduces many new terms & concepts. A m2 dictionnary would help (note: the m2 book is on its way, so that may come in the form of its index)
• plugins style consistency: m2 developers should perhaps setup a structure to review initial design proposals for externaly hosted plugins to make sure they are developed to fit correctly within the m2 model and philosophy.
• for plugin developers, I spent quite some time trying to find out which dependencies I needed to add in my pom, and which version. When I didn’t have my IDE opened on the maven project itself, I spent most of my time doing locate ClassName to find out the module the class was pertaining to then check in the logs whether the required pom was there or whether it had the right version. That worked because I had the whole m2 source on my machine, as well as plexus source, doxia source… We maybe need a sort of versioned jarhoo for maven 2 related projects?
• I think that it’s very easy to not find out about particular maven issues if you have a locally deployed dependency and a broken maven that incorrectly searches for the dependency. For similar reasons, I wonder if m2 developers tend to have an environment that doesn’t let them find out the issues user face (in particular dependencies issues). I may be wrong there, so I won’t extrapolate too much. Now I wonder how this issues will affect Continuous Integration setups. For the moment I recommend using a different repository for each independent project your are using. You should try to avoid any side effect your local repository may introduce.
• upgrading m2 without cleaning your repository can let you in a dependency mess (in particular if you had a snapshot version installed). Cleaning your full repository forces you to manually reinstall all some artifacts (jdbc, jta, …). Be use to set up a corporate repository soon enough.
• slow slow slow ibiblio. Anyway, you shouldn’t completely depend on the Internet. Use a mirror and even better maven proxy.
• maven now has many plugins which are small projects themselves. These projects have their own release cycle. That’s a big departure from the way traditional open source frameworks have been released (ant, cruisecontrol, etc…). There are obvisouly benefits (easy to push a fix to the users) but that creates other issues. You may introduce breakage if your poms haven’t been correctly specified and a regression is introduced. Similarly it’s up to the bug reporters to make sure their issues are taken into account. Sometimes one will have to poll the developer list and notify the developers that a particular issue is to be included in the release. That will go hopefully away when the code will mature. Still one can see the typical dependency issues that have plagued Linux distributions for years.

Good parts of m2

Not all is that bad!

• the m2 developpers have been very quick at empowering me. They gave me committing access to the mojo-sandbox where I’ve been able to upload various plugins required for me to work that could be useful to the community. I believe this to be a very good move from the official developers of the m2 related toolset. The m2 community is indeed very active, and allowing people to be gradually involved in the project can only benefit them. I am also pretty satisfied with mojo community. I’ve received a lot of enhancements in the form of patches and ideas that now will soon make some of the plugins really production ready
• I find the standard lifecycle and directory layout a breeze
• mailing list/IRC support: mostly good, even if it tends to be less efficient when a people are busy (preparing a release, writing a book…)
• the quality of the project (and documentation) is increasing. The trunk seems to be broken less than before, and the developers are willing to make maven 2 a better product.

Conclusion

m2 shows great promises. Its current state is not completely ready for production (at least not for all kinds of projects), altought it is getting closer and closer. Depending on how you will use m2, you may encounter many many small irritating bits, or slightly broken features there and there. When trying to migrate a small and relatively heterogeneous project, you might spend too many resources fixing internal issues or adding missing functionality through plugins. On the other side, the bigger your project, the more homogeneous it might be. The economy of scale will reduce these migration costs. With regard to starting a new project today, I definitively advice you to consider using m2, especially if you don’t need fancy features. E.g. if your project is a back-end sort of things, like combining layers of jars together into a single project that do not require many customizations, m2 is definitively a tool to look at for your infrastructure. As with other types of projects, in particular big J2EE projects, work is being done to produce sets of guidelines that should make your life easier.

The plugins I looked at were of differing quality but most were good. For developers, if you want to develop a plugin, be sure to spend time understanding the concept of life cycle. I definitively lost some time here jumping too early into the code (for my defense, the documentation was shallower when I started). Spend also time understanding the way dependencies and release mecanism work.

Many people are coming to maven because of dependency management. If you just need dependency management, maybe you can get away by using the maven2 ant tasks. Even with those, dependencies issues are problems that Linux distributions have had since their inception and that won’t disappear tomorrow. If you choose to use transitive dependencies, you just trade one problem for another one, hopefully smaller. But remember, dependency management is only as good as your poms and release process can be!

All in all, I believe m2 is going to rock, but it will take some time, because it changes the way we do things. M2 brings in a long awaited set of features really useful for managing big projects. Its philosophy can sometimes feel to be in your way to gets things done when a particular feature is missing or an issue affects your setup, but I think it pays a lot on the long run to follow these somewhat enforced guidelines. And if you use maven/m2 just remember one thing: don’t try to fight it.

I will give it a 3.5 stars out of 5, but I believe it will be worth 4 or 4.5 within the year.

So you started using maven / maven2 in your organization. Now you need a place to deploy your company specific artifacts. You also don’t want to depend on an external repository to perform your build.

So setting up a corporate repository is a natural step. Unfortunately, I’ve found that the information describing how to do this is scattered around several places. So here’s a single page version that works for us.

The following short guideline explains how to deploy artifacts to a remote server using scp and retrieve them using https.

Note: I am using apache 2.0.55 under Linux on the server and maven 2.0.2 and SDK 1.5 under Linux on the client. In this example, the serve’s named dev.domaine.com. There are some variations for Windows.

Note2: it is possible to setup your system differently. maven2 is flexible. This just describe one way to make it work, which was I believe, the simplest and most flexible way for us to achieve our goal.

Deploying artifacts on your server

Most of this information comes from this page. My preferred setup is to use scp and an ssh agent for handling my keys.

• on your server, create a place where to put your artifacts. E.g. /home/m2/public_html
• use suitable permissions for developers to be able to deploy artifacts there and for your web server to be able to serve the files later on
• 2 cases:
  • If you have control over the pom.xml of the project you are trying to deploy, deploying your artifact is easy. Just add the following information to your project’s pom.xml:

    <distributionmanagement>
      <!-- use the following if you're not using a snapshot version. -->
      <repository>
        <id>ssh-repo</id>
        <name>Repository Name</name>
        <url>scp://host/path/to/repo</url>
      </repository>
      <!-- use the following if you ARE using a snapshot version. -->
      <snapshotrepository>
        <id>ssh-repo</id>
        <name>Repository Name</name>
        <url>scp://host/path/to/repo</url>
      </snapshotrepository>
    </distributionmanagement>
    

    Then add an entry in your ~/.m2/settings.xml:

    <settings>
      ...
      <servers>
        ...
        <server>
          <id>ssh-repo</id>
          <username>...</username>
          ...
        </server>
      </servers>
      ...
    </settings>
    

    And finally, deploy doing mvn deploy.

  • When you don’t have control over the pom, you will need to specify the repository related information on the command line. You will use the deploy:deploy-file goal.
    • If you already have a pom for your artifact, it is best to instruct the plugin to pick the information from the pom. You will end up doing something like:
      mvn deploy:deploy-file -Durl=scpexe://dev.domaine.com/home/m2/public_html -DrepositoryId=ssh-repo pomFile=pom.xml -Dfile=the.jar
    • If you don’t have a pom, you will have to specify all the other arguments:
      mvn deploy:deploy-file -DgroupId=... -DartifactId=... -Dversion=... .... -Durl=scpexe://dev.domaine.com/home/m2/public_html -DrepositoryId=ssh-repo pomFile=pom.xml -Dfile=the.jar
  • In both cases you will need to add a <server> entry to your settings.xml file, as specified above.

Make these artifacts accessible from your client

• Generate/sign an ssl certificate for your server. On Linux, it will probably be stored somewhere like /etc/ssl/apache2/server.crt
• Make sure your repository directory is available under apache. E.g.

  Alias /m2 /home/m2/public_html
  <directory /home/m2/public_html>
    AllowOverride None
    Options Indexes FollowSymLinks
    Order allow,deny
    ....
  </directory>
  

• Protect this directory using usual Apache access rights management tools
• Test. Point your browser to https://dev.domaine.com/m2. The browser should ask for username/password. You may or not need to prepend a slash to your url depending on your apache configuration.

Let’s make this work with maven2!

• Import your server ssl certificate into your JVM keystorre. As root:
  keytool -keystore $JAVA_HOME/jre/lib/security/cacerts -import -file server.crt. You will need the keystore password to modify it. By default this password is ‘changeit‘. Note: If you change SDK, you will need to redo this operation. Note: if you already have a certificate under the “mykey” alias, you may have the following error:
  keytool error: java.lang.Exception: Certificate not imported, alias already exists

  In that case, import the other one under a new alias (-alias newalias) or remove your old certificate (keytool -keystore $JAVA_HOME/jre/lib/security/cacerts -delete -alias mykey).
• on your project: add a repository to your pom:

      <repository>
        <id>https-repo</id>
        <name>My Company Repository</name>
        <url>https://dev.domaine.com/m2</name>
      </repository>
  

• if you host plugins on your corporate repository, you may also need to add something like:

    <pluginRepositories>
      ...
      <pluginRepository>
        <id>https-repo</id>
        <url>https://dev.domaine.com/m2</url>
      </pluginRepository>
  
    </pluginRepositories>
  

• Modify your ~/.m2/settings.xml. That’s where you will add your personnal authentication information:

  <settings>
    ...
    <servers>
      ...
      <server>
        <id>https-repo</id>
        <username>...</username>
        <password>...</password>
      </server>
    </servers>
    ...
  </settings>
  

• clean up your local maven repository to test downloading from the remote one. E.g. rm ~/.m2/repository/com/domaine/...
• Test building your project. e.g. m2 install.

If you have any issue and you think it might be ssl related, enable debugging using the javax.net.https unsupported option. You might want to try m2 -Djavax.net.https=ssl install. -Djavax.net.https=all for more info and -Djavax.net.https=help for usage information.

Troubleshooting and common mistakes

• Note: trying to use a a full basic auth url (https://user:password@dev.domaine.com/m2), either in the pom or on the command line won’t work. You have to externalize your login/password into an external file
• if you are deploying a multi-project, and one module is not deployed and you get a generic error that says something like:

  
  Failed to configure plugin parameters for: org.apache.maven.plugins:maven-deploy-plugin:2.1
  
  check that the following section of the pom.xml is present and correct:
  
  <distributionManagement>
    <!-- use the following if you're not using a snapshot version. -->
    <repository>
      <id>repo</id>
      <name>Repository Name</name>
      <url>scp://host/path/to/repo</url>
    </repository>
    <!-- use the following if you ARE using a snapshot version. -->
    <snapshotRepository>
      <id>repo</id>
      <name>Repository Name</name>
      <url>scp://host/path/to/repo</url>
    </snapshotRepository>
  </distributionManagement>
  
  Cause: Class 'org.apache.maven.artifact.repository.ArtifactRepository' cannot be instantiated
  [INFO] ----------------------------------------------------------------------------
  [...]
  Caused by: java.lang.InstantiationException: org.apache.maven.artifact.repository.ArtifactRepository
          at java.lang.Class.newInstance0(Class.java:335)
          at java.lang.Class.newInstance(Class.java:303)
          at org.codehaus.plexus.component.configurator.converters.AbstractConfigurationConverter.instantiateObject(AbstractConfigurationConverter.java:111 )
          ... 24 more
  

  then be sure to follow the recommendations in the error log. A classical mistake is to have forgotten to inherit the module’s pom from the parent pom where this information was stored.

• when deploying a module, maven searches the repositories referenced in the pom for the dependencies of the module. I think the idea is to identify whether the required dependency versions are already deployed. If you try to deploy a multi project whose modules depend on each other, you will probably have to make sure that the target repository is listed in the <repositories> section of your pom, otherwise the deploy operation might fail, not finding a suitable version for your module, even though the dependency might just have been uploaded there. Maven has no way to know that fact (especially as you refer to a repository differently when you upload or retrieve information from it, and also because each module deployment is treated independently of the others, even in the case of a multi-project).

Last advice: you might want to have one repository for snapshots and one for releases. That might simplify backing up operations on the server if you only want to backup your releases.

Voila! m2 should now download your artifacts from new your shiny corporate repository. Good luck!

update: added information on how to update the imported certificate.

Now that you’ve secured your ssh (*), you may want to be able to access that from the jira svn plugin.

Jira-svn uses the javasvn library under the hood.

• First thing is to create a user that will access the repository. This user. Let’s call it svn-jira.
• Make sure this user can retrieve the code from your svn repository without ssh. E.g. do a svn co file:///your/path/to/svn/repository
• Try ssh svn-jira@localhost. If you get a password prompt, then your ssh server is not secure enough.
• Add this user to the list of allowed users in your sshd_config and restart your ssh daemon
• Create a ssh key for that user. E.g. use ssh-keygen -t dsa -b 2048. I didn’t put a password.
  You should end up with:

  ....
  Your identification has been saved in /home/svn-jira/.ssh/id_dsa.
  Your public key has been saved in /home/svn-jira/.ssh/id_dsa.pub
  ...
  

• Now make sure this key will be accepted:

  cat /home/svn-jira/.ssh/id_dsa.pub >> /home/svn-jira/.ssh/authorized_keys2
  chmod 600 /home/svn-jira/.ssh/authorized_keys2
  

• Test connection. With ssh svn-jira@localhost, you should now login correctly. Check /var/log/auth.log
• Now let’s configure Jira-SVN plugin. We have to pass the appropriate JVM system property so that the private key is used. But the private key has to be in a readable location. In my case, I copied the private key in /etc/tomcat5/svn-jira_id_dsa and modified the tomcat5 start up script in /etc/tomcat5/tomcat5.conf, adding “-Djavasvn.ssh2.key=/etc/tomcat5/svn-jira_id_dsa” to the JAVA_OPTS definition. Make sure the files has the correct permission and restart tomcat5:

  > ls -la svn-jira_id_dsa
  -rw-r-----  1 root tomcat5 1196 Sep 22 13:15 svn-jira_id_dsa
  > /etc/init.d/tomcat5 restart
  

• Check again /var/log/auth.log to see what is going on. You should see something like:

  Sep 22 13:23:49 deesse sshd[3604]: Accepted publickey for svn-jira from ::ffff:192.168.1.9 port 52857 ssh2
  

• If you have any issue add the following lines to jira/WEB-INF/classes/log4j.properties:

  log4j.category.com.atlassian.jira.plugin.ext.subversion = DEBUG, console
  log4j.category.com.atlassian.jira.plugin.ext.subversion.additivity = false
  

• If your issues are indexed, but the subversion tab won’t appear, check your Jira project scheme permissions, they might miss the jira-developpers user associated to the View Version Control permission

You’re done!

Now what I would like to do is restrict the operations the svn-jira can do using its newly access. Just in Case. Anyone knows how to do that? One should usually modify the authorized_keys2 file, and add restrictions in front of the key, but I don’t know what to add for a javasvn/jsch ssh backend.

(Too bad Jsch doesn’t have internal logging. It would help. Strange they don’t have one as they thank openssh for having a -v -v -v command line option.)

(*) shameless link to my personal blog.