Git - Submodules

It often happens that while working on one project, you need to use another project from within it. Perhaps it’s a library that a third party developed or that you’re developing separately and using in multiple parent projects. A common issue arises in these scenarios: you want to be able to treat the two projects as separate yet still be able to use one from within the other.

Git addresses this issue using submodules. Submodules allow you to keep a Git repository as a subdirectory of another Git repository. This lets you clone another repository into your project and keep your commits separate.

One good example of where this is appropriate is the JSON java libary located at https://github.com/stleary/JSON-java which is just the Java source files and not even arranged in a project structure.

To cope with this I used git-sub-modules and built a maven project around it.

Adding the sub-module

To add the sub-module I first created my maven project as normal.
Then I created the folder /src/main/java/org/json to contain my link to the other project.
Adding the submodule is then as easy as opening a command prompt on that folder and issuing this command:

git submodule add https://github.com/stleary/JSON-java.git

You will now have a .gitmodules file in your project.
Add it to your source control to record what you have linked to.
As easy as that!

What are your sub-modules

Later you may not recall where your sub-modules are so from the project route issue this command:

git submodule status

This will tell you the folders with sub-modules.

Getting maven to play along!

It is a good idea to ensure that you are using the correct code and that you have any changes in place when building.
When you update the project your git client may ask & check sub-modules ... it may not.
So adding the following to your POM.xml ensures that it is added.

  <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>exec-maven-plugin</artifactId>
         <version>1.2.1</version>
         <executions>
             <execution>
                 <phase>initialize</phase>
                  <id>git-submodules-init-update</id>
                     <goals><goal>exec</goal></goals>
              </execution>
          </executions>
          <configuration>
              <executable>git</executable>
              <arguments>
                  <argument>submodule</argument>
                  <argument>update</argument>
                  <argument>--init</argument>
                  <argument>--recursive</argument>
              </arguments>
          </configuration>
   </plugin>

This causes Maven to execute an executable command.

In this case the sub-module init command which runs maven and initializes the content based on the project's  .gitmodules file.

And that is all it takes!

Version Numbering - Redux

The issue

I have been looking at version numbering for a project where the developers had stuck at 0.0.1-SNAPSHOT for 12 months and were starting to encounter issues with getting the correct JARS for their projects from their binary repository.

The solution was to use the features that are present in Maven and Jenkins to assist them in their processes.

I covered the basics of version numbering in an other post (May 2012), but that just gives the version strategy for Apache projects. Since then I have come across Semantic Versioning which is a great a almost definitive source on how you should work with versions.
However this does not cater for the way Maven actually treats version numbers, it is just compatible with it..

A good version number has a number of properties:

• Natural order: it should be possible to determine at a glance between two versions which one is newer
• Maven support: Maven should be able to deal with the format of the version number to enforce the natural order
• Machine incrementable: so you don't have to specify it explicitly every time

What does Maven do?

For reference, Maven version numbers are comprised as follows: <MajorVersion>.<MinorVersion>.<IncrementalVersion>-<BuildNumber | Qualifier>. Where MajorVersion, MinorVersion, IncrementalVersion and BuildNumber are all numeric and Qualifier is a string. If your version number does not match this format, then the entire version number is treated as being the Qualifier. [See]
If all the version numbers are equal, the qualifier is compared alphabetically. "RC1" and "SNAPSHOT" and sorted no differently to "a" and "b". As a result, "SNAPSHOT" is considered newer because it is greater alphabetically. See this page as a reference.
The issue on may projects is how to manage the versions and how to do so with out breaking the Maven format which will cause Maven to treat your versions as just the Qualifier which is a string (not good). Note that a.b.c-RC1-SNAPSHOT would be considered older than a.b.c-RC1, because of text comparisons.

What to use as an Incremental version number?

I think it is reasonably straight forwards for a project to determine the major and minor version as they often come directly from the business drivers for the project.
The Incremental version can give some issues as the business may not be interested and it is the developers that need it for tracking purposes.
Therefore the incremental number has be be meaningful to them.
So usefull numbers could be the database schema version, the sprint number the feature set that is being implemented (although this can be hard if you have several teals working in parallel).

The simplest way is to start at 1 and for the team leads to determine when to increase the number.

Whether to use as a Qualifier or a BuildNumber?

There appear to be several schools of thought on this and Maven simply fits with them all.
The Apache method is to not have either and using the SNAPSHOT qualifier allows you to follow this pattern.
However, SNAPSHOT does not allow your developers to know what to use the version for.
JBoss has qualifiers (alpha[n], beta[n], release candidate 'CR[n]' and Final) with optional numbers. [See].
The OSGI specification adds a further complication as does the eclipse numbering which are
MajorVersion, MinorVersion, IncrementalVersion.TIMESTAMP[-Mn]
MajorVersion, MinorVersion, IncrementalVersion.CR[n]
MajorVersion, MinorVersion, IncrementalVersion.Final
There are only two qualifiers.  The first one is for the milestone releases, and the qualifier starts with a numeric timestamp.  The project can use a timestamp as shown below as it will sort according to the compareTo method of the String class just like any other qualifier. ie YYYYMMDD
Optionally, if for some reason there is a need to make two releases in the same day, you can add a sequence number to the end of the timestamp. The next part of the qualifier is the milestone number, where M stands for milestone, and n is the milestone number.
After all the milestone releases that have added the various functional pieces are complete, and the project and any sub-projects that are integrated are at least at a candidate release stage, then a CR release will follow.  Just like in the traditional model, there may be multiple CR releases depending on the feedback from the community.

It is the above approach minus the milestone number I would advise (assuming the 'IncrementalVersion' is controlled by the development team.

Setting up Maven

Making sure your version numbers are incremented can be a pain in the arse but there is  a plugin for Maven that helps you mange this.

The Maven 'Release' Plugin

The Release plugin [See] is helpful but not essential. It is used to help a developer release a project with Maven, saving a lot of repetitive, manual work. Its best usage is to allow the developer to update their version number with out effort and correctly.

It is added to the maven project as follows:

 <project>  
     ...  
     <build>  
         <plugins>  
             ...  
             <plugin>  
                 <groupId>org.apache.maven.plugins</groupId>  
                 <artifactId>maven-release-plugin</artifactId>  
                 <version>2.5.2</version>  
             </plugin>  
             ...  
         </plugins>  
         ...  
     </build>  
     ...  
 </project>  

easy!

This allows the developer to issue a command as follows:

 mvn -B release:update-versions  

... and the version will be updated to the next increment. They then only need to commit it as part of their code.
It is always the last part of the version number that is incremented and it even works if you have a text qualifier such as CRn (see above).

Maven Versions Plugin

This plugin is much more useful when it comes to controlling your release via your CI server. [See]
(In these examples I'm going to quote Jenkins but this process should work for others.)
(I am also not sure if Maven 3.1+ doesn't include this plugin.)

Unlike the previous plugin that increments the version, this will allow a specific version number to be set in the POM, like this:

 mvn versions:set -DnewVersion=0.1.1-RC1  

Where '0.1.1-RC1' is an example of a version number.

The process

Now to tie this together.
The process we want is:

1. Jenkins checks out the latest revision from SCM (Subversion, Mercurial, Git, ...)
2. Release Plugin transforms the POMs with the new version number
3. Maven compiles the sources and runs the tests
4. Release Plugin commits the new POMs into SCM
5. Maven publishes the binaries into the Artifact Repository

Prerequisites: Jenkins with a JDK and Maven configured, and both the Git and the Workspace Cleanup, and Parameterized Trigger Plugin  plugins installed.
We're going to start by creating a new Maven job and making sure we have a fresh workspace for every build:

After assigning you SCM, the next step is to set the version upon checkout.
A good version number is both unique and chronological. We're going to use the Jenkins BUILD_NUMBER (the current build number, such as "153") as it fulfills both these criteria wonderfully.
We could use BUILD_ID which is such as "2005-08-22_23-59-59" (YYYY-MM-DD_hh-mm-ss).
or even the Git commit number using GIT_REVISION.
This is configured as follows:

and in the build step:

And that's it! Every time this job is run, a new release is produced, the artifacts will be deployed and the source code will be tagged. The version of the release will be the BUILD_NUMBER (or GIT_REVISION) of the Jenkins project. Nice and simple.

Code School

Ok I've tried to get to grips with GIT ... finally! and I have been shown a realy  nice learning resource.

Code School!


Please take a look as it is quite fascinating ... http://try.github.io

Cool tip of the day: gource

I have the funny feeling that this is one of those tools that could be really useful if you could think of a sensible way to use it.
It produces an interesting visualisation of the changes occurring in your source control system.

It works with SVN & GIT but I have only seen it work so far with GIT and that was with a team of 3.
With that set up we were able to visualise the scope of work performed over the two week sprint as part of our sprint-retrospective.

Gource is a software version control visualization tool.

See more of Gource in action on the Videos page.

Version Numbering

So many times I hear developers arguing about how to label versions so here is the "right" way to do it ... well a widely accepted way!

Software should be released with a version number based on the Apache Portable Runtime Project version guidelines. In summary, the version is denoted as three integers in the format of MAJOR.MINOR.PATCH.

Major versions are incompatible at the API level, as they can include any kind of change.
Minor versions are compatible with older versions at the API and binary level, and they can introduce new functions or deprecate old ones.
Patch versions are perfectly compatible, and they are released to fix defects.

Here are some examples to demonstrate the compatibility:

Original Version	New Version	Compatible?
2.2.3	2.2.4	Yes Compatibility across patch versions is guaranteed.
2.2.3	2.2.1	Yes Compatibility across patch versions is guaranteed.
2.2.3	2.3.1	Yes Compatibility with later minor versions is guaranteed.
2.2.3	2.1.7	No Compatibility with prior minor versions is not guaranteed.
2.2.3	3.0.0	No Compatibility with different major versions is not guaranteed.
2.2.3	1.4.7	No Compatibility with different major versions is not guaranteed.

Note: while some of the cells say "no", it is possible that the versions may be compatible, depending very precisely upon the particular APIs used by the application.

Patch Versions

To retain perfect source and binary compatibility, a patch release can only change function implementations. Changes to the API, to the signatures of public functions, or to the interpretation of function parameters is not allowed. Effectively, these releases are pure bug fix releases.

Minor Versions

Minor releases can introduce new functions, new symbolic and enumerated constants, and deprecate existing functions. In practice, this will not be a problem since upgrading to newer versions is always safe.

Major Versions

Any kind of change can be made during a major version release. Particular types of changes that might occur:

• remove or change constants
• remove (deprecated) functions

So you might find that your code stops working completly.

Minor Versions

Minor Versions
Minor