Developer branches

Note: If you have been granted workspace on the server, you will have read and electronically signed the Open Source Contributor License found at bugs.digium.com (upon signing in) and have been given an access certificate.

The SVN repositories contain a number of branches, and development team members can have their own public and private working branches.

Using svnmerge for cross-branch merging

Note: For the below purposes the 1.2, 1.4, trunk, and 1.6.0 branches are used as examples.

Note: If you encounter issues please ensure you have the most recent version of svnmerge from the repotools repository.

The svnmerge script is used for handling cross-branch merging, since it makes tracking what is available to merge easy and simple to handle. Download the script and place it somewhere on your PATH.

If you do a 'svn pl -v' while you are located in a checked out branch or trunk, you will see all the properties currently attached to the root directory of the branch or trunk. For instance, on a checked out copy of the trunk, you will see something like this:

svnmerge-blocked : /branches/1.2:47648
branch-1.4-blocked : /branches/1.4:43484,43510,43582,43626,43703,..and so on...,47856,47892
branch-1.4-merged : /branches/1.4:1-43376,43383,43386,43388,43392,...and so on...,47959,47992

and on the 1.4 branch, you will see these sort of properties:

svnmerge-blocked : /branches/1.2:47648
branch-1.2-blocked : /branches/1.2:7490,7497,7517,7529,7546,...and so on...,,47855,47910
branch-1.2-merged : /branches/1.2:1-7489,7491-7496,7498-7516,...and so on...,47958,47968

In the old days, one person could use these properties via svnmerge (if they were properly updated) to quickly merge all candidate patches into a working dir in this manner:

$ cd ~/trunk-working-copy
$ svn update
$ svnmerge avail -P branch-1.2-merged -B branch-1.2-blocked --log

(This command would provide a list of all the candidate revisions. The available revisions, and the changes they make, could be reviewed. If any are not acceptable, either 'svnmerge block -P branch-1.2-merged -B branch-1.2-blocked -r ' them permanently or a list of the revisions numbers to be merged could be made and then supplied to the merge command as shown below.)

$ svnmerge merge -P branch-1.2-merged -B branch-1.2-blocked -f\
          ../merge-commit.msg [--revision {revision list}]
$ make

This command would generate the ../merge-commit.msg file, so a nice description file could be generated to use for committing. Another any other checking/testing was performed to ensure that the merge went successfully and everything worked, and then the following was performed:

$ svn commit -F ../merge-commit.msg

After this is done, the requested revisions will have been merged, their log messages will have been combined into a single log message for the 'merge commit', and the custom properties on the trunk were updated to reflect the list of revisions that have been merged from the 1.2 branch.

The way svnmerge has been used has changed from release to release. "Back in the day", all the fixes applied to 1.2 would be accumulated and applicable fixes would be folded back into 1.0. As Asterisk has grown, this is a hard job for one person to perform. Now, each developer is expected to make sure that each fix they develop gets applied to all supported releases. The best way to do this is to apply the fix to the oldest applicable release, and then use svnmerge to help you update the newer branches or trunk. This process will keep the 'merged' and 'blocked' properties up to date and accurate. For instance, if you apply a fix to 1.2, you will then use svnmerge to apply that revision to the 1.4 branch, and once that is committed, you will then use svnmerge to apply that 1.4 revision to trunk. Even if the changes are radical enough, that conflicts result from the use of svnmerge, at least you have the 'merged' revision numbers updated... you can revert any messed up files and apply a more specific patch to make things work. If you do not choose to use svnmerge to apply a specific revision from an older tree, then you must at least use the svnmerge block command (outlined below) to add the revision number to the 'blocked' property.

The svnmerge script will prompt for your certificate passphrase quite a few times (possibly many, many times) during its operations. One option is to use this 'expect'-based script which allows you to temporarily cache your password while svnmerge runs. Another is to set the configuration variables in your ~/.subversion/servers file:

ssl-client-cert-file = /path-to-your-digium-supplied-file/username-cert.p12
ssl-client-cert-password = mypword
ssl-authority-files = /path-to-your-digium-supplied-cacert-file/Digium_SVN-cacert.pem

At this current moment in time, no one is mass-folding changes into the branches for production purposes. But for future planned usage, all developers are expected to use svnmerge as outlined.

And, also all developers with commit privileges are expected to do their best to avoid committing the following properties into the trunk, 1.4, and 1.2 branches:

• svnmerge-integrated
• automerge
• automerge-email

Creating and maintaining developer branches

Developer branches are stored in the /team/{name} directory of each project repository (and /team/{name}/private for private branches). To create a branch and prepare it for future merge tracking of the branch you created it from (this example will create a branch from Asterisk 1.2):

$ svn copy https://origsvn.digium.com/svn/asterisk/branches/1.2\
      https://origsvn.digium.com/svn/asterisk/team/jdoe/my-fun-branch
$ svn checkout https://origsvn.digium.com/svn/asterisk/team/jdoe/my-fun-branch
$ cd my-fun-branch
$ svnmerge init

(svnmerge will setup a custom property on your branch to indicate where it was copied from and which revisions it already includes)

$ svn commit

(commit the custom property changes)

To make a developer branch copied from the trunk (or any other trunk-based branch)

$ svn copy https://origsvn.digium.com/svn/asterisk/trunk\
      https://origsvn.digium.com/svn/asterisk/team/jdoe/my-fun-branch
$ svn checkout https://origsvn.digium.com/svn/asterisk/team/jdoe/my-fun-branch
$ cd my-fun-branch
$ svnmerge init

(svnmerge will setup a custom property on your branch to indicate where it was copied from and which revisions it already includes)

$ svn commit

(commit the custom property changes)

When you want to update the branch to reflect what has happened in the source branch

Remember that your branch is just a checked out copy of the branch you created on the server. It does not normally get updated on its own unless you use the 'automerge' properties (see near the bottom of this document). If you do have the automerge property attached to the root dir of your branch back on the server, then changes made to the associated trunk or branch repositories will be merged into your repository on a regular basis (the equivalent of an 'svn update' and 'svn commit' on the server). You pull these changes down into your working copy via a simple 'svn update'.

But, if you did not attach the 'automerge

(before doing this, make sure any changes you have made to your branch are committed back into the repository. Otherwise, svnmerge will report that ". has local modifications; it must be clean" and abort.)

$ cd my-fun-branch
$ svnmerge avail --log
$ svnmerge merge -f ../merge-commit.msg
$ make (etc.)
$ svn commit -F ../merge-commit.msg

Merging and Blocking Procedures

Note: It is extremely important that you have an up to date svnmerge from the repotools directory when using the below commands. If you do not some may fail to work.

Any issues should be fixed in the farthest back branch currently being maintained. As of this writing this is the 1.4 branch. The fix should then be merged up to the last branch it is applicable to. If a fix is not applicable to the branch it should be blocked instead.

An example would be an issue only applicable to the 1.4 branch. The fix would be put into the 1.4 branch and blocked from trunk, and then from 1.6.0.

When you want to merge a 1.2 branch change into the 1.4 branch

Note: the use of the svnmerge avail -P command at the top of this page would pull ALL candidate changes from 1.2 into your current trunk working copy. You
usually don't want to mess with what could be other people's changes.

To pull just one set of changes into your trunk working copy:

First, when committing your changes into the 1.2 branch, write down the revision number associated with your set of changes. Then, while in a checked out workdir of the 1.4 branch:

$ merge24 <revnum>

(Now, resolve any conflicts, make sure everything not only compiles, but also works right...)

$ svn commit -F ../merge.msg

When you want to merge a 1.4 branch change into trunk

Note: the use of the svnmerge avail -P command at the top of this page would pull ALL candidate changes from 1.4 into your current trunk working copy. You
usually don't want to mess with what could be other people's changes.

To pull just one set of changes into your trunk working copy:

First, when committing your changes into the 1.4 branch, write down the revision number associated with your set of changes. While in a trunk working copy:

$ merge4trunk <revnum>

(Now, resolve any conflicts, make sure everything not only compiles, but also works right...)

$ svn commit -F ../merge.msg

When you want to block a 1.4 branch change from trunk

To block just one set of changes from your trunk working copy:

First, when committing your changes into the 1.4 branch, write down the revision number associated with your set of changes. While in a trunk working copy:

$ block4trunk <revnum>

$ svn commit -F ../merge.msg

When you want to merge a trunk change into 1.6.0

Note: the use of the svnmerge avail -P command at the top of this page would pull ALL candidate changes from trunk into your current 1.6.0 working copy. You
usually don't want to mess with what could be other people's changes.

To pull just one set of changes into your trunk working copy:

First, when committing your changes into trunk, write down the revision number associated with your set of changes. While in a 1.6.0 working copy:

$ mergetrunk6 <revnum>

(Now, resolve any conflicts, make sure everything not only compiles, but also works right...)

$ svn commit -F ../merge.msg

When you want to block a trunk change from 1.6.0

To block just one set of changes from your 1.6.0 working copy:

First, when committing your changes into trunk, write down the revision number associated with your set of changes. While in a 1.6.0 working copy:

$ blocktrunk6 <revnum>

$ svn commit -F ../merge.msg

When you backported a fix from trunk to 1.4

Sometimes you commit a fix to trunk, and it's decided that this fix should also go into 1.4. Once committed to 1.4 instead of blocking this commit in trunk you should record it:

$ record4trunk <revnum>

When you want to create Private Branches

Private branches are branches that only you and the repository managers can create, explore, modify, and access.

You can create a private branch, much like any other branch, by creating a branch in your team/jdoe/private directory:

$ svn copy https://origsvn.digium.com/svn/asterisk/trunk\
           https://origsvn.digium.com/svn/asterisk/team/jdoe/private/my-fun-branch
$ svn checkout https://origsvn.digium.com/svn/asterisk/team/jdoe/private/my-fun-branch

You can grant others access to your private branch, if you send russell@digium.com a request for such.

It's also important to note that all work placed into developer branches for the Asterisk, Asterisk-Addons, Zaptel and Libpri projects is covered by your contributer license agreement, regardless of whether you ever offer it for inclusion into the project's main trunk or not. In addition, the repository managers have a private mailing list where see all your commits to private branches.

When you want to create Group Branches

Group branches are branches that everyone can see and explore, but only those with commit privileges can modify. Group branches are for medium-/long-term projects involving multiple developers. They are stored in the team/group area in each repository.

You can create a group branch, much like any other branch, by creating a branch in the team/group directory:

$ svn copy https://origsvn.digium.com/svn/asterisk/trunk\
      https://origsvn.digium.com/svn/asterisk/team/group/my-fun-branch
$ svn checkout https://origsvn.digium.com/svn/asterisk/team/group/my-fun-branch

Merging a branch back into the trunk

Reminder: If your branch contains new functionality please make sure you have made the proper modifications to CHANGES or UPGRADE.txt

If a developer has a branch that is ready to be merged back into the trunk, here is the process:

$ svn co https://origsvn.digium.com/svn/asterisk/trunk trunk
$ cd trunk
$ svn merge --ignore-ancestry https://origsvn.digium.com/svn/asterisk/trunk\
           https://origsvn.digium.com/svn/asterisk/team/jdoe/bug12345 .

(Note that this is NOT using the svnmerge script; this is just a normal SVN merge).

Once this is done, the working copy will contain the trunk plus the changes from the developer branch. But, if you follow the above instructions for creating branches, you have probably introduced properties to the root of the branch that will not be healthy for the trunk SVN repository.

To restore the properties as they normally should be in the trunk repository, issue the command:

$ svn revert .

If you are purposely introducing new properties, or purposely introducing new values for existing properties, then you might do the following instead, so as not to destroy your properties:

$ svn pd svnmerge-integrated .
$ svn pd automerge .
$ svn pd automerge-email .

If everything merged cleanly, you can test compile and then:

$ svn commit -m "commit message for bug 6754"

If not, 'svn revert -R.' will revert all the changes that the merge brought over.

Once the contents of your branch has been merged, please use 'svn remove' to remove it from the repository. It will still be accessible if needed by looking back in the repository history, but removing it from the current view will make the automerge and other processes run more efficiently.

Viewing cross-branch diffs

If you would like to see the differences between two branches, you can do it without even having any working copies. The following command shows the diff between a developer's team branch and the trunk:

$ svn diff http://svn.digium.com/svn/asterisk/trunk\
      http://svn.digium.com/svn/asterisk/team/jdoe/my-fun-branch

Automatic updating of developer branches

Branches you create in your developer area can automatically track the branch they are based on. To do so, follow these steps in a working copy (checkout) of your developer branch:

$ svn propset automerge '*' .

(set a property to enable automerging, the value can be anything you like)

$ svn commit

(commit the property changes)

From that point on, once per hour the Subversion server will check your branch to see if any updates need to be merged, and merge them if possible.

If conflicts occur, the merge will not be done, but the 'automerge' property will be removed from your branch to stop any future automerge attempts until you correct the situation. If you wish to receive an email when this occurs, add a property called 'automerge-email' with your desired email address to your branch, and you will get the 'svn status' output when the merge conflicts occur.

Automerge - What is, and is not Automerged

An automerge script is run on the SVN repositories on a regular basis (see above). All this script does is basically run an 'svn update', and 'svn commit' on any branches that have the 'automergesvnmerge avail' and 'svnmerge merge' commands outlined above.

A fix you commit to the 1.4 branch will NOT be automatically propagated to the trunk. You must, by hand, merge your changes yourself.

When things go wrong

Sometimes, despite our best efforts, you'll encounter error messages:

1. 'svnmerge: multiple heads found. Explicit head argument (-S/--head) required.'
   This means that you have more than one item in your svnmerge-integrated property. Possible cause: someone messed up and committed the svnmerge-integrated property into trunk, 1.2, or 1.4. Check it out.
2. 'svn: Aborting commit: '/home/murf/asterisk/bug8386-1.4' remains in conflict'
   check the dir for conflicts; also, check for the file 'dir_conflicts.prej' as there may be conflicts with the properties attached to '.'. Most likely, there was a little trouble with the svnmerge-integrated property if dir_conflicts.prej exists.
3. When you do the 'svn commit -F ../merge-commit.msg', it says you are out of date, or to restart the commit. Usually, all you have to do is 'svn update', and then the 'svn commit -F....'; just doing the commit again usually does not help. There must be some good explanation of why this happens, and why svn update solves it, but I cannot give it.