Agile git Workflow
When we started using git to manage our source code at work, we actually jumped in a little bit too fast. It seems like there is a lot of writing about how you can do lots of really neat things with git, but no real guide about one particular way of using git for your project. This post is going to describe how we use git day to day on a reasonably large “agile-style” project.
Overview
The general overview of how this works is:
- Branch off of master to work on the feature
- Work, work, work, commit, commit, commit
- Pull down any updates to master
- Rebase and squash the feature branch to master’s HEAD
- Merge the feature branch back to master
- Push up to the shared repository
Also in this post are details about using release-candidate and production branches and merging features across those branches.
Working on a feature
Let’s imagine that you need to work on Issue #12. The first thing you’d do is create a feature branch from master for the issue. This keeps your work isolated, so that you can switch what you’re working on very quickly should the need arise.
(master)jj@im-jj:~/demo$ git checkout -b issue-12
Switched to a new branch 'issue-12'
(issue-12)jj@im-jj:~/demo$
Now that you’re on the feature branch, you might do all sorts of work towards completing this issue:
(issue-12)jj@im-jj:~/demo$ pico feature.py
(issue-12)jj@im-jj:~/demo$ git add feature.py
(issue-12)jj@im-jj:~/demo$ git commit -m "Added feature"
[issue-12 765a6c0] Added feature
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 feature.py
(issue-12)jj@im-jj:~/demo$ git rm README.txt
rm 'README.txt'
(issue-12)jj@im-jj:~/demo$ git commit -m "Removed the README file"
[issue-12 e7829cb] Removed the README file
0 files changed, 0 insertions(+), 0 deletions(-)
delete mode 100644 README.txt
(issue-12)jj@im-jj:~/demo$ pico testfile.txt
(issue-12)jj@im-jj:~/demo$ git add testfile.txt
(issue-12)jj@im-jj:~/demo$ git commit -m "Added testfile.txt"
[issue-12 f9753df] Added testfile.txt
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 testfile.txt
(issue-12)jj@im-jj:~/demo$ pico feature.py
(issue-12)jj@im-jj:~/demo$ git add feature.py
(issue-12)jj@im-jj:~/demo$ git commit -m "Updated feature"
[issue-12 4e39fc7] Updated feature
1 files changed, 3 insertions(+), 1 deletions(-)
Getting feedback on your feature
If you want to share all that work with someone else as a patch (maybe for code review or something), thats is pretty simple:
(issue-12)jj@im-jj:~/demo$ git diff master
This is really saying, "compare what’s currently on my branch with what’s on master and print it to standard out.
You can redirect the output of that to a file (ie, git diff master >
issue12.diff) if you need.
Packaging up your work for the shared repository
When you’re ready to share this feature with the rest of your team, they probably won’t care about all your interim commits (for example, you added and then changed feature.py). We can package up all the commits into one big commit (sort of like the one big patch against master you used before) by using git’s rebase command.
(issue-12)jj@im-jj:~/demo$ git rebase -i master
The -i flag is “interactive” which allows you to specify what you
want git to do with each of those interim commits. It should present a text
editor that looks like this:
pick 765a6c0 Added feature
pick e7829cb Removed the README file
pick f9753df Added testfile.txt
pick 4e39fc7 Updated feature
# Rebase 309291d..4e39fc7 onto 309291d
#
# Commands:
# p, pick = use commit
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#
To pack all of these commits into a single commit, change all but the top one to
s or squash. This says “keep the change from commit, but
roll it into the parent commit”. The default as you can see is “pick” which if left
alone would leave the commits separate. If for some reason you want to “undo” a
commit that you already had, you can remove that line entirely. This makes will
remove entirely the changes introduced in that particular commit.
To “package” up the branch into a single commit, our editor would look like this:
pick 765a6c0 Added feature
s e7829cb Removed the README file
s f9753df Added testfile.txt
s 4e39fc7 Updated feature
# Rebase 309291d..4e39fc7 onto 309291d
#
# Commands:
# p, pick = use commit
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#
Once you save that file, you’ll be prompted for a message to go along with the new “re-packaged” commit. Usually it’s best to start fresh from there and go with something about the feature, the prompt I got was:
# This is a combination of 4 commits.
# The first commit's message is:
Added feature
# This is the 2nd commit message:
Removed the README file
# This is the 3rd commit message:
Added testfile.txt
# This is the 4th commit message:
Updated feature
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# Not currently on any branch.
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# deleted: README.txt
# new file: feature.py
# new file: testfile.txt
#
Which you could change to:
Issue 12 - Added new feature XYZ
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# Not currently on any branch.
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# deleted: README.txt
# new file: feature.py
# new file: testfile.txt
#
Keeping the shared repository linear
At this point, the “issue-12” branch consists of one single commit. However, if somebody else has pushed any changes to the shared repository, merging in the feature branch will have two commits: one for the feature itself, and another to merge the feature with whatever else was pushed by the rest of the team.
All those “merged my local branch” commits will look like a lot of noise for no good reason, so git allows you to “replay” your commits against the master repository, with the end goal being to make your “re-packaged” commit to appear as though you branched right off the most updated version of the code.
This is sometimes referred to as “changing history” because you are updating the parent of your current commit to be further along in the future than it actually was.
To illustrate how this might work, you can make a small commit on master, and then use rebase to replay the commit against the updated master repository.
(issue-12)jj@im-jj:~/demo$ git checkout master
Switched to branch 'master'
(master)jj@im-jj:~/demo$ pico fixed_typo.txt
(master)jj@im-jj:~/demo$ git add fixed_typo.txt
(master)jj@im-jj:~/demo$ git commit -m "Added fixed typo"
[master c348893] Added fixed typo
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 fixed_typo.txt
(master)jj@im-jj:~/demo$
Now master has one extra commit, and issue-12 has one extra commit, where each has the same parent. If you were to merge issue-12 into master you’d have the two commits we mentioned before (one for the “re-packaged” feature as a single commit, and another for the recursive merge). To get rid of that second commit that really doesn’t provide much information to the team, we’ll rebase the commit to look like it happened after the typo-fixing commit we just threw into master. This also will give us the chance to resolve any conflicts.
This is one representation before we change the parent:
(master)jj@im-jj:~/demo$ git checkout issue-12
Switched to branch 'issue-12'
(issue-12)jj@im-jj:~/demo$ git rebase master
First, rewinding head to replay your work on top of it...
Applying: Issue 12 - Added new feature XYZ
This is one representation after the rebase changed the parent:
Now we can merge the change back into master with a fast forward:
(issue-12)jj@im-jj:~/demo$ git checkout master
Switched to branch 'master'
Your branch is ahead of 'origin/master' by 1 commit. # <-- Always be 1 commit ahead.
(master)jj@im-jj:~/demo$ git merge issue-12
Updating c348893..d0e9912
Fast forward
feature.py | 3 +++
testfile.txt | 1 +
2 files changed, 4 insertions(+), 0 deletions(-)
delete mode 100644 README.txt
create mode 100644 feature.py
create mode 100644 testfile.txt
Now let’s double check that we can still fast-forward on the shared repository:
(master)jj@im-jj:~/demo$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 2 commits. # <-- 1 for issue-12, 1 for the typo-fix.
#
nothing to commit (working directory clean)
(master)jj@im-jj:~/demo$ git remote show origin
* remote origin
URL: my-server:/git/demo.git
HEAD branch: master
Remote branch:
master tracked
Local branch configured for 'git pull':
master merges with remote master
Local ref configured for 'git push':
master pushes to master (fast forwardable) # <-- Always be able to fast-forward.
Since we can fast forward, it’s just a git push away:
(master)jj@im-jj:~/demo$ git push origin
Counting objects: 8, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (7/7), 699 bytes, done.
Total 7 (delta 0), reused 0 (delta 0)
To my-server:/git/demo.git
309291d..d0e9912 HEAD -> master
And since we don’t need that issue branch anymore, we can delete it:
(master)jj@im-jj:~/demo$ git branch -d issue-12
Deleted branch issue-12 (was d0e9912).
What if other people push stuff in the meantime?
If somebody else happens to push some code in the time between your rebase and
your git remote show origin (you’ll know this because the push
won’t be “fast forwardable”) you should first pull down any changes, and then
do another rebase. All this will do is (again) replay your commits ontop of the
new ones that got pushed so that the shared repository doesn’t have lots of
noise from merging your feature branch. Since you’ve already got the commit
packaged up the way you like, you can leave off the -i and just do:
(issue-12)jj@im-jj:~/demo$ git rebase master
First, rewinding head to replay your work on top of it...
Applying: Issue 12 - Added new feature XYZ
The moral of the story is that if you want to keep your shared repository full of only the important stuff, make sure all your pushes are fast-forwards. This is as easy as always remembering to rebase immediately before you push.
Staging and production
Now that you have the concept of feature branches down pat, another thing that we struggled with was how to deal with branches that signified different versions of our code.
Ideally we’d like to have a “trunk” (aka master) and then a branch for the latest release-candidate (say “rc-1.0”) and then the official release branch (say “1.0”).
The basic idea is to treat the release-candidate branch just like master (re-packaging commits, feature branches, etc) and then the production branch as a “merge-only” or “cherry-pick-only” branch. Some people prefer to use tags for this, however using a tag means that anything currently in testing will hold up moving things into production. This is a choice you’ll have to make, a branch gives a bit more independence from the RC than a tag.
Creating the branch
It seems like the first place to start would be to create the RC branch:
(master)jj@im-jj:~/demo$ git checkout -b rc-1.0
Switched to a new branch 'rc-1.0'
(rc-1.0)jj@im-jj:~/demo$ git push origin rc-1.0
Total 0 (delta 0), reused 0 (delta 0)
To my-server:/git/demo.git
* [new branch] rc-1.0 -> rc-1.0
(rc-1.0)jj@im-jj:~/demo$ git remote show origin
* remote origin
URL: my-server:/git/demo.git
HEAD branch (remote HEAD is ambiguous, may be one of the following):
master
rc-1.0
Remote branches:
master tracked
rc-1.0 tracked
Local branch configured for 'git pull':
master merges with remote master
Local refs configured for 'git push':
master pushes to master (up to date)
rc-1.0 pushes to rc-1.0 (up to date)
Now you have a new release candidate branch that you can push changes to. Follow the same procedure discussed above to make sure that stays linear and everything should work out just fine.
This is what you’re RC branch might look like:
What if somebody else already created the branch?
Maybe someone else on your team already created the branch and you just need to
check it out locally. This is pretty simple in git. The format is git
checkout -b mybranch -t origin/mybranch. For example, if someone already
created the rc-1.0 branch, we could pull it down like this:
(master)jj@im-jj:~/demo$ git checkout -b rc-1.0 -t origin/rc-1.0
Branch rc-1.0 set up to track remote branch rc-1.0 from origin.
Switched to a new branch 'rc-1.0'
(rc-1.0)jj@im-jj:~/demo$ git remote show origin
* remote origin
URL: my-server:/git/demo.git
HEAD branch: master
Remote branches:
master tracked
rc-1.0 tracked
Local branches configured for 'git pull':
master merges with remote master
rc-1.0 merges with remote rc-1.0
Local refs configured for 'git push':
master pushes to master (up to date)
rc-1.0 pushes to rc-1.0 (up to date)
Merging between branches
If everything is going fine, there will come a point when it’s time to merge all
the changes you made on your RC branch back to the master branch. This is nothing
more than a git merge. (Note that this is one place where you won’t
be fast-forwarding, but always should have a merge commit to show that you merged
the RC branch back to master.)
(rc-1.0)jj@im-jj:~/demo$ git checkout master
Switched to branch 'master'
(master)jj@im-jj:~/demo$ git merge rc-1.0
Merge made by recursive.
mynewfile.txt | 1 +
mynewfile2.txt | 1 +
2 files changed, 2 insertions(+), 0 deletions(-)
create mode 100644 mynewfile.txt
create mode 100644 mynewfile2.txt
(master)jj@im-jj:~/demo$ git push
Counting objects: 4, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 331 bytes, done.
Total 2 (delta 1), reused 0 (delta 0)
To my-server:/git/demo.git
af218a9..b4c7ac5 HEAD -> master
After things are merged back, your repository will look something like this:
Ready to release
When your release candidate has made it through all the testing it needs, you should be ready to create a production branch. This is just like the previous example where all you need to do is create the remote branch off of the RC branch.
(master)jj@im-jj:~/demo$ git checkout rc-1.0
Switched to branch 'rc-1.0'
(rc-1.0)jj@im-jj:~/demo$ git checkout -b 1.0
Switched to a new branch '1.0'
(1.0)jj@im-jj:~/demo$ git push origin 1.0
Total 0 (delta 0), reused 0 (delta 0)
To my-server:/git/demo.git
* [new branch] 1.0 -> 1.0
Usually this branch will stay up to date with the RC branch, and as bugs are fixed in the RC they are just merged over into the production branch.
(1.0)jj@im-jj:~/demo$ git checkout rc-1.0
Switched to branch 'rc-1.0'
(rc-1.0)jj@im-jj:~/demo$ echo 'print "fixing a bug"' >> feature.py
(rc-1.0)jj@im-jj:~/demo$ git add -A && git commit -m "Added bug fix"
[rc-1.0 52a6d49] Added bug fix
1 files changed, 1 insertions(+), 0 deletions(-)
(rc-1.0)jj@im-jj:~/demo$ git push
Counting objects: 5, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 338 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
To my-server:/git/demo.git
97a0f2a..52a6d49 HEAD -> rc-1.0
And once somebody verifies that the bug is then fixed:
(rc-1.0)jj@im-jj:~/demo$ git checkout 1.0
Switched to branch '1.0'
(1.0)jj@im-jj:~/demo$ git merge rc-1.0
Updating 97a0f2a..52a6d49
Fast forward
feature.py | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
(1.0)jj@im-jj:~/demo$ git push
Total 0 (delta 0), reused 0 (delta 0)
To my-server:/git/demo.git
97a0f2a..52a6d49 HEAD -> 1.0
The same thing applies when you want to pull back this new change into master:
(1.0)jj@im-jj:~/demo$ git checkout master
Switched to branch 'master'
(master)jj@im-jj:~/demo$ git merge rc-1.0
Merge made by recursive.
feature.py | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
(master)jj@im-jj:~/demo$ git push
Counting objects: 4, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 285 bytes, done.
Total 2 (delta 1), reused 0 (delta 0)
To my-server:/git/demo.git
b4c7ac5..4834481 HEAD -> master
What if I just want one commit?
There may be times when the production branch needs an important fix, but there are other features on the branch that haven’t been adequately tested. This problem is pretty well solved by “cherry-picking”. Merging won’t work because a merge carries with it all the parent commits until a common ancestor, which you definitely don’t want if those parent commits haven’t been tested.
The basic idea cherry-picking is that git will take the change-set, package it up as a different commit, and move it over to another branch, without taking anything else.
Let’s make two small bug fixes on rc-1.0, and then take the last one over with a cherry pick.
(master)jj@im-jj:~/demo$ git checkout rc-1.0
Switched to branch 'rc-1.0'
(rc-1.0)jj@im-jj:~/demo$ echo '# bugfix1!' >> feature.py
(rc-1.0)jj@im-jj:~/demo$ git add -A && git commit -m "Added bugfix number 1"
[rc-1.0 92ab376] Added bugfix number 1
1 files changed, 1 insertions(+), 0 deletions(-)
(rc-1.0)jj@im-jj:~/demo$ echo '# bugfix2!' >> feature2.py
(rc-1.0)jj@im-jj:~/demo$ git add -A && git commit -m "Added bugfix number 2"
[rc-1.0 1dec2d3] Added bugfix number 2
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 feature2.py
(rc-1.0)jj@im-jj:~/demo$ git push
Counting objects: 8, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 629 bytes, done.
Total 6 (delta 3), reused 0 (delta 0)
To my-server:/git/demo.git
52a6d49..1dec2d3 HEAD -> rc-1.0
If we just wanted to merge over bug fix number 2, the merge would also carry bug fix number 1 along with it since that’s the common parent. We haven’t tested bug fix number 1, so we’ll need to cherry pick number 2 over onto the 1.0 branch by itself.
(rc-1.0)jj@im-jj:~/demo$ git log --pretty=oneline --abbrev-commit
1dec2d3 Added bugfix number 2 # <-- This is the commit we want, note the short-hash.
92ab376 Added bugfix number 1
52a6d49 Added bug fix
97a0f2a Minor commit
099214d Minor commit
d0e9912 Issue 12 - Added new feature XYZ
c348893 Added fixed typo
309291d Initial Commit
(rc-1.0)jj@im-jj:~/demo$ git checkout 1.0
Switched to branch '1.0'
(1.0)jj@im-jj:~/demo$ git cherry-pick 1dec2d3
Finished one cherry-pick.
[1.0 8e3a5f9] Added bugfix number 2
1 files changed, 1 insertions(+), 0 deletions(-)
create mode 100644 feature2.py
(1.0)jj@im-jj:~/demo$ git push
Counting objects: 4, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 289 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
To my-server:/git/demo.git
52a6d49..8e3a5f9 HEAD -> 1.0
You now have the change-set from bug fix number 2, but it’s technically a fully different commit (with its own commit hash and everything). This allows you to move things over from the RC branch independently of other things in the branch. Also, when you do decide to merge over the rest of the RC branch, that shouldn’t be a problem at all since I guess git knows that the new commit has the same content and just a different parent.
Help, I broke things and want to start over
Another common thing I find myself doing all the time is having to “try again”
after I accidentally merge over the wrong commit, or make a change that just
isn’t right. This is where the git reset command comes in handy.
If you just committed before you were ready, the standard git reset HEAD~1
will work just fine. This is git-speak for “rewind things by 1 commit, but don’t
change the contents of the files on disk”.
If you do want to undo any changes to files locally, you can use the --hard
flag to change that as well: git reset --hard HEAD~1.
The HEAD~1 is notation for the head of the current branch minus 1
commit. So HEAD~4 would mean “four commits ago”.
You can also point out a specific commit by it’s short hash to rewind to that
point in time. And remember the --hard flag will bring you back to
that point with the exact code from then as well.
(1.0)jj@im-jj:~/demo$ git log --abbrev-commit --pretty=oneline
8e3a5f9 Added bugfix number 2
52a6d49 Added bug fix
97a0f2a Minor commit
099214d Minor commit
d0e9912 Issue 12 - Added new feature XYZ
c348893 Added fixed typo
309291d Initial Commit
(1.0)jj@im-jj:~/demo$ git reset --hard 52a6d49
HEAD is now at 52a6d49 Added bug fix
(1.0)jj@im-jj:~/demo$ git log --abbrev-commit --pretty=oneline
52a6d49 Added bug fix
97a0f2a Minor commit
099214d Minor commit
d0e9912 Issue 12 - Added new feature XYZ
c348893 Added fixed typo
309291d Initial Commit
I accidentally started my work on master…
Sometimes I’ll make a couple of small changes on master and one thing leads to another and all the sudden I realize I’ve got a lot of changes right on master instead of a feature branch. Let’s get into that situation real quick:
(rc-1.0)jj@im-jj:~/demo$ git checkout master
Switched to branch 'master'
(master)jj@im-jj:~/demo$ touch myothernewfile.txt
(master)jj@im-jj:~/demo$ git add myothernewfile.txt
(master)jj@im-jj:~/demo$ git commit -m "Added another new file"
[master 617f46d] Added another new file
0 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 myothernewfile.txt
(master)jj@im-jj:~/demo$ touch onemorefile.txt
(master)jj@im-jj:~/demo$ git add onemorefile.txt
(master)jj@im-jj:~/demo$ git commit -m "Added one more new file"
[master 23290ed] Added one more new file
0 files changed, 0 insertions(+), 0 deletions(-)
create mode 100644 onemorefile.txt
OK, now we are on master, with two commits that we accidentally did right on master instead of a feature branch. The way we’ll get these onto a feature branch, and get master back to normal is with by branching and resetting.
First, we’ll create a branch from master which will have those two commits, then we’ll reset our master branch back to before we accidentally made those commits.
(master)jj@im-jj:~/demo$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 2 commits.
#
nothing to commit (working directory clean)
(master)jj@im-jj:~/demo$ git branch feature1
(master)jj@im-jj:~/demo$ git reset --hard origin/master
HEAD is now at 4834481 Merge branch 'rc-1.0'
(master)jj@im-jj:~/demo$ git status
# On branch master
nothing to commit (working directory clean)
This keeps your changes off on the “feature1” branch, and then moves the “master” pointer back to the master pointer on the shared repository. If you checkout the feature1 branch, you’ll see that the changes you made originally on master are in place over there, and you can easily merge them back into master when the time comes to commit them.
That’s all
That’s all I have for now. If there’s anything missing that you’d like to see feel free to e-mail me at jj@(this website).
Extra thanks to Mark Chadwick for being the inspiration behind this post. Also if you’re going to be working with lots of git repositories and lots of different branches, see my previous post (http://geewax.org/2009/11/15/git-workspace-magic.html) for how to make your PS1 update based on what branch you’re currently on.
