So, you have an AVI file and a SRT file, and you want to put them together so that you don't have to carry around the two files... (and you're on Ubuntu...)
Since we were acquired by Google, I figured that it would be a fitting 20% project to work with some App Engine engineers to move GAE Testbed into the App Engine Python SDK. The benefits are pretty obvious:
Managing lots of different App Engine applications gets pretty tricky, so I threw together a little Makefile to make it easy to run the debug server, start the development console, deploy the application, etc.
In the early days of Invite Media we always talked about being acquired but I didn’t really think it would ever happen. Amazingly, back in June we were acquired by Google. (You can read about it in the official DoubleClick blog post.) We’re all still getting ourselves situated at Google — our NYC engineers are happily settling into Google’s NYCoffice and Google is setting up a new office in Center City for the Philly engineers — but for the most part the work is the same. I’m still trying to find my way around, but everybody I’ve met so far has been really helpful. Looks like I officially work for Google. Let the excitement begin…
We use Jira for a whole lot of things, but one of the major uses is for Bug reports. That said, sometimes people forget important information when filing a bug so I wanted to have the default description for Bug reports have some prompts for what information we’d need to reproduce the bugs (sort of like Google Code’s bug report interface).
Unfortunately, there’s no nifty UI to edit the default value for the description field in Jira, but there’s a niceKnowledge Base article which describes how you can do it by editing the template file used to render the description field. That file is: /path/to/atlassian-jira/WEB-INF/classes/templates/jira/issue/field/description-edit.vm
It wasn’t entirely clear how to do different things based on the issue type, etc so I had to poke around a bit to figure out what’s different, the section I wrote ended up as:
This makes the entire description-edit.vm file look like this:
Hopefully this helps someone else who wants to provide defaults for system fields like Description. In the meantime, please go vote on the feature request so that this blog post won’t be needed anymore.
Keeping track of scripts that run regularly and e-mail the output is pretty simple when only one person is managing the schedules, scripts, and server. When you throw many more people into the mix, keeping track of who wants to see the output of which jobs and when jobs should run, you end up with a nasty crontab, and lots of cooks in the kitchen.
I went looking around for a simple way to throw a GUI ontop of a crontab, and stumbled across the Chronograph Django application. It had most of the stuff that we needed (including some really advanced scheduling options thanks to the dateutil library) so I tried spinning it up. What came out was a really nice user interface where we could see just about everything we wanted.
There were a few tweaks I wanted to make for my own purposes, but one major thing that was missing was a list of e-mail subscribers. This was surprisingly simple to throw in, and works really well since Django does pretty much all of the work for me.
Anyway, I’m hoping these changes can get pulled back into the main repository on Google Code, but if not, I’ve got a fork going on GitHub for anybody that’s interested:
http://github.com/jgeewax/django-chronograph
You can install this via easy install:
$ easy_install http://github.com/downloads/jgeewax/django-chronograph/django-chronograph-0.1...
Here's what the fork looks like:
When we started using git to manage our source code at work, we actually jumped in a little bit too fast. It seems like there is a lot of writing about how you can do lots of really neat things with git, but no real guide about one particular way of using git for your project. This post is going to describe how we use git day to day on a reasonably large “agile-style” project.
The general overview of how this works is:
Also in this post are details about using release-candidate and production branches and merging features across those branches.
Let’s imagine that you need to work on Issue #12. The first thing you’d do is create a feature branch from master for the issue. This keeps your work isolated, so that you can switch what you’re working on very quickly should the need arise.
Now that you’re on the feature branch, you might do all sorts of work towards completing this issue:
If you want to share all that work with someone else as a patch (maybe for code review or something), thats is pretty simple:
This is really saying, "compare what’s currently on my branch with what’s on master and print it to standard out.
You can redirect the output of that to a file (ie, git diff master > issue12.diff) if you need.
When you’re ready to share this feature with the rest of your team, they probably won’t care about all your interim commits (for example, you added and then changed feature.py). We can package up all the commits into one big commit (sort of like the one big patch against master you used before) by using git’s rebase command.
The -i flag is “interactive” which allows you to specify what you want git to do with each of those interim commits. It should present a text editor that looks like this:
To pack all of these commits into a single commit, change all but the top one to s or squash. This says “keep the change from commit, but roll it into the parent commit”. The default as you can see is “pick” which if left alone would leave the commits separate. If for some reason you want to “undo” a commit that you already had, you can remove that line entirely. This makes will remove entirely the changes introduced in that particular commit.
Keep in mind that you might not want to package everything as a single commit, and anything that you want to keep separate, just leave that line as pick and it will be left alone. For the purposes of this article, we’re assuming that each branch is a completely isolated topic, where the intermediate commits are for your personal use, and not of any use to the rest of your team. This may not always be the case.
To “package” up the branch into a single commit, our editor would look like this:
Once you save that file, you’ll be prompted for a message to go along with the new “re-packaged” commit. Usually it’s best to start fresh from there and go with something about the feature, the prompt I got was:
Which you could change to:
At this point, the “issue-12” branch consists of one single commit. However, if somebody else has pushed any changes to the shared repository, merging in the feature branch will have two commits: one for the feature itself, and another to merge the feature with whatever else was pushed by the rest of the team.
All those “merged my local branch” commits will look like a lot of noise for no good reason, so git allows you to “replay” your commits against the master repository, with the end goal being to make your “re-packaged” commit to appear as though you branched right off the most updated version of the code.
This is sometimes referred to as “changing history” because you are updating the parent of your current commit to be further along in the future than it actually was.
To illustrate how this might work, you can make a small commit on master, and then use rebase to replay the commit against the updated master repository.
Now master has one extra commit, and issue-12 has one extra commit, where each has the same parent. If you were to merge issue-12 into master you’d have the two commits we mentioned before (one for the “re-packaged” feature as a single commit, and another for the recursive merge). To get rid of that second commit that really doesn’t provide much information to the team, we’ll rebase the commit to look like it happened after the typo-fixing commit we just threw into master. This also will give us the chance to resolve any conflicts.
This is one representation before we change the parent:
This is one representation after the rebase changed the parent:
Now we can merge the change back into master with a fast forward:
Now let’s double check that we can still fast-forward on the shared repository:
Since we can fast forward, it’s just a git push away:
And since we don’t need that issue branch anymore, we can delete it:
If somebody else happens to push some code in the time between your rebase and your git remote show origin (you’ll know this because the push won’t be “fast forwardable”) you should first pull down any changes, and then do another rebase. All this will do is (again) replay your commits ontop of the new ones that got pushed so that the shared repository doesn’t have lots of noise from merging your feature branch. Since you’ve already got the commit packaged up the way you like, you can leave off the -i and just do:
The moral of the story is that if you want to keep your shared repository full of only the important stuff, make sure all your pushes are fast-forwards. This is as easy as always remembering to rebase immediately before you push.
Now that you have the concept of feature branches down pat, another thing that we struggled with was how to deal with branches that signified different versions of our code.
Ideally we’d like to have a “trunk” (aka master) and then a branch for the latest release-candidate (say “rc-1.0”) and then the official release branch (say “1.0”).
The basic idea is to treat the release-candidate branch just like master (re-packaging commits, feature branches, etc) and then the production branch as a “merge-only” or “cherry-pick-only” branch. Some people prefer to use tags for this, however using a tag means that anything currently in testing will hold up moving things into production. This is a choice you’ll have to make, a branch gives a bit more independence from the RC than a tag.
It seems like the first place to start would be to create the RC branch:
Now you have a new release candidate branch that you can push changes to. Follow the same procedure discussed above to make sure that stays linear and everything should work out just fine.
This is what you’re RC branch might look like:
Maybe someone else on your team already created the branch and you just need to check it out locally. This is pretty simple in git. The format is git checkout -b mybranch -t origin/mybranch. For example, if someone already created the rc-1.0 branch, we could pull it down like this:
If everything is going fine, there will come a point when it’s time to merge all the changes you made on your RC branch back to the master branch. This is nothing more than a git merge. (Note that this is one place where you won’t be fast-forwarding, but always should have a merge commit to show that you merged the RC branch back to master.)
After things are merged back, your repository will look something like this:
When your release candidate has made it through all the testing it needs, you should be ready to create a production branch. This is just like the previous example where all you need to do is create the remote branch off of the RC branch.
Usually this branch will stay up to date with the RC branch, and as bugs are fixed in the RC they are just merged over into the production branch.
And once somebody verifies that the bug is then fixed:
The same thing applies when you want to pull back this new change into master:
There may be times when the production branch needs an important fix, but there are other features on the branch that haven’t been adequately tested. This problem is pretty well solved by “cherry-picking”. Merging won’t work because a merge carries with it all the parent commits until a common ancestor, which you definitely don’t want if those parent commits haven’t been tested.
The basic idea cherry-picking is that git will take the change-set, package it up as a different commit, and move it over to another branch, without taking anything else.
Let’s make two small bug fixes on rc-1.0, and then take the last one over with a cherry pick.
If we just wanted to merge over bug fix number 2, the merge would also carry bug fix number 1 along with it since that’s the common parent. We haven’t tested bug fix number 1, so we’ll need to cherry pick number 2 over onto the 1.0 branch by itself.
You now have the change-set from bug fix number 2, but it’s technically a fully different commit (with its own commit hash and everything). This allows you to move things over from the RC branch independently of other things in the branch. Also, when you do decide to merge over the rest of the RC branch, that shouldn’t be a problem at all since I guess git knows that the new commit has the same content and just a different parent.
Another common thing I find myself doing all the time is having to “try again” after I accidentally merge over the wrong commit, or make a change that just isn’t right. This is where the git reset command comes in handy.
If you just committed before you were ready, the standard git reset HEAD~1 will work just fine. This is git-speak for “rewind things by 1 commit, but don’t change the contents of the files on disk”.
If you do want to undo any changes to files locally, you can use the --hard flag to change that as well: git reset --hard HEAD~1.
The HEAD~1 is notation for the head of the current branch minus 1 commit. So HEAD~4 would mean “four commits ago”.
You can also point out a specific commit by it’s short hash to rewind to that point in time. And remember the --hard flag will bring you back to that point with the exact code from then as well.
Sometimes I’ll make a couple of small changes on master and one thing leads to another and all the sudden I realize I’ve got a lot of changes right on master instead of a feature branch. Let’s get into that situation real quick:
OK, now we are on master, with two commits that we accidentally did right on master instead of a feature branch. The way we’ll get these onto a feature branch, and get master back to normal is with by branching and resetting.
First, we’ll create a branch from master which will have those two commits, then we’ll reset our master branch back to before we accidentally made those commits.
This keeps your changes off on the “feature1” branch, and then moves the “master” pointer back to the master pointer on the shared repository. If you checkout the feature1 branch, you’ll see that the changes you made originally on master are in place over there, and you can easily merge them back into master when the time comes to commit them.
That’s all I have for now. If there’s anything missing that you’d like to see feel free to e-mail me at jj@(this website).
Extra thanks to Mark Chadwick for being the inspiration behind this post. Also if you’re going to be working with lots of git repositories and lots of different branches, see my previous post (http://geewax.org/2009/11/15/git-workspace-magic.html) for how to make your PS1 update based on what branch you’re currently on.
There are a few discussions going on over at Reddit and HackerNews, some of the feedback has been incorporated into the article since then, but hopefully the comments on these forums can help you to adapt this workflow to best suit the needs of your team.
You can find these discussions here:
