bzr, git, and hg performance on the Linux tree

OK, so I just did a historical comparison of git and bzr performance using the Linux source tree. One of the comments I got was “what about Mercurial?” Fair enough. I’ve really never done much with Mercurial because Ubuntu primarily uses bzr and git is what most of the other people I know using a DVCS use. However, there are a lot of projects using Mercurial, Mozilla being probably the most notable one. So, here’s a comparison of bzr and hg. You may want to read my previous post for details on the steps I’m doing.

Repo Initialization:
git                bzr                hg
0m0.086s     0m0.334s     0m0.137s
1        :         3.88      :      1.59

Add 2.6.0 Linux tree:
git                bzr                hg
0m14.269s   0m4.852s      0m2.526s
5.65      :      1.92      :       1

Commit 2.6.0 Linux tree:
git                bzr                 hg
0m10.263s   0m43.968s    0m30.890s
1         :        4.28       :       3.01

Diff after copying in 2.6.25.2 Linux tree:
git                bzr                hg
0m24.425s   0m51.158s    0m37.846s
1        :         2.09      :      1.55

Committing large changes:
git                bzr                hg
0m28.468s   1m8.627s     0m47.948s
1        :         2.41      :        1.68

Diff after no changes:
git                bzr                hg
0m0.343s     0m47.448s    0m1.340s
1         :        138       :       3.91

Getting repo status after no changes:
git                bzr                hg
0m1.230s     0m4.027s     0m1.077s
1.14       :      3.74      :     1

Committing a trivial change:
git                bzr                hg
0m0.397s    0m9.010s      0m1.913s
1        :        22.7       :       4.82

Repository size (just VCS control directory):
git (gc)        bzr (pack)      hg
92 MB         112 MB          179 MB

So, Mercurial performs quite well. It generally sits somewhere between git and bzr. Hg runs somewhere around 2.75 times slower than git in the tested operations. Bzr runs around 5 times slower with the notable exception that bzr diff when there are no changes is 138 times slower than git and 35 times slower than Hg.

Posted in Ubuntu. Tags: , , .

12 Responses to “bzr, git, and hg performance on the Linux tree”

  1. bartman Says:

    Thanks for the benchmarks.

    Just curious… did you prime (or flush) the disk cache before running tests to make sure that all tests started with the same setup?

    Also, were time measurements ran on a packed/gc’ed repository?

    Lastly, can you quote the versions you used?

    Cheers,
    -Bart

  2. laserjock Says:

    No, I didn’t flush the disk cache. I’m sure there’s lots more that can be done to get really good benchmarks, but this isn’t my area of expertise as I’m just a curious VCS user :-)

    I did the packing and gc’ing after all the timing. I figure it’s highly unlikely that people are going to repack/gc after each operation.

    The git/bzr/hg versions are the ones currently found in Ubuntu 8.04:
    git (1.5.4.3), hg (0.9.5), and bzr (1.3.1)

  3. dap Says:

    Do you remember what order git/bzr/hg were tested in, and was the ordering the same for the different tests?

    Since the cache wasn’t flushed, the first test will be at a disadvantage compared to subsequent tests which might have lots of filesystem data already in memory. Thanks for posting these!

  4. Robin Says:

    Thanks for also benchmarking Mercurial! The results are about what I expected.

  5. Bill Gates Says:

    Good work. Thanks for the comparison. Git is the best as I expected.

  6. gerardo Says:

    Good benchmark, and , as you can see, mercurial performs quite well.
    Using the benchmark, and doing some additional comparison about the features of those VCS, we could conclude that:

    * Hg performs better than bzr in the tests
    * Git doesn’t have a clean plugin support
    * bzr and hg has have a nice plugin support.

    So, with mercurial, you have an excellent performance and a good plugin support! :D

  7. Vadim P. Says:

    Now, how many people work with the Linux kernel tree…

    … not a lot, so it’s the bells and whistles that matter! :)

  8. Pieter Says:

    These type of benchmarks don’t really say anything. They show how the tools behave in importing two specific versions of a single project, which is not a typical workflow for developers.

    It would be more interesting to see benchmarks on repositories with thousands of commits. How does repository size increase for each commit? Do the tools get any slower when you have a lot of commits? How about diffing versus the last revision, or 100 revisions ago? Those are operations that can be slow in some tools, and would be worthwile to test

  9. Pieter Says:

    I started a blog describing my adventures into benchmarking. It might be interesting to follow if you’re looking into this type of benchmarks.

  10. Dazed Says:

    Somehow i missed the point. Probably lost in translation :) Anyway … nice blog to visit.

    cheers, Dazed.

  11. In Traction » Blog Archive » Bzr vs git, the sequel Says:

    [...] did another comparison with bzr, git and hg, which showed that hg is actually fairly close to git in terms of [...]

  12. Jakub Narebski Says:

    It is not mentioned how time was measured, is it user+sys result, or wallclock (real) time; see for example historical comparison of old versions of git and Bazaar-NG (or is it now GNU Bazaar?), where for Bazaar (bzr) real time and user+sys time differed *wildly*.

    Also the details of hardware, operating system and filesystem used are not given.

    BTW. I have added benchmark results from those two articles on Git Wiki: http://git.or.cz/gitwiki/GitBenchmarks#head-ac6a277715384041898e90485848b14b48f6c142

Leave a Reply