Web frameworks: a free software oriented study

The web2.0 era has put the web application frameworks at the center of the free software (aka FLOSS) community attention. Various opinions (1,2) and performance (1,2) comparisons have been published by free software enthusiasts trying to rank the quality and the potential of different web frameworks.

In this post we use standard data mining and statistical techniques applied to source code repositories to evaluate the strength, commitment and creativity of the community behind popular web framework projects. The metrics we will be using will be all the information found in source control revision software used by those projects.

We choose to study 3 web frameworks: Django, Ruby on Rails and Seam. This choice is by nature arbitrary but we think the projects are diverse enough for the study to capture the free software movement behind web frameworks. Feel free to comment on this choice if you like, we might consider extending the study to other frameworks ;-).

Projects overview

language: python
#SLOC(1) : 56K Python / 60K Total
starting date(2) : 13 Jul 2005

Ruby on Rails (RoR)
Language: ruby
#SLOC: 100K Ruby / 116K Total
starting date: 24 Nov 2004

language: Java
#SLOC: 118K Java / 313K Total
starting date: 12 Aug 2005

(1): we consider lines of source code (SLOC) in the sense that cloc counts them (featuring automatic language discovery, duplicate code line detection, blank line detection etc.)

(2): public source repository launch date

Source code evolution overview

To grasp a first overview of the projects, we plot the #SLOC considering the total number of SLOC (including configuration, html, js files etc.) and the core #SLOC: Java, Ruby and Python for Django, RoR and Seam respectively.

Figure #1: #SLOC evolution

Remarks :

Contributor diversity

For each of the projects, we plot the density of the number of lines in each commit, presented in a cumulative way by author (each author is represented by a different color). The goal here is to have an idea of the number of the commiters behind the project.

Figure #2: Seam code modification by author

Figure #3: RoR code modification by author

Figure #4: Django code modification by author

What we see clearly here is that all three projects, despite their different nature, present the same characteristics: the vast majority of the code is produced or at least checked in by a few “gatekeepers”, the project creators plus one or two other persons.

Maybe we can note that Django seems the least diverse in terms of the number of important commiters; maybe a weakness that might be overcomed by the recent Google app engine announcement and the endorsement of Django that will definitely attract much attention to this framework.

Code base stability

Here we plot the density of the number of files impacted by individual commits (using the Kernel density estimation method).

Figure #5: Commit impact

Site Map The goal of every code project is to split logically the code into components as disconnected from each other as possible. Usually, this has a direct translation into the various files of the project. Hence we estimate the impact of each changeset (a changeset is supposed to be a logical modification limited to one feature or bug fix etc.) by plotting the number of files modified by commits. Arguably this is a measure of how well the code is structured: vast commits impacting many files are a bad sign in terms of code structure efficiency (of course, branches merging into trunk will always affect plenty of files but this should only happen now and then). To avoid initialization artifacts, we start the plot a few weeks after each project start.

We see that the code base of Seam has been fairly refactored during the lifetime of the project. On the contrary, Django and Ruby on rails are much more stable in this regard: they seem to pay much more attention to backward compatibility and avoid deep refactoring of the source code base.


We hope that this study might help some newcomers to web development to pick up a framework. For others, it might be an insight into the world of free software development. For those interested we heartily recommend the reading of the findings of the flosspols study that is full of statistics and insightful analysis of the free software phenomenon at large.

If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.


Nice article!

Maybe you could include PHP’s Symfony in your study to cover one of the most popular LAMP frameworks too.

The reason we did not include PHP’s Symphony in this study is that we don’t know this framework enough to be sure that it could be compared to the others.

But if the perimeter is quite the same as the others (django,RoR and seam) we could do the same source code analysis in a future post. Thanks for your remark.

[…] django-ror-seam összehasonlítás http://www.mininglabs.com/2008/05/13/web-frameworks-a-free-software-oriented-study/ « előző | Hangya — 2008. 05. 15. […]

Interesting approach to web frameworks indeed!

Following the suggestion of Peter Nash, including other frameworks in such a study would be very interesting, especially when it comes to “judge” a framework with different criteria than technical merits.

I second Symfony as well as CakePHP which both are the most featured and popular among PHP developpers.

In the Python world, I also suggest Pylons

Yeah skipping any PHP frameworks seemed a bit odd since there’s so much development done in PHP compared to RoR and Django. These are all OOP frameworks and so is Symfony so that might be a good choice to add.

[…] Web frameworks: a free software oriented study […]

Where is Gaia in this list?
All though it’s a “.Net Framework” it has 100% compatibility towards Mono and runs perfectly on all Linux distros and it’s also GPL.

“RoR is still twice the size (#SLOC) of Django, although this might come from python well known compactness it came as a shock to us and we haven’t figured out why it is so.”

Is it possible that the relatively small number of “gatekeepers” goes alongside a philosophy to keep the codebase small and manageable?

It might be interesting to follow up with a comparison of a few javascript frameworks.

Leave a comment



Белье мы рекомендуем секс корсеты белье 3 секс шоп нижнее белье Вам