I discovered Subversion and Revision control software in general a little over a year ago. I’d heard of it all before but that was when I began to learn about and use it. The appeal, once I took the time to think about, was significant. As a freelance website developer (and reluctant designer), being able to version my code meant time saved from searching through backups (when I’d deleted stuff) and cleaner space (from when I wouldn’t have deleted stuff; see previous point). It particularly allowed me to get rid of goofy versioning mechanisms I had employed (file names with 1,2, etc at the end, e.g.).

Having used Subversion extensively this past year (although certainly not all of its features), I have a few suggestions for newcomers regarding the repository setup. It turns out, looking back at it, that the online manual could have told me this just as well (more or less), particularly the section about Repository Deployment and Externals Definitions. If you’re seriously looking for repository layout advice, I recommend reading those more than the rest of this article. On the other hand, personal experience can at times be more immediate and useful that a manual. (By the way, I haven’t looked closely to see how well my suggestions really match up with the manual; rather, I’ve noted in re-reading sections that they make a lot more sense to me now.)

When I setup my repository, I set up one, single repository for all my projects. And life was good. The root directory contained a directory for each project. This was manageable up to about ten projects. Then, I begun dividing them into groups. My logic for a single repo was:

1. Aesthetic reasons: for better or worse, I just liked the idea of a single repository better.
2. Ability to svn copy files/directories between projects: A number of the websites I’ve made have very similar code that is not quite similar enough to just create a library, and the chance to save space appealed to me.
3. I could setup a frame project (for Rails apps) and when I started a new site, just svn copy).

I still aesthetically like the idea of a single repository, but that’s no justification for a business decision (at least when there are weightier factors). What made me change my mind otherwise? Simply, it became a headache. One of the problems was that the project group directories I created early on are no longer as useful groupings as they were, and I don’t really want the change to be versioned. When going back to earlier revisions, the path to the project would be uselessly different. But the straw that did in that design was one project. This project contains a lot of media. I didn’t want to put all that media in my main repository. I like the fact that it doesn’t take up a lot of space. But I’ve found a need to start versioning many of those large media files. So, I decided to use svndumpfilter to break out the project. And with all the path moves, I realized I was facing a serious chore. I also had a bunch of project folders that were basically empty, unused, I’d created in a fit of “planning”. There was no reason to have them at all, so I wanted to completely remove them from the repository.

Over the past couple of weeks, I’ve spent many hours writing a lengthy script to break up the repository. Because I realized it’s best to get it done now. Some major difficulties have been the result of my earlier reasons for a single repository. The copies between projects and setting up projects by copying from a frame create extra dumps that must be done. And for very little gain, it’s turned out.

Here’s how my repositories will now be:

From the view of svn-serve–at least, for now–the repository won’t look much different. But the project group folders will generally be regular folders with the filesystem, holding project folders, which will each be (generally) a single repository. There are a few exceptions, where the group is the root of the repository, holding multiple projects. But I plan for minimal, if any, path moving once they’re created (if needed, I plan to do it using dumps, but I will try to plan to avoid it).

Anyway, my five key points of advice:

1. Except for small, easily grouped projects (rails plugins, for example), each project has it’s own repository.
2. In any case, project heirarchy (divsion into groups) should not be within the repository. There’s no use in having that versioned and it gives much more freedom.
3. Use svn:externals rather than svn copy for sharing code between projects, even within your own repository realm. And when you do, use -r option to explictly set the revision (or set the external to a tag). Setting to the trunk without a -r is asking to break your code.
4. If you need a frame for creating new projects, make the frame itself a repository. Copy the repository (within the filesystem), rather than a project within a repository. Much more manageable. Or use a script.
5. It’s worth the extra space to copy large chunks from one project to another in different repositories, when a library shared via svn:externals won’t work. It’s better than the headache.

Yes, the proceeding blog entry was ~80% introduction. test

For those of you using canofcode (for the record, that’s me and googlebot), it is back up. There’s a story to that statement, of course.

I discovered, oh, maybe a month ago that my impression that the Subversion to Ruby SWIG bindings did not support remote repositories was just plain wrong. Had I looked at the code, I would have found the Svn::Ra module. Relying heavily on the unit tests, I started work on creating a library that used the Svn::Ra library to replace my previous work on directly implementing the svn and svn-over-WebDAV protocols.

About this time, googlebot started roaming through the site and errors started showing up regularly in my email box (I’d done a few, but obviously imperfect, unit tests). A number of these were directly related to my aforementioned implementations, so rather than fix them, I took down the site until I had the new Ra-based library ready. And now it is!

My next step for canofcode is to work on the UI (such as making some ‘potential’ links into actual links), along with better functional testing, etc. I will also continue working on CVS and Darcs implementations.

In addition, I am nearing completion on a CSCM adapter using Svn::Ra. CSCM is a library that has recently been created by Johan Sørensen, creator of Collaboa, assumedly for future use in Collaboa. I’m considering using the CSCM library for canofcode, part of the reason I’m developing the aforementioned library.

A couple of weeks ago, I mentioned that I started working on a for-fun project to interact with a remote subversion repository using svnserve. I’ve continued working on it (probably more than I ought to), and have started to add support for access via WebDAV. That said, I’m going to list some of the resources I’ve been utilizing, for anyone interested, and for my future reference. So, basically, a content-less blog post. Yay!

Subversion Repository Access

• RA Summary
• Subversion access through WebDAV
• SVN protocol specification
• Subversion SWIG/API Information

I have the impression that the Ruby SWIG bindings can’t be used for remote repositories, because I haven’t seen it happen, but I may be wrong on that (Update 2007-02-26. I was wrong. See here). I need to investigate more. After all, if I can use the SWIG bindings for all types of access, I needn’t develop the SVN and WebDAV options, except for fun.

Web-based Repository Browsing Projects

Some projects I’ve looked at for inspiration:

• Trac
• ViewVC - Used by sourceforge, for example
• Collaboa
• Retrospectiva
• bsSvnBrowser

These are all open-source. The last three are rails projects. I believe those three all use the SWIG bindings. I generally like the interfaces for all but ViewVC.

There’s sort of a point to this article, but it may take me a few paragraphs. Feh, isn’t blogging great? I could write an appropriately structured article, but since I am the lone editor here, I can also conduct a non-quite stream-of-conciousness discourse, then, at the last possible moment, suddenly introduce a thesis. And you say, thanks for that meandering and irrelevant comment.

For those of you using subversion for version control, particularly for your rails applications, Collaboa is a useful tool. It includes repository browsing and a ticket system for projects stored in subversion, and is a rails application. Functionally, it’s similar to trac, but since I’m accustomed to rails, Collaboa is a better resource for me.

In reference to the title, there have actually been a couple of forks of Collaboa.

• Titra, if I remember right, was forked in order to add in time tracking features. I’ve never tried it.
• Retrospectiva is a fork created a month or two ago, with some different design goals, I guess.

Anyway, probably all three are good tools, although Collaboa is what I use for lack of any pressing reason to switch.

One issue that interests me is that none of these projects aim to provide a functionality like sourceforge, rubyforge or Google code, that is to provide the functionality for multiple, disassociated projects (To clarify, at least Collaboa has multiple projects support, but it’s a web application that organizations install and use, rather than a web service hosting multiple projects. Right, if that made sense.). Another thing I’ve noticed is how projects hosted by these large web service sites tend to have much less attractive and functional repository browsing than that offered by the above projects.

Okay, actually approaching the point. Despite telling myself I should do so, I haven’t contributed anything to Collaboa. However, I did get the bug this week to start playing around with a rails project interacting with a subversion repository. Collaboa uses Subversion bindings for ruby via SWIG (or that’s my understanding) to interact with a repository on the local machine. I wanted (for fun) to set up to interact with a remote repository via svn-serve. Once I found the protocol specification it was quite simple, by which I mean, I had no idea what I was doing. But, over the past couple of days, I’ve learned a lot about Ruby’s TCPSocket library, and about Subversion. The code I’ve written can actually do a directory listing now. So, it’s been a lot of fun, and maybe I can release it someday. I think I saw that Collaboa will soon have the repository interaction abstracted, so other repository types can be used, so maybe this will be something I can contribute. We’ll see, eh?

I don’t know if anyone’s noticed, but I write in this blog almost exactly twice a week. Once for a general blog entry, once for a From Genesis entry. Basically, my reason for blogging is to give me an excuse to write, and I make sure that I take the time to do that twice a week. But I’m not sure what to discuss this week.

Okay, here’s a thought. Version control for writing, say, a novel. I’ve actually written the first draft to one full novel and have fifty-plus pages (I think) of a couple of others, as well as several dozen (I imagine) lesser attempts.

One of the downsides is that I’ll tend to work on one for a while, then stop, then pick it back up later. Leaving me with the question of whether I copy and start a new file (or, in most cases for me, a new database record) and have multiple copies or lose the historical record of what I originally wrote (and I like historical records).

So, naturally, when I discovered subversion, and started using it with source code, I began to ponder using it with novels. The major upside is that it deals well with the issues of the previous paragraph. The downside is that I’m not sure how to make the best use of it. Specifically: as best I understand, subversion (or diff, I suppose) checks lines for changes in text files. This is great for source code, but in a novel, a “line” is a paragraph. Seems inefficient, and might make checking changes more difficult. On the other hand, saving it as an ODF text document would lead to a binary diff (and I have no clue on how that works), which could be more efficient, and allows formatting (not that I would use it), but probably removes the possibility of just glancing through a change set.

Anybody have any thoughts? Experience with this? Anywho, I’m planning to start giving it a try with a collection of novels and stories I’ve been working on for several years.