Subversion: The Definitive Guide

Subversion is a free/open-source version control system. That is, Subversion manages files over time. The files are placed into a central repository. The repository is much like an ordinary file server, except that it remembers every change ever made to your files. This allows you to recover older versions of your files, or browse the history of how your files changed. Many people think of a version control system as a sort of ``time machine.''

Some version control systems are also software configuration management (SCM) systems. These systems are specifically tailored to manage trees of source code, and have many features that are specific to software development (such as natively understanding programming languages). Subversion, however, is not one of these systems; it is a general system that can be used to manage any sort of collection of files, including source code.

Subversion aims to be the successor to the Concurrent Versions System (CVS). You can find more information about CVS at http://www.cvshome.org/.

At the time of writing, CVS is the standard Free version control system used by the open-source community. It has a hard-earned, well-deserved reputation as stable and useful software, and has a design that makes it perfect for open-source development. However, it also has a number of problems that are difficult to fix.

Subversion's original designers settled on a few simple goals. First, it was decided that Subversion should be a functional replacement for CVS. That is, it should do everything that CVS does, preserving the same development model while fixing the most obvious flaws. Secondly, with existing CVS users as the first target audience, Subversion should be written such that any CVS user should be able to start using it with little effort.

Collabnet http://www.collab.net/ provided the initial funding in 2000 to begin development work, and the effort has now blossomed into a large, open-source project backed by a community of free software developers.

All version control systems have to solve the same fundamental problem: how will the system allow users to share information, but prevent them from accidentally stepping on each other's feet? It's all too easy for users to accidentally overwrite each other's changes in the repository.

Consider this scenario: suppose we have two co-workers, Jane and Joe. They each decide to edit the same repository file at the same time. If Joe saves his changes to the repository first, then it's possible that (a few moments later) Jane could accidentally overwrite them with her own new version of the file. While Joe's version of the file won't be lost forever (because the system remembers every change), any changes Joe made won't be present in Jane's newer version of the file, because she never saw Joe's changes to begin with. Joe's work is still effectively lost—or at least missing from the latest version of the file—and probably by accident. This is definitely a situation we want to avoid!

Figure 2.2. The problem to avoid

Many version control systems use a lock-modify-unlock model to address this problem, which is a very simple solution. In such a system, the repository allows only one person to change a file at a time. First Joe must "lock" the file before he can begin making changes to it. Locking a file is a lot like borrowing a book from the library; if Joe has locked a file, then Jane cannot make any changes to it. If she tries to lock the file, the repository will deny the request. All she can do is read the file, and wait for Joe to finish his changes and release his lock. After Joe unlocks the file, his turn is over, and now Jane can take her turn by locking and editing.

Figure 2.3. The lock-modify-unlock solution

The problem with the lock-modify-unlock model is that it's a bit restrictive, and often becomes a roadblock for users:

Subversion, CVS, and other version control systems use a copy-modify-merge model as an alternative to locking. In this model, each user's client reads the repository and creates a personal working copy of the file or project. Users then work in parallel, modifying their private copies. Finally, the private copies are merged together into a new, final version. The version control system often assists with the merging, but ultimately a human being is responsible for making it happen correctly.

Here's an example. Say that Joe and Jane each create working copies of the same project, copied from the repository. They work concurrently, and make changes to the same file "A" within their copies. Jane saves her changes to the repository first. When Joe attempts to save his changes later, the repository informs him that his file A is out-of-date. In other words, that file A in the repository has somehow changed since he last copied it. So Joe asks his client to merge any new changes from the repository into his working copy of file A. Chances are that Jane's changes don't overlap with his own; so once he has both sets of changes integrated, he saves his working copy back to the repository.

Figure 2.4. The copy-modify-merge solution

Figure 2.5. ...copy-modify-merge continued

But what if Jane's changes do overlap with Joe's changes? What then? This situation is called a conflict, and it's usually not much of a problem. When Joe asks his client to merge the latest repository changes into his working copy, his copy of file A is somehow flagged as being in a state of conflict: he'll be able to see both sets of conflicting changes, and manually choose between them. Note that software can't automatically resolve conflicts; only humans are capable of understanding and making intelligent choices. Once Joe has manually resolved the overlapping changes (perhaps by discussing the conflict with Jane!), he can safely save the hand-merged file back to the repository.

The copy-modify-merge model may sound a bit chaotic, but in practice, it runs extremely smoothly. Users can work in parallel, never waiting for one another. When they work on the same files, it turns out that most of their concurrent changes don't overlap at all; conflicts are infrequent. And the amount of time it takes to resolve conflicts is far less than the time lost by a locking system.

In the end, it all comes down to one critical factor: user communication. When users communicate poorly, both syntactic and semantic conflicts increase. No system can force users to communicate perfectly, and no system can detect semantic conflicts. So there's no point in being lulled into a false promise that a locking system will somehow prevent conflicts; in practice, locking seems to inhibit productivity more than anything else.

You've already read about working copies; now we'll demonstrate how the Subversion client creates and uses them.

A Subversion working copy is an ordinary directory tree on your local system, containing a collection of files. You can edit these files however you wish, and if they're source code files, you can compile your program from them in the usual way. Your working copy is your own private work area: Subversion will never incorporate other people's changes, nor make your own changes available to others, until you explicitly tell it to do so.

After you've made some changes to the files in your working copy and verified that they work properly, Subversion provides you with commands to "publish" your changes to the other people working with you on your project (by writing to the repository). If other people publish their own changes, Subversion provides you with commands to merge those changes into your working directory (by reading from the repository).

A working copy also contains some extra files, created and maintained by Subversion, to help it carry out these commands. In particular, each directory in your working copy contains a subdirectory named .svn, also known as the working copy administrative directory. The files in each administrative directory help Subversion recognize which files contain unpublished changes, and which files are out-of-date with respect to others' work.

A typical Subversion repository often holds the files (or source code) for several projects; usually, each project is a subdirectory in the repository's filesystem tree. In this arrangement, a user's working copy will usually correspond to a particular subtree of the repository.

For example, suppose you have a repository that contains two software projects.

Figure 2.6. The repository's filesystem

In other words, the repository's root directory has two subdirectories: paint and write.

To get a working copy, you must check out some subtree of the repository. (The term "check out" may sound like it has something to do with locking or reserving resources, but it doesn't; it simply creates a private copy of the project for you.) For example, If you check out /write, you will get a working copy like this:

$ svn checkout http://svn.example.com/repos/write
A  write
A  write/Makefile
A  write/document.c
A  write/search.c

$ ls -a write
Makefile  document.c  search.c  .svn/

Your working copy is a personal copy of the repository's /write directory, with one additional entry—.svn—which holds the extra information needed by Subversion, as mentioned earlier.

Suppose you make changes to search.c. Since the .svn directory remembers the file's modification date and original contents, Subversion can tell that you've changed the file. However, Subversion does not make your changes public until you explicitly tell it to. The act of publishing your changes is more commonly known as committing or (checking in) changes to the repository.

To publish your changes to others, you can use Subversion's commit command:

$ svn commit search.c
Sending search.c
Transmitting file data..
Committed revision 57.

Now your changes to search.c have been committed to the repository; if another user checks out a working copy of /write, they will see your changes in the latest version of the file.

Suppose you have a collaborator, Felix, who checked out a working copy of /write at the same time you did. When you commit your change to search.c, Felix's working copy is left unchanged; Subversion only modifies working directories at the user's request.

To bring his project up to date, Felix can ask Subversion to update his working copy, by using the Subversion update command. This will incorporate your changes into his working copy, as well as any others that have been committed since he checked it out.

$ pwd
/home/felix/write

$ ls -a 
.svn/ Makefile document.c search.c

$ svn update
U search.c

The output from the svn update command indicates that Subversion updated the contents of search.c. Note that Felix didn't need to specify which files to update; Subversion uses the information in the .svn directory, and further information in the repository, to decide which files need to be brought up to date.

A svn commit operation can publish changes to any number of files and directories as a single atomic transaction. In your working copy, you can change files' contents, create, delete, rename and copy files and directories, and then commit the complete set of changes as a unit.

In the repository, each commit is treated as an atomic transaction: either all the commit's changes take place, or none of them take place. Subversion tries to retain this atomicity in the face of program crashes, system crashes, network problems, and other users' actions.

Each time the repository accepts a commit, this creates a new state of the filesystem tree, called a revision. Each revision is assigned a unique natural number, one greater than the number of the previous revision. The initial revision of a freshly created repository is numbered zero, and consists of nothing but an empty root directory.

A nice way to visualize the repository is as a series of trees. Imagine an array of revision numbers, starting at 0, stretching from left to right. Each revision number has a filesystem tree hanging below it, and each tree is a "snapshot" of the way the repository looked after each commit.

Figure 2.7. The repository

It's important to note that working copies do not always correspond to any single revision in the repository; they may contain files from several different revisions. For example, suppose you check out a working copy from a repository whose most recent revision is 4:

write/Makefile:4
      document.c:4
      search.c:4

At the moment, this working directory corresponds exactly to revision 4 in the repository. However, suppose you make a change to search.c, and commit that change. Assuming no other commits have taken place, your commit will create revision 5 of the repository, and your working copy will now look like this:

write/Makefile:4
      document.c:4
      search.c:5

Suppose that, at this point, Felix commits a change to document.c, creating revision 6. If you use svn update to bring your working copy up to date, then it will look like this:

write/Makefile:6
      document.c:6
      search.c:6

Felix's changes to document.c will appear in your working copy, and your change will still be present in search.c. In this example, the text of Makefile is identical in revisions 4, 5, and 6, but Subversion will mark your working copy of Makefile with revision 6 to indicate that it is still current. So, after you do a clean update at the top of your working copy, it will generally correspond to exactly one revision in the repository.

For each file in a working directory, Subversion records two essential pieces of information in the .svn/ administrative area:

Given this information, by talking to the repository, Subversion can tell which of the following four states a working file is in:

This may sound like a lot to keep track of, but as you will learn in a later chapter, the svn status command will show you the state of any item in your working copy.

When working on a project with a team, you'll want to update your working copy: that is, receive any changes from other developers on the project. svn update brings your working copy in sync with the latest revision in the repository.

$ svn up
U  ./foo.c
U  ./bar.c
Updated to revision 2.
      

In this case, someone else checked in modifications to both foo.c and bar.c since the last time you updated, and Subversion has updated your working copy to include those changes.

Let's examine the output of svn update a bit more. When the server sends changes to your working copy, a letter code is displayed next to each item to let you know what actions Subversion performed to bring your working copy up to date:

Now you can to get to work and make changes in your working copy. It's usually most convenient to create a “task” for yourself, such as writing a new feature, fixing a bug, etc. The Subversion commands that you will use here are svn add, svn delete, svn copy, and svn move. However, if you are merely editing a file (or files) that is already in Subversion, you may not need to use any of these commands until you commit.

To make file changes, just use your text editor, word processor, or graphics program—whatever tool you would normally use. A file needn't be in text-format; Subversion handles binary files just as easily as it handles text file (and just as efficiently too).

Here is an overview of the four Subversion subcommands that you'll use most often to make tree changes (we'll cover svn import and svn mkdir later).

We've already seen how svn status -u can predict conflicts. Suppose you run svn update and some interesting things occur:

$ svn update
U  ./INSTALL
G  ./README
C  ./bar.c
      

The U and G codes are no cause for concern; those files cleanly absorbed changes from the repository. The files marked with U contained no local changes but were Updated with changes from the repository. The G stands for merGed, which means that the file had local changes to begin with, but the changes coming from the repository didn't overlap in any way.

But the C stands for conflict. This means that the changes from the server overlapped with your own, and now you have to manually choose between them.

Whenever a conflict occurs, your Subversion client does the following:

For example, Sally makes changes to the file sandwich in the repository. Harry has just changed the file in his working copy and checked it in. Sally updates her working copy before checking in and she gets a conflict:

$ svn update
C  sandwich
Updated to revision 2.
$ ls -1
sandwich                       
sandwich.59528.00001.r2        
sandwich.59524.00001.r1
sandwich.59532.00001.mine
$
      

At this point, Subversion will not allow you to commit the file sandwichuntil the three temporary files are removed.

$ svn ci --message "Add a few more things"
subversion/libsvn_client/commit.c:654: (apr_err=155015, src_err=0)
svn: A conflict in working copy obstructs the current operation
svn: Commit failed (details follow):
subversion/libsvn_client/commit_util.c:180: (apr_err=155015, src_err=0)
svn: Aborting commit: '/home/sally/svn-work/sandwich' remains in conflict.
      

If you get a conflict, you need to either (1) hand-merge the conflicted text (by examining and editing the conflict markers within the file), (2) copy one of the tmpfiles on top of your working file, or (3) run svn revert <filename> to throw away all of your local changes.

$ cp sandwich.59532.00001.mine sandwich
      

Once you've resolved the conflict, you need to let Subversion know by removing the three temporary files. (The svn resolve command, by the way, is a shortcut that does nothing but automatically remove the three temporary files for you.) When those files are gone, Subversion no longer considers the file to be in a state of conflict anymore.

$ svn resolve sandwich
Resolved conflicted state of sandwich
      

Now you're ready to check in your changes

$ svn ci --message "Add a few more things"
Sending        sandwich
Transmitting file data .
Committed revision 3.
      

Note thatsvn resolve, unlike most of the other commands we've dealt with in this chapter, requires an argument. In any case, you want to be careful and only run svn resolve when you're certain that you've fixed the conflict in your file—once the temporary files are removed, Subversion will even let you check in a file with conflict markers.

Finally! Your edits are finished, you've merged all updates from the server, and you're ready to commit your changes to the repository.

The svn commit command sends all (or, if you specify files or directories, some) of your changes to the repository. When you commit a change, you need to supply a log message, describing your change. Your log message will be attached to the new revision you create. If your log message is brief, you may wish to supply it on the command line using the --message (or -m):

$ svn commit --message "Corrected number of cheese slices."
Sending        sandwich
Transmitting file data .
Committed revision 3.
      

However, if you've been composing your log message as you work, you may want to tell Subversion to get the message from a file by passing the filename with the --file switch:

svn ci --file logmsg 
Sending        sandwich
Transmitting file data .
Committed revision 4.
      

If you fail to specify either the --message or --file switch, then Subversion will automatically launch your favorite editor (As defined in the environment variable $EDITOR) for composing a log message.

The repository doesn't know or care if your changes make any sense as a whole; it only checks to make sure that nobody else has changed any of the same files that you did when you weren't looking. If somebody has done that, the entire commit will fail with a message informing you that one or more of your files is out-of-date:

$ svn commit --message "Add another rule"
Sending        rules.txt
subversion/libsvn_client/commit.c:655: (apr_err=160028, src_err=0)
svn: Transaction is out of date
svn: Commit failed (details follow):
subversion/libsvn_repos/commit.c:112: (apr_err=160028, src_err=0)
svn: out of date: `rules.txt' in txn `g'
$

At this point, you need to run svn update, deal with any merges or conflicts that result, and attempt your commit again.

That covers the basic work cycle for using Subversion. There are many other features in Subversion that you can use to manage your repository and working copy, but you can get by quite easily using only the commands that we've discussed so far in this chapter.

To find out information about the history of a file or directory, use the svn log command. svn log will provide you with a record of who made changes to a file or directory, at what revision it changed, the time and date of that revision, and, if it was provided, the log message that accompanied the commit.

$ svn log
------------------------------------------------------------------------
rev 3:  sally | Mon, 15 Jul 2002 18:03:46 -0500 | 1 line

Added include lines and corrected # of cheese slices.
------------------------------------------------------------------------
rev 2:  harry | Mon, 15 Jul 2002 17:47:57 -0500 | 1 line

Added main() methods.
------------------------------------------------------------------------
rev 1:  sally | Mon, 15 Jul 2002 17:40:08 -0500 | 2 lines

Initial import
------------------------------------------------------------------------
      

Note that the log messages are printed in reverse chronological order by default. If you wish to see a different range of revisions in a particular order, or just a single revision, pass the --revision (-r) switch:

$ svn log --revision 5:19
… # shows logs 5 through 19 in chronological order
$ svn log -r 19:5
… # shows logs 5 through 19 in reverse order
$ svn log -r 8
…
      

You can also examine the log history of a single file or directory. For example:

$ svn log foo.c
…
$ svn log http://foo.com/svn/trunk/code/foo.c
…
      

These will display log messages only for those revisions in which the working file (or URL) changed.

If you want even more information about a file or directory, svn log also takes a --verbose (-v) switch. Because Subversion allows you to move and copy files and directories, it is important to be able to track path changes in the filesystem, so in verbose mode, svn log will include a list of changed-paths in a revision in its output:

$ svn log -r 8 -v
------------------------------------------------------------------------
rev 8:  sally | 2002-07-14 08:15:29 -0500 | 1 line
Changed paths:
U /trunk/code/foo.c
U /trunk/code/bar.h
A /trunk/code/doc/README

Frozzled the sub-space winch.

------------------------------------------------------------------------
      

We've already seen svn diff above—it displays file differences in unified diff format; it was used to show the local modifications made to our working copy before committing to the repository.

In fact, it turns out that there are three distinct uses of svn diff:

$ svn diff
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth closed
+Chew with your mouth closed
+Listen when others are speaking
$
        

$ svn diff --revision 3 rules.txt 
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth closed
+Chew with your mouth closed
+Listen when others are speaking
$
        

$ svn diff --revision 2:3 rules.txt 
Index: rules.txt
===================================================================
--- rules.txt	(revision 2)
+++ rules.txt	(revision 3)
@@ -1,4 +1,4 @@
 Be kind to others
-Freedom = Chocolate Ice Cream
+Freedom = Responsibility
 Everything in moderation
 Chew with your mouth closed 
$
        

$ svn diff --revision 4:5 http://svn.red-bean.com/repos/example/trunk/text/rules.txt
…
$
        

If you want to examine an earlier version of a file and not necessarily the differences between two files, you can use svn cat:

$ svn cat --revision 2 rules.txt 
Be kind to others
Freedom = Chocolate Ice Cream
Everything in moderation
Chew with your mouth closed
$
      

You can also redirect the output directly into a file:

$ svn cat --revision 2 rules.txt > rules.txt.v2
$
      

Now you're probably wondering why we don't just use svn update --revision to update the file to the older revision. Well, there are several reasons that we might prefer to use svn cat.

First, you may want to see the differences between 2 revisions of a file using an external diff program (perhaps a graphical one, or perhaps your file is in such a format that the output ofunified diff is nonsensical). In this case, you'll need to grab a copy of the old revision, redirect it to a file, and pass both that and the file in your working copy to your external diff program.

Second, you may have local changes to the file and don't want to lose them.

Finally, sometimes it's easier to look at an older revision in its entirety as opposed to just the differences between it and another revision.

When Subversion modifies your working copy (or any information within .svn), it tries to do so as safely as possible. Before changing anything, it writes its intentions to a logfile, executes the commands in the logfile, then removes the logfile (This is similar in design to a journaled filesystem). If a Subversion operation is interrupted (If you hit Control-C, or if the machine crashes, for example), the logfiles remain on disk. By re-executing the logfiles, Subversion can complete the previously started operation, and your working copy can get itself back into a consistent state.

And this is exactly what svn cleanup does: it searches your working copy and runs any leftover logs, removing locks in the process. If Subversion ever tells you that some part of your working copy is “locked”, then this is the command that you should run. Also, svn status will display an L next to locked items:

$ svn status
  L    ./somedir
M      ./somedir/foo.c 

$ svn cleanup
$ svn status
M      ./somedir/foo.c
      

In general, we try to discourage you from directly reading the .svn/entries file used to track items in a Subversion working copy. Instead, you can get tons of information about an item in your working copy by using the svn info command to display most of the tracked information:

$ svn info rules.txt
Path: rules.txt
Name: rules.txt
Url: http://svn.red-bean.com/repos/example/trunk/text/rules.txt
Revision: 8
Node Kind: file
Schedule: normal
Last Changed Author: harry
Last Changed Rev: 7
Last Changed Date: 2002-11-15 23:03:54 -0500 (Fri, 15 Nov 2002)
Text Last Updated: 2002-11-16 08:48:04 -0500 (Sat, 16 Nov 2002)
Properties Last Updated: 2002-11-16 08:48:03 -0500 (Sat, 16 Nov 2002)
Checksum: 8sfaU+5dqyOgkhuSdyxGrQ==
      

The import command is a quick way to move an unversioned tree of files into a repository.

There are two ways to use this command:

$ svnadmin create /usr/local/svn/newrepos
$ svn import file:///usr/local/svn/newrepos mytree
Adding  mytree/foo.c
Adding  mytree/bar.c
Adding  mytree/subdir
Adding  mytree/subdir/quux.h
Transmitting file data....
Committed revision 1.
      

The above example places the contents of directory mytree directly into the root of the repository:

/foo.c
/bar.c
/subdir
/subdir/quux.h
      

If you give svn import a third argument, it will use the argument as the name of a new subdirectory to create within the URL.

$ svnadmin create /usr/local/svn/newrepos
$ svn import file:///usr/local/svn/newrepos mytree fooproject
Adding  mytree/foo.c
Adding  mytree/bar.c
Adding  mytree/subdir
Adding  mytree/subdir/quux.h
Transmitting file data....
Committed revision 1.
      

The repository would now look like this:

/fooproject/foo.c
/fooproject/bar.c
/fooproject/subdir
/fooproject/subdir/quux.h
      

The export command is a quick way to create an unversioned tree of files from a repository directory—a tree that looks like a typical working copy, but doesn't contain the .svn directories:

$ svn export file:///usr/local/svn/newrepos/fooproject
A  fooproject/foo.c
A  fooproject/bar.c
A  fooproject/subdir
A  fooproject/subdir/quux.h
Checked out revision 3.
      

The resulting directory will not contain any .svn administrative areas, and all property metadata will be lost. (Hint: don't use this technique for backing up; it's probably better for rolling source distributions.)

The svn list command shows you what files are in a repository directory without actually downloading the files to your local machine:

$ svn list http://svn.collab.net/repos/svn
README
branches/
clients/
tags/
trunk/
      

If you want a more detailed listing, pass the --verbose (-v) flag to get output like this.

$ svn list --verbose http://svn.collab.net/repos/svn
_    2755   kfogel     1331 Jul 28 02:07 README
_    2773  sussman        0 Jul 29 15:07 branches/
_    2769 cmpilato        0 Jul 29 12:07 clients/
_    2698   rooneg        0 Jul 24 18:07 tags/
_    2785    brane        0 Jul 29 19:07 trunk/
      

The columns tell you if a file has any properties (“P” if it does, “_” if it doesn't), the revision it was last updated at, the user who last updated it, the size if it is a file, the date it was last updated, and the item's name.

This is another convenience command, and it has two uses.

First, it can be used to simultaneously create a new working copy directory and schedule it for addition:

$ svn mkdir new-dir
A     new-dir
      

Or, it can be used to instantly create a directory in a repository (no working copy needed):

$ svn mkdir file:///usr/local/svn/newrepos/branches --message "made new dir"
Committed revision 1123.
      

Again, this is a form of immediate commit, so some sort of log message is required.

So how do we create a branch? Very simple -- you make a copy of the project in the repository using the svn copy command. Subversion is not only able to copy single files, but whole directories as well. In this case, you want to make a copy of the /trunk/write directory. Where should the new copy live? It doesn't really matter; Subversion doesn't care. It's a matter of project policy. Let's say that your team has a policy of creating branches in the /branches/write area of the repository, and you want to name your branch "my-write-branch". So you want to create a new directory, /branches/write/my-write-branch, which starts as a copy of /trunk/write.

There are two different ways to make a copy. We'll demonstrate the messy way first, just to make the concept clear. To begin, check out a working copy of the root (/) of the repository:

$ svn checkout http://svn.example.com/repos bigwc
A  bigwc/branches/
A  bigwc/branches/write
A  bigwc/branches/paint
A  bigwc/trunk/
A  bigwc/trunk/write
A  bigwc/trunk/write/Makefile
A  bigwc/trunk/write/document.c
A  bigwc/trunk/write/search.c
A  bigwc/trunk/paint
A  bigwc/trunk/paint/Makefile
A  bigwc/trunk/paint/canvas.c
A  bigwc/trunk/paint/brush.c
Checked out revision 340.

And now it's simply a matter of giving two working-copy paths to the svn copy command:

$ cd bigwc
$ svn copy trunk/write branches/write/my-write-branch
$ svn status
A  +   branches/write/my-write-branch

In this case, the svn copy command recursively copied the trunk/write working directory to a new working directory, branches/write/my-write-branch. As you can see from the svn status command, the new directory is now scheduled for addition to the repository. But also notice the '+' next to the letter A. This indicates that the scheduled addition is a copy of something, not something new. When you commit your changes, Subversion will create /branches/write/my-write-branch in the repository by copying /trunk/write, rather than resending all of the working copy data over the network:

$ svn commit -m "Creating a private branch of /trunk/write."
Adding      branches/write/my-write-branch
Committed revision 341.

And now the easier method of creating a branch, which we should have told you about in first place: the svn copy is able to operate on two URLs.

$ svn copy http://svn.example.com/repos/trunk/write \
           http://svn.example.com/repos/branches/write/my-write-branch \
      -m "Creating a private branch of /trunk/write"

Committed revision 341.

There's really no difference between these two methods. Both procedures create a new directory in revision 341, and the new directory is a copy of /trunk/write. Notice that the second method, however, performs an immediate commit. It's an easier procedure, because it doesn't require you to check out a large mirror of the repository. In fact, this technique doesn't even require you to have a working copy at all!

Figure 4.3. Repository with new copy

Now that you've created a new branch of the project, you can check out a new working copy to start using it:

$ svn checkout http://svn.example.com/repos/branches/write/my-write-branch
A  my-write-branch/Makefile
A  my-write-branch/document.c
A  my-write-branch/search.c
Checked out revision 341.

There's nothing special about this working copy; it simply mirrors a different location of the repository. When you commit changes, however, Felix won't ever see them when he updates. His working copy is of /trunk/write.

Let's pretend that a week goes by, and the following commits happen:

Things get interesting when you look at the history of changes made to your copy of document.c:

$ pwd
/home/user/my-write-branch

$ svn log document.c
------------------------------------------------------------------------
rev 343:  user | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines

* document.c:  frozzled the wazjub.

------------------------------------------------------------------------
rev 303:  felix | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines

* document.c:  changed a docstring.

------------------------------------------------------------------------
rev 98:  felix | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines

* document.c:  adding this file to the project.

------------------------------------------------------------------------

Notice that Subversion is tracing the history of your document.c all the way back through time, traversing the point where it was copied. (Remember that your branch was created in revision 341.) Now look what happens when Felix runs the same command on his copy of the file:

$ pwd
/home/felix/write

$ svn log document.c
------------------------------------------------------------------------
rev 344:  felix | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines

* document.c:  fix a bunch of spelling errors.

------------------------------------------------------------------------
rev 303:  felix | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines

* document.c:  changed a docstring.

------------------------------------------------------------------------
rev 98:  felix | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines

* document.c:  adding this file to the project.

------------------------------------------------------------------------

Felix sees his own revision 344 change, but not the change you made in revision 343. As far as Subversion is concerned, these two commits affected different files in different repository locations. However, Subversion does show that the two files share a common history. Before the branch-copy was made in revision 341, they used to be the same file. That's why you and Felix both see revisions 303 and 98.

There are now two independent lines of development happening. A diagram makes it easier to visualize:

Figure 4.4. The branching of one file's history

There are two important lessons that you should remember from this section.

Conceptually speaking, a Subversion repository is a sequence of directory trees. Each tree is a snapshot of how the files and directories versioned in your repository looked at various points in time. These snapshots are created as a result of client operations, and are called revisions.

Every revision begins life as a transaction tree. When doing a commit, a client builds a Subversion transaction that mirrors their local changes (plus any additional changes that might have been made to the repository since the beginning of the client's commit process), and then instructs the repository to store that tree as the next snapshot in the sequence. If the commit succeeds, the transaction is effectively promoted into a new revision tree, and is assigned a new revision number. If the commit fails for some reason, the transaction is destroyed and the client is informed of the failure.

At the moment, updates work in a similar way. The client builds a temporary transaction tree that mirrors the state of the working copy. The repository then compares that transaction tree with the revision tree at the request revision (usually the most recent, or “youngest” tree), and sends back information that informs the client about what changes are needed to transform their working copy into a replica of that revision tree. After the update completes, the temporary transaction is deleted.

The use of transaction trees is the only way to make permanent changes to repository's versioned filesystem. However, it's important to understand that the lifetime of a transaction is completely flexible. In the case of updates, transactions are temporary trees that are immediately destroyed. In the case of commits, transactions are transformed into permanent revisions (or removed if the commit fails). In the case of an error or bug, it's possible that a transaction can be accidentally left lying around in the repository (not really affecting anything, but still taking up space).

In theory, someday whole workflow applications might revolve around more fine-grained control of transaction lifetime. It is feasible to imagine a system whereby each transaction slated to become a revision is left in stasis well after the client finishes describing its changes to repository. This would enable each new commit to be reviewed by someone else, perhaps a manager or engineering QA team, who can choose to promote the transaction into a revision, or abort it.

What does all of this have to do with repository administration? The answer is simple: if you're administering a Subversion repository, you're going to have to examine revisions and transactions as part of monitoring the health of your repository.

Transactions and revisions in the Subversion repository can have properties attached to them. These properties are generic key-to-value mappings, and are used to store information about the tree to which they are attached. The names and values of these properties are stored in the repository's filesystem, along with the rest of your tree data.

Revision and transaction properties are useful for associating information with a tree that is not strictly related to the files and directories in that tree—the kind of information that isn't managed by client working copies. For example, when a new commit transaction is created in the repository, Subversion adds a property to that transaction named svn:date—a datestamp representing the time that the transaction was created. By the time the commit process is finished, and the transaction is promoted to a permanent revision, the tree has also been given a property to store the username of the revision's author (svn:author) and a property to store the log message attached to that revision (svn:log).

Revision and transaction properties are unversioned properties—as they are modified, their previous values are permanently discarded. Also, while revision trees themselves are immutable, the properties attached to those trees are not. You can add, remove, and modify revision properties at any time in the future. If you commit a new revision and later realize that you had some misinformation or spelling error in your log message, you can simply replace the value of the svn:log property with a new, corrected log message.

A hook is a program triggered by some repository event, such as the creation of a new revision or the modification of an unversioned property. Each hook is handed enough information to tell what that event is, what target(s) it's operating on, and the username of the person who triggered the event. Depending on the hook's output or return status, the hook program may continue the action, stop it, or suspend it in some way.

The hooks subdirectory is, by default, filled with templates for various repository hooks.

$ ls repos/hooks/
post-commit.tmpl          pre-revprop-change.tmpl  write-sentinels.tmpl
post-revprop-change.tmpl  read-sentinels.tmpl
pre-commit.tmpl           start-commit.tmpl

There is one template for each hook that the Subversion repository implements, and by examining the contents of those template scripts, you can see what triggers each such script to run and what data is passed to that script. Also present in many of these templates are examples of how one might use that script, in conjunction with other Subversion-supplied programs, to perform common useful tasks. To actually install a working hook, you need only place some executable program or script into the repos/hooks directory which can be executed as the name (like start-commit or post-commit) of the hook.

On Unix platforms, this means supplying a script or program (which could be a shell script, a Python program, a compiled C binary, or any number of other things) named exactly like the name of the hook. Of course, the template files are present for more than just informational purposes—the easiest way to install a hook on Unix platforms is to simply copy the appropriate template file to a new file that lacks the .tmpl extension, customize the hook's contents, and ensure that the script is executable. Windows, however, uses file extensions to determine whether or not a program is executable, so you would need to supply a program whose basename is the name of the hook, and whose extension is one of the special extensions recognized by Windows for executable programs, such as .exe or .com for programs, and .bat for batch files.

Currently there are five true hooks implemented by the Subversion repository.

Subversion will attempt to execute hooks as the same user who owns the process which is accessing the Subversion repository. In most cases, the repository is being accessed via Apache HTTP server and mod_dav_svn, so this user is the same user that Apache runs as. The hooks themselves will need to be configured with OS-level permissions that allow that user to execute them. Also, this means that any file or programs (including the Subversion repository itself) accessed directly or indirectly by the hook will be accessed as the same user. In other words, be alert to potential permission-related problems that could prevent the hook from performing the tasks you've written it to perform.

The Berkeley DB environment has it own set of default configuration values for things like the number of locks allowed to be taken out at any given time, or the size cutoff for Berkeley's journaling log file, etc. Subversion's filesystem code additionally chooses default values for some of the Berkeley DB configuration options. However, sometimes your particular repository, with its unique collection of data and access patterns, might require a different set of configuration option values.

The folks at Sleepycat (the producers of Berkeley DB) understand that different databases have different requirements, and so they have provided a mechanism for runtime overriding of many of the configuration values for the Berkeley DB environment. Berkeley checks for the presence of a file named DB_CONFIG in each environment directory, and parses the options found in that file for use with that particular Berkeley environment.

The Berkeley configuration file for your repository is located in the db environment directory, at repos/db/DB_CONFIG. Subversion itself creates this file when it creates the rest of the repository. The file initially contains some default options, as well as pointers to the Berkeley DB online documentation so you can read about what those options do. Of course, you are free to add any of the supported Berkeley DB options to your DB_CONFIG file. Just be aware that while Subversion never attempts to read or interpret the contents of the file, and makes no use of the option settings in it, you'll want to avoid any configuration changes that may cause Berkeley DB to behave in a fashion that is unexpected by the rest of the Subversion code.

Your Subversion repository will generally require very little attention once it is configured to your liking. However, there are times when some manual assistance from an administrator might be in order. The svnadmin utility provides some helpful functionality to assist you in performing such tasks as

Perhaps the most commonly used of svnadmin's subcommands is setlog. When a transaction is committed to the repository and promoted to a revision, the descriptive log message associated with that new revision (and provided by the user) is stored as an unversioned property attached to the revision itself. In other words, the repository remembers only the latest value of the property, and discards previous ones.

Sometimes a user will have an error in her log message (a misspelling or some misinformation, perhaps). If the repository is configured (using the pre-revprop-change and post-revprop-change hooks; see the section called “Hook scripts”) to accept changes to this log message after the commit is finished, then the user can “fix” her log message remotely using the svn program's propset command (see Chapter 8). However, because of the potential to lose information forever, Subversion repositories are not, by default, configured to allow changes to unversioned properties— except by an administrator.

If a log message needs to be changed by an administrator, this can be done using svnadmin setlog. This command changes the log message (the svn:log property) on a given revision of a repository, reading the new value from a provided file.

$ echo "Here is the new, correct log message" > newlog.txt
$ svnadmin setlog myrepos newlog.txt -r 388

Another common use of svnadmin is to query the repository for outstanding—possibly dead—Subversion transactions. In the event that commit should fail, the transaction is usually cleaned up. That is, the transaction itself is removed from the repository, and any data associated with (and only with) that transaction is removed as well. Occasionally, though, a failure occurs in such a way that the cleanup of the transaction never happens. This could happen for several reasons: perhaps the client operation was inelegantly terminated by the user, or a network failure might have occurred in the middle of an operation, etc. Regardless of the reason, these dead transactions serve only to clutter the repository and consume resources.

You can use svnadmin's lstxns command to list the names of the currently outstanding transactions.

$ svnadmin lstxns myrepos
19
3a1
a45
$

Each item in the resultant output can then be used with svnlook (and its --transaction option) to determine who created the transaction, when it was created, what types of changes were made in the transaction—in other words, whether or not the transaction is a safe candidate for removal! If so, the transaction's name can be passed to svnadmin rmtxns, which will perform the cleanup of the transaction. In fact, the rmtxns subcommand can take its input directly from the output of lstxns!

$ svnadmin lstxns myrepos | svnadmin rmtxns myrepos
$

If you use these two subcommands like this, you should consider making your repository temporarily inaccessible to clients. That way, no one can begin a legitimate transaction before you start your cleanup. The following is a little bit of shell-scripting that can quickly generate information about each outstanding transaction in your repository:

#!/bin/sh

### Generate informational output for all outstanding transactions in
### a Subversion repository

SVNADMIN=/usr/local/bin/svnadmin
SVNLOOK=/usr/local/bin/svnlook

REPOS=${1}
if [ x$REPOS = x ] ; then
  echo "usage: $0 REPOS_PATH"
  exit
fi

for TXN in `${SVNADMIN} lstxns ${REPOS}`; do 
  echo "---[ Transaction ${TXN} ]-------------------------------------------"
  ${SVNLOOK} info ${REPOS} --transaction ${TXN}
done

You can run the previous script using /path/to/script /path/to/repos. The output is basically a concatenation of several chunks of svnlook info output (see the section called “svnlook”), and will look something like:

$ txn-info.sh myrepos
---[ Transaction 19 ]-------------------------------------------
sally
2001-09-04 11:57:19 -0500 (Tue, 04 Sep 2001)
0
---[ Transaction 3a1 ]-------------------------------------------
harry
2001-09-10 16:50:30 -0500 (Mon, 10 Sep 2001)
39
Trying to commit over a faulty network.
---[ Transaction a45 ]-------------------------------------------
sally
2001-09-12 11:09:28 -0500 (Wed, 12 Sep 2001)
0
$

Usually, if you see a dead transaction that has no log message attached to it, this is the result of a failed update (or update-like) operation. These operations use Subversion transactions under the hood to mimic working copy state. Since they are never intended to be committed, Subversion doesn't require a log message for those transactions. Transactions that do have log messages attached are almost certainly failed commits of some sort. Also, a transaction's datestamp can provide interesting information—for example, how likely is it that an operation begun nine months ago is still active?

In short, transaction cleanup decisions need not be made unwisely. Various sources of information—including Apache's error and access logs, the logs of successful Subversion commits, and so on—can be employed in the decision-making process. Finally, an administrator can often simply communicate with a seemingly dead transaction's owner (via email, for example) to verify that the transaction is, in fact, in a zombie state.

In order to protect the data in your repository, the database back-end uses a locking mechanism. This mechanism ensures that portions of the database are not simultaneously modified by multiple database accessors, and that each process sees the data in the correct state when that data is being read from the database. When a process needs to change something in the database, it first checks for the existence of a lock on the target data. If the data is not locked, the process locks the data, makes the change it wants to make, and then unlocks the data. Other processes are forced to wait until that lock is removed before they are permitted to continue accessing that section of the database.

In the course of using your Subversion repository, fatal errors (such as running out of disk space or available memory) or interruptions can prevent a process from having the chance to remove the locks it has placed in the database. The result is that the back-end database system gets “wedged”. When this happens, any attempts to access the repository hang indefinitely (since each new accessor is waiting for a lock to go away—which isn't going to happen).

First, if this happens to your repository, don't panic. Subversion's filesystem takes advantage of database transactions and checkpoints and pre-write journaling to ensure that only the most catastrophic of events ^[3] can permanently destroy a database environment. A sufficiently paranoid repository administrator will be making off-site backups of the repository data in some fashion, but don't call your system administrator to restore a backup tape just yet.

Secondly, use the following recipe to attempt to “unwedge” your repository:

This procedure fixes almost every case of repository lock-up. Make sure that you run this command as the user that owns and manages the database, not just as root. Part of the recovery process might involve recreating from scratch various database files (shared memory regions, for example). Recovering as root will create those files such that they are owned by root, which means that even after you restore connectivity to your repository, regular users will be unable to access it.

If the previous procedure, for some reason, does not successfully unwedge your repository, you should do two things. First, move your broken repository out of the way and restore your latest backup of it. Then, send an email to the Subversion developer list (at <dev@subversion.tigris.org>) describing your problem in detail. Data integrity is an extremely high priority to the Subversion developers.

A Subversion filesystem has its data spread throughout various database tables in a fashion generally understood by (and of interest to) only the Subversion developers themselves. However, circumstances may arise that call for all, or some subset, of that data to be collected into a single, portable, flat file format. Subversion provides such a mechanism, implemented in a pair of svnadmin subcommands: dump and load.

The most common reason to dump and load a Subversion repository is due to changes in Subversion itself. As Subversion matures, there are times when certain changes made to the back-end database schema cause Subversion to be incompatible with previous versions of the repository. The recommended course of action when you are upgrading across one of those compatibility boundaries is a relatively simple process:

svnadmin dump will output a range of repository revisions that are formatted using Subversion's custom filesystem dump format. The dump format is printed to the standard output stream, while informative messages are printed to the standard error stream. This allows you to redirect the output stream to a file while watching the status output in your terminal window. For example:

$ svnadmin youngest myrepos
26
$ svnadmin dump myrepos > dumpfile
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
…
* Dumped revision 25.
* Dumped revision 26.

At the end of the process, you will have a single file (dumpfile in the previous example) that contains all the data stored in your repository in the requested range of revisions.

The other subcommand in the pair, svnadmin load, parses the standard input stream as a Subversion repository dump file, and effectively replays those dumped revisions into the target repository for that operation. It also gives informative feedback, this time using the standard output stream.

$ svnadmin load newrepos < dumpfile
<<< Started new txn, based on original revision 1
     * adding path : A ... done.
     * adding path : A/B ... done.
     …
------- Committed new rev 1 (loaded from original rev 1) >>>

<<< Started new txn, based on original revision 2
     * editing path : A/mu ... done.
     * editing path : A/D/G/rho ... done.

------- Committed new rev 2 (loaded from original rev 2) >>>

…

<<< Started new txn, based on original revision 25
     * editing path : A/D/gamma ... done.

------- Committed new rev 25 (loaded from original rev 25) >>>

<<< Started new txn, based on original revision 26
     * adding path : A/Z/zeta ... done.
     * editing path : A/mu ... done.

------- Committed new rev 26 (loaded from original rev 26) >>>

Note that because svnadmin uses standard input and output streams for the repository dump and load process, people who are feeling especially saucy can try things like this (perhaps even using different versions of svnadmin on each side of the pipe):

$ svnadmin create newrepos
$ svnadmin dump myrepos | svnadmin load newrepos

We mentioned previously that svnadmin dump outputs a range of revisions. Use the --revision option to specify a single revision to dump, or a range of revisions. If you omit this option, all the existing repository revisions will be dumped.

$ svnadmin dump myrepos --revision 23 > rev-23.dumpfile
$ svnadmin dump myrepos --revision 100:200 > revs-100-200.dumpfile

As Subversion dumps each new revision, it outputs only enough information to allow a future loader to re-create that revision based on the previous one. In other words, for any given revision in the dump file, only the items that were changed in that revision will appear in the dump. The only exception to this rule is the first revision that is dumped with the current svnadmin dump command.

By default, Subversion will not express the first dumped revision as merely differences to be applied to the previous revision. For one thing, there is no previous revision in the dump file! And secondly, Subversion cannot know the state of the repository into which the dump data will be loaded (if it ever, in fact, occurs). To ensure that the output of each execution of svnadmin dump is self-sufficient, the first dumped revision is by default a full representation of every directory, file, and property in that revision of the repository.

However, you can change this default behavior. If you add the --incremental option when you dump your repository, svnadmin will compare the first dumped revision against the previous revision in the repository, the same way it treats every other revision that gets dumped. It will then output the first revision exactly as it does the rest of the revisions in the dump range—mentioning only the changes that occurred in that revision. The benefit of this is that you can create several small dump files that can be loaded in succession, instead of one large one, like so:

$ svnadmin dump myrepos 0 1000 > dumpfile1
$ svnadmin dump myrepos 1001 2000 --incremental > dumpfile2
$ svnadmin dump myrepos 2001 3000 --incremental > dumpfile3

These dump files could be loaded into a new repository with the following command sequence:

$ svnadmin load newrepos < dumpfile1
$ svnadmin load newrepos < dumpfile2
$ svnadmin load newrepos < dumpfile3

Another neat trick you can perform with this --incremental option involves appending to an existing dump file a new range of dumped revisions. For example, you might have a post-commit hook that simply appends the repository dump of the single revision that triggered the hook. Or you might have a script like the following that runs nightly to append dump file data for all the revisions that were added to the repository since the last time the script ran.

#!/usr/bin/perl

$repos_path  = '/path/to/repos';
$dumpfile    = '/usr/backup/svn-dumpfile';
$last_dumped = '/var/log/svn-last-dumped';
 
# Figure out the starting revision (0 if we cannot read the last-dumped file,
# else use the revision in that file incremented by 1).
if (open LASTDUMPED, "$last_dumped")
{
    $new_start = <LASTDUMPED>;
    chomp $new_start;
    $new_start++;
    close LASTDUMPED;
}
else
{
    $new_start = 0;
}

# Query the youngest revision in the repos.
$youngest = `svnlook youngest $repos_path`;
chomp $youngest;

# Do the backup.
`svnadmin dump $repos_path $new_start $youngest --incremental >> $dumpfile`;

# Store a new last-dumped revision
open LASTDUMPED, "> $last_dumped" or die;
print LASTDUMPED "$youngest\n";
close LASTDUMPED;

# All done!

Used like this, svnadmin's dump and load commands can be a valuable means by which to backup changes to your repository over time in case of a system crash or some other catastrophic event.

Finally, another possible use of the Subversion repository dump file format is conversion from a different storage mechanism or version control system altogether. Because the dump file format is, for the most part, human-readable, ^[4] it should be relatively easy to describe generic sets of changes—each of which should be treated as a new revision—using this file format.

To network your repository, you basically need four components, available in two packages. You'll need Apache httpd 2.0, the mod_dav DAV module that comes with it, Subversion, and the mod_dav_svn filesystem provider module distributed with Subversion. Once you have all of those components, the process of networking your repository is as simple as:

You can accomplish the first two items either by compiling httpd and Subversion from source code, or by installing pre-built binary packages of them on your system. For the most up-to-date information on how to compile Subversion for use with Apache HTTP Server, as well as how to compile and configure Apache itself for this purpose, see the INSTALL file in the top level of the Subversion source code tree.

Once you have all the necessary components installed on your system, all that remains is the configuration of Apache via its httpd.conf file. Instruct Apache to load the mod_dav_svn module using the LoadModule directive. This directive must precede any other Subversion-related configuration items. If your Apache was installed using the default layout, your mod_dav_svn module should have been installed in the modules subdirectory of the Apache install location (often /usr/local/apache2). The LoadModule directive has a simple syntax, mapping a named module to the location of a shared library on disk:

LoadModule dav_svn_module     modules/mod_dav_svn.so

Note that if mod_dav was compiled as a shared object (instead of statically linked, directly to the httpd binary), you'll need a similar LoadModule statement for it, too.

At a later location in your configuration file, you now need to tell Apache where you keep your Subversion repository (or repositories). The Location directive has an XML-like notation, starting with an opening tag, and ending with a closing tag, with various other configuration directives in the middle. The purpose of the Location directive is to instruct Apache to do something special when handling requests that are directed at a given URL or one of its children. In the case of Subversion, you want Apache to simply hand off support for URLs that point at versioned resources to the DAV layer. You can instruct Apache to delegate the handling of all URLs whose path portions (the part of the URL that follows the server's name and the optional port number) begin with /repos/ to a DAV provider whose repository is located at /absolute/path/to/repository using the following httpd.conf syntax:

<Location /repos>
  DAV svn
  SVNPath /absolute/path/to/repository
</Location>

If you plan to support multiple Subversion repositories that will reside in the same parent directory on your local disk, you can use an alternative directive, the SVNParentPath directive, to indicate that common parent directory. For example, if you know you will be creating multiple Subversion repositories in a directory /usr/local/svn that would be accessed via URLs like http://my.server.com/svn/repos1, http://my.server.com/svn/repos2, and so on, you could use the following httpd.conf configuration syntax:

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
</Location>

Using the previous syntax, Apache will delegate the handling of all URLs whose path portions begin with /svn/ to the Subversion DAV provider, which will then assume that any items in the directory specified by the SVNParentPath directive are actually Subversion repositories. This is a particularly convenient syntax in that, unlike the use of the SVNPath directive, you don't have to restart Apache in order to create and network new repositories.

Now, at this stage, you should strongly consider the question of permissions. If you've been running Apache for some time now as your regular web server, you probably already have a collection of content—web pages, scripts and such. These items have already been configured with a set of permissions that allows them to work with Apache, or more appropriately, that allows Apache to work with those files. Apache, when used a Subversion server, will also need the correct permissions to read and write to your Subversion repository.

You will need to determine a permission system setup that satisfies Subversion's requirements without messing up any previously existing web page or script installations. This might mean changing the permissions on your Subversion repository to match those in use by other things the Apache serves for you, or it could mean using the User and Group directives in httpd.conf to specify that Apache should run as the user and group that owns your Subversion repository. There is no single correct way to set up your permissions, and each administrator will have different reasons for doing things a certain way. Just be aware that permission-related problems are perhaps the most common oversight when configuring a Subversion repository for use with Apache.

And while we are speaking about permissions, we should address how the authorization and authentication mechanisms provided by Apache fit into the scheme of things. Unless you have some system-wide configuration of these things, the Subversion repositories you make available via the Location directives will be generally accessible to everyone. In other words,

If you want to restrict either read or write access to a repository as a whole, you can use Apache's built-in access control features. The easiest such feature is the Basic authentication mechanism, which simply uses a username and password to verify that a user is who she says she is. Apache provides an htpasswd utility for managing the list of acceptable usernames and passwords, those to whom you wish to grant special access to your Subversion repository. Let's grant commit access to Sally and Harry. First, we need to add them to the password file.

$ ### First time: use -c to create the file
$ htpasswd -c /etc/svn-auth-file harry
New password: ***** 
Re-type new password: *****
Adding password for user harry
$ htpasswd /etc/svn-auth-file sally
New password: *******
Re-type new password: *******
Adding password for user sally
$

Next, you need to add some more httpd.conf directives inside your Location block to tell Apache what to do with your new password file. The AuthType directive specifies the type of authentication system to use. In this case, we want to specify the Basic authentication system. AuthName is an arbitrary name that you give for the authentication domain. Most browsers will display this name in the pop-up dialog box when the browser is querying the user for his name and password. Finally, use the AuthUserFile directive to specify the location of the password file you created using htpasswd.

After adding these three directives, your <Location> block should look something like this:

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file
</Location>

Now, at this stage, if you were to restart Apache, any Subversion operations which required authentication would harvest a username and password from the Subversion client, which would either provide previously cached values for these things, or prompt the user for the information. All that remains is to tell Apache which operations actually require that authentication.

You can restrict access on all repository operations by adding the Require valid-user directive to your <Location> block. Using our previous example, this would mean that only clients that claimed to be either harry or sally, and which provided the correct password for their respective username, would be allowed to do anything with the Subversion repository.

Sometimes you don't need to run such a tight ship. The repository at http://svn.collab.net/repos/svn which holds the Subversion source code, for example, allows anyone in the world to perform read-only repository tasks (like checking out working copies and browsing the repository with a web browser), but restricts all write operations to authenticated users. To do this type of selective restriction, you can use the Limit and LimitExcept configuration directives. Like the Location directive, these blocks have starting and ending tags, and you would nest them inside your <Location> block.

The parameters present on the Limit and LimitExcept directives are HTTP request types that are affected by that block. For example, if you wanted to disallow all access to your repository except the currently supported read-only operations, you would use the LimitExcept directive, passing the GET, PROPFIND, OPTIONS, and REPORT request type parameters. Then the previously mentioned Require valid-user directive would be placed inside the <LimitExcept> block instead of just inside the <Location> block.

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file
  <LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
  </LimitExcept>
</Location>

These are only a few simple examples. For more in-depth information about Apache access control, take a look at the Security section of the Apache documentation's tutorials collection at http://httpd.apache.org/docs-2.0/misc/tutorials.html.

Subversion makes use of the COPY request type to perform server-side copies of files and directories. As part of the sanity checking done by the Apache modules, the source of the copy is expected to be located on the same machine as the destination of the copy. To satisfy this requirement, you might need to tell mod_dav the name you use as the hostname of your server. Generally, you can use the ServerName directive in httpd.conf to accomplish this.

ServerName svn.red-bean.com

If you are using Apache's virtual hosting support via the NameVirtualHost directive, you may need to use the ServerAlias directive to specify additional names that your server is known by. Again, refer to the Apache documentation for full details.

Several of the features already provided by Apache in its role as a robust Web server can be leveraged for increased functionality or security in Subversion as well. Subversion communicates with Apache using Neon, which is a generic HTTP/WebDAV library with support for such mechanisms as SSL (the secure socket layer) and Deflate compression (the same algorithm used by the gzip and PKZIP programs to “shrink” files into smaller chunks of data). You need only to compile support for the features you desire into Subversion and Apache, and properly configure the programs to use those features.

This means that SSL-enabled Subversion clients can access SSL-enabled Apache servers and perform all communication using an encrypted protocol, all by using https: URLs with their Subversion clients instead of http: ones. Businesses that need to expose their repositories for access outside the company firewall should be conscious of the possibility that unauthorized parties could be “sniffing” their network traffic. SSL makes that kind of unwanted attention less likely to result in sensitive data leaks. Apache can be configured such that only SSL-enabled Subversion clients can communicate with the repository.

Deflate compression places a small burden on the client and server to compress and decompress network transmissions as a way to minimize the size of the actual transmission. In cases where network bandwidth is in short supply, this kind of compression can greatly increase the speed at which communications between server and client can be sent. In extreme cases, this minimized network transmission could be the difference between an operation timing out or completing successfully.

Less interesting, but equally useful, are other features of the Apache and Subversion relationship, such as the ability to specify a custom port (instead of the default HTTP port 80) or a virtual domain name by which the Subversion repository should be accessed, or the ability to access the repository through a proxy. These things are all supported by Neon, so Subversion gets that support for free.

While Subversion allows you to move around versioned files and directories without any loss of information, doing so can still disrupt the workflow of those who access the repository often and come to expect things to be at certain locations. Try to peer into the future a bit; plan ahead before placing your data under version control. By “laying out” the contents of your repositories in an effective manner the first time, you can prevent a load of future headaches.

There are a few things to consider when setting up Subversion repositories. Let's assume that as repository administrator, you will be responsible for supporting the version control system for several projects. The first decision is whether to use a single repository for multiple projects, or to give each project its own repository, or some compromise of these two.

There are benefits to using a single repository for multiple projects, most obviously the lack of duplicated maintenance. A single repository means that there is one set of hook scripts, one thing to routinely backup, one thing to dump and load if Subversion releases an incompatible new version, and so on. Also, you can easily move data between projects easily, and without losing any historical versioning information.

The downside of using a single repository is that different projects may have different commit mailing lists or different authentication and authorization requirements. Also, remember that Subversion uses repository-global revision numbers. Some folks don't like the fact that even though no changes have been made to their project lately, the youngest revision number for the repository keeps climbing because other projects are actively adding new revisions.

A middle-ground approach can be taken, too. For example, projects can be grouped by how well they relate to each other. You might have a few repositories with a handful of projects in each repository. That way, projects that are likely to want to share data can do so easily, and as new revisions are added to the repository, at least the developers know that those new revisions are at least remotely related to everyone who uses that repository.

After deciding how to organize your projects with respect to repositories, you'll probably want to think about directory hierarchies in the repositories themselves. Because Subversion uses regular directory copies for branching and tagging (see Chapter 4), the Subversion community recommends using one of two different approaches here. Both approaches employ the use of directories named trunk, meaning the directory under which the main project development occurs, and branches, which is a directory in which to create various named branches of the main development line.

The first is to place each project in a subdirectory of the root filesystem directory, with trunk and branches directories immediately under each project directory, as demonstrated in Figure 5-1.

Figure 5.1. One suggested repository layout.

The second is do the reverse—to have the trunk and branches directories immediately in the top level of the filesystem, each with subdirectories for all the projects in the repository, as demonstrated in Figure 5-2.

Figure 5.2. Another suggested repository layout.

Lay out your repository in whatever way you see fit. Subversion does not expect or enforce a layout schema—in its eyes, a directory is a directory is a directory. Ultimately, you should choose the repository arrangement that meets your needs and those of the projects that live there.

After deciding how to arrange the projects in your repository, you'll probably want to actually populate the repository with that layout and with initial project data. There are a couple of ways to do this in Subversion. You could use the svn mkdir command (see the section called “svn mkdir”) to create each directory in your skeletal repository layout, one-by-one. A quicker way to accomplish the same task is to use the svn import command (see the section called “svn import”. By first creating the layout in a temporary location on your drive, you can import the whole layout tree into the repository in a single commit:

$ mkdir tmpdir
$ cd tmpdir
$ mkdir projectA
$ mkdir projectA/trunk
$ mkdir projectA/branches
$ mkdir projectB
$ mkdir projectB/trunk
$ mkdir projectB/branches
…
$ svn import file:///path/to/repos . --message 'Initial repository layout'
Adding         projectA
Adding         projectA/trunk
Adding         projectA/branches
Adding         projectB
Adding         projectB/trunk
Adding         projectB/branches

Committed revision 1.
$ cd ..
$ rm -rf tmpdir
$

Once you have your skeletal layout in place, you can begin importing actual project data into your repository, if any such data exists yet. Once again, there are several ways to do this. You could use the svn import command. You could checkout a working copy from your new repository, move and arrange project data inside the working copy, and use the svn add and svn commit commands. But once we start talking about such things, we're no longer discussing repository administration. If you aren't already familiar with the svn client program, see Chapter 3.

At the time of writing, the configuration area only contains one item: a proxies file. By setting values in this file, your Subversion client can operate through an http proxy. (Read the file itself for details; it should be self-documenting.)

Soon—very soon—a config file will exist in this area for defining general user preferences. For example, the preferred $EDITOR to use, options to pass through to svn diff, preferences for date/time formats, and so on.

On Unix, an administrator can create “global” Subversion preferences by creating and populating an /etc/subversion/ area. The per-user ~/.subversion/ configuration will still override these defaults, however.

On Win32, an administrator has the option of creating three other locations: a global Subversion folder in the “All Users” area, a collection of global registry settings, or a collection of per-user registry settings. The registry settings are set in:

HKCU\Software\Tigris.org\Subversion\Proxies
HKCU\Software\Tigris.org\Subversion\Config
etc.
      

To clarify, here is the order Subversion searches for run-time settings on Win32. Each subsequent location overrides the previous one:

Subversion has no particular policy regarding properties; they can be used for any purpose. The only restriction is that Subversion has reserved the svn: name prefix for itself. A number of special “magic” properties begin with this prefix. We'll cover these features here.

$ svn status
M  ./foo.c
?  ./foo.o
?  ./foo.c~
        

$ svn propget svn:ignore .
*.o
*~
        

Here is the latest report from the front lines.
$LastChangedDate$
Cumulus clouds are appearing more frequently as summer approaches.
        

Here is the latest report from the front lines.
$LastChangedDate: 2002-07-22 21:42:37 -0700 (Mon, 22 Jul 2002) $
Cumulus clouds are appearing more frequently as summer approaches.
        

$ svn propset svn:keywords "Date Author" foo.c
property `svn:keywords' set on 'foo.c'
        

When referring to Subversion's Repository Layer, we're generally talking about two libraries—the repository library, and the filesystem library. These libraries provide the storage and reporting mechanisms for the various revisions of your version-controlled data. This layer is connected to the Client Layer via the Repository Access Layer, and is, from the perspective of the Subversion user, the stuff at the "other end of the line."

The Subversion Filesystem is accessed via the libsvn_fs API, and is not a kernel-level filesystem that one would install in an operating system (like the Linux ext2 or NTFS), but a virtual filesystem. Rather than storing "files" and "directories" as real files and directories (as in, the kind you can navigate through using your favorite shell program), it uses a database system for its back-end storage mechanism. Currently, the database system in use is Berkeley DB. ^[6] However, there has been considerable interest by the development community in giving future releases of Subversion the ability to use other back-end database systems, perhaps through a mechanism such as Open Database Connectivity (ODBC).

The filesystem API exported by libsvn_fs contains the kinds of functionality you would expect from any other filesystem API: you can create and remove files and directories, copy and move them around, modify file contents, and so on. It also has features that are not quite as common, such as the ability to add, modify, and remove metadata ("properties") on each file or directory. Furthermore, the Subversion Filesystem is a versioning filesystem, which means that as you make changes to your directory tree, Subversion remembers what your tree looked like before those changes. And before the previous changes. And the previous ones. And so on, all the way back through versioning time to (and just beyond) the moment you first started adding things to the filesystem.

All the modifications you make to your tree are done within the context of a Subversion transaction. The following is a simplified general routine for modifying your filesystem:

Once you have committed your transaction, your filesystem modifications are permanently stored as historical artifacts. Each of these cycles generates a single new revision of your tree, and each revision is forever accessible as an immutable snapshot of "the way things were."

Most of the functionality provided by the filesystem interface comes as an action that occurs on a filesystem path. That is, from outside of the filesystem, the primary mechanism for describing and accessing the individual revisions of files and directories comes through the use of path strings like /foo/bar, just as if you were addressing files and directories through your favorite shell program. You add new files and directories by passing their paths-to-be to the right API functions. You query for information about them by the same mechanism.

Unlike most filesystems, though, a path alone is not enough information to identify a file or directory in Subversion. Think of a directory tree as a two-dimensional system, where a node's siblings represent a sort of left-and-right motion, and descending into subdirectories a downward motion. Figure 7-2 shows a typical representation of a tree as exactly that.

Figure 7.2. Files and directories in two dimensions

Of course, the Subversion filesystem has a nifty third dimension that most filesystems do not have—Time! ^[7] In the filesystem interface, nearly every function that has a path argument also expects a root argument. This svn_fs_root_t argument describes either a revision or a Subversion transaction (which is usually just a revision-to-be), and provides that third-dimensional context needed to understand the difference between /foo/bar in revision 32, and the same path as it exists in revision 98. Figure 7-3 shows revision history as an added dimension to the Subversion filesystem universe.

Figure 7.3. Revisioning Time—the third dimension!

As we mentioned earlier, the libsvn_fs API looks and feels like any other filesystem, except that it has this wonderful versioning capability. It was designed to be usable by any program interested in a versioning filesystem. Not coincidentally, Subversion itself is interested in that functionality. But while the filesystem API should be sufficient for basic file and directory versioning support, Subversion wants more—and that is where libsvn_repos comes in.

The Subversion repository library (libsvn_repos) is basically a wrapper library around the filesystem functionality. This library is responsible for creating the repository layout, making sure that the underlying filesystem is initialized, and so on. Libsvn_repos also implements a set of hooks—scripts that are executed by the repository code when certain actions take place. These scripts are useful for notification, authorization, or whatever purposes the repository administrator desires. This type of functionality, and other utility provided by the repository library, is not strictly related to implementing a versioning filesystem, which is why it was placed into its own library.

Developers who wish to use the libsvn_repos API will find that it is not a complete wrapper around the filesystem interface. That is, only certain major events in the general cycle of filesystem activity are wrapped by the repository interface. Some of these include the creation and commit of Subversion transactions, and the modification of revision properties. These particular events are wrapped by the repository layer because they have hooks associated with them. In the future, other events may be wrapped by the repository API. All of the remaining filesystem interaction will continue to occur directly with libsvn_fs API, though.

For example, here is a code segment that illustrates the use of both the repository and filesystem interfaces to create a new revision of the filesystem in which a directory is added. Note that in this example (and all others throughout this book), the SVN_ERR macro simply checks for a non-successful error return from the function it wraps, and returns that error if it exists.

/* Create a new directory at the path NEW_DIRECTORY in the Subversion
   repository located at REPOS_PATH.  Perform all memory allocation in
   POOL.  This function will create a new revision for the addition of
   NEW_DIRECTORY.  */
static svn_error_t *
make_new_directory (const char *repos_path,
                    const char *new_directory,
                    apr_pool_t *pool)
{
  svn_error_t *err;
  svn_repos_t *repos;
  svn_fs_t *fs;
  svn_revnum_t youngest_rev;
  svn_fs_txn_t *txn;
  svn_fs_root_t *txn_root;
  const char *conflict_str;

  /* Open the repository located at REPOS_PATH.  */
  SVN_ERR (svn_repos_open (&repos, repos_path, pool));

  /* Get a pointer to the filesystem object that is stored in
     REPOS.  */
  fs = svn_repos_fs (repos);

  /* Ask the filesystem to tell us the youngest revision that
     currently exists.  */
  SVN_ERR (svn_fs_youngest_rev (&youngest_rev, fs, pool));

  /* Begin a new transaction that is based on YOUNGEST_REV.  We are
     less likely to have our later commit rejected as conflicting if we
     always try to make our changes against a copy of the latest snapshot
     of the filesystem tree.  */
  SVN_ERR (svn_fs_begin_txn (&txn, fs, youngest_rev, pool));

  /* Now that we have started a new Subversion transaction, get a root
     object that represents that transaction.  */
  SVN_ERR (svn_fs_txn_root (&txn_root, txn, pool));
  
  /* Create our new directory under the transaction root, at the path
     NEW_DIRECTORY.  */
  SVN_ERR (svn_fs_make_dir (txn_root, new_directory, pool));

  /* Commit the transaction, creating a new revision of the filesystem
     which includes our added directory path.  */
  err = svn_repos_fs_commit_txn (&conflict_str, repos, 
                                 &youngest_rev, txn);
  if (err == SVN_NO_ERROR)
    {
      /* No error?  Excellent!  Print a brief report of our success.  */
      printf ("Directory '%s' was successfully added as new revision "
              "'%" SVN_REVNUM_T_FMT "'.\n", new_directory, youngest_rev);
    }
  else if (err->apr_err == SVN_ERR_FS_CONFLICT))
    {
      /* Uh-oh.  Our commit failed as the result of a conflict
         (someone else seems to have made changes to the same area 
         of the filesystem that we tried to modify).  Print an error
         message.  */
      printf ("A conflict occured at path '%s' while attempting "
              "to add directory '%s' to the repository at '%s'.\n", 
              conflict_str, new_directory, repos_path);
    }
  else
    {
      /* Some other error has occurred.  Print an error message.  */
      printf ("An error occured while attempting to add directory '%s' "
              "to the repository at '%s'.\n", 
              new_directory, repos_path);
    }

  /* "Disconnect" from the repository.  */
  (void) svn_repos_close (repos);

  /* Return the result of the attempted commit to our caller.  */
  return err;
} 

As you can see in the previous code segment, calls were made to both the repository and filesystem interfaces. Note that we could just as easily have committed the transaction using svn_fs_commit_txn. But the filesystem API knows nothing about the repository library's hook mechanism. If you want your Subversion repository to automatically perform some set of non-Subversion tasks every time you commit a transaction (like, for example, sending an email that describes all the changes made in that transaction to your developer mailing list), you need to use the libsvn_repos-wrapped version of that function—svn_repos_fs_commit_txn. This function will actually first run the "pre-commit" hook script if one exists, then commit the transaction, and finally will run a "post-commit" hook script. The hooks provide a special kind of reporting mechanism that does not really belong in the core filesystem library itself.

The hook mechanism requirement is but one of the reasons for the abstraction of a separate repository library from the rest of the filesystem code. The libsvn_repos API provides several other important utilities to Subversion. These include the abilities to:

As Subversion continues to evolve, the repository library will grow with the filesystem library to offer increased functionality and configurable option support.

If the Subversion Repository Layer is at "the other end of the line", the Repository Access Layer is the line itself. Charged with marshalling data between the client libraries and the repository, this layer includes the libsvn_ra module loader library, the RA modules themselves (which currently includes libsvn_ra_local and libsvn_ra_dav), and any additional libraries needed by one or more of those RA modules, such as the mod_dav_svn Apache module with which libsvn_ra_dav communicates.

Since Subversion uses URLs to identify its repository resources, the protocol portion of the URL schema (usually http:, https:, or file:) is used to determine which RA module will handle the communications. Each module registers a list of the protocols it knows how to "speak" so that the RA loader can, at runtime, determine which module to use for the task at hand. You can determine which RA modules are available to the Subversion command-line client, and what protocols they claim to support, by running svn --version. In the following example, both of the RA modules that are included with Subversion have been successfully located by the client program.

$ svn --version
svn, version 0.14.3 (dev build)
   compiled Oct  7 2002, 07:52:08

Copyright (C) 2000-2002 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/

The following repository access (RA) modules are available:

* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
  - handles 'http' schema
  - handles 'https' schema
* ra_local : Module for accessing a repository on local disk.
  - handles 'file' schema

For the majority of this section we will be highlighting libsvn_ra_dav, which is the most commonly used RA module for real-world repository communications. Unlike libsvn_ra_local, which offers no networking capabilities at all, libsvn_ra_dav is designed for use by clients that are being run on different machines than the servers with which they communicating, specifically machines reached using URLs that contain the http: or https: protocol portions. To understand how this module works, we should first mention a couple of other key components in this particular configuration of the Repository Access Layer—the powerful Apache HTTP Server, and the Neon HTTP/WebDAV client library.

Subversion's official server is the Apache HTTP Server. Apache is a time-tested, extensible open-source server process that is ready for serious use. It can sustain a high network load and runs on many platforms. The Apache server supports a number of different standard authentication protocols, and can be extended through the use of modules to support many others. It also supports optimizations like network pipelining and caching. By using Apache as a server, Subversion gets all of these features for free. And since most firewalls already allow HTTP traffic to pass through, sysadmins typically don't even have to change their firewall configurations to allow Subversion to work.

Subversion uses WebDAV (with DeltaV) as its network protocol. You can read more about this in the WebDAV section of this chapter, but in short, WebDAV and DeltaV are extensions to the standard HTTP 1.1 protocol that enable sharing and versioning of files over the web. Apache 2.0 comes with mod_dav, an Apache module that understands the DAV extensions to HTTP. Subversion itself supplies mod_dav_svn, though, which is another Apache module that works in conjunction with (really, as a back-end to) mod_dav to provide Subversion's specific implementations of WebDAV and DeltaV.

When communicating with a repository over HTTP, the RA loader library chooses libsvn_ra_dav as the proper access module. The Subversion client makes calls into the generic RA interface, and libsvn_ra_dav maps those calls (which embody rather large-scale Subversion actions) to a set of HTTP/WebDAV requests. Using the Neon library, libsvn_ra_dav transmits those requests to the Apache server. Apache receives these requests (exactly as it does generic HTTP requests that your web browser might make), notices that the requests are directed at a URL that is configured as a DAV location (using the Location directive in httpd.conf), and hands the request off to its own mod_dav module. When properly configured, mod_dav knows to use Subversion's mod_dav_svn for any filesystem-related needs, as opposed to the generic mod_dav_fs that comes with Apache. So ultimately, the client is communicating with mod_dav_svn, which binds directly to the Subversion Repository Layer.

That was a simplified description of the actual exchanges taking place, though. For example, the Subversion repository might be protected by Apache's authorization directives. This could result in initial attempts to communicate with the repository being rejected by Apache on authorization grounds. At this point, libsvn_ra_dav gets back the notice from Apache that insufficient identification was supplied, and calls back into the Client Layer to get some updated authentication data. If the data is supplied correctly, and the user has the permissions that Apache seeks, libsvn_ra_dav's next automatic attempt at performing the original operation will be granted, and all will be well. If sufficient authentication information cannot be supplied, the request will ultimately fail, and the client will report the failure to the user.

By using Neon and Apache, Subversion gets free functionality in several other complex areas, too. For example, if Neon finds the OpenSSL libraries, it allows the Subversion client to attempt to use SSL-encrypted communications with the Apache server (whose own mod_ssl can "speak the language"). Also, both Neon itself and Apache's mod_deflate can understand the "deflate" algorithm (the same used by the PKZIP and gzip programs), so requests can be sent in smaller, compressed chunks across the wire. Other complex features that Subversion hopes to support in the future include the ability to automatically handle server-specified redirects (for example, when a repository has been moved to a new canonical URL) and taking advantage of HTTP pipelining.

Of course, not all communications with a Subversion repository require a powerhouse server process and a network layer. For users who simply wish to access the repositories on their local disk, they may do so using file: URLs and the functionality provided by libsvn_ra_local. This RA module binds directly with the repository and filesystem libraries, so no network communication is required at all.

And for those who wish to access a Subversion repository using still another protocol—well, that is precisely why the Repository Access Layer is modularized! Developers can simply write a new library that implements the RA interface on one side and communicates with the repository on the other. Your new protocol could use straight socket connections (bypassing the likes of Apache), or inter-process communication (IPC) calls, or—let's get crazy, shall we?—you could even implement an email-based protocol. Subversion supplies the APIs; you supply the creativity.

On the client side, the Subversion working copy is where all the action takes place. The bulk of functionality implemented by the client-side libraries exists for the sole purpose of managing working copies—directories full of files and other subdirectories which serve as a sort of local, editable "reflection" of one or more repository locations—and propogating changes to and from the Repository Access layer.

Subversion's working copy library, libsvn_wc, is directly responsible for managing the data in the working copies. To accomplish this, the library stores administrative information about each working copy directory within a special subdirectory. This subdirectory, named .svn is present in each working copy directory and contains various other files and directories which record state and provide a private workspace for administrative action. For those familiar with CVS, this .svn subdirectory is similar in purpose to the CVS administrative directories found in CVS working copies. For more information about the .svn administrative area, see the section called “Inside the Working Copy Administration Area”in this chapter.

The Subversion client library, libsvn_client, has the broadest responsibility; its job is to mingle the functionality of the working copy library with that of the Repository Access Layer, and then to provide the highest-level API to any application that wishes to perform general revision control actions. For example, the function svn_client_checkout takes a URL as an argument. It passes this URL to the RA layer and opens an authenticated session with a particular repository. It then asks the repository for a certain tree, and sends this tree into the working copy library, which then writes a full working copy to disk (.svn directories and all).

The client library is designed to be used by any application. While the Subversion source code includes a standard command-line client, it should be very easy to write any number of GUI clients on top of the client library. New GUIs (or any new client, really) for Subversion need not be clunky wrappers around the included command-line client—they have full access via the libsvn_client API to same functionality, data, and callback mechanisms that the command-line client uses.

Along with Subversion's own datatype, you will see many references to datatypes that begin with "apr_"—symbols from the Apache Portable Runtime (APR) library. APR is Apache's portability library, originally carved out of its server code as an attempt to separate the OS-specific bits from the OS-independent portions of the code. The result was a library that provides a generic API for performing operations that differ mildly—or wildly—from OS to OS. While Apache HTTP Server was obviously the first user of the APR library, the Subversion developers immediately recognized the value of using APR as well. This means that there are practically no OS-specific code portions in Subversion itself. Also, it means that the Subversion client compiles and runs anywhere that the server does. Currently this list includes all flavors of Unix, Win32, BeOS, OS/2, and Mac OS X.

In addition to providing consistent implementations of system calls that differ across operating systems, ^[8] APR gives Subversion immediate access to many custom datatypes, such as dynamic arrays and hash tables. Subversion uses these types extensively throughout the codebase. But perhaps the most pervasive APR datatype, found in nearly every Subversion API prototype, is the apr_pool_t—the APR memory pool. Subversion uses pools internally for all its memory allocation needs, and while a person coding against the Subversion APIs is not required to do the same, they are required to provide pools to the API functions that need them. This means that users of the Subversion API must also link against APR, must call apr_initialize() to initialize the APR subsystem, and then must acquire a pool for use with Subversion API calls. See the section called “Programming with Memory Pools” for more information.

With remote version control operation as the whole point of Subversion's existence, it makes sense that some attention has been paid to internationalization (i18n) support. After all, while "remote" might mean "across the office", it could just as well mean "across the globe." To facilitate this, all of Subversion's public interfaces that accept path arguments expect those paths to be canonicalized, and encoded in UTF-8. This means, for example, that any new client binary that drives the libsvn_client interface needs to first convert paths from the locale-specific encoding to UTF-8 before passing those paths to the Subversion libraries, and then re-convert any resultant output paths from Subversion back into the locale's encoding before using those paths for non-Subversion purposes. Fortunately, Subversion provides a suite of functions (see subversion/include/svn_utf.h) that can be used by any program to do these conversions.

Also, Subversion APIs require all URL parameters to be properly URI-encoded. So, instead of passing file:///home/username/My File.txt as the URL of a file named My File.txt, you need to pass file:///home/username/My%20File.txt. Again, Subversion supplies helper functions that your application can use—svn_path_uri_encode and svn_path_uri_decode, for URI encoding and decoding, respectively.

If you are interested in using the Subversion libraries in conjunction with something other than a C program—say a Python script or Java application—Subversion has some initial support for this via the Simplified Wrapper and Interface Generator (SWIG). The SWIG bindings for Subversion are located in subversion/bindings/swig and are slowly maturing into a usable state. These bindings allow you to call Subversion API functions indirectly, using wrappers that translate the datatypes native to your scripting language into the datatypes needed by Subversion's C libraries.

There is an obvious benefit to accessing the Subversion APIs via a language binding—simplicity. Generally speaking, languages such as Python and Perl are much more flexible and easy to use than C or C++. The sort of high-level datatypes and context-driven typechecking provided by these languages are often better at handling information that comes from users. As you know, only a human can botch up the input to a program as well as they do, and the scripting-type language simply handle that misinformation more gracefully. Of course, often that flexibility comes at the cost of performance. That is why using a tightly-optimized, C-based interface and library suite, combined with a powerful, flexible binding language is so appealing.

Let's look at an example that uses Subversion's Python SWIG bindings. Our example will do the same thing as our last example. Note the difference in size and complexity of the function this time!

from svn import fs
import os.path

def crawl_filesystem_dir (root, directory, pool):
  """Recursively crawl DIRECTORY under ROOT in the filesystem, and return
  a list of all the paths at or below DIRECTORY.  Use POOL for all 
  allocations."""

  # Get the directory entries for DIRECTORY.
  entries = fs.dir_entries(root, directory, pool)

  # Initialize our returned list with the directory path itself.
  paths = [directory]

  # Loop over the entries
  names = entries.keys()
  for name in names:
    # Calculate the entry's full path.
    full_path = os.path.join(basepath, name)

    # If the entry is a directory, recurse.  The recursion will return
    # a list with the entry and all its children, which we will add to
    # our running list of paths.
    if fs.is_dir(fsroot, full_path, pool):
      subpaths = crawl_filesystem_dir(root, full_path, pool)
      paths.extend(subpaths)

    # Else, it is a file, so add the entry's full path to the FILES list.
    else:
      paths.append(full_path)

  return paths

An implementation in C of the previous example would stretch on quite a bit longer. The same routine in C would need to pay close attention to memory usage, and need to use custom datatypes for representing the hash of entries and the list of paths. Python has hashes and lists (called "dictionaries" and "sequences", respectively) as built-in datatypes, and provides a wonderful selection of methods for operating on those types. And since Python uses reference counting and garbage collection, users of the language don't have to bother themselves with allocating and freeing memory.

In the previous section of this chapter, we mentioned the libsvn_client interface, and how it exists for the sole purpose of simplifying the process of writing a Subversion client. The following is a brief example of how that library can be accessed via the SWIG bindings. In just a few lines of Python, you can check out a fully functional Subversion working copy!

#!/usr/bin/env python
import sys
from svn import util, _util, _client

def usage():
  print "Usage: " + sys.argv[0] + " URL PATH\n"
  sys.exit(0)

def run(url, path):
  # Initialize APR and get a POOL.
  _util.apr_initialize()
  pool = util.svn_pool_create(None)

  # Checkout the HEAD of URL into PATH (silently)
  _client.svn_client_checkout(None, None, url, path, -1, 1, None, pool)

  # Cleanup our POOL, and shut down APR.
  util.svn_pool_destroy(pool)
  _util.apr_terminate()

if __name__ == '__main__':
  if len(sys.argv) != 3:
    usage()
  run(sys.argv[1], sys.argv[2])

Currently, it is Subversion's Python bindings that are the most complete. Some attention is also being given to the Java bindings. Once you have the SWIG interface files properly configured, generation of the specific wrappers for all the supported SWIG languages (which currently includes versions of Tcl, Python, Perl, Java, Ruby, Mzscheme, Guile, and PHP) should theoretically be trivial. Still, some extra programming is required to compensate for complex APIs that SWIG needs some help generalizing. For more information on SWIG itself, see the project's website at http://www.swig.org.

Perhaps the single most important file in the .svn directory is the entries file. The entries file is an XML document which contains the bulk of the administrative information about a versioned resource in a working copy directory. It is this one file which tracks the repository URLs, pristine revision, file checksums, pristine text and property timestamps, scheduling and conflict state information, last-known commit information (author, revision, timestamp), local copy history—practically everything that a Subversion client is interested in knowing about a versioned (or to-be-versioned) resource!

The following is an example of an actual entries file:

<wc-entries
<entry
   committed-rev="1"
   name="svn:this_dir"
   committed-date="2002-09-24T17:12:44.064475Z"
   url="file:///home/cmpilato/tests/.greek-repo/A/D"
   kind="dir"
   revision="1"/>
<entry
   committed-rev="1"
   name="gamma"
   text-time="2002-09-26T21:09:02.000000Z"
   committed-date="2002-09-24T17:12:44.064475Z"
   checksum="QSE4vWd9ZM0cMvr7/+YkXQ=="
   kind="file"
   prop-time="2002-09-26T21:09:02.000000Z"/>
<entry
   name="zeta"
   kind="file"
   schedule="add"
   revision="0"/>
<entry
   url="file:///home/cmpilato/tests/.greek-repo/A/B/delta"
   name="delta"
   kind="file"
   schedule="add"
   revision="0"/>
<entry
   name="G"
   kind="dir"/>
<entry
   name="H"
   kind="dir"
   schedule="delete"/>
</wc-entries>

As you can see, the entries file is essentially a list of entries. Each entry tag represents one of three things: the working copy directory itself (noted by having its name attribute set to "svn:this-dir"), a file in that working copy directory (noted by having its kind attribute set to "file"), or a subdirectory in that working copy (kind here is set to "dir"). The files and subdirectories whose entries are stored in this file are either already under version control, or (as in the case of the file named zeta above) are scheduled to be added to version control when the user next commits this working copy directory's changes. Each entry has a unique name, and each entry has a node kind.

Developers should be aware of some special rules that Subversion uses when reading and writing its entries files. While each entry has a revision and URL associated with it, note that not every entry tag in the sample file has explicit revision or url attributes attached to it. Subversion allows entries to not explicitly store those two attributes when their values are the same as (in the revision case) or trivially calculable from ^[9] (in the url case) the data stored in the "svn:this-dir" entry. Note also that for subdirectory entries, Subversion stores only the crucial attributes—name, kind, url, revision, and schedule. In an effort to reduce duplicated information, Subversion dictates that the method for determining the full set of information about a subdirectory is to traverse down into that subdirectory, and read the "svn:this-dir" entry from its own .svn/entries file. However, a reference to the subdirectory is kept in its parent's entries file, with enough information to permit basic versioning operations in the event that the subdirectory itself is actually missing from disk.

As mentioned before, the .svn directory also holds the pristine "text-base" versions of files. Those can be found in .svn/text-base. The benefits of these pristine copies are multiple—network-free checks for local modifications and "diff" reporting, network-free reversion of modified or missing files, smaller transmission of changes to the server—but comes at the cost of having each versioned file stored at least twice on disk. These days, this seems to be a negligible penalty for most files. However, the situation gets uglier as the size of your versioned files grows. Some attention is being given to making the presence of the "text-base" an option. Ironically though, it is as your versioned files' sizes get larger that the existence of the "text-base" becomes more crucial—who wants to transmit a huge file across a network just because they want to commit a tiny change to it?

Similar in purpose to the "text-base" files are the property files and their pristine "prop-base" copies, located in .svn/props and .svn/prop-base respectively. Since directories can have properties, too, there are also .svn/dir-props and .svn/dir-prop-base files. Each of these property files ("working" and "base" versions) uses a simple "hash-on-disk" file format for storing the property names and values.

We will wrap up our peek at the working copy internals by noting the .svn/auth directory. Here is where Subversion stores a cache of various data used when authenticating against a server that requires such. Subversion uses OS-level permissioning to secure the contents of these files, but also gives users the option of disabling this cache altogether (via a per-use command-line argument, or more conveniently via the user's configuration files).

The first step in community participation is to find a way to stay on top of the latest happenings. To do this most effectively, you will want to subscribe to the main developer discussion list (<dev@subversion.tigris.org>) and commit mail list (<svn@subversion.tigris.org>). By following these lists even loosely, you will have access to important design discussions, be able to see actual changes to Subversion source code as they occur, and be able to witness peer reviews of those changes and proposed changes. These e-mail based discussion lists are the primary communication media for Subversion development. See the Mailing Lists section of the website for other Subversion-related lists you might be interested in.

But how do you know what needs to be done? It is quite common for a programmer to have the greatest intentions of helping out with the development, yet be unable to find a good starting point. After all, not many folks come to the community having already decided on a particular itch they would like to scratch. But by watching the developer discussion lists, you might see mentions of existing bugs or feature requests fly by that particularly interest you. Also, a great place to look for outstanding, unclaimed tasks is the Issue Tracking database on the Subversion website. There you will find the current list of known bugs and feature requests. If you want to start with something small, look for issues marked as "bite-sized".

To edit the code, you need to have the code. This means you need to check out a working copy from the public Subversion source repository. As straightforward as that might sound, the task can be slightly tricky. Because Subversion's source code is versioned using Subversion itself, you actually need to "bootstrap" by getting a working Subversion client via some other method. The most common methods inclue downloading the latest binary distribution (if such is available for your platform), or downloading the latest source tarball and building your own Subversion client. If you build from source, make sure read the INSTALL file in the top level of the source tree for instructions.

After you have a working Subversion client, you are now poised to checkout a working copy of the Subversion source repository from http://svn.collab.net/repos/svn/trunk: ^[10]

$ svn checkout http://svn.collab.net/repos/svn/trunk subversion
A  HACKING
A  INSTALL
A  README
A  autogen.sh
A  build.conf
...

The above command will checkout the bleeding-edge, latest version of the Subversion source code into a subdirectory named subversion in your current working directory. Obviously, you can adjust that last argument as you see fit. Regardless of what you call the new working copy directory, though, after this operation completes, you will now have the Subversion source code. Of course, you will still need to fetch a few helper libraries (apr, apr-util, etc.)—see the INSTALL file in the top level of the working copy for details.

Now that you have a working copy containing the latest Subversion source code, you will most certainly want to take a cruise through the HACKING file in that working copy's top-level directory. The HACKING file contains general instructions for contributing to Subversion, including how to properly format your source code for consistency with the rest of the codebase, how to describe your proposed changes with an effective change log message, how to test your changes, and so on. Commit priveleges on the Subversion source repository are earned—a government by meritocracy. ^[11] The HACKING file is an invaluable resource when it comes to making sure that your proposed changes earn the praises they deserve without being rejected on technicalities.

With the code and community policy understanding in hand, you are ready to make your changes. It is best to try to make smaller but related sets of changes, even tackling larger tasks in stages, instead of making huge, sweeping modifications. Your proposed changes will be easier to understand (and therefore easier to review) if you disturb the fewest lines of code possible to accomplish your task properly. After making each set of proposed changes, your Subversion tree should be in a state in which the software compiles with no warnings.

Subversion has a fairly thorough ^[12] regression test suite, and your proposed changes are expected to not cause any of those tests to fail. By running make check (in Unix) from the top of the source tree, you can sanity-check your changes. The fastest way to get your code contributions rejected (other than failing to supply a good log message) is to submit changes that cause failure in the test suite.

In the best-case scenario, you will have actually added appropriate tests to that test suite which verify that your proposed changes actually work as expected. In fact, sometimes the best contribution a person can make is solely the addition of new tests. You can write regression tests for functionality that currently works in Subversion as a way to protect against future changes that might trigger failure in those areas. Also, you can write new tests that demonstrate known failures. For this purpose, the Subversion test suite allows you to specify that a given test is expected to fail (called an XFAIL), and so long as Subversion fails in the way that was expected, a test result of XFAIL itself is considered a success. Ultimately, the better the test suite, the less time wasted on diagnosing potentially obscure regression bugs.

After making your modifications to the source code, compose a clear and concise log message to describe those changes and the reasons for them. Then, send an email to the developers list containing your log message and the output of svn diff (from the top of your Subversion working copy). If the community members consider your changes acceptable, someone who has commit priveleges (permission to make new revisions in the Subversion source repository) will add your changes to the public source code tree. Recall that permission to directly commit changes to the repository is granted on merit—if you demonstrate comprehension of Subversion, programming competency, and a "team spirit", you will likely be awarded that permission.