I'm a lazy linux piper

Mar

2009

Submitted by Nick on Wed, 11th Mar 2009, 11:20 and has been commented on 9 times.

Tagged in Drupal, Geek, Howto, Linux, Svn & Tip

This morning I updated a site to the latest release of Drupal 5.16. Nothing special there at all. I've done that many times as has (hopefully) mabye other drupal devs… However, I'm a bit of a newbie when it comes to SVN. Didn't I mention this drupal site was in an SVN repository? ;-)

So, I've svn copy'd the trunk to an "update_to_5.16" branch, checked out this branch and done a cvs up -dP -r DRUPAL-5-16. Everything is going according to plan so far. Next I run svn status to get a list of files which I need to mark as added or deleted (or to list anything else which has gone wrong). What happens next is I get a list of hundreds of CVS Template files which have been added to the CVS folders. For example…

?      profiles/default/CVS/Template
M      profiles/default/CVS/Entries
M      profiles/default/CVS/Tag
?      profiles/CVS/Template
M      profiles/CVS/Tag
?      themes/bluemarine/CVS/Template
M      themes/bluemarine/CVS/Entries
M      themes/bluemarine/CVS/Tag
M      themes/engines/phptemplate/phptemplate.engine
?      themes/engines/phptemplate/CVS/Template
M      themes/engines/phptemplate/CVS/Entries
M      themes/engines/phptemplate/CVS/Tag
?      themes/engines/CVS/Entries.Log
?      themes/engines/CVS/Template
M      themes/engines/CVS/Tag
?      themes/garland/images/CVS/Template
M      themes/garland/images/CVS/Entries
M      themes/garland/images/CVS/Tag
?      themes/garland/minnelli/CVS/Template
M      themes/garland/minnelli/CVS/Entries
M      themes/garland/minnelli/CVS/Tag
?      themes/garland/minnelli/color/CVS/Template
M      themes/garland/minnelli/color/CVS/Entries
M      themes/garland/minnelli/color/CVS/Tag
?      themes/garland/CVS/Template
&hellip;
&hellip;

So as you can see from this (shortened) list, several Entries & Tag files have been modified (along with 1 core file). The files labelled with a ? are ones which are new in the Working Copy which are not in the SVN repository.

Now those who know me or read my blog may know I dont enjoy tedious jobs. I think the definition of tedious could be described as typing svn add themes/garland/minnelli/CVS/Template hundreds of times (for each ? file in list which was much longer that than). What do I do when I have to do tedious jobs? I find a bash script which will do it for me and then blog about it!

So, firstly, I tried piping the svn st results through grep to filter out the results which need adding. I also add a filter to only add Template entries to start with.

svn st | grep ^? | grep Template$

The <em>svn st</em> is short-hand for svn status.

Next we need to strip out the question marks and spaces at the beginning of each line. Enter gawk. I'm not an expert at gawk, but this time it seemed to Just Work^TM.

svn st | grep ^? | grep Template$ | gawk '{ print $2 }'

I then trid to just pipe that into svn add. Subversion did not appreciate this and complained about there not being enough arguments. "Ok", I thought — I've seen this problem before when trying to work with the results of a list of files produced by the find command… What about the xargs command? Would you adam and eve it? It worked!

$ svn st | grep ^? | grep Template$ | gawk '{ print $2 }' | xargs svn add
A         profiles/default/CVS/Template
A         profiles/CVS/Template
A         themes/bluemarine/CVS/Template
A         themes/engines/phptemplate/CVS/Template
A         themes/engines/CVS/Template
A         themes/garland/images/CVS/Template
A         themes/garland/minnelli/CVS/Template
A         themes/garland/minnelli/color/CVS/Template
A         themes/garland/CVS/Template
A         themes/garland/color/CVS/Template
A         themes/CVS/Template
A         themes/chameleon/CVS/Template
A         themes/chameleon/marvin/CVS/Template
A         themes/pushbutton/CVS/Template
A         scripts/CVS/Template
A         sites/default/CVS/Template
A         sites/all/CVS/Template
A         sites/CVS/Template
A         misc/farbtastic/CVS/Template
A         misc/CVS/Template
A         CVS/Template
A         includes/CVS/Template
A         modules/aggregator/CVS/Template
A         modules/blog/CVS/Template
A         modules/system/CVS/Template
A         modules/upload/CVS/Template
A         modules/filter/CVS/Template
A         modules/node/CVS/Template
A         modules/drupal/CVS/Template
A         modules/help/CVS/Template
A         modules/forum/CVS/Template
A         modules/book/CVS/Template
A         modules/block/CVS/Template
A         modules/statistics/CVS/Template
A         modules/contact/CVS/Template
A         modules/CVS/Template
A         modules/tracker/CVS/Template
A         modules/path/CVS/Template
A         modules/ping/CVS/Template
A         modules/locale/CVS/Template
A         modules/profile/CVS/Template
A         modules/watchdog/CVS/Template
A         modules/comment/CVS/Template
A         modules/menu/CVS/Template
A         modules/legacy/CVS/Template
A         modules/search/CVS/Template
A         modules/throttle/CVS/Template
A         modules/poll/CVS/Template
A         modules/blogapi/CVS/Template
A         modules/color/images/CVS/Template
A         modules/color/CVS/Template
A         modules/taxonomy/CVS/Template
A         modules/user/CVS/Template

Brilliant! One line of commands joined together with a pipe and I have saved many, many minutes of tedious and bording typing!

9 Comments

The most recent comment was on Wed, 3rd Feb 2010, 22:44

syncvsvn

Submitted by Wim Leers (not verified) on Wed, 11th Mar 2009, 16:30.

I wrote syncvsvn, which does more than what your one-line script: it also deletes files from SVN that no longer exist in CVS and doesn't add the useless Entries.log files that CVS creates.

See http://wimleers.com/blog/syncvsvn-synchronize-cvs-working-copy-with-svn-working-copy.

once again, great minds think alike

Submitted by greggles (not verified) on Wed, 11th Mar 2009, 17:27.

It's a repeat of last time ;)

I wrote about this just weeks ago http://growingventuresolutions.com/blog/fun-command-line-add-all-files-directory-tree-svn-subversion

Use svn add

Submitted by yaph (not verified) on Wed, 11th Mar 2009, 19:44.

When run "svn add DIRECTORYNAME" it will recurse through DIRECTORYNAME and all subdirectories and add all files that have not been added. If the directories contain files you don't want to add you can add entries for files or file patters to an .svnignore file.
Also see http://svnbook.red-bean.com/en/1.0/re01.html

That's why I use CVS for core and SVN for the waist down

Submitted by Victor Kane (not verified) on Wed, 11th Mar 2009, 23:18.

Meaning ./sites, of course.

That way CVS can do its thing, while SVN can be the master of ./sites on down.

There are other solutions, of course,

Victor Kane
http://awebfactory.com.ar
http://projectflowandtracker.com

Simpler grep

Submitted by Freso (not verified) on Tue, 28th Apr 2009, 08:34.

Just curious, but wouldn't it be possible to do grep '^?.*Template$' instead of grep ^? | grep Template$? That way, you'd only have to run it through grep once. Of course, you're not going to run this very often, so it's not likely to cause performance problems. I'm just hinking aloud here, and thought I'd archive my thoughts. :)

Good point!

Submitted by Nick on Thu, 30th Apr 2009, 09:09.

Based on the above suggestion, I now tend to do…

[joe@blah folder]$ svn st | grep ^?  | gawk '{ print $2 }' | xargs svn add
[joe@blah folder]$ svn st | grep ^\! | gawk '{ print $2 }' | xargs svn del

The first line marks all missing files for deletion and all new files for addition.

Like I said, I'm a lazy linux piper!

EDIT

… Or, on one line…

$ svn st | grep ^\! | gawk '{ print $2 }' | xargs svn del && svn st | grep ^? | gawk '{ print $2 }' | xargs svn add

Use svn add --force

Submitted by Trevor (not verified) on Mon, 15th Jun 2009, 05:33.

You can avoid the need for pipes if you use 'svn add --force' to the root directory of your repository (or any dir in the repository). It will recursively add all of the missing files below it.

Alternatively if you are on a windows machine and need something fancier with pipes but don't have xargs (because you are using the dos shell) you can also use the '-targets' option to read the list of filenames from a file.

True...

Submitted by Nick on Mon, 15th Jun 2009, 09:05.

That's correct - but you have to be aware that it will add EVERYTHING. One of my posts above lets you use grep to filter out/in certain filenames (such as Template or Entries.Log)...

Or even without grep

Submitted by Anonymous (not verified) on Wed, 3rd Feb 2010, 22:44.

You can do without grep:

svn st | awk '/^?.*Template$/ { print $2 }' | xargs svn add

And even without xargs, at least in the newer svn versions (tested with 1.6.5)

svn st | awk '/^?.*Template$/ { print $2 }' | svn add --targets -

I'm a lazy linux piper

9 Comments

syncvsvn

once again, great minds think alike

Use svn add

That's why I use CVS for core and SVN for the waist down

Simpler grep

Good point!

Use svn add --force

True...

Or even without grep

Post new comment

Hot Content

Follow Me

Recent comments

Site Nav

Recent Posts

Tag Cloud

Recent Photos