I am currently in the process of migrating several dozen sites between two servers. I tried using scp command to copy the sites over however scp is a very slow command when transferring many small files.
I did a little research on how to use tar over an ssh connection and realised that you could specify the stdout on tar.
How?
[adsense:468x60:4496506397]
Using this method effectively sends the compressed tarball to the terminal. You then pipe that into an ssh session which is running the extract version of the previous tar function along with the change directory argument. This, essentially, sends the compressed tarball into a decompression process at the other end over a secure ssh "pipe".
The result is a pretty quick file transfer which - as the data is being sent in a compressed GZIP form (of BZip2 if you replace the z with a j in the tar functions) you save on bandwidth too.
Here an an example of how to do this, assuming you are in (for example) /var/www/html/ and the website you want to transfer is the folder www.example.com.
tar czf - www.example.com/ | ssh joebloggs@otherserver.com tar xzf - -C ~/
This will send the entire www.example.com folder over to the home folder on your target server in compressed form over and encrypted connection.
10 Comments
how to get directory over ssh :)
If you need to get a directory from a server (you must to have an account on it) do such way:
Enjoy :)
Nice!
That's a really handy way of doing it...
It's like a "pull" (your script) compared to a "push" (my script).
The f - part is redundant;
The
f -part is redundant; Tar will use stdin/stdout b default. So the command becomes:Remote backup stream?
What if I wanted to 'pull' it, but leave it as a compressed tarball? I.E. Remote 'pull' backup.
In other words, create a tar/7z file by streaming the tar/7z output into ssh to another box where it resides as the compressed archive.
You would do that like this..
If you want to do a remote backup and then pull that to your system...
You can test that this pulled backup works by running (on the system you pulled the backup onto)
That's a pretty nifty technique!!
The way i usually do it is:
The way I usually do it is:
Hi - these are great
tar + split and scp ?
Hi - these are great techniques. But how about, copying a directory by tar-ing it - > splitting it into smaller chunks - > scp to destination - > Merge them - > untar to desination folder
Is it possible too?
I would like to point out
I would like to point out another comment that people don't seem to be noticing. tar automatically uses stdout/stdin for the x/c flags (respectively). This means f - is redundant; 'tar czf -' is identical to 'tar cz', and 'tar xzf -' is identical to 'tar xz'.
Good point, Another Anon!
Good point, Another Anon!
When I tried the suggestions
When I tried the suggestions on how to *pull* from a remote server, I ended up with the entire folder structure (in my case it was something like /home/content/a/b/site instead of just site).
This worked for me:
Perhaps it was something weird about the server (the site is on GoDaddy).
Thanks for the tips, they got me on the right track.
Add new comment