43

What is the idiomatic way to do the following

  • tar to stdout
  • read this tar output from stdout and extract to some other folder.

My solution is tar --to-stdout .. | tar -C somefolder -xvf -

But may be there is more idiomatic way to do it.

Jin
  • 4,333
shabunc
  • 820

3 Answers3

57

The same -f - option works for tarring as well.

tar -cf - something | tar -C somefolder -xvf -

GNU tar uses stdio by default:

tar -c something | tar -C somefolder -xv

rsync is also popular.

rsync -av something/ somefolder/
u1686_grawity
  • 452,512
11

Just adding another use-case here. I had a large directory structure on a system nearly out of disk space and wanted to end up with a tar.gz file of the directory structure on another machine with lots of space.

tar -czf - big-dir | ssh user@host 'cat > /path/to/big-dir.tar.gz'

This saves on network overhead and means you don't have to tar on the other side in case you'd wanted to use rsync for the transfer instead.

quickshiftin
  • 279
  • Netcat is perfect for this. (Cat from one host to another host). – Hennes Jun 25 '15 at 16:40
  • 5
    @Hennes: With its lack of authentication, integrity checking, data encryption, as well as having to manually start it on both sides for each individual transfer (i.e. 2× the work), it sounds a bit less than perfect – u1686_grawity Jul 23 '15 at 07:00
  • Most of the time I gzip it before dumping it over the network. Any integretiy failures are likely to show up as decompression errors (though I never got any when I used it). As to starting two programs: Aye, true. – Hennes Jul 23 '15 at 07:35
  • 3
    It may be more work, but for sending a large compressed archive over a link during a time-sensitive operation between machines in a secured local network or over a VPN, piping through nc will be significantly faster than SSH (over a 1Gb network, often by a factor of 2). Send over an md5 sum of the archive for integrity checking. – Spooler Jan 05 '19 at 19:14
  • whats the netcat command? – Dave Ankin Feb 28 '21 at 01:48
  • Why not use tar's built-in remote access functionality? – ReeseWang Sep 05 '22 at 17:54
  • @ReeseWang how about an example command? When I check man tar and search for "remote" there are no matches. Personally I didn't know tar had remote capabilities. – quickshiftin Sep 06 '22 at 14:49
  • 1
    @quickshiftin tar --zstd -cf 192.168.1.2:/home/user/backup.tar.zstd --rsh-command=/usr/bin/ssh ./ see this man page. – ReeseWang Sep 07 '22 at 16:26
  • Ah, looks like it's a bit different on a Mac. Rang up the man page on a Linux box today and I see the remote options. Your point is well taken, may answer is ignorance lol. – quickshiftin Sep 07 '22 at 17:27
4

I don't have enough reputation to post a comment about using netcat.

On the receiving end run: netcat -l 5555 > /path/to/dest.tar.gz or netcat -l 5555 | tar -C /path/to/expanded/tar -xz

On the sending side run: tar -C /path/to/source -cz files | netcat <target IP/hostname> 5555

If you are on a fast network, don't bother compressing and decompressing the tar. Used some variant of this over the years for all kinds of recovery.

Robert LeBlanc
  • 41
  • 2
    Robert, thank you for this addition. I think it's fine per se as an answer, so it's even better - I hope you'll pretty soon will have reputation for comments as well though ) – shabunc May 12 '21 at 21:31