0

Disclaimer: Yes, finding files in a script with ls is bad, but find can't sort by modification date.

Using ls and xargs with echo everything is fine:

$ ls -t1 ssl-access*.gz | xargs -n 1 echo
ssl-access.log.2.gz
ssl-access.log.3.gz
ssl-access.log.4.gz
[...]

Changing echo to zcat:

$ ls -t1 ssl-access*.gz | xargs -n 1 zcat
gzip: ssl-access.log.2.gz.gz: No such file or directory
gzip: ssl-access.log.3.gz.gz: No such file or directory
gzip: ssl-access.log.4.gz.gz: No such file or directory
[...]

Duplicate file suffixes?! What is going on here?

UPDATE:

OS is Debian 5.

zcat is a shell script at /bin/zcat:

#!/bin/sh
PATH=${GZIP_BINDIR-'/bin'}:$PATH
exec gzip -cd "$@"
Christoph Wurm
  • 5,798
  • Unable to reproduce. Perhaps try taking a look at your zcat executable (probably a shell script) and see if it is auto-appending a .gz extension. – jw013 Oct 21 '11 at 20:51
  • 3
    You actually can sort by modified time if you use -printf %t (gnu-only) or -exec stat -c '%X' {}\; then pipe it to sort. – Shawn J. Goff Oct 21 '11 at 21:10
  • What happens on your system when you zcat a file that is not compressed? – Shawn J. Goff Oct 21 '11 at 23:14
  • @ShawnJ.Goff: I get the error message gzip: test.txt: not in gzip format with return code 1. – Christoph Wurm Oct 21 '11 at 23:19
  • 2
    See Shell script for moving oldest files? for better ways to sort files by date. The only advantage of xargs is running the command for a batch of files at once, and you're not using this capability, so don't even consider using it. About the strange behavior you observe: what does type zcat show? What is the *full* output from ls -t1 ssl-access*.gz? – Gilles 'SO- stop being evil' Oct 22 '11 at 02:11
  • the zcat that is part of the gzip package is a binary file, not a shell script. the binary zcat is functionally identical to gunzip -c. On systems that have a compress package installed, in there is already a zcat program that reads .Z files, so the gzip edition of zcat is installed as gzcat (which you noted below you don't have. can you modify your script to use ls -t1 ssl-access*.gz | xargs -n 1 gunzip -c ? – Tim Kennedy Oct 22 '11 at 04:52
  • Ugh, iPhone keyboard and I cannot click on edit. Could you post the output of ls -alF ssl-access*.gz? Maybe they are symlinks pointing to missing files. – janmoesen Oct 22 '11 at 07:12
  • Try adding --verbose to the xargs and zcat invocations. It should at least tell you which program is adding the extra .gz. – James Sneeringer Oct 23 '11 at 00:53
  • I was just about to post the output to all the commands you requested when it hit me. In my shell configuration I'm aliasing the ls command with ls -C --color=always --indicator-style=slash --group-directories-first. The --color=always seems to screw everything up when the output of ls is reused. I use this option to view colored output even when piping into less (got it from this answer) What is the correct way of handling this question? – Christoph Wurm Oct 23 '11 at 08:07

2 Answers2

4

I found this problem on my system caused by color ls. In my .bash_profile, I had this:

alias ls="ls --color"

I found the result by sending it to stat, which printed something handy:

$ ls local4.notice-201207* | xargs -n1 -P4 -I{} stat {}
stat: cannot stat `\033[0mlocal4.notice-20120711.gz\033[0m': No such file or directory

Look at those null color codes! It was confusing zcat, which attempted to add a .gz suffix to find a file. The problem was easily solved by changing the ls to color=auto, which disables color output when STDOUT is glued to another process instead of a terminal

alias ls="ls --color=auto"

Good luck!

Aaron Kondziela
  • 56
1

You might be using an OS where zcat is for compressed files while gzcat is for gzipped ones. In such case, this should work:

   ls -t ssl-access*.gz | xargs -n 1 gzcat

Edit:

I reproduced a similar but not identical behavior on Solaris:

$ ls -t1 *.gz | xargs -n 1 zcat          
c.gz.Z: No such file or directory
b.gz.Z: No such file or directory
a.gz.Z: No such file or directory

On a Debian based distribution (Ubuntu), I do not reproduce your problem.

This might work anyway:

cat $(ls -t ssl-access*.gz) | zcat

or the equivalent:

zcat <(cat $(ls -t ssl-access*.gz))

or this simpler one that avoids an unnecessary cat

zcat $(ls -t ssl-access*.gz)

None of these will handle odd filenames like the ones with embedded spaces.

jlliagre
  • 61,204