1

I have a part of a bash script, which I am executing as root:

rm -f ../../include/profile.h; \
cp profile.h ../../include/profile.h

the file ../../include/profile.h in question has permissions:

-rw-r--r-- 1 root root 12178 Nov  5 02:00 ../../include/profile.h

Most of the time, the snippet executes fine, but very rarely, impossible to reproduce, it fails:

cp: cannot create regular file `../../include/profile.h': File exists

I checked, there is no other part of the program, that would be executed in parallel, which could write the file in between the two statements, thus creating a race condition which would explain the behaviour. There is no other place than the above, which is executed only once, which would write this file.

The system is

kernel:

Linux dev64 2.6.32.63+drm33.26-64.128-a10 #6 SMP Fri Jul 25 15:21:56 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux

distro:

CentOS release 6.3 (Final)

What could possibly cause this failure to appear??

Giacomo1968
  • 55,001
user322908
  • 779
  • 1
    The only thing that springs to mind is disc write caching, which may not have completed after the rm before the cp is executed. Perhaps calling sync between the two commands will cure it. – AFH Dec 13 '15 at 03:44
  • @AFH can you tell me, do I have a bug in the script, where sync is necessary for the script to be correct? In other words, is sync always necessary to call between two shell commands that depends on each others effect on the filesystem? I want to know is this my error, or is this a bug in Linux kernel and/or implementation of cp, which can be mitigated with the use of sync. – user322908 Dec 13 '15 at 04:00
  • Update to newest CentOS release 6.7. Then check whether the error still occurs. – Cyrus Dec 13 '15 at 08:08
  • @Cyrus thank you I can do that, but, please read my question carefully - it is meant literally. I am not so much interested in "try x and see if the problem disappears" because that will not bring me understanding of what is happening. I want to understand what is happening first and foremost, and based on that, I can then "do the right thing". – user322908 Dec 13 '15 at 08:35
  • @user322908 - It is certainly not a script bug and it is only a guess that it may be a caching bug: inserting sync will provide some evidence of the latter. If so, the cache in question may be in the disc driver or in the disc controller itself (I experienced this some years ago and the disc drives had to be replaced by the manufacturer). Unfortunately, I run Ubuntu, which is Debian-based, whereas CentOS is RedHat-based, so I cannot run any meaningful tests. I have given you a possible cause, as you asked: I'm not sure I can do more. – AFH Dec 13 '15 at 13:56
  • 1
    Normally cp foo bar will overwrite bar if it exists. The error cp: cannot create regular file … File exists indicates something deeper than only the old file not having been removed yet. Is FUSE involved? I believe it can introduce race conditions in some circumstances (see this answer). Or maybe it's a remote filesystem and the root of the problem is on the server. I notice the question is old, the setup in question may no longer exist. – Kamil Maciorowski Mar 09 '21 at 08:44

2 Answers2

0

Use

strace cp profile.h ../../include/profile.h 2>logfile

and look for:

stat("../../include/profile.h", 0x7ffdca347950) = -1 ENOENT (No such file or directory)
open("../../include/profile.h", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4

That will tell you what's going on. In this example the file did not exist before the cp (which it determined), hence the O_CREAT|O_EXCL.

ctrl-d
  • 136
-2

It may be a race condition when different processes are writing to a same file location.

cp process A:

  • action1: found the file not exist
  • action2: create the file

cp process B:

  • action1: found the file not exist
  • action2: create the file

When B.action2 happen between A.action1 and A.action2, A will report "cp: cannot create regular file '???': File exists"

Chunlin Zhang
  • 1
  • 3
    A quote from the question: "I checked, there is no other part of the program, that would be executed in parallel, which could write the file in between the two statements, thus creating a race condition which would explain the behaviour. There is no other place than the above, which is executed only once, which would write this file" – gronostaj Mar 09 '21 at 08:08