Exchanging two files

By Jonathan Corbet
October 2, 2013

The renameat() system call changes the name of the file given as an argument, possibly replacing an existing file in the process. This operation is atomic; the view of the filesystem from user space will reflect the situation before or after the renameat() call, but it will never expose an intermediate state. Things work well when one file is involved, but what happens when multiple rename operations need to be run as a single atomic operation? That is a big problem, but, thanks to a patch from Miklos Szeredi, we might have a solution to a smaller subset.

The problem Miklos is trying to solve is the problem of exchanging two files — both files continue to exist, but their names have been swapped. To achieve this, he has posted a patch set adding a new renameat2() system call:

    int renameat2(int olddir, const char *oldname, 
		  int newdir, const char *newname, unsigned int flags);

This system call differs from renameat() in that it has the new flags argument; if flags is zero, renameat2() behaves exactly as renameat(). If, instead, flags contains RENAME_EXCHANGE, an existing file at newname will not be deleted; instead, it will be renamed to oldname. Thus, with this flag, renameat2() can be used to atomically exchange two files. The main use case for renameat2() is to support union filesystems, where it is often desirable to atomically replace files or directories with "whiteouts" indicating that they have been deleted. One could imagine other possibilities as well; Miklos suggests atomically replacing a directory with a symbolic link as one of them.

No review comments have been posted as of this writing.

Index entries for this article
Kernel	renameat2()

Exchanging two files

Posted Oct 3, 2013 14:44 UTC (Thu) by rvfh (guest, #31018) [Link] (7 responses)

I thought the number was now meant to represent the number of arguments? Meaning renameat5 and not renameat2. Am I wrong?

Exchanging two files

Posted Oct 3, 2013 17:34 UTC (Thu) by rwmj (subscriber, #5474) [Link] (6 responses)

Yes. The number is just a sequential number indicating a newer system call with a similar name and semantics to the old one.

This has been going on for a very long time, eg: dup & dup2 (dup2, I am told, first appeared in Unix v7 so that would be 1979).

Exchanging two files

Posted Oct 3, 2013 21:33 UTC (Thu) by khim (subscriber, #9252) [Link] (5 responses)

Well, dup has one argument, dup2 has two and dup3 has three thus I've always thought about this number as about number of arguments.

Exchanging two files

Posted Oct 4, 2013 10:36 UTC (Fri) by nix (subscriber, #2304) [Link] (4 responses)

It certainly indicates number of arguments in wait3() and wait4() (note that there is no wait2()).

i.e., this was a kludge workaround for C not supporting C++ function overloading, not a way for people to not bother to think up good names for their syscalls. (That way like Microsoft's OpenEx2() horrors.)

(I started writing a comment much like the original one, but it seemed uncharitable because I couldn't think of a better name myself. renameat_with_flags(), bleah, no. frenameat(), no. renameat+(), not a valid C identifier and people would shoot me and they'd be right.)

Exchanging two files

Posted Oct 4, 2013 11:03 UTC (Fri) by dlang (guest, #313) [Link]

it all depends on the person who named the function.

some people use the number to indicate the number of parameters, some people use it as a generation number

I suspect some of it is just the personality of the people involved, but that some of it may also be how much they expect the new version to completely replace the old version (as opposed to both versions being used for writing new software)

Exchanging two files

Posted Oct 4, 2013 23:57 UTC (Fri) by giraffedata (guest, #1954) [Link] (2 responses)

I think the reason you can't think of a better name for renameat2 is that it does two barely related things. This would be an abuse of C++ function overloading if that were an option.

I think the proper naming would be to keep renameat for renaming and add exchangeat for exchanging.

Exchanging two files

Posted Oct 7, 2013 21:04 UTC (Mon) by nix (subscriber, #2304) [Link]

Agreed. Swapping files is not renaming in any real sense: no names change.

Exchanging two files

Posted Jun 9, 2014 12:25 UTC (Mon) by roblucid (guest, #48964) [Link]

Actually the new call is not just about exchanging :

0 - same as renameat() behaviour
RENAME_NOREPLACE - avoids overwriting
RENAME_EXCHANGE - atomic exchange

So it's a modal renameat(2).

Exchanging two files

Posted Oct 3, 2013 14:59 UTC (Thu) by Seegras (guest, #20463) [Link] (3 responses)

I like this.

Now I can move incoming to archive-20130309 and create a new incoming in one go; while files are incoming ;).

Exchanging two files

Posted Oct 3, 2013 20:33 UTC (Thu) by epa (subscriber, #39769) [Link] (2 responses)

So, does mv(1) expose the atomic behaviour? Or does it continue in the Unix tradition of link-then-unlink?

Exchanging two files

Posted Oct 3, 2013 22:04 UTC (Thu) by khim (subscriber, #9252) [Link] (1 responses)

AFAICS mv(1) does not have any options which can actually be used for this behavior. Perhaps they could be added in the future, but as it is now it just changes the name of file, it can not swap two filenames!

Exchanging two files

Posted Oct 4, 2013 2:17 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

I'd actually rather that mv(1) *not* gain an option for this. It seems something more fitting for a swap(1) (or similar) tool instead. Related, I have a little tool I use locally called 'relocate' which takes a tree and does mv(1) on the contents, but recurses into directories when the target directory already exists (it's main use is merging in new music files into the existing "archive" tree without using rsync to do a smarter cp(1) and wasting time).

Exchanging two files

Posted Oct 3, 2013 18:21 UTC (Thu) by Yorick (guest, #19241) [Link] (2 responses)

It may be useful for saving documents while retaining a back-up copy of the original—right now, one has to muck about with hard links if atomicity is desired.

It's almost like the old MacOS function FspExchangeFiles, except that the latter only swapped the data, not the file identities or metadata. Keeping the creation date, permissions and so on is quite useful, but clearly not what renameat2 is about.

I'm curious to know what happens if two processes try to swap the same files simultaneously. Will they both fail (nothing happens), both succeed (same), or one succeed (accomplishing the task for either, but arbitrarily signalling a failure to one of them)?

Exchanging two files

Posted Oct 4, 2013 12:37 UTC (Fri) by fishface60 (subscriber, #88700) [Link] (1 responses)

As I understand it (I could easily be wrong), filesystem access is sequential, so even if two threads executed the syscall at the same time, one would end up with the lock first, so they'd both succeed at the swap.

Exchanging two files

Posted Oct 6, 2013 8:29 UTC (Sun) by jzbiciak (guest, #5246) [Link]

If they were both doing swap(a, b) then yes I'd expect the outcome to be a double-swap, leaving a and b in their original state. A perhaps more interesting thought experiment is if one process tries to do swap(a, b) in a race with another process trying to do swap(a, c). You get two possible outcomes:

   // swap(a,b) happens before swap(a,c).  a', b', and c' are the new files
   a' = c
   b' = a
   c' = b

and

   // swap(a,c) happens before swap(a,b).  a', b', and c' are the new files
   a' = b
   b' = c
   c' = a

I can't immediately think of why this might be useful, but it could certainly lead to some head scratching.

Exchanging two files

Posted Oct 4, 2013 7:42 UTC (Fri) by hansl (subscriber, #5086) [Link]

Something like this may also be useful for Wine to emulate junctions points, which basically redirect one directory to another on the local filesystem. So you would exchange a directory with a symlink to another directory.

A complication is that the original directory must be empty, and checking that with a separate call reintroduces a race condition. So renameat2 would need to gain a flag saying that it must fail if oldname is a non-emtpy directory.