unix: can i write to the same file in parallel wit

Question

I wrote a script that executes commands in parallel. I let them all write an entry to the same log file. It does not matter if the order is wrong or entries are interleaved, but i noticed that some entries are missing. I should probably lock the file before writing, however, is it true that if multiple processes try to write to a file simultaneously, it will result in missing entries?

Answer 1

In a script you should use ">> file" (double greater than) to append output to that file. The interpreter will open the destination in "append" mode. If your program also wants to append, follow the directives below:

Open a text file in "append" mode ("a+") and give preference to printing only full lines (don't do multiple 'print' followed by a final 'println', but print the entire line with a single 'println').

The fopen documentation states this:

DESCRIPTION
     The fopen() function opens the file whose  pathname  is  the
     string  pointed to by filename, and associates a stream with
     it.

     The argument mode points to a string beginning with  one  of
     the following sequences:

     r or rb             Open file for reading.


     w or wb             Truncate to  zero  length or create file
                         for writing.


     a or ab             Append; open or create file for  writing
                         at end-of-file.


     r+ or rb+ or r+b    Open file for update (reading and  writ-
                         ing).


     w+ or wb+ or w+b    Truncate to zero length or  create  file
                         for update.


     a+ or ab+ or a+b    Append; open or create file for  update,
                         writing at end-of-file.

The character b has no effect, but is allowed for ISO C standard conformance (see standards(5)). Opening a file with read mode (r as the first character in the mode argument) fails if the file does not exist or cannot be read.

Opening a file with append mode (a as the first character in the mode argument) causes all subsequent writes to the file to be forced to the then current end-of-file, regardless of intervening calls to fseek(3C). If two separate processes open the same file for append, each process may write freely to the file without fear of destroying output being written by the other. The output from the two processes will be intermixed in the file in the order in which it is written.

It is because of this intermixing that you want to give preference to using only 'println' (or its equivalent).

Answer 2

Yes, if different processes independently open and write to the same file, it may result in overlapping writes and missing data. This happens because each process will get its own file pointer, that advances only by local writes.

Instead of locking, a better option might be to open the log file once in an ancestor of all worker processes, have it inherited across fork(), and used by them for logging. This means that there will be a single shared file pointer, that advances when any of the processes writes a new entry.