sed command with -i option (in-place editing) work

Question

This question already has an answer here:

• I need my sed -i command for in-place editing to work with both GNU sed and BSD/OSX sed 7 answers

I know nothing about Sed but need this command (which works fine on Ubuntu) to work on a Mac OSX:

sed -i "/ $domain .*#drupalpro/d" /etc/hosts

I'm getting:

sed: 1: "/etc/hosts": extra characters at the end of h command

Answer 1

Ubuntu ships with GNU sed, where the suffix for the -i option is optional. OS X ships with BSD sed, where the suffix is mandatory. Try sed -i ''

Answer 2

man is your friend.

OS X

 -i extension
         Edit files in-place, saving backups with the specified extension.
         If a zero-length extension is given, no backup will be saved.  It
         is not recommended to give a zero-length extension when in-place
         editing files, as you risk corruption or partial content in situ-
         ations where disk space is exhausted, etc.

Answer 3

To complement microtherion's helpful, to-the-point answer:

• with a portable solution
• with background information

tl;dr:

The equivalent of this GNU sed (standard on most Linux distros) command:

sed -i    's/foo/bar/' file

is this BSD/macOS sed command:

sed -i '' 's/foo/bar/' file  # Note the '' as a *separate argument*

With BSD/macOS sed, the following commands do not work at all or not as intended:

sed -i    's/foo/bar/' file  # Breaks; script is misinterpreted as backup-file suffix
sed -i''  's/foo/bar/' file  # Ditto
sed -i -e 's/foo/bar/' file  # -e is misinterpreted as backup-file suffix

For a discussion of all the differences between GNU sed and BSD/macOS sed, see this answer of mine.

Portable approach:

^{Note: Portable here means that the command works with both implementations discussed. It is not portable in a POSIX sense, because the -i option is not POSIX-compliant.}

# Works with both GNU and BSD/macOS Sed, due to a *non-empty* option-argument:
# Create a backup file *temporarily* and remove it on success.
sed -i.bak 's/foo/bar/' file && rm file.bak

For an explanation, see below; for alternative solutions, including a POSIX-compliant one, see this related answer of mine.

---

Background information

In GNU sed (standard on most Linux distros) and BSD/macOS sed, the -i option, which performs in-place updating^[1] of its input files, accepts an option-argument that specifies what suffix (filename extension) to use for the backup file of the file being updated.

E.g., in both implementations, the following keeps the original file, file, as backup file file.bak:

sed -i.bak 's/foo/bar/' file  # Keep original as 'file.bak'; NO SPACE between -i and .bak

Even though with GNU sed the suffix argument is optional, whereas with BSD/macOS sed it is mandatory, the above syntax works with both implementations, because directly adjoining the option-argument (.bak) to the option (-i) - -i.bak, as opposed to -i .bak - works both as an optional and a mandatory option-argument:

• Syntax -i.bak is the only form that works for an optional option-argument.
• Syntax -i.bak also works as a mandatory option-argument, as an alternative to -i .bak, i.e., specifying the option and its argument separately.

Not specifying a suffix - which is often the case - means that no backup file should be kept, and that is where the incompatibility arises:

• With GNU sed, not specifying a suffix means simply using -i by itself.

• With BSD/macOS sed, not specifying a suffix means specifying the empty string as the - mandatory - suffix, and for technical reasons, the empty string can only be passed as a separate argument: that is, -i '' not -i''.

-i'' doesn't work, because to sed it is indistinguishable from just -i, because the shell effectively removes the empty quotes (it concatenates -i and '' and removes quotes with syntactical function), and passes just -i in both cases.

With (effectively) just -i specified, it is the next argument that is interpreted as the option-argument:

sed -i 's/foo/bar/' file # BREAKS with BSD/macOS Sed

's/foo/bar/' - intended to the the Sed script (command) - is now interpreted as the suffix, and the word file is interpreted as the script.
Interpreting such a word as script then leads to obscure error message such as
sed: 1: "file": invalid command code f,
because the f is interpreted as a Sed command (function).

Similarly, with:

sed -i -e 's/foo/bar/' file # CREATES BACKUP FILE 'file-e'

-e is interpreted as the suffix argument, and NOT as Sed's -e option (which can be used to specify multiple commands, if needed).
As as result, instead of keeping NO backup, you get a backup file with suffix -e.

That this command doesn't work as intended is less obvious, because the in-place updating does succeed, given that the syntax requirement of the suffix argument is satisfied by the -e argument.

That the accidental creation of these backup files easily goes unnoticed is the likeliest explanation for Crt's incorrect answer and this incorrect answer to a similar question having received so many up-votes (as of this writing).

---

^{[1] Strictly speaking, a temporary file is created behind the scenes that then replaces the original file; this approach can be problematic: see the bottom half of this answer of mine.}

Answer 4

In OS X, you can use the GNU version of sed: gsed.

# if using brew
brew install gnu-sed

#if using ports
sudo port install gsed

Then, if your script should be portable, depending on your OS you can define which command to use.

SED=sed
unamestr=`uname`
if [[ "$unamestr" == "Darwin" ]] ; then
    SED=gsed
    type $SED >/dev/null 2>&1 || {
        echo >&2 "$SED it's not installed. Try: brew install gnu-sed" ;
        exit 1;
    }
fi
# here your sed command, e.g.:
$SED -i "/ $domain .*#drupalpro/d" /etc/hosts