How to fix “No newline at end of file” warning for

2019-01-14 04:11发布

I have a huge number of source files that are all lacking a newline at the end.

How do I automatically add a newline to the end of each of them?

Some may already have a newline, so it should only be added if necessary.

I'm probably not looking for code, per se, but just something I can run in Terminal to add the necessary newlines (or some kind of programming or development tool).

11条回答
欢心
2楼-- · 2019-01-14 04:41
pcregrep --recursive --exclude-dir=.git \
  --files-without-match --multiline '\n\z' . |
  while read k ; do echo >> "$k"; done

There are several steps involved here:

  1. Recursively find files
  2. Detect which files lack a trailing new line
  3. Loop over each of those files
  4. Append the newline

Step 1 is traditionally done with find (following the Unix tradition of "each tool doing one thing and doing it well"), but since pcregrep has builtin support, I'm comfortable using it. I'm careful to avoid messing around with the .git folder.

Step 2 is done with a multiline regular expression matching files that do have a final newline, and printing the names of files that don't match.

Step 3 is done with a while/read loop rather than a for/in, since the latter fails for filenames with spaces and for extremely long lists of files.

Step 4 is a simple echo, following @norman-ramsey's approach.

h/t @anthony-bush https://stackoverflow.com/a/20687956/577438 for the pcregrep suggestion.

查看更多
ら.Afraid
3楼-- · 2019-01-14 04:43

Converted Norman's answer to a split one-liner for convenience.

for i in * ; do  echo $i; \
 if diff /dev/null "$i" | tail -1 | \
  grep '^\\ No newline' > /dev/null; then echo >> "$i"; \
 fi; done

Replace * with whatever file pattern you want, eg *.c

And another to just tell you which files are broken:

for i in * ; do \
 if diff /dev/null "$i" | tail -1 | \
  grep '^\\ No newline' > /dev/null; then  echo $i; \
 fi; done
查看更多
冷血范
4楼-- · 2019-01-14 04:51

After finding the tool do this job with no luck. I decide to write my own

This is my python script to do that job

It only append (\r\n) to file not contains (\n) at the end of file

https://github.com/tranhuanltv/append_newline

Usage: append_newline.py .c ./projects ./result_dir

Make Pull Requests if you want to

查看更多
太酷不给撩
5楼-- · 2019-01-14 04:53

If you have access to Unix tools, you can run diff to find out which files lack a newline and then append it:

#!/bin/sh
for i
do
  if diff /dev/null "$i" | tail -1 | grep '^\\ No newline' > /dev/null
  then 
    echo >> "$i"
  fi
done

I'm relying on diff to produce the message with a \ in the first column, tail to give me the last line of diff's output, and grep to tell me if the last line is the message I'm looking for. If all that works, then the echo produces a newline and the >> appends it to the file "$i". The quotes around "$i" make sure things still work if the filename has spaces in it.

查看更多
倾城 Initia
6楼-- · 2019-01-14 04:54

find -type f | while read f; do [[ `tail -c1 "$f"` ]] && echo >> "$f"; done

I'm using find instead of for f in * as it is recursive and the question was about "huge number of source files".

I'm using while read instead of find -exec or xargs for performance reasons, it saves spawning shell process every time.

I'm taking advantage of the fact that backtick operator is returning output of command "with any trailing newlines deleted" man bash, so for properly terminated files backtick will be empty and echo will be skipped.

The find | read couple will fail on filenames that contain newlines, but it's easy to fix if required:

find -type f -print0 | while read -d $'\0' f; do [[ `tail -c1 "$f"` ]] && echo >> "$f"; done

查看更多
我命由我不由天
7楼-- · 2019-01-14 04:57

OK, after complaining in the comments, there is my better solution. First, you want to know, which files are missing newlines:

find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -v 0a$" ';' -print

Not super fast (calling a couple of processes for each file), but it's OK for practical use.

Now, when you have it, you may as well add the newline, with another -exec:

find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -v 0a$" ';' -exec sh -c "echo >> {}" ';'

Possible gotchas:

  • if filenames are bad, e.g. they have spaces, you may need tail -1 \"{}\". Or does find do it right?

  • you may want to add more filtering to find, like -name \*py, or the like.

  • think about possible DOS/Unix newlines mess before use (fix that first).

EDIT:

If you don't like the output from these commands (echoing some hex), add -q to grep:

find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -q -v 0a$" ';' -print
find -type f -exec sh -c "tail -1 {} | xxd -p | tail -1 | grep -q -v 0a$" ';' -exec sh -c "echo >> {}" ';'
查看更多
登录 后发表回答