I have a somewhat straightforward awk used for the purpose described here:
Append multiple header information fields to file until next header found
The awk only works on the data after I copy/paste it into a new file. If I direct the output of head into a new file, the awk still does not work, for instance. The awk only works if I copy/paste the file into a new file.
`head -40 file.csv > output.csv`
This is the awk:
`awk -F, '/"Serial No."/ {sn = $2}
/"Location:"/ {loc = $2}
/"([0-9]{1,2}\/){2}[0-9]{4} [0-9]{2}:[0-9]{2}"/
{$0 = loc FS sn FS $0}1' file.csv>master1.csv`
If I copy/paste the data and compare it to the original data, the output indicates a difference in every single line, but does not say where. If you look at a diff between a head output and a copy/paste files you get:
`diff trap4_top.csv trap4_again.csv'
:
< 1,25c1,24
< "Serial No.","0700000036022821"
< "Location:","LS_trap_2c"
< "High temperature limit (�C)",20
< "Low temperature limit (�C)",0
< "Date - Time","Temperature (�C)"
< "5/28/2015 08:00",24.0
< "5/28/2015 10:00",29.5
< "5/28/2015 12:00",28.0
< "5/28/2015 14:00",28.5
< "5/28/2015 16:00",27.0
< "5/28/2015 18:00",24.5
< "5/28/2015 20:00",23.0
< "5/28/2015 22:00",22.5
< "5/29/2015 00:00",21.5
< "5/29/2015 02:00",21.0
< "5/29/2015 04:00",20.0
< "5/29/2015 06:00",20.0
< "5/29/2015 08:00",24.5
< "5/29/2015 10:00",26.0
< "5/29/2015 12:00",27.5
< "5/29/2015 14:00",30.0
< "5/29/2015 16:00",29.0
< "5/29/2015 18:00",25.5
< "5/29/2015 20:00",23.5
< "5/29/2015 22:00",23.0
---
> "Serial No.","0700000036022821"
> "Location:","LS_trap_2c"
> "High temperature limit (°C)",20
> "Low temperature limit (°C)",0
> "Date - Time","Temperature (°C)"
> "5/28/2015 08:00",24.0
> "5/28/2015 10:00",29.5
> "5/28/2015 12:00",28.0
> "5/28/2015 14:00",28.5
> "5/28/2015 16:00",27.0
> "5/28/2015 18:00",24.5
> "5/28/2015 20:00",23.0
> "5/28/2015 22:00",22.5
> "5/29/2015 00:00",21.5
> "5/29/2015 02:00",21.0
> "5/29/2015 04:00",20.0
> "5/29/2015 06:00",20.0
> "5/29/2015 08:00",24.5
> "5/29/2015 10:00",26.0
> "5/29/2015 12:00",27.5
> "5/29/2015 14:00",30.0
> "5/29/2015 16:00",29.0
> "5/29/2015 18:00",25.5
> "5/29/2015 20:00",23.5`
I see special characters in the diff but I'm not if they're involved, or how exactly to remove them, other than copy/paste so far.
head trap4.csv | cat -vte
gives:
"Serial No.","0700000036022821"^M$
"Location:","LS_trap_2c"^M$
"High temperature limit (M-0C)",20^M$
"Low temperature limit (M-0C)",0^M$
"Date - Time","Temperature (M-0C)"^M$
"5/28/2015 08:00",24.0^M$
"5/28/2015 10:00",29.5^M$
"5/28/2015 12:00",28.0^M$
"5/28/2015 14:00",28.5^M$
"5/28/2015 16:00",27.0^M$