maildir headers problem

Question

I have the followign bash script to update mtimes for maildir files:

#!/bin/bash

for i in /test/emailfile

do
    date=$(sed -n '/Received: from/ { :a; n; /;/ {s/.*; //p;q}; b a }' "$i")
    newdate=$(date -d "$date" +'%Y%m%d%H%M.%S')
    touch -t "$newdate" "$i"
done

This script has always worked fine, with standard headers like these:

Delivery-date: Sun, 22 Apr 2007 00:15:13 -0600
Received: from an-out-0708.google.com ([209.85.132.243])
    by x.xxxx.com with esmtp (Exim 4.63)
    (envelope-from <[e-mail]>)
    id 1HfVLs-0002Io-RQ
    for [e-mail]; Sun, 22 Apr 2007 00:15:13 -0600

Which has a 2007 delivery date. If I touch that file so the file date is from today, and then run my script, the file date is restored to the received from date.

However, when attempting to run my script on an email with the following headers:

Delivery-date: Mon, 15 Dec 2008 17:26:37 -0800
Received: from xxxxxx ([130.194.13.165])
    by xxxxxxx with esmtp (Exim 4.69)
    (envelope-from <[e-mail]>)
    id 1LCOhp-0006fm-2g
    for [e-mail]; Mon, 15 Dec 2008 17:26:37 -0800

The date is apparantly not restored. I can not see that the headers are obviously different in any way. I need to reset the mtimes, because many mail clients use the filetime to display as the received from time. My script has worked on over 3000 emails, ensuring all clients displays the emails in the right order aftr moving servers and all files having the same date, but for some reason it will not work on a particular email. Have I left something obvious out of the script?

edit: the date is apparently restored from the script, however clients that rely on mtimes will not display this message regardless of what the script sets the date to. The permissions are the same, as is the layout and filename format. Below is a posting from ls -la

-rw-rw----   1 username username    11769 Dec 14 21:25 1229318728.H329820P11297.xxxxx.serverxxxxx.com:2,S
-rw-rw----   1 username username     3366 Dec 15 17:26 1229390797.H476913P25671.xxxxx.serverxxxxx.com:2,S
-rw-rw----   1 username username     1149 Dec 22 05:39 1229953142.H901034P11016.xxxxx.serverxxxxx.com:2,S
-rw-rw----   1 username username     7557 Dec 23 15:43 1230075791.H700954P8392.xxxxx.serverxxxxx.com:2,S

The file that will not display correctly is the second from the top. Is there any way to debug why this is happening at all?

Answer 1

It appears to work fine for me, try this little ditty and see what you get.

#!/bin/bash
echo 'Delivery-date: Sun, 22 Apr 2007 00:15:13 -0600
Received: from an-out-0708.google.com ([209.85.132.243])
    by x.xxxx.com with esmtp (Exim 4.63)
    (envelope-from <[e-mail]>)
    id 1HfVLs-0002Io-RQ
    for [e-mail]; Sun, 22 Apr 2007 00:15:13 -0600' | sed -n '/Received: from/ { :a; n; /;/ {s/.*; //p;q}; b a }'
date -d "Sun, 22 Apr 2007 00:15:13 -0600" +'%Y%m%d%H%M.%S'
echo 'Delivery-date: Mon, 15 Dec 2008 17:26:37 -0800
Received: from xxxxxx ([130.194.13.165])
    by xxxxxxx with esmtp (Exim 4.69)
    (envelope-from <[e-mail]>)
    id 1LCOhp-0006fm-2g
    for [e-mail]; Mon, 15 Dec 2008 17:26:37 -0800' | sed -n '/Received: from/ { :a; n; /;/ {s/.*; //p;q}; b a }'
date -d "Mon, 15 Dec 2008 17:26:37 -0800" +'%Y%m%d%H%M.%S'

I get (as expected):

Sun, 22 Apr 2007 00:15:13 -0600
200704221415.13
Mon, 15 Dec 2008 17:26:37 -0800
200812161026.37

So I'm not sure why yours isn't working, although the output of that script should be a good starting point.

The other thing you can do is temporarily modify your script as follows:

: : : : :
echo touch -t "$newdate" "$i"
touch -t "$newdate" "$i"
: : : : :

This will output the actual command you're trying.

Also check file permissions on the directory and individual files.

What does "ls -l" (mtime) and "ls -lc" (ctime) give you on the actual suspect mail file? It may be that your script is working fine vut the users email client is not using the mtime/ctime.

After edit:

Quoting "however clients that rely on mtimes will not display this message regardless of what the script sets the date to", my response is that the client is not relying on mtimes.

Paraphrasing Sherlock himself, once you have eliminated all the other possibilities, whatever's left, however unlikely, must be the case. And one thing I noticed: did you just state that the message isn't being displayed at all (or that it's got the wrong date as I've thought we were talking about)?

If the latter, you need to look into the mail client to see where it's actually getting its date from. Just out of interest, what date does the client think it is (and does it match any date within the header?

If the former, that's a different problem we can discuss.

I think that's (the client) where you need to go from here and, since my expertise is more in scripting than email clients, I may not be able to help further. Still, I'm interested to see how it turns out, so I'll favorite this question.

Answer 2

The following scripts used the Date rather then the Received Date attribute of the mail message. This sets the date to match the date used by some mail applications and simplifies the sed statement as well. It also includes improved error handling when an incorrect / invalid date is encountered.

#!/bin/bash
echo "Process folder : $@"
for i in $( ls -1 $@ )
do
#    date=$(sed -n '/Received: from/ { :a; n; /;/ {s/.*; //p;q}; b a }' "$i") 
#    date=$(sed -n -f ~/scripts/fix.sed "$i")
    date=$(sed  -n '/^Date: / {s/^Date: //p;q}' "$i")
    newdate=$(date -d "$date" +'%Y%m%d%H%M.%S')
    returnCode=$?
    if [ "$returnCode" != "0" ]
    then
      echo "Date Return Code : $returnCode"
      echo "Message file : $i received at : $date"
    fi
touch -c -t "$newdate" "$i"
done

Answer 3

Are you sure - it works for me.

The only changes I made were:

#!/bin/bash

for i in "$@"

do
    date=$(sed -n '/Received: from/ { :a; n; /;/ {s/.*; //p;q}; b a }' "$i")
    newdate=$(date -d "$date" +'%Y%m%d%H%M.%S')
    touch -c -t "$newdate" "$i"
done

touch -c so that files don't get created by touch.

for i in "$@" so that it runs on the command line arguments rather than a fixed file.

$ touch data2 ; ls -l data2
-rw-r--r-- 1 leeder leeder 250 2009-01-22 11:43 data2
$ bash test.sh data data2 ; ls -l data2
-rw-r--r-- 1 leeder leeder 250 2008-12-16 01:26 data2