Going from twitter date to Python datetime date

Question

I am receiving twitter messages that are sent at a certain date in the following format from twitter:

Tue Mar 29 08:11:25 +0000 2011

I want to store these dates in 'timestamp with time zone' field in postgresql with djangos DateTimeField field. When I store that string however I get this error:

ValidationError: [u'Enter a valid date/time in YYYY-MM-DD HH:MM[:ss[.uuuuuu]] format.']

can I automatically transform the twitter datetype to a python datetime time (that does work elsewhere in my app for saving dates).

Answer 1

Give this a go. It assumes the date format from twitter is RFC822 compliant (see the question linked to by @Adrien).

A naive datetime object is constructed (i.e. no timezone info). It is adjusted according to the timezone offset to UTC. Unless you have a need to keep the original timezone, I'd store the date time as UTC and format to local time when you display it.

from datetime import datetime, timedelta
from email.utils import parsedate_tz

s = 'Tue Mar 29 08:11:25 +0000 2011'

def to_datetime(datestring):
    time_tuple = parsedate_tz(datestring.strip())
    dt = datetime(*time_tuple[:6])
    return dt - timedelta(seconds=time_tuple[-1])

Answer 2

Using a similar strategy as SoFolichon proposed, in Python 3.x you can also use pytz like:

from datetime import datetime, timezone
import pytz

datetime.strptime(tweets["created_at"], '%a %b %d %H:%M:%S %z %Y').replace(
tzinfo=timezone.utc).astimezone(pytz.timezone('US/Eastern')).strftime(
'%Y-%m-%d %H:%M:%S')

Answer 3

Writing something like this should convert a twitter date to a timestamp.

import time

ts = time.strftime('%Y-%m-%d %H:%M:%S', time.strptime(tweet['created_at'],'%a %b %d %H:%M:%S +0000 %Y'))

Answer 4

How about this? It doesn't need any formatting strings.

import datetime
from email.utils import mktime_tz, parsedate_tz

def parse_datetime(value):
    time_tuple = parsedate_tz(value)
    timestamp = mktime_tz(time_tuple)

    return datetime.datetime.fromtimestamp(timestamp)

print(parse_datetime('Tue Mar 29 08:11:25 +0000 2011'))
#2011-03-29 10:11:25

My system is at GMT +2 hence the difference included.

Answer 5

The following code will print a nice date (local time) from a Twitter date (UTC).

from datetime import datetime
from datetime import timezone    

datetime.strptime(mydata["created_at"], '%a %b %d %H:%M:%S %z %Y').replace(
            tzinfo=timezone.utc).astimezone(tz=None).strftime('%Y-%m-%d %H:%M:%S'))

Answer 6

you can convert the date using datetime.strptime(), or time.strptime(). however, those two functions cannot parse the timezone offset (see this bug).

so, the only solution i see is to split the date yourself, remove the timezone offset, feed the rest to strptime(), and process the offset manually...

have a look at this question, where you will find some hints on how to parse the offset yourself.