How to do Realtime/fast processing of GPS data in

Question

I am writing a web application for mapping Real-time GPS coordinates on Google maps coming from a GPS device, for fleet managment.

Since the flow of data is very fast from the GPS device to web application for database it becomes very heavy and the database is being queried every 5 seconds(via AJAX from web browser running the website) it becomes more heavy.

Keeping the updates in real-time is becoming very difficult a lagging of 30 seconds to 60 seconds is created between the actually update and its visibility on the website.

I am using Django + Apache + MySQL on CentOS 6.4 64 bit.

Any advice in what direction i should move to make the processing/visibility of data in more real-time would be helpful.

Answer 1

Some tipps:

• Avoid xml, especially a DOM based xml parser (this blows up data by a factor of 100). (A lat long coordinate without time, needs 8 bytes, not more)
• favor a binary represenattion of the coordinates, and parse them by hand, instead using an slow generated parsing code taht probaly uses reflection to parse.
• try to minimize the use of databases, especially relational ones. raise the intervall that clients are sending: e.g evry 20min instead evry 5.
• if you use a db, minimize the transactions, try to do all processing in one transaction.

Answer 2

I would suggest you to use NoSql database like MongoDB. It would really help you to achieve real time application performance.

Have a look at Django-With-MonoDB.

And if possible try to replace default python interpreter to PyPy.

I think these two are enough to give you best performance. :)

Understanding Django-using-PyPy

Also for front-end you should use KnockoutJS or AngularJS.