MySQL like query runs extremly slow for 5000 recor

Question

I have this issue on our production server. The application stack is,

• Java Web App on Tomcat 6.0.18
• iBatis data access layer
• MySQL 5.0 database
• CentOS

The system is deployed on virtual server having around 256 MB memory.

Real problem:

The query like,

select * from customer

executes in around 10 seconds however if the following query is executed,

select * from customer where code like '%a%'

right after executing the above query, the system goes into indefinite processing and ultimately forces Tomcat to restart !.

Table statistics: - No. of records : 5000 - Primary Key : code

The same query PHP MyAdmin executes in around 4 seconds.

Do you think it could be MySQL problem ? Any idea to debug this. I am right now enabling the detailed logs and will keep updating this question with my findings but would appreciate your db insights.

Answer 1

I recently encountered a similar issue with MySQL in one of my production systems.

As a commenter noted above, the issue is the wildcard searching on the text field, and in particular the leading % in the search.

We dropped the leading % and reduced time take for a search query by several orders of magnitude (from a server grinding 60seconds+ to "no time at all").

Alternatives would be to use a Full-Text index or a system like Lucene for searching.

Answer 2

Have you tried creating an autoincrement primary field, and making code a separate unique index.

I've had issues with text as primary key being very slow in certain environments before.

Answer 3

The % in the beginning of the search is causing the table to never use an index. The % is a wildcard so it has to go through every single record in the database. Combine this with the fact that tomcat is running as well as a MySQL is running makes it even worse. There is not a lot of room to put an index in memory to read from.

You need to up the amount of memory on that box to let MySQL create the necessary memory caches it needs and room to allow the queries to work fast.

Answer 4

It seems your MySQL server is not setup correctly, taking 10 seconds for 5000 records should not be acceptable.

How are you accessing your db, via java code? In that case please give your high level logic here, maybe you are not efficiently reusing the db handlers.

Answer 5

In the question the poster states that the same query runs in 4 seconds in PHP MyAdmin, so the problem is not in MySql or the inability to use an index due to the %, but in Tomcat or the data access layer.