SQLite problem “unable to open the database file”

2019-08-03 06:45发布

问题:

I come to you after a lot of hours googling and reading other discussions about SQLite on StackOverflow, but I definitely can't find any explanation to my problem, so here it is :

  • The context :
    I'm developping an application for iPad wich has to deal with some "large" amounts of data, in several occasions. In one of them, I must import points coordinates from a .kml file (Google's xml for geographical data) into my database, in order to reuse them later with a MKMapView and load them faster than by parsing xml when it needs to show a specific layer.

  • The details :
    The import thing is quite easy : when dealing with those files, I'm only concerned with 2 tables :

    • One containing zones definitions and details : for the moment, an integer as an id, and a text for naming.
    • One containing two real for coordinate storage and an integer referencing the first table for knowing which zone point is part of.
      So as long as reading my file, I first create an entry for the new zone, and then I insert points into the second table, with ID of the last zone created in the first table...nothing complicated!

    But...

  • The problem :
    After running fine a while, I get an exception from SQLite with the famous message "Unable to open the database file", and then it comes I can't do anything more with the database. This exception can randomly occur in the zone creation or the points insertion methods.

  • My reflexions :
    Considering the numerous points in those files, I suspected memory or disk saturation but other parts of my app discarded those points (to my mind).
    First, memory : it comes that when the exception occurs, the app is using about 10 or 12 MB of RAM. It can seems quite huge, but it's due to the ~10MB .kml file loaded in memory, so it's explainable. And above it all, the MKMapView thing of my app deals with tons of high-res tiles layers above map, and so leads to memory peaks which can afford 20 or even 25MB without making the iPad to crash.
    Second, disk : when reseting my database and filling only the 2 tables described above, the db file size when the exception occurs is always about 2.2 or 2.5MB, but when I fill other tables (the other parts of my apps works well!) the db file is about 6 or 7MB, and the device doesn't complain at all.

  • So what?!
    CPU-angryness and panic? I don't think so because some of the other tables of my database are filled at the same rythm without problem... and running my app in simulator crashes too, with a core i7 just laughing at the job.
    SQLite bad use? There we go! To my mind, it's the only solution left! But I really can't understand what's going on here because I process my requests the same way I do in other app's parts which - repeating myself - work like a charm!

  • SQLite details :
    I have a DB class which is a singleton I use to avoid creating/releasing an SqliteConnection object each request I do, and all my methods dealing with database are contained in this class to be sure I don't play with the connection anywhere else without knowing it. Here are concerned methods of this class :

    public void     saveZone(ObjZone zone)  { //at this point, just creates an entry with a name and let sqlite give it a new id
        lock (connection) { //SqliteConnection object
            try {
                openConnection();
                SqliteCommand cmd = connection.CreateCommand();
                cmd.CommandText = zone.id == 0 ?
                    "insert into ZONES (Z_NAME) values (" + format(zone.name) + ") ;" :
                    "update ZONES set Z_NAME = " + format(zone.name) + " where Z_ID = " + format(zone.id) + " ;";
                cmd.ExecuteNonQuery();
                if (zone.id == 0) {
                    cmd.CommandText = "select Z_ID from ZONES where ROWID = last_insert_rowid() ;";
                    zone.id = uint.Parse(cmd.ExecuteScalar().ToString());
                }
                cmd.Dispose();
            }
            catch (Exception e) {
                Log.failure("DB.saveZone(" + zone.ToString() + ") : [" + e.GetType().ToString() + "] - " +
                    e.Message + "\n" + e.StackTrace); //custom Console.WriteLine() method with some formating
                throw e;
            }
            finally {
                connection.Close();
            }
        }
    }
    
    public void     setPointsForZone(List<CLLocationCoordinate2D> points, uint zone_id) { //registers points for a given zone
        lock (connection) {
            try {
                openConnection();
                SqliteCommand cmd = connection.CreateCommand();
                cmd.CommandText = "delete from ZONESPOINTS where Z_ID = " + format(zone_id);
                cmd.ExecuteNonQuery();
                foreach(CLLocationCoordinate2D point in points) {
                    cmd.CommandText = "insert into ZONESPOINTS values " +
                        "(" + format(zi_id) + ", " + format(point.Latitude.ToString().Replace(",", ".")) + ", "
                        + format(point.Longitude.ToString().Replace(",", ".")) + ");";
                    cmd.ExecuteNonQuery();
                    cmd.Dispose();
                }
            }
            catch (Exception e) {
                Log.failure("DB.setPointsForZone(" + zone_id + ") : [" + e.GetType().ToString() + "] - " + e.Message);  
                throw e;
            }
            finally {
                connection.Close();
            }
        }
    }
    

    And to be as clear as I can, here are some of the methods referenced in the two above (I use this custom openConnection() method because I use foreign keys constraints in most of my tables and cascading behaviours are not enabled by default, but I need them.) :

    void openConnection() {
        try {
            connection.Open();
            SqliteCommand cmd = connection.CreateCommand();
            cmd.CommandText = "PRAGMA foreign_keys = ON";
            cmd.ExecuteNonQuery();
            cmd.Dispose();
        }
        catch (Exception e) {
            Log.failure("DB.openConnection() : [" + e.GetType().ToString() + "] - " + e.Message);
            throw e;
        }
    }
    
    public static string format(object o) {
        return "'" + o.ToString().Replace("'", "''") + "'";
    }
    

Well, sorry for the novel, I may already thank you for reading all that stuff, no?! Anyway, if I missed something that could be useful, let me know and I'll document it as soon as possible. I hope someone will be able to help me, anyway, thank you by advance!
(And my apologies for my poor frenchie's english.)

EDIT
My problem is "solved"! After a few changes for debugging pourposes, no big modifications, and no success, I put back the code in the state I posted it... and now it works. But I really would appreciate if some someone could give me an explanation of what may have happened! It seems like SQLite behaviour (on iPad at least - never used it anywhere else) can be quite obscure at some times... :/

回答1:

I wouldn't cross my fingers for this but I would try two things:

  1. If possible, pre-process the KML file to a second SQLite database and use this database to import data in the main database (thinking of lower memory/processor requirements)
  2. Transaction the imported data in small batches.

HTH

EDIT: you might have checked this already, but anyway: unable to open database.