SQLite stack overflow when running a bulk INSERT O

Question

(See update at bottom of question for more to-the-point info.)

---

My code is constructing a SQL bulk INSERT OR REPLACE query with 500 rows (4 int32 and 2 int64 fields, for a total of 6 columns). From here and from experience, I know that the maximum limit for bulk insert of rows in SQLite is 500 rows.

A shortened example of the bulk insert query that my program is constructing, but with only 2 rows, is:

INSERT OR REPLACE INTO "VG_INSTANCE_DATA_B" (year, yield, ccode, month, DATETIME_ROW_START, DATETIME_ROW_END) VALUES (1800, 6.9400000000000004, 2, 1, -5364662400000, -5333126400000), (1800, 6.9400000000000004, 2, 2, -5364662400000, -5333126400000)

The query is prepared in SQLite via sqlite3_prepare_v2().

When the program builds & runs in Visual Studio 2013's debugger, and when there are 500 inserts in the bulk insert statement, the sqlite3_prepare_v2() function crashes with a stack overflow, with internal SQLite functions being called over and over in a cycle.

When I decrease the number of bulk-inserted rows to 400, however, the stack overflow condition disappears.

I attempted to increase the stack size by a factor of 10 by passing /F 10000000 to the compiler and /STACK 10000000 to the linker. However, it had no effect. I suspect that something else is going on - and that my stack is not really overflowing (it is hard for me to see how 500 rows at 20 bytes or so per row could cause a stack overflow just due to the amount of data, and as noted, there shouldn't be a stack overflow due to the internal function calls in SQLite, because otherwise SQLite would not claim to support 500 inserts in a bulk insert statement).

Can someone please explain why there is a stack overflow crash during the call to sqlite3_prepare_v2() for a bulk insert of 500 rows (each containing only 6 reasonably-sized columns), when the official SQLite limit for a bulk insert is 500? (Note: I also tested 499, and it still crashed with a stack overflow exception. It is known not to crash at 400, though.)

Thanks!

---

Update: I have debugged into sqlite3.c and by using here and here, actually added code to calculate the stack usage, and placed a breakpoint at the point where the stack overflow occurs.

Two comments.

• SQLite does a RECURSIVE call for bulk inserts. I.e., 500 rows in the bulk insert = 500 recursions into the set of functions involved, bringing the stack deeper and deeper. (I do not know if the Release-mode compiler optimizes away the recursion, or not.)

• The stack is legitimately reaching the default 1 MB limit in Debug mode. The crash definitely occurs at the single function call that brings the stack size over 1 MB (I know this by using the above-linked code to calculate the stack usage at the point of the stack overflow).

---

Addendum 2

Important info: my bulk query is actually INSERT OR REPLACE, and the stack overflow crash occurs in a recursive SELECT query. This SELECT query is related to the REPLACE part of my bulk INSERT statement. Therefore, I think it's likely that the relevant cause of the problem is the REPLACE, not the INSERT. (See comments below this question for some more details, including timing.)

I have updated the title to reflect this.