While looking at some of the websocket methods that stack exchange offers, I wanted to save a few data points into a MySQL database. However, when I attempt to run an executemany command, I get the following error:
_mysql_exceptions.OperationalError: (1241, 'Operand should contain 1 column(s)')
While looking around SO, I found many examples of this error, but they have dealt with removing parenthesis on SELECT statements. I'm not using a SELECT. I'm attempting to INSERT.
A short, contained, example of my code looks like this:
import MySQLdb as mdb
db = mdb.connect(host='localhost',user='myuser',db='qs',passwd='mypass',use_unicode=True,charset='utf8')
cur = db.cursor()
db_qry = """INSERT IGNORE INTO questions (id, site_base, title, body_sum, tags, last_act_dt, url, owner_url, owner, api_site) values (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)"""
parms = [(u'mathematica.stackexchange.com',
43248,
u'How to plot “Normalized distance” in this problem',
u"Problem: there are two particles whose equationsof motion both satisfy -n Abs[x[t]]^n/x[t] == x''[t]. But their initial conditions are different: one is x'[0] == 0, x[0] == 2;another is x'[0] == 0, ...",
[u'plotting', u'equation-solving', u'differential-equations', u'numerical-integration', u'notebooks'],
1393801095,
u'http://mathematica.stackexchange.com/questions/43248/how-to-plot-normalized-distance-in-this-problem',
u'http://mathematica.stackexchange.com/users/12706/lawerance', u'Lawerance', u'mathematica')]
cur.executemany(db_qry, parms)
cur.commit()
Am I using the executemany incorrect? Or missing another aspect of the parms list that I need to clean before passing to executemany?
The problem was the data going into the tags column. It was attempting to pass a list instead of a string.
For the sample in my original question, I used this code to convert it to a string.
','.join([u'plotting', u'equation-solving', u'differential-equations', u'numerical-integration', u'notebooks'])
It should also be noted that I messed up my commit line. It should be db.commit() not cur.commit()
I've run into this problem while trying to save a df with a column of lists to mysql via pandas.DataFrame.to_sql which can automate some of this process. Instead of using join, I typically convert the lists to encoded strings using json.dumps(). The json package also makes it easy to load it all back in to its original format with json.loads().
In the case of the original example I would convert parms to a dataframe
parms = pd.DataFrame(parms)
parms
0 1 \
0 mathematica.stackexchange.com 43248
2 \
0 How to plot “Normalized distance” ...
3 \
0 Problem: there are two particles whose equatio...
4 5 \
0 [plotting, equation-solving, differential-equa... 1393801095
6 \
0 http://mathematica.stackexchange.com/questions...
7 8 9
0 http://mathematica.stackexchange.com/users/127... Lawerance mathematica
Then convert the column of lists to json strings:
parms[4] = parms[4].apply(json.dumps)
parms
0 mathematica.stackexchange.com 43248
2 \
0 How to plot “Normalized distance” ...
3 \
0 Problem: there are two particles whose equatio...
4 5 \
0 ["plotting", "equation-solving", "differential... 1393801095
6 \
0 http://mathematica.stackexchange.com/questions...
7 8 9
0 http://mathematica.stackexchange.com/users/127... Lawerance mathematica
Fire up a MySQL connection (using details provided above):
import sqlalchemy
call = 'mysql+mysqldb://myuser:mypass@localhost:3306/qs'
engine = sqlalchemy.create_engine(call)
Then dump it using the built in pandas function:
parms.to_sql('questions', engine, if_exists='append', chunksize=5000)
