Flask_SQLAlchemy, MySQL, store Swedish characters

Question

I can't sto re Swedish characters in MySQL by using Flask_SQLAlchemy :( I have tried to find a solution for a week now and I really need help as it feels like I have reached a dead end. I think it could be something wrong with the version compatibilities abong my tools, but I don't hope so! I am trying to build an website using Flask, Flask_SQLAlchemy and MySQL (5.5.3). If this is unsolvable, I am considering changing Flask_SQLAlchemy to something else.. (I have taken one course in programming (Python) and the rest is self thought so I would be very happy if you could be as detailed as possible in your answers, very thankful for any advice as well!)

(competeEnv) C:\>conda list
 # packages in environment at C:\Users\MyName\Anaconda3.1\envs\competeEnv:
 #
click                     6.6                      py27_0
flask                     0.11.1                   py27_0
Flask-SQLAlchemy          2.1                       <pip>
itsdangerous              0.24                     py27_0
jinja2                    2.8                      py27_1
markupsafe                0.23                     py27_2
mysql-python              1.2.5                    py27_0
pip                       9.0.1                    py27_0
python                    2.7.12                        0
setuptools                27.2.0                   py27_1
sqlalchemy                1.1.4                    py27_0
vs2008_runtime            9.00.30729.1                  2
werkzeug                  0.11.11                  py27_0
wheel                     0.29.0                   py27_0
(competeEnv) C:\>

Here is the code in testAlchemy.py file

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app=Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:myPassword@myServer/firstdb'
app.config['SQLALCHEMY_ECHO'] = False
app.config['SQLALCHEMY_TRACK_MODIFICATIONS']=True
app.config['MYSQL_DATABASE_CHARSET'] = 'utf8mb4'
db = SQLAlchemy(app)


class Users(db.Model):
    __tablename__='users'
    id=db.Column('iduser', db.Integer, primary_key=True)
    name=db.Column('column_name', db.String(193))

    def __init__(self, name):
        self.name=name

    def __repr__(self):
        return self.name

db.create_all()
db.session.commit()

president1=Users('Obama')
president2=Users('Trump')
db.session.add(president1)
db.session.add(president2)
db.session.commit()

Here should be some hints..

(competeEnv) C:\Users\MyName\Anaconda3.1\envs>python
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jun 29 2016, 11:07:13)[MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> from testAlchemy import Users, db
>>> db.session.add(Users('Federer'))
>>> db.session.commit()
>>> Users.query.all()
[Obama, Trump, Federer]
>>> db.session.add(Users('ä'))
>>> db.session.commit()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\scoping.py", line 157, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\session.py", line 801, in commit
self.transaction.commit()
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\session.py", line 392, in commit
self._prepare_impl()
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\session.py", line 372, in _prepare_impl
self.session.flush()
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\session.py", line 2019, in flush
self._flush(objects)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\session.py", line 2137, in _flush
transaction.rollback(_capture_exception=True)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\util\langhelpers.py", line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\session.py", line 2101, in _flush
flush_context.execute()
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\unitofwork.py", line 373, in execute
rec.execute(self)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\unitofwork.py", line 532, in execute
uow
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\persistence.py", line 174, in save_obj
mapper, table, insert)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\orm\persistence.py", line 800, in _emit_insert_statements
execute(statement, params)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\engine\base.py", line 914, in execute
return meth(self, multiparams, params)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\sql\elements.py", line 323, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\engine\base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\engine\base.py", line 1146, in _execute_context
context)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\engine\base.py", line 1341, in _handle_dbapi_exception
exc_info
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\util\compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\engine\base.py", line 1139, in _execute_context
context)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\sqlalchemy\engine\default.py", line 450, in do_execute
cursor.execute(statement, parameters)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\MySQLdb\cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "C:\Users\MyName\Anaconda3.1\envs\competeEnv\lib\site-packages\MySQLdb\connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1366, "Incorrect string value: '\\x84' for column 'column_name' at row 1") [SQL: u'INSERT INTO users (column_name) VALUES (%s)'] [parameters: ('\x84',)]
>>>

Answer 1

Since you're using Python 2.7, you need to specify that your string contains unicode.

>>> db.session.add(Users(u'ä'))

You can also use a future import to treat all strings as unicode.

from __future__ import unicode_literals

Alternatively you can upgrade your version of Python. 2.7 is the last version to treat strings as bytes rather than unicode.

Edit

You'll also need to update your __repr__ so that it properly handles unicode.

def __repr__(self):
    return self.name.decode('utf-8')

or whatever encoding you want to use.

In general, you'll need to make sure you handle encoding from and decoding to unicode. I can't urge you enough to consider using a more recent version of Python. One of the largest changes in Python 3 addresses this very issue.

Answer 2

In using UTF-8 characters, try to avoid any kind of encode/decode; that just masks the real problem, which is usually in the configuration somewhere.

My notes on sqlalchemy:

db_url = sqlalchemy.engine.url.URL(drivername='mysql', host=foo.db_host,
    database=db_schema,
    query={ 'read_default_file' : foo.db_config, 'charset': 'utf8' })

json.dumps(mydict, ensure_ascii=False) avoids "\u...." strings.

https://docs.sqlalchemy.org/en/latest/dialects/mysql.html#mysql-unicode

Python 2.7 is rather old, see this about the differences with 3:

https://stackoverflow.com/a/40708131/1766831

Python tips for utf8: http://mysql.rjweb.org/doc.php/charcoll#python