We Keep Coding

py ceODBC can't fetch result of select statement

I am trying to use ceODBC to try and improve some query times, and I have a problem starting with ceODBC library.
I import the lib, connect, execute the select statement, but when running cursor.fetchall(), or similar, I get the error.
this error seems to happen only with labels that could have spaces or special characters
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc1 in position 26: invalid start byte
sample
import ceODBC
conn = ceODBC.connect('cnx string', autocommit=False)
cursor = conn.cursor()
cursor.execute("select [name], [label] from table1")
# error because of label having a "héllo world" value for eg
# none works
[print(row) for row in cursor]
print(cursor.fetchall())
I tried looking for decode/encode methods but found none on ceODBC

UnicodeError with SqlAlchemy + Firebird + FDB

I'm triying to display results from a firebird 3.x database, but get:
File
"/...../Envs/pos/lib/python3.6/site-packages/fdb/fbcore.py",
line 479, in b2u
return st.decode(charset) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd1 in position 9: invalid continuation byte
Despite I set utf-8 everywhere:
# -- coding: UTF-8 --
import os
os.environ["PYTHONIOENCODING"] = "utf8"
from sqlalchemy import *
SERVIDOR = "localhost"
BASEDATOS_1 = "db.fdb"
PARAMS = dict(
user="SYSDBA",
pwd="masterkey",
host="localhost",
port=3050,
path=BASEDATOS_1,
charset='utf-8'
)
firebird = create_engine("firebird+fdb://%(user)s:%(pwd)s#%(host)s:%(port)d/%(path)s?charset=%(charset)s" % PARAMS, encoding=PARAMS['charset'])
def select(eng, sql):
with eng.connect() as con:
return eng.execute(sql)
for row in select(firebird, "SELECT * from clientes"):
print(row)

I had the same problem.
In my situation the database was not in UTF-8.
After setting the correct charset in the connection string it worked: ?charset=ISO8859_1

I would try to use the module unidecode.
Your script is crashing when it tries to convert, so this module can help you. As they says in the module documentation:
The module exports a single function that takes an Unicode object
(Python 2.x) or string (Python 3.x) and returns a string (that can be
encoded to ASCII bytes in Python 3.x)
First you download it using pip and then try this:
import unidecode
...
if type(line) is unicode:
line = unidecode.unidecode(line)
I hope it solves your problem.

'ascii' codec can't encode character error

I request your kind assistance in tackling an error. I am trying to save MS Access database tables as CSV files using Python. I seem to be running into an error I do not know how to fix. I have looked through different posts on Stack overflow and tried them but nothing fulfilling. Please provide your kind assistance.
import pyodbc
import csv
conn_string = ("DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=C:\\Access\\permissions.accdb")
conn = pyodbc.connect(conn_string)
cursor = conn.cursor()
cursor.execute("select * from [Perm_Site Info];")
with open('C:\\Desktop\\Python Files\\Perms_Site_Info.csv','wb') as csvfile:
writer = csv.writer(csvfile)
rest_array = [text.encode("utf8") for text in cursor]
writer.writerow(rest_array)
writer.writerow([i[0] for i in cursor.description])
writer.writerows(cursor)
cursor.close()
conn.close()
print 'All done for now'
The error:
writer.writerows(cursor)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 4: ordinal not in range(128)

You should probably install and use the unicodecsv module.
https://pypi.python.org/pypi/unicodecsv/0.14.1

errors while trying to remove non-UTF characters returned from a DLL

I'm trying to fix one of the fields in a structure returned by a dll to not have any non UTF-8 characters. When I try to modify the field I'm getting errors
import _portaudio as pa
...
my_dev_info = pa.get_device_info(device_index)
my_dev_info.name = my_dev_info.name.decode('utf-8','ignore')
my_dev_info.name = my_dev_info.name.decode('utf-8','ignore')
UnicodeDecodeError: 'utf8' codec can't decode byte 0xce in position 0: invalid c
ontinuation byte
If I try to simply reassign .name to something else there is a different error
my_dev_info.name = '???'
AttributeError: Fields read-only: cannot modify values
pa is import from a dll I believe. I don't have control over its contents. Any way around this?
This works fine with a simple string, but not with the 'pa' object :
>>> s='┼δⁿτΦ foo'
>>> s = s.decode('utf-8','ignore')
>>> s
u' foo'
>>>

Why am I getting this UnicodeEncodeError when I am inserting into the MySQL database?

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 2: ordinal not in range(128)
I changed my database default to be utf-8, and not "latin"....but this error still occurs. why?
This is in my.cnf. Am I doing this wrong? I just want EVERYTHING TO BE UTF-8.
init_connect='SET collation_connection = utf8_general_ci'
init_connect='SET NAMES utf8'
default-character-set=utf8
character-set-server = utf8
collation-server = utf8_general_ci
default-character-set=utf8

MySQLdb.connect(read_default_*) options won't set the character set from default-character-set. You will need to set this explicitly:
MySQLdb.connect(..., charset='utf8')
Or the equivalent setting in your django databases settings.

If you get an exception from Python then it's nothing to do with MySQL -- the error happens before the expression is sent to MySQL. I would presume that the MySQLdb driver doesn't handle unicode.
If you are dealing with the raw MySQLdb interface this will be somewhat annoying (database wrappers like SQLAlchemy will handle this stuff for you), but you might want to create a function like this:
def exec_sql(conn_or_cursor, sql, *args, **kw):
if hasattr(conn_or_cursor):
cursor = conn_or_cursor.cursor()
else:
cursor = conn_or_cursor
cursor.execute(_convert_utf8(sql), *(_convert_utf8(a) for a in args),
**dict((n, _convert_utf8(v)) for n, v in kw.iteritems()))
return cursor
def _convert_utf8(value):
if isinstance(value, unicode):
return value.encode('utf8')
else:
return value