We Keep Coding

PYMYSQL ERROR pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax ")

Here is the full error
Traceback (most recent call last):
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/test.py", line 26, in <module>
handler()
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/test.py", line 17, in handler
cursor.execute('SELECT * FROM Reads')
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "Desktop/Work/RDS_LAMDA_Stuff/RDS_Node_Stuff/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Reads' at line 1")
For context here is my code
from multiprocessing import connection
import pymysql
from pymysql.cursors import Cursor
# RDS config
endpoint = '********'
username = '********'
password = '********'
database_name = '*********'
#connection config
connection = pymysql.connect(host=endpoint,user=username,passwd=password,db=database_name)
def handler():
cursor = connection.cursor()
cursor.execute('SELECT * FROM Reads')
rows = cursor.fetchall()
for row in rows:
print("{0} {1} {2} {3}".format(row[0], row[1], row[2], row[3]))
handler()
I have 2 other tables in my database which work fine when i query them however this table seems to be the only one causing an error.
Feel free to ask for more if this is too vague

It's worth checking manual in cases like this - and Reads is a reserved word therefore must be backticked - better still don't use reserved words or you will forever have to remember to backtick in code..

Is there a way to store pandas dataframe to a Teradata table

I have created a pandas data-frame 'df' and I am trying to store it in a 'table' using Teradata-SQL assistant.
Connection string -
conn = pyodbc.connect(
"DRIVER=Teradata;DBCNAME=tdprod;Authentication=LDAP;UID=" + username + ";PWD=" + password + ";QUIETMODE=YES",
autocommit=True, unicode_results=True)
cursor = conn.cursor().execute(sql)
Tried using: df.to_sql('table', con =conn)
This doesn't work.
Is there an easier way to store a dataframe into a table.
Any help is appreciated.
Thanks.
Traceback (most recent call last):
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\engine\base.py", line 2158, in _wrap_pool_connect
return fn()
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\pool.py", line 410, in connect
return _ConnectionFairy._checkout(self, self._threadconns)
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\pool.py", line 788, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\pool.py", line 529, in checkout
rec = pool._do_get()
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\pool.py", line 1096, in _do_get
c = self._create_connection()
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\pool.py", line 347, in _create_connection
return _ConnectionRecord(self)
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\pool.py", line 474, in __init__
self.__connect(first_connect_check=True)
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\pool.py", line 671, in __connect
connection = pool._invoke_creator(self)
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\engine\strategies.py", line 106, in connect
return dialect.connect(*cargs, **cparams)
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\sqlalchemy\engine\default.py", line 412, in connect
return self.dbapi.connect(*cargs, **cparams)
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\teradata\tdodbc.py", line 454, in __init__
checkStatus(rc, hDbc=self.hDbc, method="SQLDriverConnectW")
File "C:\Users\tripata\PycharmProjects\NLP\venv\lib\site-packages\teradata\tdodbc.py", line 231, in checkStatus
raise DatabaseError(i[2], u"[{}] {}".format(i[0], msg), i[0])
teradata.api.DatabaseError: (8017, '[28000] [Teradata][ODBC Teradata Driver][Teradata Database] The UserId, Password or Account is invalid. , [Teradata][ODBC Teradata Driver][Teradata Database] The UserId, Password or Account is invalid. ')

From the docs for to_sql:
Parameters
----------
name : string
Name of SQL table.
con : sqlalchemy.engine.Engine or sqlite3.Connection
Using SQLAlchemy makes it possible to use any DB supported by that
library. Legacy support is provided for sqlite3.Connection objects.
You can see you need sqlalchemy or sqlite3, but not pyodbc.
You need the following to create an engine for Teradata:
from sqlalchemy import create_engine
engine = create_engine(f'teradata://{username}:{password}#tdprod:22/')
You would then use it like
df.to_sql('table', engine)

I've done some digging and this solution does the job and does it quickly - using the python teradata module:
import teradata
import numpy as np
import pandas as pd
num_of_chunks = 100 #breaking the data into chunks is optional - use if you have many rows or would like to view status updates
query = 'insert into SomeDB.SomeTeraDataTable'
df = someDataframe
#set host, user, password params
host,username,password = 'hostName_or_IPaddress','username', 'password'
#connet to DB using UdaExec
udaExec = teradata.UdaExec (appName="IMC", version="1.0", logConsole=False)
with udaExec.connect(method="odbc",system=host, username=username,
password=password, driver="Teradata") as connect:
df_chunks = np.array_split(df, num_of_chunks)
for i,_ in enumerate(df_chunks):
data = [tuple(x) for x in df_chunks[i].to_records(index=False)]
connect.executemany(query, data,batch=True)
solution based on: this stackoverflow post

create_engine('teradata://' +user+':'+ password + '#'+host+':1025/'+'/'+'?authentication=LDAP')
Adding both the host name and authentication to the connection string worked for me.

Insert ERROR - mysql connector python 2.7

I am attempting to insert some sentiment analysis results (google cloud language API) into a mysql database. I am using the mysql connector.
import mysql.connector
from google.cloud import language
cnx = mysql.connector.connect(user='blahuser', password='Blahpw',
host='BlahIP',
database='FeedbackDB')
cursor = cnx.cursor(buffered=True)
CoreSQL = ("SELECT ResID, TextResp FROM Response")
cursor.execute(CoreSQL)
client = language.Client()
for row in cursor:
document = client.document_from_text(row[1])
sent_analysis = document.analyze_sentiment()
sentiment = sent_analysis.sentiment
annotations = document.annotate_text(include_sentiment=True, include_syntax=True, include_entities=True)
print(row[0], sentiment.score, sentiment.magnitude)
ResID_ =row[0]
PhraseSent_ = sentiment.score
PhraseMag_ = sentiment.magnitude
SQLInsertCmd = ("INSERT INTO PhraseAnalysis (ResID, PhraseSent, PhraseMag), VALUES (%s,%s,%s)");
cursor.execute(SQLInsertCmd, (ResID_, PhraseSent_,PhraseMag_))
cnx.commit()
cursor.close()
cnx.close()
The error I get indicates I have an issue with my INSERT statement :
python tm16.py
(1, -0.4, 2.2)
Traceback (most recent call last):
File "tm16.py", line 27, in <module>
cursor.execute(SQLInsertCmd, (ResID_, PhraseSent_,PhraseMag_))
File "/usr/lib/python2.7/dist-packages/mysql/connector/cursor.py", line 559, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/usr/lib/python2.7/dist-packages/mysql/connector/connection.py", line 494, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "/usr/lib/python2.7/dist-packages/mysql/connector/connection.py", line 396, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' VALUES (1,-0.4,2.2)' at line 1
There are lots of INSERT examples online, but I haven't be able to resolve. New to coding- no doubt something simple. Can someone point out where I am going wrong?
Mike

You have an unnecessary comma after the end of the fields list.

Pandas DataFrame.to_sql() error - not all arguments converted during string formatting

Python Version - 2.7.6
Pandas Version - 0.17.1
MySQLdb Version - 1.2.5
DataFrame.to_sql() is throwing pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': not all arguments converted during string formatting
Python Code Snippet
con = MySQLdb.connect('localhost', 'root', '', 'product_feed')
cur = con.cursor()
cur.execute("SELECT VERSION()")
connection_result = cur.fetchall()
print connection_result[0][0] #It prints 5.5.44-0ubuntu0.14.04.1
table_column = ['A', 'B', 'C']
created_data = numpy.array([numpy.arange(10)]*3).T
df = pandas.DataFrame(data=created_data ,columns=table_column)
df.to_sql('test_table', con)
The error comes at the execution of df.to_sql('test_table', con) line.
Error Details
File "/home/yogi/yogi/mlPython/product_feed/etl_pf/process_data.py", line 298, in render_df
df.to_sql('test_table', con)
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/core/generic.py", line 1003, in to_sql
dtype=dtype)
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/io/sql.py", line 569, in to_sql
chunksize=chunksize, dtype=dtype)
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/io/sql.py", line 1640, in to_sql
table.create()
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/io/sql.py", line 685, in create
if self.exists():
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/io/sql.py", line 673, in exists
return self.pd_sql.has_table(self.name, self.schema)
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/io/sql.py", line 1653, in has_table
return len(self.execute(query, [name,]).fetchall()) > 0
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/io/sql.py", line 1554, in execute
raise_with_traceback(ex)
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/pandas/io/sql.py", line 1543, in execute
cur.execute(*args)
File "/home/yogi/yogi/mlenv/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 187, in execute
query = query % tuple([db.literal(item) for item in args])
pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': not all arguments converted during string formatting
I checked that pandas 0.17.1 is mostly using .format() so this error should not arise because of % formatting.
It would be great help if someone could suggest some work around. I do not want to try this with cursor.execute()

Parameters:
con : SQLAlchemy engine or DBAPI2 connection (legacy mode)
Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported.
flavor : ‘sqlite’, default None
Deprecated since version 0.19.0: ‘sqlite’ is the only supported option if SQLAlchemy is not used.
It will be fine if you use SQLAlchemy instead of MySQLdb.

Running Py ODBC queries?

Trying to query for dates in an Access DB, and it keeps returning the follow error:
Traceback (most recent call last):
File "C:/Users/sniederriter/Desktop/SATG.py", line 10, in <module>
for row in cursor.execute(SQL):
File "C:\Python34\lib\site-packages\pypyodbc-1.3.3-py3.4.egg\pypyodbc.py", line 1605, in execute
self.execdirect(query_string)
File "C:\Python34\lib\site-packages\pypyodbc-1.3.3-py3.4.egg\pypyodbc.py", line 1631, in execdirect
check_success(self, ret)
File "C:\Python34\lib\site-packages\pypyodbc-1.3.3-py3.4.egg\pypyodbc.py", line 986, in check_success
ctrl_err(SQL_HANDLE_STMT, ODBC_obj.stmt_h, ret, ODBC_obj.ansi)
File "C:\Python34\lib\site-packages\pypyodbc-1.3.3-py3.4.egg\pypyodbc.py", line 954, in ctrl_err
raise ProgrammingError(state,err_text)
pypyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error (missing operator) in query expression 'Date *'.")
Here is my code:
import pypyodbc
DBfile = (r'C:\Users\sniederriter\Desktop\SATGexpenses.accdb')
conn = pypyodbc.connect(r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ='+DBfile)
cursor = conn.cursor()
print(DBfile)
SQL = 'SELECT Date * FROM 20142015;'
for row in cursor.execute(SQL):
print (row.Date)
cursor.close()
conn.close()

Your MySQL query is wrong. Following code selects all fields from the table:
SELECT * FROM Table
If you want specific fields write:
SELECT field1, field2, field3 FROM table
But only do one of the things. Either all or specified fields.