I am able to connect to aws-redshift with psycopg2 using python, I can query tables and get data back, etc...
However, when I try to run a create udf fucntion through psycopg2, nothing happens, no error returns but nothing gets created.
Here's my code:
def _applyFunctionToDB():
con=psycopg2.connect(dbname = redhsiftDatabase, host = redshiftHost, port = '5439', user = redshiftUser, password = redshiftPwd)
cur = con.cursor()
udf=_fileOpenWrite(udfFile)
size = os.stat(udfFile).st_size
udfCode=udf.read(size)
cur.execute(udfCode)
con.close()
I have run it through the debugger and all the pieces are there, but nothing happens when the "execute" method is invoked on the cursor.
If anyone has any advice and/or ideas on what might be going on here, please advise.
Thanks!
found answer just after posting here: Copying data from S3 to AWS redshift using python and psycopg2
I need to invoke a commit.
So, add con.commit in above code after execute.
Related
I'm trying to execute many (~1000) MERGE INTO statements into oracledb 11.2.0.4.0(64bit) using python 3.9.2(64bit) and pyodbc 4.0.30(64bit). However, all the statements return an exception:
HY000: The driver did not supply an error
I've tried everything I can think of to solve this problem, but no luck. I tried changing code, encodings/decodings and ODBC driver from oracle home 12.1(64bit) to oracle home 19.1(64bit). I also tried using pyodbc 4.0.22 in which case the error just changed into:
<class 'pyodbc.ProgrammingError'> returned a result with an error set
Which is not any more helpful error than the first one. The issue I assume cannot be the MERGE INTO statement itself, because when I try running them directly in the database shell, it completes without issue.
Below is my code. I guess I should also mention the commands and parameters are read from stdin before being executed, and oracledb is using utf8 characterset.
cmds = sys.stdin.readlines()
comms = json.loads(cmds[0])
conn = pyodbc.connect(connstring)
conn.setencoding(encoding="utf-8")
cursor = conn.cursor()
cursor.execute("""ALTER SESSION SET NLS_DATE_FORMAT='YYYY-MM-DD"T"HH24:MI:SS.....'""")
for comm in comms:
params = [(None) if str(x)=='None' or str(x)=='NULL' else (x) for x in comm["params"]]
try:
cursor.execute(comm["sql"],params)
except Exception as e:
print(e)
conn.commit()
conn.close()
Edit: Another things worth mentioning for sure - this issue began after python2.7 to 3.9.2 update. The code itself didn't require any changes at all in this particular location, though.
I've had my share of HY000 errors in the past. It almost always came down to a syntax error in the SQL query. Double check all your double and single quotes, and makes sure the query works when run independently in an SQL session to your database.
For details I'm running a very recent version of python and PyODBC and am trying to remotely instantiate a backup on SQL. I don't know much about SQL. I've tried googling the issue and have tried a few tricks below including using .nextset() and trying different encodings just in case. I see a large surge on the local network of the PC with the SQL server but it very quickly tapers to nothing. I also can't seem to run a stored procedure as it will never seem to find it despite it existing and being able to run from my local machine using SSMS or the SSMS on the remote PC. Also I've given complete access to the remote account accessing the sql server as far as I know
Here is the relevant function:
def back_up_and_restore(self):
if self.Arbin not in [3, 4]:
self.errorString = "Not remote server!"
return self.errorString
self.cnxn = pyodbc.connect(self.connectionString(self.db[0]))
self.cursor = self.cnxn.cursor()
self.cnxn.autocommit = True
if self.Arbin == 3:
folder = "Fred"
else:
folder = "George"
query = r"BACKUP DATABASE [%s] TO DISK = N'\\10.130.130.5\TestLab\DATA\%s\ArbinMasterData.bak' WITH NOFORMAT, INIT, NAME = N'ArbinMasterData-Full Database Backup', SKIP, NOREWIND, NOUNLOAD, STATS = 10" % ("ArbinMasterData", folder)
#query = "{CALL BackUp%s2}" % folder
query.encode(encoding='UTF-8')
print(query)
self.cursor.execute(query)
time.sleep(2)
while self.cursor.nextset():
pass
self.cnxn.close
When I run the function it just dies quickly and doesn't say anything. If I try running a stored procedure it says that the stored procedure doesn't exist. Any help would be great.
This code is working great if you need an example of PyODBC making a backup. I unfortunately was looking in the George folder instead of Fred.
I have created a stored procedure usuarios_get , I test it in oracle console and work fine. This is the code of stored procedure
create or replace PROCEDURE USUARIOS_GET(
text_search in VARCHAR2,
usuarios_list out sys_refcursor
)
AS
--Variables
BEGIN
open usuarios_list for select * from USUARIO
END USUARIOS_GET;
The python code is this:
with connection.cursor() as cursor:
listado = cursor.var(cx_Oracle.CURSOR)
l_query = cursor.callproc('usuarios_get', ('',listado)) #in this sentence produces error
l_results = l_query[1]
The error is the following:
NotSupportedError: Variable_TypeByValue(): unhandled data type VariableWrapper
I've also tried with other stored procedure with a out parameter number type and modifying in python code listado= cursor.var(cx_Oracle.NUMBER) and I get the same error
NotSupportedError: Variable_TypeByValue(): unhandled data type VariableWrapper
I work with
python 2.7.12
Django 1.10.4
cx_Oracle 5.2.1
Oracle 12c
Can any help me with this?
Thanks
The problem is that Django's wrapper is incomplete. As such you need to make sure you have a "raw" cx_Oracle cursor instead. You can do that using the following code:
django_cursor = connection.cursor()
raw_cursor = django_cursor.connection.cursor()
out_arg = raw_cursor.var(int) # or raw_cursor.var(float)
raw_cursor.callproc("<procedure_name>", (in_arg, out_arg))
out_val = out_arg.getvalue()
Then use the "raw" cursor to create the variable and call the stored procedure.
Looking at the definition of the variable wrapper in Django it also looks like you can access the "var" property on the wrapper. You can also pass that directly to the stored procedure instead -- but I don't know if that is a better long-term option or not!
Anthony's solution works for me with Django 2.2 and Oracle 12c. Thanks! Couldn't find this solution anywhere else on the web.
dcursor = connection.cursor()
cursor = dcursor.connection.cursor()
import cx_Oracle
out_arg = cursor.var(cx_Oracle.NUMBER)
ret = cursor.callproc("<procedure_name>", (in_arg, out_arg))
I'm trying to use ipython-cypher to run Neo4j Cypher queries (and return a Pandas dataframe) in a Python program. I have no trouble forming a connection and running a query when using IPython Notebook, but when I try to run the same query outside of IPython, as per the documentation:
http://ipython-cypher.readthedocs.org/en/latest/introduction.html#usage-out-of-ipython
import cypher
results = cypher.run("MATCH (n)--(m) RETURN n.username, count(m) as neighbors",
"http://XXX.XXX.X.XXX:xxxx")
I get the following error:
neo4jrestclient.exceptions.StatusException: Code [401]: Unauthorized. No permission -- see authorization schemes.
Authorization Required
and
Format: (http|https)://username:password#hostname:port/db/name, or one of dict_keys([])
Now, I was just guessing that that was how I should enter a Connection object as the last parameter, because I couldn't find any additional documentation explaining how to connect to a remote host using Python, and in IPython, I am able to do:
%load_ext cypher
results = %cypher http://XXX.XXX.X.XXX:xxxx MATCH (n)--(m) RETURN n.username,
count(m) as neighbors
Any insight would be greatly appreciated. Thank you.
The documentation has a section for the API. When used outside of IPython and in need to connect to a different host, just using the parameter conn and passing a string should work.
import cypher
results = cypher.run("MATCH (n)--(m) RETURN n.username, count(m) as neighbors",
conn="http://XXX.XXX.X.XXX:xxxx")
But also consider that with the new authentication support in Neo4j 2.2, you need to set the new password before connecting from ipython-cypher. I will fix this as soon as I implement the forcing password change mechanism in neo4jrestclient, the library underneath.
Hello guys I am currently working in a python project at my school. First I want to make clear that I'm not a python programmer (I was just called to put out the flames in this project because no one else would and I was brave enough to say yes).
I have the following problem here. I have to write a method that connects to an existing localhost MySQL database (I'm using connector version 1.0.12 and python 2.6) and then does pretty basic stuff. The parameters are sent by a GTK-written GUI (I didn't write that interface). So I wrote my method like this:
def compMySQL(self, user, database, password, db_level, table_level, column_level):
sql_page_textview = self.mainTree.get_widget('sql_text_view')
sql_page_textview.modify_font(pango.FontDescription("courier 10"))
sql_page_buffer = sql_page_textview.get_buffer()
#Gonna try connecting to DB
try:
print("Calling conn with U:{0} P:{1} DB:{2}".format(user,password,database))
cnxOMC = mysql.connector.connect(user, password,'localhost',database)
except:
print "Error: Database connection failed. User name or Database name may be wrong"
return
#More code ...
But when I run my code I get this:
Calling conn with U:root P:PK17LP12r DB:TESTERS
Error: Database connection failed. User name or Database name may be wrong
And I don't know why, since the arguments sent are the same arguments that get printed (telling me that the GUI the other guy coded works fine) and they are valid login parameters. If I hardcode the login parameters directly insetad of using the GUI everything goes ok and the functions executes properly; the following code executes nice and smooth:
def compMySQL(self, user, database, password, db_level, table_level, column_level):
sql_page_textview = self.mainTree.get_widget('sql_text_view')
sql_page_textview.modify_font(pango.FontDescription("courier 10"))
sql_page_buffer = sql_page_textview.get_buffer()
#Gonna try hardcoding
try:
#print("Calling conn with U:{0} P:{1} DB:{2}".format(user,password,database))
cnxOMC = mysql.connector.connect(user="root", password='PK17LP12r', host='localhost', database='TESTERS')
print 'No prob with conn'
except:
print "Error: Database connection failed. User name or Database name may be wrong"
return
#more code ...
Console output:
No prob with conn
Any ideas guys? This one is killing me. I'm just learning Python but I imagine the problem to be something very easy for a seasoned python developer so any help would be strongly appreciated.
Thanks in advance.
The difference between the two versions is not that you are hard coding the parameters in the second one, it is that you are calling by keyword args rather than positional ones. The docs for MySQL connector don't seem to give the actual positional order, and there's no reason to think they are in the order you've given, so looks like you should always call by kwarg:
cnxOMC = mysql.connector.connect(user=user, password=password,host=host,database=database)