I've been trying to automate the installation of an Open Street Map Server since no one has published one yet and the task is pretty tedious. In order to do this I'm dealing with PostgreSQL databases in a script, which I left Python in charge of.
Here's the situation: Basically I'm running python scripts dealing with the database throughout bash code. I'm trying to make the install as user friendly as possible, part of that is automating the PostgreSQL setup. I prompt the user, in bash, for a password they would like to use for the postgres database that already comes with PostgreSQL. I then send their password as a command line argument to a Python script.
This is the part of the script I'm having problems with:
import psycopg2
import sys
con = None
code = sys.argv[1]
try:
con = psycopg2.connect(database='postgres', user='postgres')
cur = con.cursor()
cur.execute("ALTER USER postgres WITH PASSWORD '%s'" % code)
Basically: On the bottom line where I change the password for the postgres database, it doesn't actually work. I know this because later I am prompted in my bash script to enter the password and it results in an authentication failure.
I'm pretty new to this, so if anyone has some good advice, it would be greatly appreciated.
Please use the below code, you can generate random passwords and update them
NOTE: For this code to work, the readwrite1 user has to be present in database prior using this
from psycopg2 import Error
import psycopg2
import random
#password generation
def password_generator(password_length):
# maximum length of password needed
characters = string.ascii_letters + string.digits + '!##$%^&*()'
password = ''.join(random.choice(characters) for i in range(password_length))
return password
#define a function that handles and parses psycopg2 exceptions
def print_psycopg2_exception(err):
err_type, err_obj, traceback = sys.exc_info()
# get the line number when exception occured
line_num = traceback.tb_lineno
# print the connect() error
print ("\npsycopg2 ERROR:", err, "on line number:", line_num)
print ("psycopg2 traceback:", traceback, "-- type:", err_type)
# psycopg2 extensions.Diagnostics object attribute
print ("\nextensions.Diagnostics:", err.diag)
# print the pgcode and pgerror exceptions
print ("pgerror:", err.pgerror)
print ("pgcode:", err.pgcode, "\n")
def update_password():
password=password_generator(10)
try:
con = psycopg2.connect(host="host here",database="dbhere", user="username",password="password")
cur = con.cursor()
cur.execute("alter user readwrite1 with password %(password)s;", {'password': password})
con.commit()
except Exception as err:
# pass exception to function
print_psycopg2_exception(err)
exit(1)
finally:
print('password is: ', password)
if con:
con.close()
return password
Related
I am attempting to execute some basic SQL via Python using the teradatasql module. The code appears to run and the SQL is executed: however, the execution of the Python itself ends with an error on the end of the code reproduced below. Currently, I need to run additional data preprocessing steps using pandas on the output of the SQL, but the larger program will not continue past the Operational Error (not even via a try/except block excepting the teradatsql.OperationalError). Therefore even though the SQL executes fine with this issue, I need to resolve it.
Any suggestions? Thank you!
Error:
teradatasql.OperationalError: 1 is not a valid connection pool handle
Code:
import teradatasql
import os
def refresh_table():
usr = ****1
with open(f'C:\\Users\\{usr}\\Documents\\my_td_password.txt', 'r') as my_pwd_f:
pw = my_pwd_f.read()
with teradatasql.connect(host = '*******2'
, user = usr
, password = pw
, ) as con:
with con.cursor() as cur:
with open('C:\\Users\\****1\\Documents\\test.sql', 'r') as my_sql:
sql_script = my_sql.read()
for sql_block in sql_script.split(';'):
try:
cur.execute(sql_block)
print("Block executed")
except ValueError:
print("Failure to execute block: ValueError")
finally:
print(sql_block)
my_sql.close()
print("SQL file closed")
con.close()
print("Connection closed")
refresh_table()
Fixed by removing con.close() from the end - as Fred pointed out, the with block implicitly closes the connection when it finishes execution
https://stackoverflow.com/users/11552426/fred
I have two machines: local_machine, server_machine. I have mysql server on server_machine and sftp server on local_machine. I am trying to send sritest.csv file (UTF-8) from local_machine to server_machine using python. These are the contents of sritest.csv:
1,2,3
I have the sql query saved in sritest.sql and these are the contents of the file:
LOAD DATA INFILE '{}'
INTO TABLE TESTBED_STAGING.test
COLUMNS TERMINATED BY ','
;
This is the python script I have now:
import MySQLdb
import os
import string
# Open database connection
db = MySQLdb.connect (host="1.2.3.4",port=3306,user="app_1",\
passwd="passwd",db="TESTBED_STAGING")
cursor=db.cursor()
#Query under testing
sql = open('sritest.sql','r').read()
print sql
l = os.listdir(".")
for file_name in l:
if file_name.endswith('sritest.csv'):
print 'the csv file we are reading is: '+file_name
#try:
cursor = db.cursor()
print 'filename is '+sql.format(file_name)
cursor.execute(sql.format(file_name))
db.commit()
'''
except Exception:
# Rollback in case there is any error
db.rollback()
print 'ERROR - So, rollback :( :( '
'''
# disconnect from server
db.close()
In the above script, I commented try,except so I can see the error where it breaks. Currently the code is breaking at cursor.execute(sql.format(file_name)) line with this error:
OperationalError: (1045, "Access denied for user 'app_1'#'%' (using password: YES)")
I have been playing around but not able to fix it. Any suggestions/ideas?
For starters, creating cursor at every loop is not a good idea. You've already created a cursor earlier, so you can remove the cursor declaration in the for loop.
Second, I think your error is due to lack of access on MySQL server at 1.2.3.4 remotely using user app_1. Try this on the server's MySQL console,
GRANT ALL PRIVILEGES ON TESTBED_STAGING.* TO 'app_1'#'%';
Lastly, try and avoid using print "line" notation and start switching to the print("line") notation for compatibility with Python 3.x
I figured out the answer and decided to leave this question open for those who might face the similar problem:
In the MySQL server (server_machine), make sure you do this after you start mysql:
mysql>grant all privileges on *.* to 'app_1'#'%' identified by 'passwd';
change LOAD DATA INFILE '{}' in sritest.sql to LOAD DATA LOCAL INFILE '{}'
In the python code, edit the MySQLdb.connect statement as:
db = MySQLdb.connect (host="1.2.3.4",port=3306,user="app_1",\
passwd="passwd",db="TESTBED_STAGING", local_infile=1)
All errors are eliminated and data is transferred.
I'm currently writing a chat bot with plugin functionality and at the moment, I'm working on a permission system.
However, my insert query into my database somehow doesn't work. If I do it by hand, it works flawless.
Here's that piece of code... hopefully you see what I try here:
def dothis(message):
if message.content.split()[1].lower() == "op":
user = get_member_by_name(message, message.content.split()[2])
try:
pmcon = mdb.connect(db_server, db_user, db_pass, db_name)
pmcur = pmcon.cursor()
pmcur.execute("INSERT INTO users (username,userid,hasop) VALUES (\'{}\',\'{}\',{})".format(message.content.split()[2], user.id, "TRUE"))
except: mdb.Error, e:
print "Error %d: %s" % (e.args[0], e.args[1])
finally:
if pmcon:
pmcon.close()
I already tried putting the query in a string and let it be printed out, but I don't see an error.
Am I doing something wrong?
if your database connection is not configured to autocommit, you need to commit your statements:
pmcon.commit()
(after the execute statement.)
i'm making a login system for part of my coursework, where it querys the database to see if the data is correct. It works fine until I introduce a login that's not in the database. Is there anyway I can make the while working even after the mycursor.fetchone() gives the error for not fetching anything?
The code below is the subroutine I'm having issues with:
#the error message I'm receiving
username, password = mycursor.fetchone()
TypeError: 'NoneType' object is not iterable
#the subroutine I'm having trouble with
def accept(self):
conn = mysql.connector.connect(user, password, host, database)
#create cursor between python and mysql
mycursor = conn.cursor()
query = """
SELECT
username, password
FROM
UserInfo.User
WHERE
username = %s AND password = %s
"""
mycursor.execute(query, (self.textName.text(),self.textPass.text(), ))
global username
username, password = mycursor.fetchone()
#self.textName.text() and self.textPass.text() is the username and password entered by the user
if username == self.textName.text() and password == self.textPass.text():
self.GoToLogin()
else:
#the error message I want to display
QtGui.QMessageBox.warning(
self, 'Error', 'Username and password are incorrect')
Add try except block and ignore the exception that happens when the record is not found or if multiple records are found.
try:
#db call
except (SpecificException, AnotherException):
pass
It's good to catch specific exceptions than catch all exceptions but in this case you can just catch the global Exception
Also there is no reason to check again if the user and password are equal, since you're already passing that as part of the query.
This is my first project using psycopg2 extensively. I'm trying to find a way to extract the psql error message for whenever a connection attempt fails. I've tested the code below will work if all the variables are set correctly, however whenever an error condition occurs (e.g. user chooses a database that doesn't exist), Python will give me the following:
I am unable to connect to the database
None
Traceback (most recent call last):
File "./duplicate_finder.py", line 163, in <module>
main(sys.argv[1:])
File "./duplicate_finder.py", line 142, in main
print e.diag.message_detail
AttributeError: 'OperationalError' object has no attribute 'diag'
Is there a simple, catch-all method to catch whatever error message psql generates when a connection fails, or do I need to write except blocks for multiple psycopg2 exceptions?
Extract from my script:
import sys, getopt, os, time, csv, psycopg2
...
...
conn_string = "host=" + dbhost + " dbname=" + database + " user=" + dbuser + " password=" + dbpass
try:
conn = psycopg2.connect(conn_string)
except psycopg2.Error as e:
print "Unable to connect!"
print e.pgerror
print e.diag.message_detail
sys.exit(1)
else:
print "Connected!"
cur = conn.cursor()
cur.execute("SELECT id, lastname, firstname, location FROM test ORDER BY ctl_upd_dttm DESC;")
print cur.fetchone()
...
conn.close()
When I try to catch exceptions, e.pgerror is always None for connection errors. The following code block gets around this by directly printing 'e'.
try:
conn = psycopg2.connect(conn_string)
except psycopg2.OperationalError as e:
print('Unable to connect!\n{0}').format(e)
sys.exit(1)
else:
print('Connected!')
# do stuff
For example, in the case of password authentication failure:
Unable to connect!
FATAL: password authentication failed for user "user"
I realize this question is a year old but hopefully might help someone in the future
Ended up here because of
class 'psycopg2.errors.InvalidCursorName'
on Django. If that's your case, be sure to makemigrations
You are catching all exceptions with the base class psycopg2.Error. Your problem is probably that the diag attribute is new in psycopg2 2.5. What is your version?
>>> print psycopg2.__version__
2.5.1 (dt dec pq3 ext)
Since Python 3.9 (.removesuffix(), f-strings) I use
except psycopg2.Error as e:
log.error(f"{type(e).__module__.removesuffix('.errors')}:{type(e).__name__}: {str(e).rstrip()}")
if conn: conn.rollback()
where log is logger.
Connection errors lives directly in psycopg2 module while syntax errors in psycopg2.errors submodule. There is an unwanted newline at the end of every psycopg2 error message.