I have been having major trouble connecting my python shell to my postgres. I am doing this on windows. I have downloaded psycopg2 and everything for this to process, however it still is not working.
import psycopg2
conn=psycopg2.connect("dbname = 'test' user ='postgres' host ='localhost' password = 'mypassword'")
It gives me an error telling me that the database "test" does not exist, however it does! If you guys have any advice at all on what I should test out, that would be amazing. Thank you!
You can layout connection parameters as a string and pass it to the connect() function as like:
conn = psycopg2.connect("dbname=test user=postgres password=postgres")
Or you can use a list of keyword arguments like
conn = psycopg2.connect(host="localhost",database="test", user="postgres", password="postgres")
If its still fails then you should check on PostgreSQL side. You should try to connect the db in question using command line and see if error re appears or not. if it appears then something is missing on DB server side.
Related
I need to check oracle connectivity using python script.
My oracle connection string is in below format.
jdbc:oracle:thin:#ldap://ovd.mycomp.com:38901/cn=Oraclecontext,o=eus,dc=mycomp,dc=com/pidev ldap://ovd- mwdc.mycomp.com:38901/cn=Oraclecontext,o=eus,dc=mycomp,dc=com/pidev.
I tried https://dbajonblog.wordpress.com/2019/12/18/python-and-cx_oracle-for-oracle-database-connections/ but that did not help.
Could you please help me.
Thanks in advance.
The general cx_Oracle documentation on working with JDBC and Oracle SQL Developer Connection Strings has some info however if you're using LDAP you'll need to do some extra configuration. See https://stackoverflow.com/a/32151099/4799035 and https://github.com/oracle/node-oracledb/issues/1212#issuecomment-591940440 The steps are the same for cx_Oracle. Also see Connect to DB using LDAP with python cx_Oracle
I created following script using cx_oracle, which works fine.
only restriction is the dns name should be TNS entry.
Script:
import cx_Oracle
import sys
from botocore.exceptions import ClientError
connection = None
def isOracleHealthy(dbname, username, password, dns, log):
try:
sys.stderr.write(dns)
connection=cx_Oracle.connect("{}/{}#{}".format(username, password, dns))
cur=connection.cursor()
for result in cur.execute("SELECT * FROM dual"):
log.info(result)
return True
except Exception as e:
sys.stderr.write(dbname+' Oracle health check failed.\n')
log.error(dbname+' Oracle health check failed.')
return False
I am attempting to create a flask web application that uses mysql to store users and some other data. When I use the following code
import MySQLdb
def connection():
conn = MySQLdb.connect(host='localhost', user='root', passwd='password', database='data')
c = conn.cursor()
return c, conn
I get a 500 internal service error. This code will connect on the command line, and my flask site works fine until I add the line
from dbconnect import connection
I have tried using different ways to connect to the database, (python connector, pymysql). They all give the same error. I have also tried updating permissions from inside mysql to the user. I have been following tutorials from https://pythonprogramming.net/flask-connect-mysql-using-mysqldb-tutorial/ but they did not get these errors in the tutorial and I have followed most of the tutorial exactly.
I've been trying for some days to connect my python 3 script to PostgresSQL database(psycopg2) in Heroku, without Django.
I found some article and related questions, but I had to invest a lot of time to get something that I thought should be very straightforward, even for a newbie like me.
I eventually made it work somehow but hopefully posting the question (and answer) will help other people to achieve it faster.
Of course, if anybody has a better way, please share it.
As I said, I had a python script that I wanted to make it run from the cloud using Heroku. No Django involved (just a script/scraper).
Articles that I found helpful at the beginning, even if they were not enough:
Running Python Background Jobs with Heroku
Simple twitter-bot with Python, Tweepy and Heroku
Main steps:
1. Procfile
Procfile has to be:
worker: python3 folder/subfolder/myscript.py
2. Heroku add-on
Add-on Heroku Postgres :: Database has to be added to the appropriate personal app in the heroku account.
To make sure this was properly set, this was quite helpful.
3. Python script with db connection
Finally, to create the connection in my python script myscript.py, I took this article as a reference and adapted it to Python 3:
import psycopg2
import urllib.parse as urlparse
import os
url = urlparse.urlparse(os.environ['DATABASE_URL'])
dbname = url.path[1:]
user = url.username
password = url.password
host = url.hostname
port = url.port
con = psycopg2.connect(
dbname=dbname,
user=user,
password=password,
host=host,
port=port
)
To create a new database, this SO question explains it. Key line is:
con.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
You can do it using the SQLALCHEMY library.
First, you need to install the SQLALCHEMY library using pip, if you don't have pip on your computer install, you will know-how using a simple google search
pip install sqlalchemy
Here is the code snippet that do what you want:
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
import os
# Put your URL in an environment variable and connect.
engine = create_engine(os.getenv("DATABASE_URL"))
db = scoped_session(sessionmaker(bind=engine))
# Some variables you need.
var1 = 12
var2 = "Einstein"
# Execute statements
db.execute("SELECT id, username FROM users WHERE id=:id, username=:username"\
,{"id": var1, "username": var2}).fetchall()
# Don't forget to commit if you did an insertion,etc...
db.commit()
I wasn't able to parse the DATABASE_URL provided by Heroku with the urllib.parse as suggested above, but the following worked for me:
The URL I retrieved from Heroku was in the format:
postgres://username:password#host:port/database
for example:
postgres://jticiuimwernbk:ff78903549d4c6ec13a53a8ffefcd201b937d54c35d976
#ec2-52-123-182-987.compute-1.amazonaws.com:5432/dbsd4fdf6c1awq
So I manually dissected it as follows:
user = 'jticiuimwernbk'
password = 'ff78903549d4c6ec13a53a8ffefcd201b937d54c35d976'
host = 'ec2-52-123-182-987.compute-1.amazonaws.com'
port = '5432'
database = 'dbsd4fdf6c1awq'
#Then created the connection using the above:
con = psycopg2.connect(database=database,
user=user,
password=password,
host=host,
port=port)
# and now I was able to perform queries:
cur = conn.cursor()
results = cur.execute("<some SQL query>;").fetchall()
cur.close()
conn.close()
Is it possible to make SQLAlchemy do cross server joins?
If I try to run something like
engine = create_engine('mssql+pyodbc://SERVER/Database')
query = sql.text('SELECT TOP 10 * FROM [dbo].[Table]')
with engine.begin() as connection:
data = connection.execute(query).fetchall()
It works as I'd expect. If I change the query to select from [OtherServer].[OtherDatabase].[dbo].[Table] I get an error message "Login failed for user 'NT AUTHORITY\\ANONYMOUS LOGON"
Looks like there's an issue with how you authenticate to SQL server.
I believe you can connect using the current Windows user, the URI syntax is then mssql+pyodbc://SERVER/Database?trusted_connection=yes (I have never tested this, but give it a try).
Another option is to create a SQL server login (ie. a username/password that is defined within SQL server, NOT a Windows user) and use the SQL server login when you connect.
The database URI then becomes: mssql+pyodbc://username:password#SERVER/Database.
mssql+pyodbc://SERVER/Database?trusted_connection=yes threw an error when I tried to it. It did point me in the right direction though.
from sqlalchemy import create_engine, sql
import urllib
string = "DRIVER={SQL SERVER};SERVER=server;DATABASE=db;TRUSTED_CONNECTION=YES"
params = urllib.quote_plus(string)
engine = create_engine('mssql+pyodbc:///?odbc_connect={0}'.format(params))
query = sql.text('SELECT TOP 10 * FROM [CrossServer].[datbase].[dbo].[Table]')
with engine.begin() as connection:
data = connection.execute(query).fetchall()
It's quite complicated if you suppose to alter different servers through one connection.
But if you need to perform a query to a different server under different credentials you should add linked server first with sp_addlinkedserver. Then it should be added credentials to the linked server with sp_addlinkedsrvlogin. Have you tried this?
I have recently installed Microsoft SQL Server 2014 on my PC as I want to create a database for a web application I am building (I have been learning Python for a year and have very basic experience with SQLite).
After installing SQL Server 2014 and creating a database called Users, I am just trying to run some very basic commands to my database but I am falling at the first hurdle over and over!
I have installed pymssql and pyodbc and tried running commands directly with these but have failed. (e.g. pymssql gives me a TypeError: argument of type 'NoneType' is not iterable when I set the variable conn = pymssql.connect(server, user, password, "tempdb")
My latest attempt is to use SQLalchemy to achieve my long awaited connection with SQL database. However, after installing this, it is failing on the following error:
"sqlalchemy.exc.OperationalError: (pymssql.OperationalError) (20009, 'DB-Lib error message 20009, severity 9:\nUnable to connect: Adaptive Server is unavailable or does not exist\nNet-Lib error during Unknown error (10035)\n')"
The question I need answering is, how do I start talking to my database using SQLalchemy?
The code I am using is as follows:
from sqlalchemy import *
engine = create_engine('mssql+pymssql://Han & Lew:#SlugarPlum:1433/Users')
m = MetaData()
t = Table('t', m,
Column('id', Integer, primary_key=True),
Column('x', Integer))
m.create_all(engine)
Yes, my PC is called SlugarPlum. User is Han & Lew. And my server is called THELROYSERVER. DSN = 1433. No password. (I don't know if it is wise that I am giving this information online but the data I have is not sensitive so I guess it's worth a shot.)
Also, if anyone can direct me to an ultra-beginners resource for Python-SQL server that would be awesome as I am getting beaten up by how complex this seems to be!
Here's a connect function and example for connecting via pyodbc. Connecting via pymssql should be as easy as formatting the connecting string for pymssql. I've provided Windows and Linux options, but only tested on Linux. I hope it helps.
def sqlalchemy_connect(connect_string):
""" Connect to the database via ODBC, start SQL Alchemy engine. """
def connect():
return pyodbc.connect(connect_string, autocommit=True)
db = create_engine('mssql://', creator=connect)
db.echo = False
return db
def main():
global DBCONN
# Linux with FreeTDS
connect_string = "DRIVER={FreeTDS};SERVER=<server name>;PORT=<port num>;DATABASE=<db>;UID=<user>;PWD=<password>;TDS_Version=<version num>;"
# Windows with SQL Server
connect_string = "DRIVER={SQL Server};SERVER=<server name>;PORT=<port num>;DATABASE=<db>;UID=<user>;PWD=<password>;"
DBCONN = sqlalchemy_connect(connect_string)
if __name__ == "__main__":
main()