SQLAlchemy error connecting to Google Cloud SQL with proxy - python

system: ubuntu
Cloud sql instance: db
Cloud sql user: admin
Cloud sql pass: pass
cloud sql db name: test
cloud_sql_proxy installed and executed by ./cloud_sql_proxy -dir=/cloudsql -instances=prj:asia-northeast1:db -credential_file=path/to/credential
The account in the credential file has all needed roles, and successfully connected to the db from a nodejs server (typeorm).
But with sqlalchemy, I tried
sqlalchemy.create_engine("postgresql+psycopg2://admin:pass#/test?host=/cloudsql/prj:asia-northeast1:db")
and
sqlalchemy.create_engine("postgres+pg8000://admin:pass#/test?unix_sock=%2Fcloudsql%2Fprj%3Aasia-northeast1%3Adb%2F.s.PGSQL.5432")
but both complains about FATAL: password authentication failed for user "admin"
What have I did wrong?

This error states that the db user and db password that you are using to connect to Cloud SQL server is wrong.
I would recommend to create a new Cloud SQL user and password and try again.
Creating and managing MySQL users
If you succeed with the new user, this will confirm my first hypothesis.

Related

How do I edit Azure SQL Database using pyodbc/odbc?

I wrote a Python program that web scraped a website and added the results to a Microsoft Access database. I now want to run the script again, with it adding the data to an Azure SQL database. I keep getting this error.
A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections.
I have tried t edit the settings of the database to no avail. Could someone tell me what settings to apply to the database? I also tried to see if there was a way to run the Python script inside azure to try to avoid the problem. Is this possible?
cnxn = pyodbc.connect(r'Driver={ODBC Driver 18 for SQL Server};Server=tcp:servername.database.windows.net,1433;Database=sizedb3;Uid={your_user_name};Pwd={your_password_here};Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;Authentication=ActiveDirectoryPassword')
I tried this driver. I have downloaded the driver from Microsoft's website. This driver is a connection string in the ODBC section of the Azure SQL database in the Azure portal.
I tried running the below python code to connect to Azure SQL DB with Azure AD authentication.
Code:-
import pyodbc
conn = pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};'
'Server=tcp:siliconserver.database.windows.net,1433;'
'Database=silicondb;'
'Uid=xxxser#sid24desaioutlook.onmicrosoft.com;'
'Pwd=xxxxxxxxxx#123;'
'authentication=ActiveDirectoryPassword')
cursor = conn.cursor()
cursor.execute('SELECT * FROM StudentReviews')
for i in cursor:
print(i)
cursor.close()
conn.close()
Output:-
Make sure you have allowed your Client IP in your Azure SQL server Networking tab like below:-
I tried to remove one syntax/spelling from my Azure SQL connection string and got the same error code as yours like below:-
You can validate your connection string server spelling and syntax from your Azure SQL server DB > Connection String > like below:-
Also, Make sure you have added your client IP and allowed it in your Azure SQL Server like below:-

Oracle Connection in Python Using create_engine

I am trying to connect to an oracle database in Python using create_engine. This database does not have a username or password.
I see this is the format now:
oracle_db = sqlalchemy.create_engine('oracle://user:pass#server').
However, if this connection has NO username or password, how would the connection string look? I've tried DMIT_connection = create_engine('oracle+cx_oracle://#....) with no luck. When I go to write a pandas df to the database using to_sql I get the error below because I cannot get the connection right give that there is no username or password.
The error occurs because this database has no username (picked up from the localhost machine) and there is no password.
The error I get is this: DatabaseError: (cx_Oracle.DatabaseError) ORA-12545: Connect failed because target host or object does not exist (Background on this error at: http://sqlalche.me/e/14/4xp6)
Let me know authentication type used. If its external authentication , picking credentials from wallet, you can try sample code mentioned here
How to use SqlAlchemy to connect Database similar to cx_oracle when we use external authorization like wallets with TNS(net service name)

Use Managed Identity to authenticate Azure App Service to SQL Database

I am trying to connect a Python Flask app running in Azure App Service Web App to an Azure SQL Database.
The works just fine when I use SQL authentication with username and password.
Now I want to move to using the Web Apps managed identity.
I have activated the system-assigned managed identity, created a user for it in SQL and added it to the db_datareader role.
I am connecting with SqlAlchemy using a connection string like this
params = urllib.parse.quote_plus(os.environ['SQL_CONNECTION_STRING'])
conn_str = 'mssql+pyodbc:///?odbc_connect={}'.format(params)
engine_azure = db.create_engine(conn_str,echo=True)
The connection string is stored as an application setting, and its value is
"Driver={ODBC Driver 17 for SQL Server};Server=tcp:<server>.database.windows.net,1433;Database=<database>;Authentication=ActiveDirectoryMsi;"
I expected this to be all I need to do, but now my app is not starting.
The logs report a timeout when connecting to the database.
How can I fix this?
I know this is quite an old post, but it may help people like me who are looking for a solution.
You could modify the connection string by adding "Authentication" parameters as "ActiveDirectoryMsi", no need to use endpoint and headers.
(Works with Azure SQL, for other databases like Postgress you may need to use the struct token)
import pyodbc
pyodbc.connect(
"Driver="
+ driver
+ ";Server="
+ server
+ ";PORT=1433;Database="
+ database
+ ";Authentication=ActiveDirectoryMsi")
I wrote a quick article for those who are interested in Azure MSI:
https://hedihargam.medium.com/python-sql-database-access-with-managed-identity-from-azure-web-app-functions-14566e5a0f1a
If you want to connect Azure SQL database with Azure MSI in python application, we can use the SDK pyodbc to implement it.
For example
Enable system-assigned identity for your Azure app service
Add the MSi as contained database users in your database
a. Connect your SQL database with Azure SQL AD admin (I use SSMS to do it)
b. run the following the script in your database
CREATE USER <your app service name> FROM EXTERNAL PROVIDER;
ALTER ROLE db_datareader ADD MEMBER <your app service name>
ALTER ROLE db_datawriter ADD MEMBER <your app service name>
ALTER ROLE db_ddladmin ADD MEMBER <your app service name>
Code
import os
import pyodbc
import requests
import struct
#get access token
identity_endpoint = os.environ["IDENTITY_ENDPOINT"]
identity_header = os.environ["IDENTITY_HEADER"]
resource_uri="https://database.windows.net/"
token_auth_uri = f"{identity_endpoint}?resource={resource_uri}&api-version=2019-08-01"
head_msi = {'X-IDENTITY-HEADER':identity_header}
resp = requests.get(token_auth_uri, headers=head_msi)
access_token = resp.json()['access_token']
accessToken = bytes(access_token, 'utf-8');
exptoken = b"";
for i in accessToken:
exptoken += bytes({i});
exptoken += bytes(1);
tokenstruct = struct.pack("=i", len(exptoken)) + exptoken;
conn = pyodbc.connect("Driver={ODBC Driver 17 for SQL Server};Server=tcp:andyserver.database.windows.net,1433;Database=database2", attrs_before = { 1256:bytearray(tokenstruct) });
cursor = conn.cursor()
cursor.execute("select ##version")
row = cursor.fetchall()
For more details, please refer to the
https://github.com/AzureAD/azure-activedirectory-library-for-python/wiki/Connect-to-Azure-SQL-Database
https://learn.microsoft.com/en-us/azure/app-service/overview-managed-identity
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-aad-authentication-configure

Why am I getting peer authentication failed error when trying to connect using psycopg2?

So I have a successful install of Postgres on Ubuntu and am trying to do some basic connection and creating a table in a db using another username other than the default (postgres) and coming up short. From what I can gather I think it may have something to do with permissions? What I want is to be able to use some superuser other than postgres to create tables and do stuff.
"psql example3" and "\l" shows the example3 db was created successfully. I now have a list of databases include the default postgres, template0, template1 and example3 all with owner as postgres. What I run into problems then is running demoscript.py gives a fatal "peer authentication failed for user 'thisuser'"
#Create the db and user with superuser permissions
sudo -u postgres -i
createdb example3
createuser --interactive --pwprompt
#role:thisuser
#pwd:thispass
#superuser?:Y
#demoscript.py
import psycopg2
connection = psycopg2.connect('dbname=example3 user=thisuser password=thispass')
cursor = connection.cursor()
cursor.execute('DROP TABLE IF EXISTS todos;')
cursor.execute('''
CREATE TABLE todos(
id serial PRIMARY KEY,
description VARCHAR NOT NULL
);
''')
connection.commit()
cursor.close()
connection.close()
Expected result is that the todos table should show as created after looking for it in the example3 db. But I just get the fatal error.
When you don't specify a host in your connection, it tries to connect via Unix sockets. By default, PostgreSQL is set up to use peer authentication on those, which means it compares the PostgreSQL username to the currently logged in OS user. If you change your connection to:
connection = psycopg2.connect('dbname=example3 user=thisuser password=thispass host=localhost')
that should cause it to use the authentication settings for TCP/IP connections, which default to password authentication on most systems I've used.

Connect to SQL Server with user credentials of another domain

How can I connect to a SQL Server database using user login/password that is in another domain?
If I use my account to connect to DB, it works fine:
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=server_name;DATABASE=testdb;UID=MY_Domain\\username;PWD=pass; Trusted connection=YES')
But I need to use another user's credentials like
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=server_name;DATABASE=testdb;UID=Another_Domain\\username;PWD=pass; Trusted connection=YES')
When I try the latter I get an error of "failed login for MY_Domain\username" rather than
for the user "Another_Domain\username".
In both cases by running SQL Server Management Studio I can use Windows Authentication to connect to the db.
You can not pass a UID and Password and set Trusted_connection=True (your second connection string) to connect as a (impersonated) windows user. You can either connect as a SQL Server user (username and password) or as a windows authenticated user (trusted connection).
Your code should impersonate the windows user (as SSMS does) and then set Trusted_connection=True only.
This MSDN page WindowsIdentity.Impersonate has an example.
Since this works from SSMS it suggests there is the necessary trust between the domains.

Categories