### ESTABLISHING CONNECTION TO SNOWFALKE
#Installing libraries
##pip install snowflake-connector-python==2.3.8
#Importing all the required libraries
import getpass
import snowflake.connector
import pandas as pd
#Getting user imputs to establish connection to snowflake
USER = input("PLEASE ENTER YOUR SNOWFLAKE USERNAME ")
PASSWORD = getpass.getpass("ENTER YOUR SNOWFLAKE PASSWORD ")
ACCOUNT = input("PLEASE ENTER YOUR SNOWFLAKE ACCOUNT NAME ")
#create snowflake database connection
conn= snowflake.connector.connect(
user=USER,
password=PASSWORD,
account=ACCOUNT,
warehouse='COHORT_XL',
database = 'IMV_IMMUNOVACCINE_INC_DLBCL_RITUXIMAB_ANY_ANTHRACYCLINE',
schema = 'SANDBOX'
)
cur=conn.cursor()
#Checking the conenction
cur
import pandas as pd
original = r"C:\Users\aseem.malik\Downloads\TX.csv" # <- Replace with your path.
delimiter = "," # Replace if you're using a different delimiter.
total = pd.read_csv(original, sep = delimiter)
write_pandas(conn, total, 'test')
The write pandas funtion is not working keeps giving the below error.
Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'.
A suitable version of pyarrow or fastparquet is required for parquet support.
Trying to import the above resulted in these errors:
- Missing optional dependency 'pyarrow'. pyarrow is required for parquet support. Use pip or conda to install pyarrow.
- Missing optional dependency 'fastparquet'. fastparquet is required for parquet support. Use pip or conda to install fastparquet.
Please install those packages:
pip install pyarrow
pip install fastparquet
Related
Since the optuna documentation does not address which modules are required from MySQL, I installed everything of MySQL on my Windows 10 machine. I looked for MySQL on my PC (in which folder the installation takes place is not revealed during installation) and updated the Path variables to
C:\Program Files\MySQL\MySQL Server 8.0\bin
I have successfully created the mysqltestexample database.
Using python SQL connectors, I can reproduce the output using:
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="root",
password="Start123"
)
print(mydb)
mycursor = mydb.cursor()
mycursor.execute("SHOW DATABASES")
for x in mycursor:
print(x)
mydb = mysql.connector.connect(
host="localhost",
user="root",
password="Start123",
database="mysqltesteexample"
)
Connection to the mysqltesteexample does not raise an error - so everything seems to be fine. However, optuna is not able to connect to my database
My python script looks like this. It is the code from the optuna documentation, I just altered the name of the test database.
study0 = optuna.create_study(storage="mysql://root#localhost/mysqltesteexample",study_name="distributed-example")
study0 = optuna.create_study(storage="mysql+pymysql://root:Start123#localhost:3306/mysqltesteexample",study_name="distributed-example")
All attempts to modify the URL string according to https://docs.sqlalchemy.org/en/14/core/engines.html failed with the following error: ImportError: Failed to import DB access module for the specified storage URL. Please install appropriate one.
Can you please help me to get it done? Thank you in advance, please don't be too harsh.
Finally, I made it. I have to install some further packages from the cmd:
py -3.8 -m easy_install mysql-python
py -3.8 -m pip install mysqlclient
Python packages - as well documented as they are eyes rolling
I am trying to connect to Informix database using Python package ibm_db but am getting the below error.
Error:
builtins.Exception: [IBM][CLI Driver] SQL0902C A system error occurred. Subsequent SQL statements cannot be processed. IBM software support reason code: "". SQLSTATE=58005
SQLCODE=-902
Code:
import sys
import ibm_db
import ibm_db_dbi
import pyodbc
import subprocess
import os
import string
import pandas
try:
conn= ibm_db.connect("HOSTNAME=xxxxxx;PORT=1900;PROTOCOL=onsoctcp ;DATABASE=webrpt; INSTANCE=vec_sandbox;UID=xxxx;PWD=xxxx","","")
except:
print ("Transaction couldn't be completed:" , ibm_db.stmt_errormsg())
else:
print ("Transaction complete.")
I was able to connect using Pyodbc package and installing client SDK on the Linux maching and configuring /etc/odbc.ini , /etc/odbcinst.ini and /app/informix/etc/sqlhosts files.
thanks!!!
I am using Jupyter Notebooks to learn Python. I would like to connect to a MySQL db hosted locally hosted through MAMP. How would I approach this?
import os
import pymysql
import pandas as pd
host = os.getenv('MYSQL_HOST')
port = os.getenv('MYSQL_PORT')
user = os.getenv('MYSQL_USER')
password = os.getenv('MYSQL_PASSWORD')
database = os.getenv('MYSQL_DATABASE')
conn = pymysql.connect(
host=host,
port=int(3306),
user="root",
passwd=password,
db="[YOUR_DB_NAME]",
charset='utf8mb4')
df = pd.read_sql_query("SELECT * FROM YOUR_TABLE",
conn)
df.tail(10)
Assuming you have MySQL installed (instructions here for macOS using HomeBrew), you need to:
Install pip3 install ipython-sql
pip3 install mysqlclient
now you should be able to run these cells and get pretty-printed HTML output:
# %%
%load_ext sql
# %%
%sql mysql+mysqldb://<user>:<password>#localhost/<dataBase>
# %%
%%sql
SELECT *
FROM <table>;
import pymysql
import pandas as a
conn=pymysql.connect(host='localhost',port=int(3306),user='root',passwd='YOUR_PASSWORD',db='YOUR_DATABASENAME')
df=a.read_sql_query("SELECT * FROM 'YOUR_TABLENAME' ",conn)
print(df)
Yes, you can. You can use the MySQL Connector library. Simply install it using pip, and then you can use it to interact with your database. See the sample code below:
import mysql.connector
db = mysql.connector.connect(
host="localhost",
user="mamp",
passwd=""
)
print(db)
I am not able to open the access file using python. I am not sure if the problem is with the mdb file or the python commands.
In [1]: import sys, subprocess
In [2]: DATABASE = 'Exam_BackUp.mdb'
In [3]: table_names = subprocess.Popen(["mdb-tables", "-1", DATABASE], stdout=subprocess.PIPE).communicate()[0]
Couldn't open database.
How do I know if the file is microsoft access file?
I have checked that mdbtools is installed on my Ubuntu server.
I need to open the (access or fortran) file and save the contents to csv.
Why not try opening it with an ODBC driver?
A good example is here, reproducing it for your case would be along the lines of:
import pyodbc
DBfile = 'Exam_BackUp.mdb'
conn = pyodbc.connect('FILEDSN='+DBfile)
cursor = conn.cursor()
# Do whatever you want with SQL selects, etc
cursor.close()
conn.close()
You can convert it by the Terminal using mdbtool like this:
Install mdbtools and upgrade it:
pip install mdbtools
pip install --upgrade pip
Then look for the name of the table inside the mdb file:
home/Docs$ mdb-tables 'file.mdb'
And finally convert the file to .csv with this line:
home/Docs$ mdb-export 'file.mdb' 'name_of_table' > 'file.csv'
I found below link
http://python.projects.pgfoundry.org/docs/1.0/driver.html#connection-keywords
it says...
import postgresql.driver as pg_driver
is the way to import
i used -
import postgresql.driver as pg_driver
pg_driver.connect(user = self.username, password = self.password, host = self.host, port = self.port,sslmode = 'verify-full', sslrootcert=self.ssl_cert)
but it gives 'ImportError: No module named postgresql.driver'
i tried 'pip install postgresql'
but it gives 'Could not find any downloads that satisfy the requirement postgresql
No distributions at all found for postgresql'
how can i fixed that ???
You can download latest version of py-postgresql and use 'Python34\python.exe setup.py install' to install it.
You can check Python34\Lib\site-packages\ if there is a folder called postgresql which proves it's installed correctly. Then try import postgresql.driver as pg_driver again and tell me if it works.