Someone tell me that what kind of changes I in my computer or file have to do for running py files in web page by xampp server, I have a simple code of getting data from database and it is also working properly, it is giving me tuple in list and no error is there while running it with cmd or vs code terminal
#! C:/Users/ashok/AppData/Local/Programs/Python/Python38-32
print("Content-Type:text/html")
import mysql.connector
print('<h1>Welcome to the Python</h1>')
conn= mysql.connector.connect(host='localhost', username='root', password='', database='first_db_py')
cur = conn.cursor()
query = "SELECT * FROM test"
cur.execute(query)
print(cur.fetchall())
conn.commit()
conn.close()
if you are able to understand this code so please tell me because I have watched so many tutorials on youtube but I didn't get
I tested some sample code that I found online. This is it.
import pandas as pd
import pypyodbc
conn = "DRIVER={SQL Server Native Client 11.0};SERVER=name.database.windows.net;DATABASE=db_name;UID=my_id;PWD=my_pwd"
df = pd.read_sql_query('select * from dbo.main_table', conn)
So, I would expect this to import the dataset, that sites in SQL Server, into the dataframe, but I'm getting this error.
ArgumentError: Could not parse rfc1738 URL from string 'DRIVER={SQL Server Native Client 11.0};SERVER=etc., etc., etc.
I'm working in a Databricks environment. Thanks for the look.
Unable to understand why my sql query is throwing an exception of [Oracle][ODBC][Ora]ORA-00936: missing expression.
The case is that the code seems to be working fine when I'm using
select* from reports.ORDERS_NOW.
So it's letting me pull all the data, but for my case, I want only specific columns for which I'm writing the query. Please look at the code below and let me know what's wrong with it.
import pyodbc
import pandas as pd
conn = conn = pyodbc.connect('DSN=abcd;UID=xxxxxx;PWD=xxxxxx')
if conn:
print("Connection is successful")
db query
sql = '''
select [QUANTITY] from reports.ORDERS_NOW
'''
df = pd.read_sql(sql,conn)
i think [] is not allowed in oracle so remove it
select QUANTITY from reports.ORDERS_NOW
I tried retrieving data from a Microsoft SQL database using pypyodbc 1.3.3 with Python 3.5 on Windows but got a pypyodbc.ProgrammingError '[24000] [Microsoft] [SQL Server Native Client 11.0] Invalid cursor state' using the following code:
import pypyodbc
conn = pypyodbc.connect(r'DRIVER={SQL Server Native Client 11.0};SERVER=server;DATABASE=database;UID=uid;PWD=pwd')
cursor = conn.cursor()
sql = '''USE database;
SELECT R0
FROM table;'''
cursor.execute(sql)
results = cursor.fetchone()
print(results)
The SQL works in Microsoft SQL Server Management Studio, the connection and executing worked in another script i wrote to insert into the same database and also works if i remove
results = cursor.fetchone()
So far I tried cursor.fetchone(), cursor.fetchall() and list(cursor) but all produced the same result which leads me to believe that the command itself isn't the problem.
According to this microsoft site it means that there isn't an open cursor, but I can get it's description, so from my understanding there has to be.
It's not a matter of being unable to execute a USE ... statement at all, it's just that we cannot do that as part of a multi-statement batch. So, this will not work ...
crsr.execute("""\
USE master;
SELECT TOP 2 name FROM sys.tables ORDER BY name;
""")
rows = crsr.fetchall() # error
... but this will work fine
crsr.execute("USE master")
crsr.execute("SELECT TOP 2 name FROM sys.tables ORDER BY name")
rows = crsr.fetchall()
(Tested with both pypyodbc 1.3.4 and pyodbc 4.0.21)
I had a similar issue. I was able to resolve this by removing the "USE Database" statement.
You already connected to your db here:
conn = pypyodbc.connect(r'DRIVER={SQL Server Native Client 11.0};SERVER=server;DATABASE=database;UID=uid;PWD=pwd')
I am trying to understand how python could pull data from an FTP server into pandas then move this into SQL server. My code here is very rudimentary to say the least and I am looking for any advice or help at all. I have tried to load the data from the FTP server first which works fine.... If I then remove this code and change it to a select from ms sql server it is fine so the connection string works, but the insertion into the SQL server seems to be causing problems.
import pyodbc
import pandas
from ftplib import FTP
from StringIO import StringIO
import csv
ftp = FTP ('ftp.xyz.com','user','pass' )
ftp.set_pasv(True)
r = StringIO()
ftp.retrbinary('filname.csv', r.write)
pandas.read_table (r.getvalue(), delimiter=',')
connStr = ('DRIVER={SQL Server Native Client 10.0};SERVER=localhost;DATABASE=TESTFEED;UID=sa;PWD=pass')
conn = pyodbc.connect(connStr)
cursor = conn.cursor()
cursor.execute("INSERT INTO dbo.tblImport(Startdt, Enddt, x,y,z,)" "VALUES (x,x,x,x,x,x,x,x,x,x.x,x)")
cursor.close()
conn.commit()
conn.close()
print"Script has successfully run!"
When I remove the ftp code this runs perfectly, but I do not understand how to make the next jump to get this into Microsoft SQL server, or even if it is possible without saving into a file first.
For the 'write to sql server' part, you can use the convenient to_sql method of pandas (so no need to iterate over the rows and do the insert manually). See the docs on interacting with SQL databases with pandas: http://pandas.pydata.org/pandas-docs/stable/io.html#io-sql
You will need at least pandas 0.14 to have this working, and you also need sqlalchemy installed. An example, assuming df is the DataFrame you got from read_table:
import sqlalchemy
import pyodbc
engine = sqlalchemy.create_engine("mssql+pyodbc://<username>:<password>#<dsnname>")
# write the DataFrame to a table in the sql database
df.to_sql("table_name", engine)
See also the documentation page of to_sql.
More info on how to create the connection engine with sqlalchemy for sql server with pyobdc, you can find here:http://docs.sqlalchemy.org/en/rel_1_1/dialects/mssql.html#dialect-mssql-pyodbc-connect
But if your goal is to just get the csv data into the SQL database, you could also consider doing this directly from SQL. See eg Import CSV file into SQL Server
Python3 version using a LocalDB SQL instance:
from sqlalchemy import create_engine
import urllib
import pyodbc
import pandas as pd
df = pd.read_csv("./data.csv")
quoted = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 11.0};SERVER=(localDb)\ProjectsV14;DATABASE=database")
engine = create_engine('mssql+pyodbc:///?odbc_connect={}'.format(quoted))
df.to_sql('TargetTable', schema='dbo', con = engine)
result = engine.execute('SELECT COUNT(*) FROM [dbo].[TargetTable]')
result.fetchall()
Yes, the bcp utility seems to be the best solution for most cases.
If you want to stay within Python, the following code should work.
from sqlalchemy import create_engine
import urllib
import pyodbc
quoted = urllib.parse.quote_plus("DRIVER={SQL Server};SERVER=YOUR\ServerName;DATABASE=YOur_Database")
engine = create_engine('mssql+pyodbc:///?odbc_connect={}'.format(quoted))
df.to_sql('Table_Name', schema='dbo', con = engine, chunksize=200, method='multi', index=False, if_exists='replace')
Don't avoid method='multi', because it significantly reduces the task execution time.
Sometimes you may encounter the following error.
ProgrammingError: ('42000', '[42000] [Microsoft][ODBC SQL Server
Driver][SQL Server]The incoming request has too many parameters. The
server supports a maximum of 2100 parameters. Reduce the number of
parameters and resend the request. (8003) (SQLExecDirectW)')
In such a case, determine the number of columns in your dataframe: df.shape[1]. Divide the maximum supported number of parameters by this value and use the result's floor as a chunk size.
I found that using bcp utility (https://learn.microsoft.com/en-us/sql/tools/bcp-utility) works best when you have a large dataset. I have 2.7 million rows that inserts at 80K rows/sec. You can store your data frame as csv file (use tabs for separator if your data doesn't have tabs and utf8 encoding). With bcp, I've used format "-c" and it works without issues so far.
This worked for me on Python 3.5.2:
import sqlalchemy as sa
import urllib
import pyodbc
conn= urllib.parse.quote_plus('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
engine = sa.create_engine('mssql+pyodbc:///?odbc_connect={}'.format(conn))
frame.to_sql("myTable", engine, schema='dbo', if_exists='append', index=False, index_label='myField')
"As the Connection represents an open resource against the database, we want to always limit the scope of our use of this object to a specific context, and the best way to do that is by using Python context manager form, also known as the with statement."
https://docs.sqlalchemy.org/en/14/tutorial/dbapi_transactions.html
The example would then be
from sqlalchemy import create_engine
import urllib
import pyodbc
connection_string = (
"Driver={SQL Server Native Client 11.0};"
"Server=myserver;"
"UID=myuser;"
"PWD=mypwd;"
"Database=mydb;"
)
quoted = urllib.parse.quote_plus(connection_string)
engine = create_engine(f'mssql+pyodbc:///?odbc_connect={quoted}')
with engine.connect() as cnn:
df.to_sql('mytable',con=cnn, if_exists='replace', index=False)
Following is what worked for me using sqlalchemy. Pay attention to the last part ?driver=SQL+Server'.
import sqlalchemy
import pyodbc
engine = sqlalchemy.create_engine('mssql+pyodbc://MyUser:MyPWD#dataserver.sandbox.myserver/MY_DB?driver=SQL+Server')
dt.to_sql("PatientResultTest", engine,if_exists='append')
The SQL table needs an index column at the beginning to store the index value of dataframe.
# using class function
import pandas as pd
import pyodbc
import sqlalchemy
import urllib
class data_frame_to_sql():
def__init__(self,dataFrame,sql_table_name):
self.dataFrame=dataFrame
self.sql_table_name=sql_table_name
def conversion(self):
params = urllib.parse.quote_plus("DRIVER={SQL Server};"
"SERVER=######;"
"DATABASE=####;"
"UID=#####;"
"PWD=###;")
try:
engine = sqlalchemy.create_engine("mssql+pyodbc:///?odbc_connect={}".format(params))
return f"Table '{self.sql_table_name}' added sucsessfully in database" ,self.dataFrame.to_sql(self.sql_table_name, engine)
except Exception as e :
e=str(e).replace(".","")
print(f"{e} in Database." )
data={"BusinessEntityID":["1","2","3"],"FirstName":["raj","abhi","amir"],"LastName":["kapoor","bachn","khhan"]}
df = pd.DataFrame(data, columns= ['BusinessEntityID','FirstName','LastName'])
ab=data_frame_to_sql(df,"ab").conversion()
print(ab)
It's not necessary to use sqlamchemy, one could create a connection with pyodbc directly to use it with pandas, as below: `with pyodbc.connect('DRIVER={ODBC Driver 18 for SQL Server};SERVER='+server
+';DATABASE='+database+';UID='+username+';PWD='+ password) as newconn:
df = pd.read_sql(,newconn)
`