We Keep Coding

How to store a data frame in a sql server database with to_sql?

I'm writing code to store a dataframe in my sql server database. The code is as follows:
from sqlalchemy import create_engine
server='my_server'
database='my_database'
driver='ODBC Driver 17 for SQL Server'
database_con=f'mssql+pyodbc://#{server}/{database}?driver={driver}'
engine=create_engine(database_con)
con=engine.connect()
data = [ 'test','test1', 'tests']
df_data = pd.DataFrame(data)
df_data.to_sql('dbo.test', con, index=False)
However, when I run the code I get an error:
ProgrammingError: (pyodbc.ProgrammingError) ('42000', '[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The specified schema name "name#email.nl" either does not exist or you do not have permission to use it. (2760) (SQLExecDirectW)')
[SQL:
CREATE TABLE [dbo.hoi] (
[0] VARCHAR(max) NULL
)
]
(Background on this error at: http://sqlalche.me/e/f405)
Does somebody know how to solve the problem?
Cheers,
Vincent

Create table and Insert to SQL Server using Python

I have a huge table (147 columns) and I would like to know if it is possible to create the table in SQL server from my pandas dataframe so I don't have to CREATE TABLE for 147 columns.
I am trying to follow what is answered in most of related questions:
params = urllib.parse.quote_plus("DRIVER={ODBC Driver 17 for SQL Server};SERVER=DESKTOP-LFOSSEF;DATABASE=test;UID=xxxx;PWD=xxx")
engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
connection = engine.raw_connection()
df.to_sql("table_name", connection,index=False)
The user and password work because that's what I am using to sign in into sqlserver.
When I run this, I get:
Execution failed on sql 'SELECT name FROM sqlite_master WHERE
type='table' AND name=?;': ('42S02', "[42S02] [Microsoft][ODBC Driver
17 for SQL Server][SQL Server]Invalid object name 'sqlite_master'.
(208) (SQLExecDirectW); [42S02] [Microsoft][ODBC Driver 17 for SQL
Server][SQL Server]Statement(s) could not be prepared. (8180)")
If I remove the connection = engine.raw_connection() and use engine directly, I get:
AttributeError: 'Engine' object has no attribute 'cursor'
Any idea what is wrong? Do I need to create an empty table first? I've been troubleshooting for hours and can't find any reason why this isnt working.

Do it like this.
import pyodbc
engine = "mssql+pyodbc://Your_Server_Name/Your_DB_Name?driver=SQL Server Native Client 11.0?trusted_connection=yes"
df.to_sql(x, engine, if_exists='append', index=True)
df = name of your dataframe & x = name of your table in SQL Server

Unable To Connect mdb VIEW Table via pyodbc Python

I am now facing a difficulty connecting VIEW table in mdb file using pyodbc.
Current code is as follows:
(Previous Provider: Microsoft.Jet.OLEDB.4.0)
import pyodbc
pyodbc.pooling = False
conn_str = (
r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};'
r'DBQ=C:***.mdb;'
)
cnxn = pyodbc.connect(conn_str)
crsr = cnxn.cursor()
for table_info in crsr.tables(tableType='TABLE'):
print(table_info.table_name)
strSQL = "SELECT A,B FROM tableC WHERE ** ORDER BY B"
crsr.execute(strSQL)
for row in crsr.fetchall():
print(row)
When it works, for table_info in crsr.tables(tableType='TABLE'): seems to have no issue.
This is because print(table_info.table_name) displays several table names.
However, when crsr.execute(strSQL) is done, exeption error has raised and following message
('HY000', "[HY000] [Microsoft][ODBC Microsoft Access Driver] ODBC--connection to 'VIEW' failed. (-2001) (SQLExecDirectW)")
is shown.
It would be appreciated to instruct me how to overcome this issue.
Thanks.

"Cannot find data type READONLY" error on UPDATE to SQL Server

I've been stuck in this problem for a long time, i hope somebody could enlighten me. I have a sql database i would like to update. Here are some pieces of the code. I extracted data from sql to Python, then apply function hex_to_string & slicing bin and I plan to update the SQL database. I don't have any ID in the database, but I have the DATETIME which differentiates the entry.
query = """ select P from Table """
cnxn = pyodbc.connect(conn_str)
cnxn.add_output_converter(pyodbc.SQL_VARBINARY, hexToString)
cursor: object = cnxn.cursor()
cursor.execute(query)
dtbs= cursor.fetchall()
row_list=[]
ln = len(dtbs)
cursor.execute(query)
for i in range(ln):
row=cursor.fetchval()
result=slicing_bin(row)
result_float = [float("{0:.2f}".format(i)) for i in result]
row_list.append(result_float)
crsr = cnxn.cursor()
crsr.execute(query)
aList = [item[0] for item in crsr.fetchall()]
for aValue in aList:
crsr.execute("""UPDATE Table SET P=? WHERE DATETIME=?""", (row_list, aValue))
crsr.close()
cnxn.commit()
When I run this code, I got an error message,
File
"C:/Users/r/.PyCharmCE2018.3/config/scratches/Finalcombined2.py",
line 64, in
crsr.execute("""UPDATE Access.dbo.M_PWA SET P_PULSES=? WHERE DATETIME=?""", (row_list, aValue)) pyodbc.ProgrammingError: ('42000',
"[42000] [Microsoft][ODBC Driver 11 for SQL Server][SQL Server]Column,
parameter, or variable #1: Cannot find data type READONLY. (2715)
(SQLExecDirectW); [42000] [Microsoft][ODBC Driver 11 for SQL
Server][SQL Server]Statement(s) could not be prepared. (8180); [42000]
[Microsoft][ODBC Driver 11 for SQL Server][SQL Server]Parameter or
variable '#P1' has an invalid data type. (2724)")
Please Help, thanks.

Hummm, I would have guessed that Gord was right. That's certainly the first thing I'd look at. Ok, here is a small sample of how I do updates in MS Access, from Python.
#import pypyodbc
import pyodbc
# MS ACCESS DB CONNECTION
pyodbc.lowercase = False
conn = pyodbc.connect(
r"Driver={Microsoft Access Driver (*.mdb, *.accdb)};" +
r"Dbq=C:\\path_here\\Northwind.mdb;")
# OPEN CURSOR AND EXECUTE SQL
cur = conn.cursor()
# Option 1 - no error and no update
cur.execute("UPDATE dbo_test SET Location = 'New York' Where Status = 'Scheduled'");
conn.commit()
cur.close()
conn.close()
Can you adapt this to your specific scenario?

SQLAlchemy df.to_sql MSSQL

I am trying to move data from a database I own to a database I only have read/write permissions on (as a temp table). However, I get an error. Please see the below code:
import pandas as pd
import pyodbc
import sqlalchemy
con = pyodbc.connect(r'Driver={SQL Server};Server=MYServer;Database=DB_People;Trusted_Connection=yes;)
sSQL = "SELECT * FROM tbl_People;"
df = pd.read_sql(sSQL, con)
con.close()
con = pyodbc.connect(r'Driver={SQL Server};Server=NOT_MyServer;Database=DB_People;Trusted_Connection=yes;)
sSQL = "CREATE TABLE [##People Table] (p_Name varchar(25), p_DOB char(10));"
con.execute(sSQL)
con.committ()
engine = sqlalchemy.create_engine('mssql://NOT_MyServer/DB_People?Driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
df.to_sql("##People Table")
I get the error:
DBAPIError: (pyodbc.Error) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
Any clues as to what the issue is? I specified a Driver in my connection and I know the Temp table exists because when I connect via SSMS I can query the table (although there are no records at this point.
I

It seems as if I have found the problem(s):
The "Create Table" function is unneeded as the df.to_sql creates the table
Instead of the "+" in the Driver name, I was able to replace them with %20
This worked for me.