We Keep Coding

TypeError when try to load information form website

I have a problem with getting information from a website to my python program. Thats what I tried:
import pandas as pd
import pyodbc as odbc
import geopandas as gpd
import shapely
import shapefile
import sys
import datetime
server = '....'
database = '<database>'
username = '<username >'
password = '<password>'
driver={'ODBC Driver 13 for SQL Server'}
sql_conn = odbc.connect('DRIVER='+driver+';SERVER='+server+';PORT=....;DATABASE='+database+';UID='+username+';PWD='+ password)
query = "select * from view;"
df = pd.read_sql(query, sql_conn)
df.head()
Error:
TypeError: can only concatenate str (not "set") to str
Does anybody know what I did wrong? I jusst want to collect the information and save it for further processing. I googled but could not find a mistake...

driver={'ODBC Driver 13 for SQL Server'} driver is a python set.
change to driver='ODBC Driver 13 for SQL Server'
See https://snakify.org/en/lessons/sets/#:~:text=Set%20in%20Python%20is%20a,union%2C%20intersection%2C%20difference).

You are trying to add str type to set type
'DRIVER='+driver+'
as you defined driver as string in curly braces, which means: set with given string as sole element. Probably you meant to define it as string, so delete braces; otherwise you, you can get (random) element from set using
next(iter(driver))

Difficulty inserting data into MySQL db using pymysql

I have written a little script using python3 that gets a RSS news feed using the feedparser library.
I then loop through the entries (dictionary) and then use a try/except block to insert the data into a MySQL db using pymysql (originally I tried to use MySQLDB but read here and other places that is does not work with Python3 or above)
I originally followed the PyMySQL example on git hub, however this did not work for me and I had to use different syntax for pymysql like they have here on digital ocean. However this worked for me when I tested out their example on their site.
But when I tried to incorporate it into my query,there was an error as it would not run the code the try block and just ran the exception code each time.
Here is my code;
#! /usr/bin/python3
# web_scraper.py 1st part of the project, to get the data from the
# websites and store it in a mysql database
import cgitb
cgitb.enable()
import requests,feedparser,pprint,pymysql,datetime
from bs4 import BeautifulSoup
conn = pymysql.connect(host="localhost",user="root",password="pass",db="stories",charset="utf8mb4")
c = conn.cursor()
def adbNews():
url = 'http://feeds.feedburner.com/adb_news'
d = feedparser.parse(url)
articles = d['entries']
for article in articles:
dt_obj = datetime.datetime.strptime(article.published,"%Y-%m-%d %H:%M:%S")
try:
sql = "INSERT INTO articles(article_title,article_desc,article_link,article_date) VALUES (%s,%s,%s,%s,%s)"
c.execute(sql,(article.title, article.summary,article.link,dt_obj.strftime('%Y-%m-%d %H:%M:%S'),))
conn.commit()
except Exception:
print("Not working")
adbNews()
I am not entirely sure what I am doing wrong. I have converted the string so that it is the format for the MySQL DATETIME type. As I originally did not have this but each time I run the program nothing gets stored in the db and the exception gets printed.
EDIT:
After reading Daniel Roseman's comments I removed the try/except block and read the errors that python gave me. It was to do with an extra argument in my sql query.
Here is he edited working code;
#! /usr/bin/python3
# web_scraper.py 1st part of the project, to get the data from the
# websites and store it in a mysql database
import cgitb
cgitb.enable()
import requests,feedparser,pprint,pymysql,datetime
from bs4 import BeautifulSoup
conn = pymysql.connect(host="localhost",user="root",password="pass",db="stories",charset="utf8mb4")
c = conn.cursor()
def adbNews():
url = 'http://feeds.feedburner.com/adb_news'
d = feedparser.parse(url)
articles = d['entries']
for article in articles:
dt_obj = datetime.datetime.strptime(article.published,"%Y-%m-%d %H:%M:%S")
#extra argument was here removed now
sql = "INSERT INTO articles(article_title,article_desc,article_link,article_date) VALUES (%s,%s,%s,%s)"
c.execute(sql,(article.title, article.summary,article.link,dt_obj.strftime('%Y-%m-%d %H:%M:%S'),))
conn.commit()
adbNews()

Python : Accessing oracle database and updating it

I am accessing oracle database and trying to update it using python. Below is my code :
import cx_Oracle
import pandas as pd
import datetime
import numpy
import math
conn = cx_Oracle.connect(conn_str)
c = conn.cursor()
def update_output_table(customer_id_list,column_name,column_vlaue_list) :
num_rows_to_add = len(customer_id_list)
conn = cx_Oracle.connect(conn_str)
c = conn.cursor()
for i in range(0,num_rows_to_add,1) :
c.execute("""UPDATE output SET """+column_name+""" = %s WHERE customer_id = %s""" %(column_vlaue_list[i],customer_id_list[i]))
total_transaction_df = pd.read_sql("""select distinct b.customer_id,count(a.transaction_id) as total_transaction from transaction_fact a,customer_dim b where a.customer_id = b.CUSTOMER_ID group by b.CUSTOMER_ID""",conn)
# Update this details to the output table
update_output_table(list(total_transaction_df['CUSTOMER_ID']),'TOTAL_TRANSACTION',list(total_transaction_df['TOTAL_TRANSACTION']))
conn.close()
My program is getting executed completely but I don't see my database table getting updated. Can someone suggest where I am going wrong?
Note : I am a newbie.Sorry for asking silly doubts. Thanks in advance.

You're missing conn.commit() before conn.close():
Here you will find some info why you need it explicitely. Without commit your code is doing update then when closing connection all non-commited changes are rolled back so you see no changes in DB.
You can also set cx_Oracle.Connection.autocommit = 1 but this is not recommended way as you're loosing control over transactions.

comparing a given variable to data in a database and checking to see if it exists

I have a sqlite db of API keys and I want to make something check and see if the given key is in the database.I'm generating the API keys using another python script named apikeygen.py. I'm using python 2.7 and pattern 2.6. This is going to be a data scraping/mining/filtering application that I'm doing just for fun and maybe have a future use for malware analysis.
I need help getting the main piece of code that we will call API.py to check and see if the given API key is in the database.
This is the code for the API.py file so far.
import os, sys; sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", ".."))
import sqlite3 as lite
from pattern.server import App
from pattern.server import MINUTE, HOUR, DAY
app = App("api")
def search_db(key=''):
con = lite.connect('apikeys.db')
with con:
cur = con.cursor()
cur.execute("SELECT * FROM keys")
while True:
row = cur.fetchone()
if row == None:
break
print row[2]

I'm still not really clear what you are asking. Why don't you explicitly query for the key, rather than iterating over your whole table?
cur.execute("SELECT * FROM keys WHERE key = ?", (key,))

Creating Python Module using pyodbc

want to make small python module that can get data from a database. I have dowload pydbc and it worked fine like this:
import pyodbc
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=MyDatabase;DATABASE=TestDB;UID='';PWD=''')
cursor = cnxn.cursor()
cursor.execute("select MeasurementValue from TAG_DATA where ItemID=10")
row = cursor.fetchone()
Now i want to put this in a module so that i can import it and i dont need to write the code evrytime or locate the file. So i tried to create this like this
import pyodbc
def testDB():
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=MyDatabase;DATABASE=TestDB;UID='';PWD=''')
cursor = cnxn.cursor()
cursor.execute("select MeasurementValue from TAG_DATA where ItemID=10")
row = cursor.fetchone()
return row
I saved it in: File "C:\Python27\lib\site-packages\testDB.py" and i tried to import it, but i got this error: SyntaxError: 'return' outside function
Im quite new to python, any ideas how i can put this as a module and be able to use import everytime i want to run that code?

As one of the comments says, your indentation is messed up. White space (indentation) is critical in Python. Try it like this:
import pyodbc
def testDB():
cnxn = pyodbc.connect("DRIVER={SQL Server};SERVER=MyDatabase;DATABASE=TestDB;UID='';PWD=''")
cursor = cnxn.cursor()
cursor.execute("select MeasurementValue from TAG_DATA where ItemID=10")
row = cursor.fetchone()
return row
Also, you have to use double quotes for your connection string, since you are using single quotes in the string itself. I've changed them above in to reflect that.
good luck,
Mike