I scraped data from a website. I want to create a table in mysql to save data. I create table with this code in my database:
create table car (Model varchar(60), Mileage varchar(60), Price varchar(60))
I also have code to create this data from truecar.com. But I con not insert this data into my table with my code. Could you help me? I face with this error:"ProgrammingError:1064(42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ('$32,000')' at line 1"
import requests
from bs4 import BeautifulSoup
import mysql.connector
url='https://www.truecar.com/used-cars-for-sale/listings/'
r=requests.get(url)
soup=BeautifulSoup(r.text,'html.parser')
cards = soup.select('div.linkable.card.card-shadow.vehicle-card._1qd1muk')
data = []
for card in cards:
vehicleCardYearMakeModel = card.find("div", {"data-test" :
"vehicleCardYearMakeModel"}).text.replace('Sponsored', '')
vehicleMileage = card.find("div", {"data-test" : "vehicleMileage"}).text
vehiclePrice = card.find("div", {"data-test" : "vehicleCardPricingBlockPrice"}).text
data.append({'price':vehiclePrice,'miles':vehicleMileage,'models':vehicleCardYearMakeModel})
print(data)
cnx = mysql.connector.connect(user='root', password='',
host='127.0.0.1',
database='cars')
cursor = cnx.cursor()
for entry in data:
cursor.execute("INSERT INTO car(Model,Mileage,Price) VALUES(\'%s\',\'%s\,\'%s\')"%
(entry['models'],entry['miles'],entry['price']))
cnx.commit()
cnx.close()
You're missing the closing quote (') after the miles value:
cursor.execute("INSERT INTO car(Model,Mileage,Price) VALUES(\'%s\',\'%s\',\'%s\')"% (entry['models'],entry['miles'],entry['price']))
# Here -----------------------------------------------------------------^
Having said that, using placeholders will save you a lot of headaches:
cursor.execute("INSERT INTO car(Model,Mileage,Price) VALUES(%s,%s,%s)", (entry['models'],entry['miles'],entry['price']))
Related
I am trying to calculate the mode value of each row and store the value in the judge = judge column, however it updates only the first record and leaves the loop
ps: Analisador is my table and resultado_2 is my db
import sqlite3
import statistics
conn = sqlite3.connect("resultado_2.db")
cursor = conn.cursor()
data = cursor.execute("SELECT Bow, FastText, Glove, Wordvec, Python, juiz, id FROM Analisador")
for x in data:
list = [x[0],x[1],x[2],x[3],x[4],x[5],x[6]]
mode = statistics.mode(list)
try:
cursor.execute(f"UPDATE Analisador SET juiz={mode} where id={row[6]}") #row[6] == id
conn.commit()
except:
print("Error")
conn.close()
You have to fetch your records after SQL is executed:
cursor.execute("SELECT Bow, FastText, Glove, Wordvec, Python, juiz, id FROM Analisador")
data = cursor.fetchall()
That type of SQL query is different from UPDATE (that you're using in your code too) which doesn't need additional step after SQL is executed.
I want to update a column in table in MySQL with some information that I retrieved from a website.The name of this column is "names" and the name of the table is "HuntsPointYelp"
I am getting ProgrammingError for my UPDATE query, which I think it is a syntax error.
Thanks!
import pymysql
conn = pymysql.connect(host='127.0.0.1',
unix_socket='/tmp/mysql.sock',user='root',
passwd=' ', db='mysql', charset='utf8')
cur = conn.cursor()
cur.execute('USE HuntsPointsBusinesses')
def storeNames (names):
cur.execute('UPDATE HuntsPointYelp SET = ("%s")', names)
cur.connection.commit()
def getNames (bs):
#names:
restGrid = bs.find_all ("ul", {"class": "lemon--ul__373c0__1_cxs
undefined list__373c0__2G8oH"})
#namesList = []
time.sleep(2)
for i in restGrid:
h3 = i.find_all ("h3")
for h in h3:
target = h.find_all ("a")
for t in target:
if "name" in t.attrs:
if t.attrs is not None:
names = t["name"]
storeNames (names)
driver.get ("https://www.yelp.com/search?
cflt=restaurants&find_loc=Hunts+Point%2C+Bronx%2C+NY+10474")
pageSource = driver.page_source
bs = BeautifulSoup (pageSource, "html.parser")
names = getNames(bs)
ProgrammingError: (1064, 'You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version for
the right syntax to use near \'= ("\'Roaming Woodfired Pizza\'")\'
at line 1')
You have two problems.
First, you're missing the name of the column to set. Second, you shouldn't put the placeholder in quotes; cur.execute() does the necessary quoting for you, so you end up with literal quotes in the value.
cur.execute('UPDATE HuntsPointYelp SET names = %s', names)
I have two data sets in my JSON API. I am unable to insert both into SQL Server. The Iteration using for loop doesnt seem to pick up the second data. Can someone please help me understand how to fix this. this is new for me, so am not able to find out whats wrong since the coding is bit different from SQL
import urllib, json
import pyodbc
#read data from API
url = "http://nagiosdatagateway.vestas.net/esq/ITE1452552/logstash- 2018.12.16/2/desc"
response = urllib.urlopen(url)
data = json.loads(response.read())
#define db connection
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=DKCDCVDCP42\DPA;"
"Database=VPDC;"
"Trusted_Connection=yes;")
cursor = cnxn.cursor()
i = 0
j = len(data)
print j
for i in range(i,j-1):
# print data[1]["_source"]["utc_timestamp"]
print i
print data[i]["_source"]["nagios_comment"]
print data[i]["_source"]["nagios_author"]
cursor.execute("insert into vpdc.pa.Pythontable(nagios_comment,nagios_author) values (?,?)",(data[i]
["_source"]["nagios_comment"],data[i]["_source"]["nagios_author"] ))
i += 1
print i
cnxn.commit()
both these two sets of values should be in the SQL table for columns
Nagios_comment & Nagios_author
307262828 Alex Christopher Ramos
307160348 Alex Christopher Ramos
the issue had been resolved by correctly indenting the cursor.execute statement in the script as below. In my original script, there was no indentation done for this line.so it was called outside the loop
import urllib, json
import pyodbc
#read data from API
url = "http://nagiosdatagateway.vestas.net/esq/ITE1452552/logstash-2018.12.16/2/desc"
response = urllib.urlopen(url)
data = json.loads(response.read())
#define db connection
cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
"Server=DKCDCVDCP42\DPA;"
"Database=VPDC;"
"Trusted_Connection=yes;")
cursor = cnxn.cursor()
i = 0
j = len(data)
print j
for i in range(0,2):
#print data[1]["_source"]["utc_timestamp"]
print data[i]["_source"]["nagios_comment"]
print data[i]["_source"]["nagios_author"]
cursor.execute("insert into vpdc.pa.Pythontable(nagios_comment,nagios_author)
values (?,?)",(data[i]["_source"]["nagios_comment"],data[i]["_source"]
["nagios_author"] ))
cnxn.commit()
I want to delete some rows from almost 500 tables in mysql database using python.
I know that the query should be something like this:
DELETE FROM (table name)
[WHERE conditions] [ORDER BY ...] [LIMIT rows]
but I am not sure what should I use for the table name when I looping over the tables!
here is my loop
from mysql.connector import (connection)
# call databasecnx = connection.MySQLConnection(user='user', password='PW',host='127.0.0.1', database= 'database')
cursor = cnx.cursor()
cursor.execute("SHOW TABLES LIKE 'options_20%'")
table_names = [tables for (tables, ) in cursor]
for t in table_names:
cursor.execute("Delete table-refs WHERE Expiration = Datadate AND UnderlyingSymbol = 'SPY'")
cnx.commit()
I got an error:
You have an error in your SQL syntax; check the manual that...etc
Test with this.... You did'nt put from on your query statement.
for table in table_names:
cursor.execute("DELETE FROM"+ table +"WHERE Expiration = Datadate AND UnderlyingSymbol = 'SPY'")
Using python and MySQLdb, how can I check if there are any records in a mysql table (innodb)?
Just select a single row. If you get nothing back, it's empty! (Example from the MySQLdb site)
import MySQLdb
db = MySQLdb.connect(passwd="moonpie", db="thangs")
results = db.query("""SELECT * from mytable limit 1""")
if not results:
print "This table is empty!"
Something like
import MySQLdb
db = MySQLdb.connect("host", "user", "password", "dbname")
cursor = db.cursor()
sql = """SELECT count(*) as tot FROM simpletable"""
cursor.execute(sql)
data = cursor.fetchone()
db.close()
print data
will print the number or records in the simpletable table.
You can then test if to see if it is bigger than zero.