We Keep Coding

Adding multiple values to MySQL inside a loop

I scrape a list of elements from the site and extract two values from it. The text and the href. I cannot figure out how to add these to the mysql DB in side a loop. I tried execute inside the loop, without using the list they are appeneded to, that failed. So, I tried executemany but I think my format might be incorrect. I saw examples where a list of tuples is fed to executemany, I don't know how to do that in this case.
for name in name_eles:
names_list.append(name.text)
n_li = name.get_attribute('href')
names_links.append(n_li)
sql = "INSERT INTO profiles(company, coprofile) VALUES (%s,%s)"
val = [(name for name in name_list),(n_li for n_li in names_links)]
cursor.executemany(sql,val)
This is the error I get.
MySQLdb._exceptions.ProgrammingError: not all arguments converted during bytes formatting

You have to pass a list of tuples in the executemany arguments.
You can simply try this:
val = list(zip(name_list, names_links))
You can read more about it here

Getting error when running a sql select statement in python

I am new to this and trying to learn python. I wrote a select statement in python where I used a parameter
Select """cln.customer_uid = """[(num_cuid_number)])
TypeError: string indices must be integers

Agree with the others, this doesn't look really like Python by itself.
I will see even without seeing the rest of that code I'll guess the [(num_cuid_number)] value(s) being returned is a string, so you'll want to convert it to integer for the select statement to process.

num_cuid_number is most likely a string in your code; the string indices are the ones in the square brackets. So please first check your data variable to see what you received there. Also, I think that num_cuid_number is a string, while it should be in an integer value.
Let me give you an example for the python code to execute: (Just for the reference: I have used SQLAlchemy with flask)
#app.route('/get_data/')
def get_data():
base_sql="""
SELECT cln.customer_uid='%s' from cln
""" % (num_cuid_number)
data = db.session.execute(base_sql).fetchall()

Pretty sure you are trying to create a select statement with a "where" clause here. There are many ways to do this, for example using raw sql, the query should look similar to this:
query = "SELECT * FROM cln WHERE customer_uid = %s"
parameters = (num_cuid_number,)
separating the parameters from the query is secure. You can then take these 2 variables and execute them with your db engine like
results = db.execute(query, parameters)
This will work, however, especially in Python, it is more common to use a package like SQLAlchemy to make queries more "flexible" (in other words, without manually constructing an actual string as a query string). You can do the same thing using SQLAlchemy core functionality
query = cln.select()
query = query.where(cln.customer_uid == num_cuid_number)
results = db.execute(query)
Note: I simplified "db" in both examples, you'd actually use a cursor, session, engine or similar to execute your queries, but that wasn't your question.

python variable invalid syntax issue

I'm having an issue getting what I think is a simple script working and I think it's just me not knowing how to get a variable working.
i'm working on a salesforce script that grabs a list of objects, then looks through the list of objects to get all fields on the table.
this query in python works perfectly for giving me my list of objects that I've pulled into the DB
query = ("SELECT obj_name FROM syncsfobjects")
cursor.execute(query)
i then loop through these records
for x in cursor:
now this is where my issue lies I want to use the obj_name that comes in from my query within the next statement
for xy in sf.%obj_name%.describe()["field"]:
what i'm having massive issues with is getting the obj name into this simple-salesforce query.
if I create a string it works fine
objectname = str(x)
sfquery = 'sf. %s .describe()["fields"]' % objectname
but then when I use the sfquery for my next loop all the loop does it run through each letter within the string instead of run the sf connection command.
any I just missing something simple?
cheers
dan

for xy in sf.%obj_name%.describe()["field"]:
Python doesn't let you do "percent substitution" outside of strings, but you can still access attributes if all you have are their names:
for xy in getattr(sf, objectname).describe()["field"]:

SQLAlchemy func issue with date and .isocalendar()

I have a table that stores tasks submitted by users, with timestamps. I would like to write a query that returns certain rows based on when they were submitted (was it this day/week/month..).
To check if it was submitted on this week, I wanted to use date.isocalendar()[1] function. The problem is, that my timestamps are datetimes, so I would need to transform those to dates.
Using func:
filter(func.date(Task.timestamp) == datetime.date(datetime.utcnow()))
works properly.
But I need the date object's isocalendar() method, so I try
filter(func.date(Task.timestamp).isocalendar()[1]==datetime.date(datetime.utcnow()).isocalendar()[1])
and it's no good, I get AttributeError: Neither 'Function' object nor 'Comparator' object has an attribute 'isocalendar'
If I make a simple query and try datetime.date(task.timestamp).isocalendar()[1] it works properly.
How do I get it to work in the query's filter?

Rule of thumb when understanding and debugging sqlalchemy queries is to always think – "How will it look in SQL?"
isocalendar() is a python function, and sqlalchemy query filters get compiled to SQL. Moreover, isocalendar() returns a tuple – and while rendering tuple comparison as SQL is probably possible, it's more trouble then it's worth. You should compare scalars and find sql date functions that suit you.
It seems you're looking to compare week number, so something like this should do the trick:
filter(func.week(Task.timestamp)==datetime.utcnow().isocalendar()[1])

Can you try sqlalchemy.extract(func.date('year', Task.timestamp)) == ... ?

You cannot mix pure python functions with those which are executed on the SQL backend. From your code it looks like you are trying to filter on the iso week. One way to do it would be to load everything from the database into memory and perform the filtering there. Obviously, it most cases it is far from efficient.
An alternative would be to use respective SQL functions, which sqlalchemy will call for you. On MySQL it looks like the function you need is weekofyear, so your filter might look similar to below:
_utcnow = datetime.utcnow().date()
_isoweek = _utcnow.isocalendar()[1]
q = db.session.query(...)
# ...
q = q.filter(db.func.weekofyear(Task.timestamp) == _isoweek)

PYMONGO - How do I use the query $in operator with MongoIDs?

So I am trying to use the $in operator in Pymongo where I want to search with a bunch of MongoIDs.
First I have this query to find an array of MongoIDs:
findUsers = db.users.find_one({'_id':user_id},{'_id':0, 'f':1})
If I print the findUsers['f'] it looks like this:
[ObjectId('53b2dc0b24c4310292e6def5'), ObjectId('53b6dbb654a7820416a12767')]
These object IDs are user ids and what I want to do is to find all users that are in the users collection with this array of ObjectID. So my thought was this:
foundUsers = db.users.find({'_id':{'$in':findUsers['f']}})
However when I print the foundUsers the outcome is this:
<pymongo.cursor.Cursor object at 0x10d972c50>
which is not what I normally get when I print a query out :(
What am I doing wrong here?
Many thanks.
Also just for you reference, I have queried in the mongo shell and it works as expected:
db.users.find({_id: {$in:[ObjectId('53b2dc0b24c4310292e6def5'), ObjectId('53b6dbb654a7820416a12767')]}})

You are encountering the difference between findOne() and find() in MongoDB. findOne returns a single document. find() returns a mongoDB cursor. Normally you have to iterate over the cursor to show the results. The reason your code works in the mongo shell is that the mongo shell treats cursors differently if they return 20 documents or less - it handles iterating over the cursor for you:
Cursors
In the mongo shell, the primary method for the read operation is the
db.collection.find() method. This method queries a collection and
returns a cursor to the returning documents.
To access the documents, you need to iterate the cursor. However, in
the mongo shell, if the returned cursor is not assigned to a variable
using the var keyword, then the cursor is automatically iterated up to
20 times [1] to print up to the first 20 documents in the results.
http://docs.mongodb.org/manual/core/cursors/
The pymongo manual page on iterating over cursors would probably be a good place to start:
http://api.mongodb.org/python/current/api/pymongo/cursor.html
but here's a piece of code that should illustrate the basics for you. After your call to find() run this:
for doc in findUsers:
print(doc)