We Keep Coding

python is saying "self" is the "missing required positional argument" when called with asyncio?

I'll try show all the functions that are being used here:
def main():
peers = [*list with many peerIDs*]
asyncio.run(async_peerImport.importPeers(peers))
async def importPeers(peers):
dividedPeers = divideList(peers, 250) # this is just a function I made to split lists into lists of smaller lists
for peers in dividedPeers:
await asyncio.gather(*[importPeer(peerID) for peerID in peers])
async def importPeer(peerID):
fetchPeerDataTask = asyncio.create_task(async_requests.fetchPeerData(peerID))
getTorStatusTask = asyncio.create_task(async_requests.fetchPeerData(peerID))
peerData = await fetchPeerDataTask
torStatus = await getTorStatusTask
if peerData is not None and torStatus is not None:
db.upsertPeer(peerID, torStatus, peerData) # the function in question
print("peer imported:", peerID)
class db:
def upsertPeer(self, peerID, torStatus, peerData):
try:
sql = "INSERT INTO peers (peerID, torStatus, peerData) VALUES (%s, %s, %s);"
self.cursor.execute(sql, [peerID, torStatus, json.dumps(peerData)])
print("peer inserted")
except psycopg2.errors.UniqueViolation:
sql = "UPDATE peers SET torStatus = %s, peerData = %s WHERE peerID = %s;"
self.cursor.execute(sql, [torStatus, json.dumps(peerData), peerID])
print("peer updated")
finally:
self.connection.commit()
Hopefully you can see what should be happening? If it is more difficult than I thought then tell me and I'll add a bunch of comments.
I don't see a reason for this not to work, but when I run it, I get this error:
File "c:\path\async_peerImport.py", line 25, in importPeer
db.upsertPeer(peerID, torStatus, peerData)
TypeError: upsertPeer() missing 1 required positional argument: 'peerData'
I tried adding some random object into the first position and moved all the other arguments up (total of 4), and it said that object has no attribute 'connection'. I have also run this upsertPeer() on its own and it does work with those 3 arguments. I am completly lost here. Am I doing something wrong?
Again, if anything I have tried to explain doesn't make sense just tell me and I'll try better.
Thanks.

This bit:
class db:
def upsertPeer(self, peerID, torStatus, peerData):
# code here
defines an instance method of a class. In order to access it, you need to access it through an instance of the db class, which might look like
my_db = db()
my_db.upsertPeer(peerId, torStatus, peerData)
In which case the value of self is my_db and it is implicitly passed without any intervention from you.
If you are attempting to make a class method, which can be used in the way your code does, try this:
class db:
#classmethod
def upsertPeer(cls, peerID, torStatus, peerData):
# code here
Note that there's still an implicit first argument, but it's the db class object, not an instance of it.

SQLAlchemy error with class but not with function

The following works when run in the context of a flask app executed in Jenkins:
A function is imported from one file to another:
def return_countries(db):
db.session.expire_on_commit = False
q = db.session.query(Country.countrycode).distinct().all()
q = [t[0] for t in q]
return q
Inside the file where it is imported I call it like this:
print(return_countries(db))
When this job is run in Jenkins, it works, no problems.
The thing is, I want one database call and then to use the returned list of country codes multiple times, not unnecessary keep querying the same thing. To keep it neat have tried to put it in a class. Yes, it could be done without the class but it may be needed in several parts of the code base, maybe with additional methods so would be cleaner if the class would work.
So I commented out that function definition and inserted the following class into the one file:
class checkCountries:
def __init__(self, db):
db.session.expire_on_commit = False
__q = db.session.query(Country.countrycode).distinct().all()
__q = [t[0] for t in __q]
print(__q)
def check(self, __q, countrycode):
if countrycode in __q:
return True
else:
return False
...updated the import statement in the other file and instantiated like this directly under the import statements:
cc = checkCountries(db)
and replaced the above print statement with this one:
print('check for DE: ', cc.check('DE'))
print('check for ES: ', cc.check('ES'))
print('check for GB: ', cc.check('GB'))
But it generated an error in the Jenkins log. The traceback is very long but below are what are probably the relevant excerpts.
Any ideas? Have a feeling something is wrong with what I have done with __q
/python3.6/site-packages/sqlalchemy/util/_collections.py", line 988, in __call__
15:26:01 return self.registry[key]
15:26:01 KeyError: 123456789123456
15:26:01
15:26:01 During handling of the above exception, another exception occurred:
15:26:01
15:26:01 Traceback (most recent call last):
/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 922, in get_app
15:26:01 raise RuntimeError('application not registered on db '
15:26:01 RuntimeError: application not registered on db instance and no application bound to current context
15:26:02 Build step 'Execute shell' marked build as failure
15:26:02 Finished: FAILURE

I got it to work, so here is the class I am using which now works:
class checkCountries:
def __init__(self, db, app):
db.init_app(app)
with app.app_context():
db.session.expire_on_commit = False
self.__q = db.session.query(Country.countrycode).distinct().all()
self.__q = [t[0] for t in self.__q]
print(self.__q)
def check(self, countrycode):
if countrycode in self.__q:
return True
else:
return False
Then, the line which instantiates it now takes 2 arguments:
cc = checkCountries(db, app)
The rest is the same as in the question above.
This answer from #mark_hildreth helped, thanks!
Yes, there were more than one thing wrong with it. The class wasn't quite right but debugged that in a simplified app and got it working but this error message specific to sqlalchemy was still there.
Maybe it had something to do with the variables in scope in the class vs in the function that stopped it binding to the Flask application context

Python - cannot access class property

I'm writing a class to access a mySQL db from python.
Here's the code:
#!/usr/bin/python
import MySQLdb
import configparser
class db():
def __init__(self):
try:
self.handler = MySQLdb.connect(host="localhost", user=user, passwd=passwd, db=dbname)
self.cur = self.handler.cursor()
except:
print "Couldn't connect to db"
def query(self, query):
self.lastQuery = query
self.res = self.cur.execute(query)
def close(self):
self.handler.close()
When I'm trying to call the class, it give me the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "db.class.py", line 6, in <module>
class db():
File "db.class.py", line 25, in db
self.res = self.cur.execute(query)
NameError: name 'self' is not defined
I've been searching and commonly there's people that forget to define the method with 'self' as an argument. But I'm including it.
Could anyone help me?
Thanks.
UPDATE:
I checked whitespaces and tabs and that is not the problem.

Your code has mixed tabs and spaces, causing Python to get confused about what statements are at what indentation level. The assignment to self.res has ended up at class level, outside of the method it was intended to be a part of.
Stop mixing tabs and spaces. Turn on "show whitespace" in your editor to make the problem more visible, and consider running Python with the -tt flag to give an error on ambiguous mixing of tabs and spaces.

argument count syntax error in Python

I'm getting a syntax error trying to make a Federated Table Builder.
Here's the offended interpreter:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "federatedTableBuilder.py", line 7, in <module>
local_public_files.generate()
File "localViewDefinition.py", line 22, in generate
self.generate_for_host(conn)
File "localViewDefinition.py", line 17, in generate_for_host
self.conn.doQuery(rsaconn,self.query)
TypeError: doQuery() takes exactly 2 arguments (3 given)
and the offending code:
import mysql as sql
from connector import Connector
import io
import traceback
class LocalViewDefinition:
...insert variables...
def doQuery(connection, query):
try:
cursor = MySQLdb.cursors.Cursor(connection)
cursor.execute(query)
except: #figure out how to handle generic and sql errors separately
traceback.print_exc()
Do you have any idea about the cause?

For class methods, Python takes an additional first argument to refer to the instance of the class. The convention is to use the word self:
def doQuery(self, connection, query):
try:
cursor = MySQLdb.cursors.Cursor(connection)
cursor.execute(query)
except: #figure out how to handle generic and sql errors separately
traceback.print_exc()
This requirement to refer to the instance of the class is because "explicit is better than implicit" in Python (cf. import this).

The other answers have already covered the fact that your instance method needs to have self1 as the first argument. However, it is worth noting that an instance method which doesn't use self maybe shouldn't be an instance method at all ...
class Example(object):
def instance_method(self):
print "I need self: %s" % self
#staticmethod
def static_method():
print "I don't need self."
#classmethod
def class_method(cls):
print "I use the class, not the instance: %s" % cls
Example.static_method() # I don't need self.
Example.class_method() # I use the class, not the instance: ...
e = Example()
e.instance_method() # I need self: ...
# can call staticmethods and classmethods from an instance as well:
e.static_method() # I don't need self.
As a final word, staticmethods in particular aren't generally super useful. Most of the time, a module level function without a class will due just fine.
1The name "self" is just convention -- You could use any name you like, but I wouldn't suggest it

You need to add another argument - name it "self" - in the method definition.
def doQuery(self, connection, query)

add self to def doQuery(connection, query)
def doQuery(self, connection, query):
that is the python class object reference.

Python descriptors to chain methods

I'm trying to figure out how to chain class methods to improve a utility class I've been writing - for reasons I'd prefer not to get into :)
Now suppose I wanted to chain a chain class methods on a class instance (in this case for setting the cursor) e.g.:
# initialize the class instance
db = CRUD(table='users', public_fields=['name', 'username', 'email'])
#the desired interface class_instance.cursor(<cursor>).method(...)
with sql.read_pool.cursor() as c:
db.cursor(c).get(target='username', where="omarlittle")
The part that's confusing is I would prefer the cursor not to persist as a class attribute after .get(...) has been called and has returned, I'd like to require that .cursor(cursor) must be first called.
class CRUD(object):
def __init__(self, table, public_fields):
self.table = table
self.public_fields = public_fields
def fields(self):
return ', '.join([f for f in self.public_fields])
def get(self, target, where):
#this is strictly for illustration purposes, I realize all
#the vulnerabilities this leaves me exposed to.
query = "SELECT {fields} FROM {table} WHERE {target} = {where}"
query.format(fields=self.fields, table=self.table, target=target,
where=where)
self.cursor.execute(query)
def cursor(self, cursor):
pass # this is where I get lost.

If I understand what you're asking, what you want is for the cursor method to return some object with a get method that works as desired. There's no reason the object it returns has to be self; it can instead return an instance of some cursor type.
That instance could have a back-reference to self, or it could get its own copy of whatever internals are needed to be a cursor, or it could be a wrapper around an underlying object from your low-level database library that knows how to be a cursor.
If you look at the DB API 2.0 spec, or implementations of it like the stdlib's sqlite3, that's exactly how they do it: A Database or Connection object (the thing you get from the top-level connect function) has a cursor method that returns a Cursor object, and that Cursor object has an execute method.
So:
class CRUDCursor(object):
def __init__(self, c, crud):
self.crud = crud
self.cursor = however_you_get_an_actual_sql_cursor(c)
def get(self, target, where):
#this is strictly for illustration purposes, I realize all
#the vulnerabilities this leaves me exposed to.
query = "SELECT {fields} FROM {table} WHERE {target} = {where}"
query.format(fields=self.crud.fields, table=self.crud.table,
target=target, where=where)
self.cursor.execute(query)
# you may want this to return something as well?
class CRUD(object):
def __init__(self, table, public_fields):
self.table = table
self.public_fields = public_fields
def fields(self):
return ', '.join([f for f in self.public_fields])
# no get method
def cursor(self, cursor):
return CRUDCursor(self, cursor)
However, there still seems to be a major problem with your example. Normally, after you execute a SELECT statement on a cursor, you want to fetch the rows from that cursor. You're not keeping the cursor object around in your "user" code, and you explicitly don't want the CRUD object to keep its cursor around, so… how do you expect to do that? Maybe get is supposed to return self.cursor.fetch_all() at the end or something?