I am using FastApi and I would like to know if I am using the dependencies correctly.
First, I have a function that yields the database session.
class ContextManager:
def __init__(self):
self.db = DBSession()
def __enter__(self):
return self.db
def __exit__(self):
self.db.close()
def get_db():
with ContextManager() as db:
yield db
I would like to use that function in another function:
def validate(db=Depends(get_db)):
is_valid = verify(db)
if not is is_valid:
raise HTTPException(status_code=400)
yield db
Finally, I would like to use the last functions as a dependency on the routes:
#router.get('/')
def get_data(db=Depends(validate)):
data = db.query(...)
return data
I am using this code and it seems to work, but I would like to know if it is the most appropiate way to use dependencies. Especially, I am not sure if I have to use 'yield db' inside the function validate or it would be better to use return. I would appreciate your help. Thanks a lot
Related
I have a few files in my code that speak to the database
This might look something like this:
def addUser():
# some code
def verifyUser():
# some code
def addStuffToDB():
# some code
In all of the above I need to use a variable - let's call it db - that holds a reference to the database (as opposed to redefining it in every function)
How would I do this? How can I have functions in one or more files that all make use of a variable (in this case db)
Thanks
If you have all this functions inside the same file, it is enough to just define variable db outside any function (this will make it global). Now all functions will be able to see db variable. But if you change db inside a function it will not change outside the function.
If you have this variable in another file you can simple import it like
from file_name import db
As #ddejohn said, you should wrap your functions in a class, so the variable self.db would have a class scope.
class DB():
def __init__(self) -> None:
self.db = "DB_connection or something..."
def addUser(self):
#Some code, acess db variable with self.db
def verifyUser(self):
#Some code, acess db variable with self.db
def addStuffToDB(self):
#Some code, acess db variable with self.db
MyDB = DB()
MyDB.addUser()
Thanks for asking the question.
You need to pass db as argument while calling the funcs like the following
db = "some referenec"
def addUser(database):
## now you can use db
# some code
def verifyUser(database):
# some code
## now you can use db
def addStuffToDB(database):
# some code
## now you can use db
## while calling each func pass db as argument like this
addUser(db)
verifyUser(db)
addStuffToDB(db)
add a db paramenter to yout funcs:
controller.py:
def addUser(db): # some code
obj.add(db)
def verifyUser(db): # some code
obj.verify(db)
def addStuffToDB(db): # some code
obj.add_stuff(db)
Then, you can use as follows:
view.py
import db
from controller import addUser
addUser(db)
I have a bit of background in Java and used static variables and methods a lot. I am new to python and learning about using #classmethod as a static method.
Here is code,
import redis
class GetClients:
r = None
#classmethod
def connect_r(cls, host, port, db):
r = redis.StrictRedis(host=host, port=port, db=db)
#classmethod
def get_clients(cls, clients_key):
return r.smembers(clients_key)
My code first create a variable r and then inside a classmethod it assigns it to a redis connection, r = redis.StrictRedis(host=host, port=port, db=db)
In the next method get_clients, I am using the value of r to invoke a function on it., but I keep getting the following error
NameError: name 'r' is not defined
here is how i am using the class,
clients = GetClients()
clients.connect_r("localhost", 6379, 0)
allc = clients.get_clients("clients")
can someone please explain why I cannot access r in the 2nd method?
The problem is r in both class methods, you should replace r with cls.r.
like this:
import redis
class GetClients:
r = None
#classmethod
def connect_r(cls, host, port, db):
cls.r = redis.StrictRedis(host=host, port=port, db=db)
#classmethod
def get_clients(cls, clients_key):
return cls.r.smembers(clients_key)
But I think that your way of implementing these methods are a little bit wrong, you should not use class methods for this demands.
can someone please explain why I cannot access` in the 2nd method?
I should say that you can not even access r in connect_r too. that r is another variable with a different scop and if you want to see what will happen just change r to cls.r in get_clients, you will see that this method will returns None and it seems that connect_r method is not setting the actual r, for accessing those variables in class methods you should use cls.
also see the difference between cls and self here.
I'm trying to make a simple test in python, but I'm not able to figure it out how to accomplish the mocking process.
This is the class and def code:
class FileRemoveOp(...)
#apply_defaults
def __init__(
self,
source_conn_keys,
source_conn_id='conn_default',
*args, **kwargs):
super(v4FileRemoveOperator, self).__init__(*args, **kwargs)
self.source_conn_keys = source_conn_keys
self.source_conn_id = source_conn_id
def execute (self, context)
source_conn = Connection(conn_id)
try:
for source_conn_key in self.source_keys:
if not source_conn.check_for_key(source_conn_key):
logging.info("The source key does not exist")
source_conn.remove_file(source_conn_key,'')
finally:
logging.info("Remove operation successful.")
And this is my test for the execute function:
#mock.patch('main.Connection')
def test_remove_execute(self,MockConn):
mock_coon = MockConn.return_value
mock_coon.value = #I'm not sure what to put here#
remove_operator = FileRemoveOp(...)
remove_operator.execute(self)
Since the execute method try to make a connection, I need to mock that, I don't want to make a real connection, just return something mock. How can I make that? I'm used to do testing in Java but I never did on python..
First it is very important to understand that you always need to Mock where it the thing you are trying to mock out is used as stated in the unittest.mock documentation.
The basic principle is that you patch where an object is looked up,
which is not necessarily the same place as where it is defined.
Next what you would need to do is to return a MagicMock instance as return_value of the patched object. So to summarize this you would need to use the following sequence.
Patch Object
prepare MagicMock to be used
return the MagicMock we've just created as return_value
Here a quick example of a project.
connection.py (Class we would like to Mock)
class Connection(object):
def execute(self):
return "Connection to server made"
file.py (Where the Class is used)
from project.connection import Connection
class FileRemoveOp(object):
def __init__(self, foo):
self.foo = foo
def execute(self):
conn = Connection()
result = conn.execute()
return result
tests/test_file.py
import unittest
from unittest.mock import patch, MagicMock
from project.file import FileRemoveOp
class TestFileRemoveOp(unittest.TestCase):
def setUp(self):
self.fileremoveop = FileRemoveOp('foobar')
#patch('project.file.Connection')
def test_execute(self, connection_mock):
# Create a new MagickMock instance which will be the
# `return_value` of our patched object
connection_instance = MagicMock()
connection_instance.execute.return_value = "testing"
# Return the above created `connection_instance`
connection_mock.return_value = connection_instance
result = self.fileremoveop.execute()
expected = "testing"
self.assertEqual(result, expected)
def test_not_mocked(self):
# No mocking involved will execute the `Connection.execute` method
result = self.fileremoveop.execute()
expected = "Connection to server made"
self.assertEqual(result, expected)
I found that this simple solution works in python3: you can substitute a whole class before it is being imported for the first time. Say I have to mock class 'Manager' from real.manager
class MockManager:
...
import real.manager
real.manager.Manager = MockManager
It is possible to do this substitution in init.py if there is no better place.
It may work in python2 too but I did not check.
How to remove DRY or to make it more Pythonic in a code like this? I know I can make the insertion as a function but I don't like that, many classes will have too many functions because code like these with different implementations are everywhere.
The injected db_conn is managed by Nameko's (microservice framework) DependencyProvider, the closing of the connection is done by the DepedencyProvider when a worker already finished the job.
I also made the db_conn object compatible with with statement. I close the connection in the __exit__ method.
Here is the current code.
class CommandChatBot(BaseChatBot):
def __init__(self, db_conn=None):
self.db_conn = None
if db_conn:
self.db_conn = db_conn
def add_interaction(self, question, answer, recipient):
if self.db_conn:
self.db_conn.use_or_create_db(db=recipient)
return self.db_conn.insert(
table=schemas.commandchatbot.TABLE,
document={
'question': question,
'answer': answer
}
)
else:
with db_connection_factory() as conn:
conn.use_or_create_db(db=recipient)
return conn.insert(
table=schemas.commandchatbot.TABLE,
document={
'question': question,
'answer': answer
}
)
The code above is also inefficient when a dependency is not injected, because each functions must instantiate their own db_conn object. I'm thinking about something like a with statement but for the whole class, is that possible?
Here's the closing of the connection when a dependency is a subclass of DependencyProvider, get_depedency() is called when a microservice dispatch a new worker, worker_teardown() will be called on worker deletion/all code inside a microservice is already executed.
class DbConnectionProvider(DependencyProvider):
def __init__(self):
self.db_conn = db_connection_factory()
def get_dependency(self, worker_ctx):
return self.db_conn
def worker_teardown(self, worker_ctx):
self.db_conn.close_conn()
One of the object generated by the db_conn_factory()
class RethinkDbAdapter(DatabaseAdapter):
def __init__(self, db=None, create_db_if_not_exist=True):
uri = build_nosqldatabase_uri(return_dict=True)
self.raw_connection = r.connect(
host=uri['host'],
port=uri['port']
)
...........
# __enter__ and __exit__ allows the use of 'with' statement
def __enter__(self):
return self
# Close connection when 'with' statement is out of scope
def __exit__(self, exc_type, exc_val, exc_tb):
self.raw_connection.close()
def close_conn(self):
"""
Only used in Nameko DependencyProvider.
"""
return self.raw_connection.close()
.........
It looks like you encapsulate the following code in another method that you can call from either the if or the else block:
insert(
table=schemas.commandchatbot.TABLE,
document={
'question': question,
'answer': answer
}
)
I have a class that I try to mock in tests. The class is located in server/cache.py and looks like:
class Storage(object):
def __init__(self, host, port):
# set up connection to a storage engine
def store_element(self, element, num_of_seconds):
# store something
def remove_element(self, element):
# remove something
This class is used in server/app.py similar to this one:
import cache
STORAGE = cache.Storage('host', 'port')
STORAGE.store_element(1, 5)
Now the problem arise when I try to mock it in the tests:
import unittest, mock
import server.app as application
class SomeTest(unittest.TestCase):
# part1
def setUp(self):
# part2
self.app = application.app.test_client()
This clearly does not work during the test, if I can't connect to a storage. So I have to mock it somehow by writing things in 'part1, part2'.
I tried to achieve it with
#mock.patch('server.app.cache') # part 1
mock.side_effect = ... # hoping to overwriting the init function to do nothing
But it still tries to connect to a real host. So how can I mock a full class here correctly? P.S. I reviewed many many questions which look similar to me, but in vain.