We Keep Coding

Pytest using development sqlite db during testing - NoneType object has no attribute drivername

I was trying to access a sqlite db I use on development server, but I got following error:
'NoneType' object has no attribute 'drivername'. It happend both during calling business logic from a view or during calling db.create_all() inside test client definition (currently removed from code).
Test file code:
import os
import json
import pytest
from flask import Flask
import run
from src.utils.config import TestingConfig
from src.models.user_model import User
class TestingConfig(Config):
TESTING = True
SQLALCHEMY_DATABASE_URI = os.getenv('SQLALCHEMY_DATABASE_URI')
SQLALCHEMY_ECHO = False
SQLALCHEMY_TRACK_MODIFICATIONS = False
#pytest.fixture
def client():
app = run.create_app(TestingConfig)
context = app.app_context()
context.push()
with context:
with app.test_client() as client:
yield client
context.pop()
def test_user_register(client):
user_data = {
'login': 'login',
'password': 'password',
'email': 'email#example.com',
'first_name': 'first_name',
'last_name': 'last_name',
'index': '000000'
}
rv = client.post('/api/v1/users/register', data=json.dumps(user_data), content_type='application/json')
user_data['user_role'] = 2
assert '' == rv.data
Create app code:
from flask import Flask, current_app
from flask_cors import CORS
from flask_swagger_ui import get_swaggerui_blueprint
from src.utils.config import Config, DevelopmentConfig, ProductionConfig
from src.utils.extensions import db, ma, migrate, cors
from src.views import user_view, task_view, task_submission_view
from src.exceptions import exceptions_view
def create_app(config):
app = Flask(__name__)
app.config.from_object(config)
app.register_blueprint(exceptions_view.handler)
app.register_blueprint(user_view.blueprint, url_prefix='/api/v1/users')
app.register_blueprint(task_view.blueprint, url_prefix='/api/v1/tasks')
app.register_blueprint(task_submission_view.blueprint, url_prefix='/api/v1/taskSubmissions')
swagger_blueprint = get_swaggerui_blueprint('/swagger', '/static/swagger.json')
app.register_blueprint(swagger_blueprint)
db.init_app(app)
ma.init_app(app)
migrate.init_app(app, db)
cors.init_app(app)
return app
.env fragment :
SQLALCHEMY_DATABASE_URI = "sqlite:///sqlite.db"

Answer:
When I created the app from an external script run.create_app() even though blueprints were registered correctly the configuration was wiped out after the script execution.
After some trial-and-error runs I found out that configurating app by function app.config.from_object(), more precisely during os.getenv(), was an issue - even after moving it to the test file it wasn't able to configurate the app.
Solution was to set the configuration manually (inside or outside the test file):
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///sqlite.db'

How to fix 'flask db.create_all() error'?

I'm have flask init with create app function. I create test file for my unitests. In unitest class add setUp function were i create new flask app and add context, push context to it. Next i create test db with create_all() function and, where i start test file, i have next error:
in _execute_for_all_tables op(bind=self.get_engine(app, bind), **extra)
TypeError: create_all() got an unexpected keyword argument 'forms'
I haven't 'forms' files or variable on models or anywere.
Before this error place in flask_sqlalchemy/init.py MetaData(bind=None), may be it some help.
from flask import Flask
from config import Config
from blocker_application.database import db
from flask_migrate import Migrate
from flask_login import LoginManager
from flask_moment import Moment
migrate = Migrate()
login = LoginManager()
login.login_view = 'user.login'
moment = Moment()
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(config_class)
db.init_app(app)
migrate.init_app(app, db)
login.init_app(app)
moment.init_app(app)
from blocker_application.main.routes import bp_main
from blocker_application.errors.handlers import bp_error
from blocker_application.reports.routes import bp_reports
from blocker_application.user.routes import bp_user
from blocker_application.applications.routes import bp_applications
app.register_blueprint(bp_main)
app.register_blueprint(bp_error)
app.register_blueprint(bp_reports, url_prefix='/reports')
app.register_blueprint(bp_user, url_prefix='/user')
app.register_blueprint(bp_applications, url_prefix='/applications')
return app
from blocker_application import models
________________________________________________________________________
/config.py
import os
basedir = os.path.abspath(os.path.dirname(__file__))
class Config:
SECRET_KEY = os.environ.get('SECRET_KEY') or 'not realy finish secret key configuration'
SQLALCHEMY_DATABASE_URI = 'mysql://some_database'
SQLALCHEMY_TRACK_MODIFICATIONS = False
class TestConfig(Config):
TESTING = True
SQLALCHEMY_BINDS = {'test': 'mysql://some_database_test'}
______________________________________________________________________
/tests.py
import unittest
from blocker_application import create_app, db
from blocker_application import models
from config import TestConfig
class UserModelCase(unittest.TestCase):
def setUp(self):
self.app = create_app(TestConfig)
self.app_context = self.app.app_context()
self.app_context.push()
db.create_all(bind='test')
def tearDown(self):
db.session.remove()
db.drop_all(bind='test')
self.app_context.pop()
def test_password(self):
u = models.User(username='Mark')
u.set_password('Mark')
self.assertTrue(u.check_password('Mark'))
self.assertFalse(u.check_password('Tony'))
if __name__ == '__main__':
unittest.main(verbosity=2)

I found decision. Unittest work ok, after recreate the virtual enveroment.

Flask, Blueprint Collisions and Pytest

I am using Flask Blueprints to create an application. I am testing it with pytest. Generating test_client in different states with pytest.fixture is causing a blueprint name collision.
This is my Flask __init__ code:
from flask import Flask
from flask_dropzone import Dropzone
from config import Config
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from flask_user import UserManager
from flask_googlemaps import GoogleMaps
import os
from app.settings import app, db
# get extensions
dropzone = Dropzone()
google_maps = GoogleMaps()
migrate = Migrate()
def create_app(config_class=Config):
# initialise extensions
google_maps.init_app(app)
dropzone.init_app(app)
db.init_app(app)
migrate.init_app(app, db)
# FLASK-User Stuff
from app.models import User
user_manager = UserManager(app, db, User)
from app.errors import bp as errors_bp
app.register_blueprint(errors_bp)
from app.main import bp as main_bp
app.register_blueprint(main_bp)
return app
I am then trying to use multiple pytests to test different aspects of the application. For each I am creating a pytest.fixture to generate a test_client. For example I'm creating two fixtures for testing the app with the db in different states.
For example.
#pytest.fixture(scope='module')
def client_state1():
app = create_app()
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://' + PATH_TO_DB_STATE_1
app.config['TESTING'] = True
client = app.test_client()
ctx = app.app_context()
ctx.push()
yield client
ctx.pop()
#pytest.fixture(scope='module')
def client_state2():
app = create_app()
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://' + PATH_TO_DB_STATE_2
app.config['TESTING'] = True
client = app.test_client()
ctx = app.app_context()
ctx.push()
yield client
ctx.pop()
def test_with_state1(client_state1):
"""Test should pass"""
assert 1 == 1
def test_with_state2(client_state2):
"""Test should pass"""
assert 1 == 1
When I run the tests I get the following error.
AssertionError: A blueprint's name collision occurred between <flask.blueprints.Blueprint object at 0x1a16b5d4e0> and <flask.blueprints.Blueprint object at 0x1a1633f6a0>. Both share the same name "googlemaps". Blueprints that are created on the fly need unique names.
I have tried splitting out tests into separate files and setting the scope of the fixture to function. But to no avail. In all cases the first test passes and the second causes the collision. What am I doing wrong?

Access mongo instance in Flask Unit Tests

I have a simple create_app function in app/__init__.py:
from flask import Flask
from flask_bootstrap import Bootstrap
from flask_login import LoginManager
from flask_pymongo import PyMongo
from .user_management import User
from config import app_config
login_manager = LoginManager()
mongo = PyMongo()
....
def create_app(config):
app = Flask(__name__, instance_relative_config=True, static_folder='static')
login_manager.init_app(app)
login_manager.login_message = 'You must be logged in to view this page'
login_manager.login_view = 'auth.login'
Bootstrap(app)
app.config.from_object(app_config[config])
app.config.from_pyfile('config.py')
if app.testing:
mongo.init_app(app, config_prefix='MONGO2')
else:
mongo.init_app(app)
....
return app
And my config:
class Config():
DEBUG = False
MONGO_HOST = 'localhost'
MONGO_PORT = 27017
....
class DevelopmentConfig(Config):
DEBUG = True
DEVELOPMENT = True
class TestingConfig(Config):
TESTING = True
DEBUG = False
CSRF_ENABLED = False
MONGO2_DBNAME = 'test'
....
app_config = {
'testing': TestingConfig,
'development': DevelopmentConfig,
'production': ProductionConfig
}
Throughout the app, I import the mongo instance from this file make use of it throughout the app. However, I can not find a find to set up a new mongo instance that uses the 'test' database without using the app context, as I'm doing here
My unit test file looks like this.
from app import create_app
import unittest
from app import mongo
class TestCase(unittest.TestCase):
def setUp(self):
app = create_app('testing')
self.app = app
def tearDown(self):
pass
def test_mongo(self):
with self.app.app_context():
assert mongo.db.name == 'test'
if __name__ == '__main__':
unittest.main()
This does not seem like the way to go at all. It also makes it impossible to use the app.test_client(). What is the proper way to instantiate the test database in a flask test setting?

I'm not sure how to do that via config files, but I have another approach that might interest you.
When I have been creating unit tests to my flask / mongo app, I have just simply read sys.args (in my '__init__.py') and decided from there if I want to use test DB or the actual DB, like this:
if "__init__.py" == sys.argv[0]:
db = LoginManager()
else:
db = TestDB() # if running unit tests
I don't know is this the most robust way to work with unit tests, but it is most certainly is super simple way to do it, and does the job.
If you are interested about the whole setup, check out my example app.

You will need to re-create the mongo object in each different unit test that needs to access a different database defined in your config. Here is how I did it and to get a real test I wrote a value to the database and made sure I could read it back two different ways:
import uuid
from datetime import datetime
from flask_testing import TestCase
from flask_pymongo import PyMongo
from project import app
class TestMongoDev(TestCase):
def create_app(self):
app.config.from_object('project.config.DevelopmentConfig')
return app
def test_mongo_development(self):
mongo = PyMongo(app)
testval = str(uuid.uuid4())
inserted_id = mongo.db.test.insert_one({
'testedAt': datetime.now(),
'testval': testval
}).inserted_id
self.assertTrue(mongo.db.name == 'dev')
self.assertTrue(mongo.db.test.find_one({'testval': testval})['testval'] == testval)
self.assertTrue(mongo.db.test.find_one({'_id': inserted_id})['testval'] == testval)
class TestMongoTest(TestCase):
def create_app(self):
app.config.from_object('project.config.TestingConfig')
return app
def test_mongo_testing(self):
mongo = PyMongo(app)
testval = str(uuid.uuid4())
inserted_id = mongo.db.test.insert_one({
'testedAt': datetime.now(),
'testval': testval
}).inserted_id
self.assertTrue(mongo.db.name == 'test')
self.assertTrue(mongo.db.test.find_one({'testval': testval})['testval'] == testval)
self.assertTrue(mongo.db.test.find_one({'_id': inserted_id})['testval'] == testval)
class TestMongoProd(TestCase):
def create_app(self):
app.config.from_object('project.config.ProductionConfig')
return app
def test_mongo_production(self):
mongo = PyMongo(app)
testval = str(uuid.uuid4())
inserted_id = mongo.db.test.insert_one({
'testedAt': datetime.now(),
'testval': testval
}).inserted_id
self.assertTrue(mongo.db.name == 'prod')
self.assertTrue(mongo.db.test.find_one({'testval': testval})['testval'] == testval)
self.assertTrue(mongo.db.test.find_one({'_id': inserted_id})['testval'] == testval)

Avoiding circular imports Python

I know this question have already been answered, but for my application, I can't resolve it. What I'm trying to do is to setup a database using MongoAlchemy instead of using MongoClient. I want to have diffrent scripts for every operation This is my main file, app.py:
import os
from flask import Flask, jsonify
from blueprints.db import insert_db
from blueprints.delete_blueprint import delete_bp
from blueprints.insert_blueprint import insert_bp
from blueprints.read_blueprint import read_bp
from blueprints.login_update_blueprint import log_update
import traceback
app = Flask(__name__)
app.register_blueprint(log_update)
app.register_blueprint(read_bp)
app.register_blueprint(insert_bp)
app.register_blueprint(delete_bp)
# database configuration
app.config['MONGOALCHEMY_DATABASE'] = 'rest_api'
# populate table with initial values
insert_db.populateTables()
# mail configuration
app.config['MAIL_SERVER'] = 'smtp.gmail.com'
app.config['MAIL_PORT'] = 465
app.config['MAIL_USERNAME'] = 'chis.simion12#gmail.com'
app.config['MAIL_PASSWORD'] = ''
app.config['MAIL_USE_TLS'] = False
app.config['MAIL_USE_SSL'] = True
#app.errorhandler(404)
def page_not_found(c):
return 'The required page does not exist'
#app.errorhandler(400)
def bad_req(c):
return 'Bad request. Please review the request'
#app.errorhandler(Exception)
def handle_Exception(error):
print("\n ______ ERROR ______\n")
print(traceback.format_exc())
print("_____________________\n")
response = dict()
response['message'] = "A PROBLEM HAS OCCURED, PLEASE TRY AGAIN LATER"
response['status_code'] = 500
response = jsonify(response)
response.status_code = 500
return response
if __name__ == '__main__':
app.secret_key = os.urandom(12)
app.run(debug=True, threaded=True)
Note that I'm modyfing on the old version, so the imports are still there. The problem with the circular imports appear when I try do import app from app.py into the script which initializes the database, described below:
from flask import current_app
from flask_mongoalchemy import MongoAlchemy
# from app import app
db = MongoAlchemy(current_app)
class People(db.Document):
Name = db.StringField()
Age = db.IntField()
Password = db.StringField()
Vms = db.AnythingField()
If I try to use current_app I get the error: RuntimeError: Working outside of application context. which means that the app is not currently initialized, and if I import the app from app.py the circular dependency appears. I'm sorry if I wasn't pretty clear, so feel free to ask for more details.

I like to place all extensions on a extensions.py file and create a function to initialize and configurate the app instance
extensions.py
from flask_mongoalchemy import MongoAlchemy
db = MongoAlchemy()
models.py
from extensions import db
class Person(db.Model):
pass
flaskr.py
from flask import Flask
from extensions import db
def create_app():
app = Flask(__name__)
db.init_app(app)
return app
app = create_app()
app.run()