im unable to get access to the peewee database from a python script other than the one writing the data (but weirdly in the interactive shell). I'm developing a data mining application with python,flask,and peewee. I'm going to cut it up into a byte size problem here, but it is large in scope than what im presenting. all python files are located in the same folder
the basic process is a python command line operation that goes out and grabs some information from the new york times and instagram and stores them into Peewee (sqllite) database. I then use a flask application to explore the data.
heres one of the models from database.py:
from peewee import *
class Story(Model):
title = CharField()
story = TextField()
date = DateField()
class Meta:
database = SqliteDatabase("newsalmost.db",threadlocals = True)
newsalmost looks something like this:
from database import *
class NewsAlmost(object):
def __init__(self):
self.db = SqliteDatabase("newsalmost.db",threadlocals = True)
if does stuff like this:
story = Story.create(title = self.feed.stories[key]["title"], story = self.feed.stories[key],date = datetime.datetime.now(), is_relative = True)
i can then run:
"python newslamost.py -g"
and it will go gather stuff and write them to the database
i then have a file called webapp.py, which is a flask app
import newsalmost
from flask import Flask
from flask import send_file
import subprocess
app = Flask(__name__)
import os
import json
from database import *
#app.route("/")
def index():
r = []
for i in Image.select():
r.append(str(i))
return json.dumps(r)
"python webapp.py"
I've tried to strip it down to the core problem. The flask app never sees anything in the database.. ever..
i KNOW its logging them correctly, because i can actually run "python" in that folder, import database and i get many stories from Stories.select()
the WEIRDER thing is that i initially had architected this in a more desirable way where the flask app merely creates a new instance of the newsalmost instance then calls functions on that to return stuff from the database and this worked.. IN DEV MODE. But when i deployed it to my web faction server (and got everything up there running), i am again greeted always with an empty response from the database. this is my attempt to try and directly reference the database in flask, thinking that maybe it was newsalmost fucking things up.. but nope.
I'm just bewildered why an sqllite database would perform exactly as expected locally, but not once deployed to a webserver, but also... why does the flask code i provided not get anything from the database, but running the same database query in the interactive shell work?
any ideas?
I don’t know if that would solve your problem, but you should use the same database object in both modules:
database.py:
from peewee import *
db = SqliteDatabase("newsalmost.db",threadlocals = True)
class Story(Model):
title = CharField()
story = TextField()
date = DateField()
class Meta:
database = db
newsalmost.py:
from database import *
class NewsAlmost(object):
def __init__(self):
self.db = db
Related
Hi!
I'm working on a flask-sqlalchemy application, and as you can imagine I'm changing database models and other thing through the process, every time that I made changes to the models I have to populate de DB again, and at the same time, as a safety measure I need to have some sort of backup restore process prepared in case of something goes wrong.
main.py
# You can image the other code...
app.config["SQLALCHEMY_DATABASE_URI"] = f"sqlite:///{DB_FILE}"
db.init_app(app)
db.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
models.py
class User_device(db.Model):
__tablename__ = "user_devices"
id = db.Column(db.String(32), primary_key=True)
user_id = db.Column(db.String(32), db.ForeignKey('users.id', ondelete="CASCADE"))
ip = db.Column(db.String(32))
agent = db.Column(db.String(100))
I'll like to have a route where to do this process
I found a solution that works for me, if you have a better alternative don't hesitate to share it:
from the terminal:
back up your current db
sqlite3 instance/pre_backup.db .dump > back.sql
delete your db
rm instance/pre_backup.db
init your project in order to create the instances of your db
python3 main.py
or with the flask run etc.
4. Restore your db
sqlite3 instance/new.db < back.sql
In my case I've got one error msg for every table that I had before, but it didn't give me any trouble to populate the db.
PD:
I'm new in Stack Overflow and I don't see the problem with starting with Hi!
thanks to nothing #PChemGuy -> This is not a letter, so you do not start your question with Hi!!!!!!!!!
I want to connect to a postgres database running in my dev docker container.
I can establish a connection by the connection String:
def create_api():
from .models import db
from .models.userModels import User
# Initialize and Configure the Flask App
app = Flask(__name__)
app.config.from_object(AppConfig)
db = SQLAlchemy(app)
I have a separate module for my routes and the related classes as well as a separate module for my models I will use for postgres model based communication.
In the init file I execute the create_api() method. Here I create the routes and connect them to a Resource class ( userResource.py).
In userReource.py I use the userModels.User for my db stuff.
Structure:
- api
__init__.py
- models
- userModels.py
- resources
- userResource.py
Model:
class User(db.Model):
id = db.Column(db.Integer,primary_key=True)
name= db.Column(db.String(100))
def __init__(self, name):
self.name=name
The docsenter link description here say that I should use the
db.create_all() method to create the table user id if not exists.
I've also read other answsers of that topic which mention to import the model class if it is placed in a separate module before executing this command. As you can see, I've imported the model before, but nothing happens in my database.
Is there something else I have to take care of?
The intent: Refactor my code into MVC (this is just the model/database part), and have the server create the database with tables on first run if the database or tables does not exist.
This works when using a "flat" file with all the classes and functions defined in that file, but after moving out the functions into a service class and the models into their own folder with model classes, the db.create_all() function does not seem to be able to detect the table class correctly any more.
Example structure, (minimum viable problem):
server.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
def main():
# Intentionally moved into the main() function to prevent import loops
from services.users import UserService
users = UserService(db)
db.create_all()
app.run(debug=True)
if __name__ == '__main__':
main()
services\users.py
# Class used to access users data using a session
from models.users import Users
class UserService:
def __init__(self, db):
self.db = db
def get_all(self):
return self.db.session.query(Users).all()
def get(self, uid):
return self.db.session.query(Users).get(uid)
def add(self, json):
user = Users(email=json['email'], password=json['password'])
self.db.session.add(user)
self.db.session.commit()
return user
models\users.py
# The actual model
from server import db
class Users(db.Model):
_id = db.Column("id", db.Integer, primary_key=True)
email = db.Column(db.Text)
password = db.Column(db.Text)
Result: The database is created, but it is just an empty file with no tables inside of it.
I have also tried placing the db.create_all() inside the service class def __init__(self, db) (grasping at straws here), both as a self reference and as an argument reference. Neither have worked.
I am sure it is something obvious I am missing, but I have boiled down my project to just the bare minimum and still fail to see why it is not working - so I have to ask. How can I get the db.create_all() to detect my table classes correctly and actually create the required tables, while using this code structure (or something similar, in case I have misunderstood MVC)?
The problem is that server.py is executed twice
when it's imported in models/users.py
when server.py is called to run the app
Each execution generates a new db instance. The db imported by the model file adds the models to its metadata, the db created when the app is run has empty metadata.
You can confirm this by printing id(db) and db.metadata.tables at the end of models/users.py and just before the call to db.create_all() in the main function.
You need to structure your code so that only one db gets created. For example, you could move the app configuration and creation code into its own module, mkapp.py (feel free to come up with a better name):
mkapp.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config['SQLALCHEMY_ECHO'] = True
db = SQLAlchemy(app)
And in server.py do
from mkapp import app, db
and in models/users.py do
from mkapp import db
As a bonus, this should also remove the import cycle.
I don't use flask much, so this solution can probably be improved on. For example, having a function create app and db and memoise the results might be better than creating them in top-level module code.
I'm attempting to modularize my flask app by splitting models, the db, routers, etc into separate files. This has been giving me some trouble, because it seems that my tables are not being created as I expect.
I have this file called flask_postgres_server.py
from . import flask_server
from flask_sqlalchemy import SQLAlchemy
from flask import jsonify, request
from . import models
Tree = models.tree.Tree_postgres.Tree
app = flask_server.app # this simply exports an instance of a flask app, so I can use it in another server with mongo
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
tree.save()
return jsonify(tree.get_dict())
I am importing a Tree model:
from ... import postgres_db
db = postgres_db.db
class Tree(db.Model):
__tablename__ = 'trees'
id = db.Column(db.Integer, primary_key=True)
label = db.Column(db.String(), nullable=False)
def save(self):
db.session.add(self)
db.session.commit()
db.session.expunge_all()
db.session.close()
def get_dict(self):
return {"label": self.label, "_id": self.id}
which in turn imports the db:
from . import flask_server
app = flask_server.app
import os
from dotenv import load_dotenv, find_dotenv
from flask_sqlalchemy import SQLAlchemy
"""
Imported by a server in order to set up a postgres connection for use by a Flask server
"""
load_dotenv(find_dotenv())
DB_NAME = os.environ.get("DB_NAME")
POSTGRES_HOST = os.environ.get("POSTGRES_HOST")
POSTGRES_PORT = os.environ.get("POSTGRES_PORT")
POSTGRES_USER = os.environ.get("POSTGRES_USER")
POSTGRES_PASSWORD = os.environ.get("POSTGRES_PASSWORD")
DB_URI = f'postgresql://{POSTGRES_USER}:{POSTGRES_PASSWORD}#{POSTGRES_HOST}:{POSTGRES_PORT}/{DB_NAME}'
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URI
db = SQLAlchemy(app)
db.create_all()
When I run my server and get post data at /trees, I get this error message:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "trees" does not exist
LINE 1: INSERT INTO trees (label) VALUES ('countries') RETURNING tre...
^
[SQL: INSERT INTO trees (label) VALUES (%(label)s) RETURNING trees.id]
[parameters: {'label': 'countries'}]
Shouldn't this relation have been created automatically?
I can go into psql and create this table manually. Then I get another error saying:
DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session
Is there something wrong with the way I've structured my code?
The trees table does not get created because db.create_all() is called before the Tree model is imported.
In the code in the question, you can fix this by moving db.create_all() to after the definition of the Tree model. In your overall structure, db.create_all() will need to be called after all the model definitions have been executed (usually by importing the files that contain them).
Note that it's not a good idea to close the session in the save method - you won't be able to access the Tree instance afterwards when calling its get_dict method. I'd suggest leaving session life-cycle management to flask-sqlalchemy, Tree.save should just add and commit.
Finally label = request.get_json['label'] in the route handler should be label = request.get_json()['label'].
DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session implicitly give users clue that the object 0x10d2facd0 that you've created is already "disconnected" with the session.
You've already closed the session with db.session.close() but then tried to access the "closed" object afterward.
It is better to save the get_dict result before you close it. It can be rewritten like this:
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
response = tree.get_dict()
tree.save()
return jsonify(response)
First time on the site, so hi to all and thanks in advance. Longtime lurker and newb.
I'm working on a web app in flask, using Flask-SqlAlchemy and SqlAlchemy-Searchable (docs-> https://sqlalchemy-searchable.readthedocs.org/en/latest/index.html). For a reason I can't figure out, when I try a similar example to the code shown on the docs page:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy, BaseQuery
from sqlalchemy_searchable import SearchQueryMixin
from sqlalchemy_utils.types import TSVectorType
from sqlalchemy_searchable import make_searchable
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql+psycopg2://usr:admin#localhost/dev'
app.config['SECRET_KEY'] = 'notreallyasecret'
db = SQLAlchemy(app)
make_searchable()
class ArticleQuery(BaseQuery, SearchQueryMixin):
pass
class Article(db.Model):
query_class = ArticleQuery
__tablename__ = 'article'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Unicode(255))
content = db.Column(db.UnicodeText)
search_vector = db.Column(TSVectorType('name', 'content'))
My search queries don't work properly. I opened a python shell and created the db, and inserted five identical articles
a= Article(name='finland',content='finland')
db.session.add(a)
db.session.commit() #a-e
with 'finland' both as name and content. According to the example:
Article.query.search(u'finland').limit(5).all()
There should be articles returned that have finland somewhere in them. In my case, I get an empty list. I get an object back if I modify the example query to:
Article.query.search(' ').first()
But it's rather useless searching for empty spaces. Any ideas?
Adding a bit more to it: I noticed in the article table, the 'search_vector tsvector' column is completely empty despite data being in the content and name columns; I'm not sure if that has anything to do with it.
I ran into this exact issue once, too, when using Flask-Script to add a manage.py management tool to my application.
The fact that the search_vector column is empty despite you having added the appropriate TSVectorType parameters means that the SQLAlchemy-Searchable trigger isn't present in the postgres DB. You can verify its absence by doing a \df+ in psql command line tool -- you will not see a trigger named article_search_vector_update. SQLAlchemy-Searchable sets up this trigger to update the content of the search_vector column when the columns named in TSVectorType(...) change.
In the case of manage.py, I had to first call:
db.configure_mappers()
Essentially, you have to configure SQLAlchemy's mappers before calling create_all(). Without doing this, SQLAlchemy-Searchable will not be given the opportunity to add its search_vector trigger to populate the TSVectorType column in the model.The SQLAlchemy-Searchable docs have more on this.
In total, a minimal manage.py that properly configures SQLAlchemy-Searchable as you require might look like:
#!/usr/bin/env python
from flask.ext.script import Manager
from app import app, db
manager = Manager(app)
#manager.command
def init_db():
"""
Drops and re-creates the SQL schema
"""
db.drop_all()
db.configure_mappers()
db.create_all()
db.session.commit()
On Collin Allen's answer: actually, the flask-sqlalchemy ''db'' exposes the configure_mappers function.
Replace:
from sqlalchemy.orm.mapper import configure_mappers
...
configure_mappers()
with:
...
db.configure_mappers()