In this code the last_seen field is being refreshed with the current time whenever the user uses the site. However, in the call to the db, he (Minuel Grindberg "Flask Web Development") adds self instead of self.last_seen, which confuses me. I understand what the basic principals of OOP are, and I (thought) understand what self is (reference to the object being created), but I do NOT understand why we don't add self.last_seen in the last line db.session.add(self)? Full code below. . .
class User(UserMixin, db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(64), unique=True, index=True)
username = db.Column(db.String(64), unique=True, index=True)
role_id = db.Column(db.Integer, db.ForeignKey('roles.id'))
password_hash = db.Column(db.String(128))
confirmed = db.Column(db.Boolean, default=False)
name = db.Column(db.String(64))
location = db.Column(db.String(64))
about_me = db.Column(db.Text())
member_since = db.Column(db.DateTime(), default=datetime.utcnow)
last_seen = db.Column(db.DateTime(), default=datetime.utcnow)
def ping(self):
self.last_seen = datetime.utcnow()
db.session.add(self)
Looks very simple and I'm sure it is, but obviously I'm missing something, or haven't learned something I should have. If i knew what to google for an answer, I would have certainly done so, but I'm not even sure what to search for other than the principals of Python OOP which I thought I already understood (I did review). Any help would be greatly appreciated because this is driving me crazy, lol.
He is adding the updated model to the DB. The model changed so db.session.add() will update the proper row behind the scene. I don't believe SQLAlchemy would allow you to add on the property of model because it wouldn't know which row to update
Perhaps an example would make this clearer. Let's take the following model:
class User(db.model):
__tablename__ = 'User'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(25))
Now there are 2 very important attributes on the model for inserting/updating it in the DB. The table name and the id. So to add that model to the DB with plain SQL we would need to do something like:
INSERT INTO User (name) VALUES ('Some string');
This is roughly what happens when you use db.session.add() on a new model. To update our model we would need to do something like:
UPDATE User
SET name='Some other String'
WHERE id=1;
Now if you were to only pass one attribute of a model to SQLAlchemy how would it be able to figure out what table you wanted to add to or which row was supposed to get changed?
If you just passed self.name to db.session.add() the query would end up looking like this:
UPDATE # There is no way to know the table
SET name='Some other String'
WHERE ; # There is no way to know which row needs to be changed
SQLAlchemy would most likely throw an exception if you tried. As for why it can't deduce the model from self that is probably way outside the scope of an SO question.
IanAuld is right-- but I'll make an effort to try and explain it in a long-winded fashion.
Lets put ourselves in SQLAlchemy's role, and lets pretend we are the db.session.add method.
self.last_seen is a datetime object, so lets pretend we're sitting at home, and an envelope comes through the door and it's addressed to db.session.add. Great, that's us, so we open it up and read the message which just says 2014-07-29 nothing else. We know we need to file it away in the filing cabinet somewhere, but we just don't have enough information to do so, all we know is we've got a datetime, we've got no idea what User it belongs to, or even if it does belong to a User at all, it's just a datetime-- we're stuck.
If instead the next thing that comes through the door is a parcel, again addressed to db.session.add, and again we open it-- this time it's a little model of a User, it's got a name, an email-- and even a last_seen datetime written on it's arm. Now it's easy-- I can go right to the filing cabinet and have a look to see if I've already got it in there, and either make a few changes to make them match, or simple file this one away if it's new.
That's the difference-- with an ORM model, you're passing these full User's or Products, or anything around, and SQLALchemy knows that it's a db.Model and therefore can know how, and where to handle it by inspecting it's details.
Related
I am trying to set up many-to-many relationship in SQLAlchemy but I am getting the error:
from shopapp import db
db.create_all()
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'shoppinglists_products.shoppinglist_id_v2' could not find table 'shoppinglist' with which to generate a foreign key to target column 'id'
My code:
from sqlalchemy import ForeignKey
from shopapp import db
shoppinglists_products = db.Table("shoppinglists_products",
db.Column("shoppinglist_id", db.Integer, ForeignKey("shoppinglist.id")),
db.Column("product_id", db.Integer, ForeignKey("product.id")))
class ShoppingList(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
products = db.relationship('Product', back_populates="shoppinglists", secondary="shoppinglists_products")
class Product(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
Where is the problem?
It seems like Flask-SQLAlchemy has problem finding the table for foreign key reference. Based on your code, here are the two ways you can fix this:
1) Fix shoppinglists_products table:
Flask-SQLAlchemy often converts the CamelCased model names into a syntax similar to this: camel_cased. In your case, ShoppingList will be referred to as shopping_list. Therefore, changing the ForeignKey("shoppinglist.id") to ForeignKey("shopping_list.id") will do the trick.
shoppinglists_products = db.Table("shoppinglists_products",
db.Column("shoppinglist_id", db.Integer, ForeignKey("shopping_list.id")), # <-- fixed
2) Change the model names:
If you'd like, you could go ahead and change the model name from ShoppingList to Shopping and later refer to this as shopping. This would prevent any confusion from rendering further. Usually, developers don't quite often go for a class name which is combined of two words, especially for the ORM cases. This is because various frameworks has different ways of interpreting the class names to create tables.
Expanding on #P0intMaN's answer - explicitly providing the SQL Alchemy table name with __tablename__ = "ShoppingList" (for example) lets you use your preferred case style and prevents SQLAlchemy from 'helping' you by changing the name of something kind of important without telling you.
class ShoppingList(db.Model):
__tablename__ = "ShoppingList"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
products = db.relationship('Product', back_populates="shoppinglists", secondary="shoppinglists_products")
In many/most Flask tutorials and books, simplistic table names (e.g. posts, comments, users) are used, which elide this issue. Thus a trap awaits for those of us who insist on meaningful CamelCased class names. This is mentioned somewhat casually in the documentation here: https://flask-sqlalchemy.palletsprojects.com/en/2.x/models/
Some parts that are required in SQLAlchemy are optional in
Flask-SQLAlchemy. For instance the table name is automatically set for
you unless overridden. It’s derived from the class name converted to
lowercase and with “CamelCase” converted to “camel_case”. To override
the table name, set the tablename class attribute.
I'm using sqlite database. I just ALTER the table called "User", adding a new column (INTEGER type) called "email_confirmed" through the console of my sqlite database.
My command was:
ALTER users ADD COLUMN email_confirmed INTEGER;
However, because I still don't add it through Python codes in my models.py, I couldn't use that variable "email_confirmed" in my program.
My codes (models.py) are below.
I'm struggling with this and if I couldn't fix this problem, I couldn't move on to another area... I would greatly appreciate it if you could show me what I should add, fix, or write, in the models.py, terminal, so that I could start using my email_confirmed variable.
-Models.py
class User(db.Model, UserMixin):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
profile_image = db.Column(db.String(64), nullable=False, default='default_profile.jpg' ) #nullable means "it cannot be blank"
email = db.Column(db.String(64), unique=True, index=True) #DON'T REALLY GET THE IDEA OF index=True
first_name = db.Column(db.String(20))
last_name = db.Column(db.String(20))
middle_name = db.Column(db.String(20), default='', nullable=True)
username = first_name+middle_name+last_name
password_hash = db.Column(db.String(128))
# email_confirmed = db.Column(db.Integer, server_default='1', nullable=False)
#Should I add the codes above?
def __init__(self,email,first_name, middle_name, last_name, password,{#email_confirmed#}):
self.email = email
self.first_name = first_name
self.middle_name = middle_name
self.last_name = last_name
self.password_hash = generate_password_hash(password)
#self.email_confirmed = email_confirmed
P.s/ When I wrote the codes that I commented out in the codes above and do the migration of the database, my terminal says: "ERROR [root] Error: Target database is not up to date."
Also, somehow, if I flask db upgrade my current migration version to the one in the picture below, I will remove the new column "email_confirmed" I added through the console.
It also seems like the "ALTER ...ADD COLUMN" command I did is shown as the code below in the migration file:
op.add_column('users', sa.Column('email_confirmed', sa.INTEGER(), nullable=True))
Thank you for your help!
The main problem is, that Django expects to be in control of the database.
How things should normally be done with Django:
You create the model and Django will create the table.
You modify a model,
You should create a migration and apply the migration, which will in turn apply all the required database changes.
Changing the database and then modifying the model is asking for trouble, as some subtle parameters might be missing in your manual alter table.
There are several ways to solve the issue.
1.) Change model, alter table manually, and perform a fake migration
Manually alter the table to exactly the same state as Django would have done
apply a fake migration
2.) dump old data, change model, create new DB and import / copy data.
create a full dump of your current database
delete it (or for sqlite just rename the db file)
edit models.py
create a new empty database with Django manage.py migrate)
try to restore your dump to the newly created database. (if field names and field types are the same this should work)
If you have any issue with the migration files created by Django and your code is only running on one server, then just delete all the migration files (and delete also all the related .pyc files in the __pycache__ directories) and recreate an initial migration from scratch.
Please tell us exactly about the current status.
Addendum 20200723:
It might help if you post the exact current schema of your current database file.
You can do this with
sqlite3 databasefile.db ".schema users"
These pics below are for #gelonida
My current migration version is at New Life 10 and somehow if I upgrade it to version NewLife 11, it will from the email_confirmed column. Will this works for both my model and database of "users" table? I just wanted to fix this mistake, go back to where I was right and do it right again. And thank you so much for helping me!
I am using Flask-SQLAlchemy with Postgres. I noticed that when I delete a record, the next record will reuse that one's id, which is not ideal for my purposes. Another SO question that this is the default behavior. In particular, his SO question discussed the sql behind the scenes. However, when I tested the solution in this problem, it did not work. In fact, postgres was not using SERIAL for the primary key. I was having to edit it in pgadmin myself. Solutions in other programs mention using a Sequence but it is not shown where the sequence is coming from.
So I would hope this code:
class Test1(db.Model):
__tablename__ = "test1"
# id = ... this is what needs to change
id = db.Column(db.Integer, primary_key=True)
would not reuse say 3 if record 3 was deleted and another was created like so:
i1 = Invoice()
db.session.add(i1)
i2 = Invoice()
db.session.add(i2)
i3 = Invoice()
db.session.add(i3)
db.session.commit()
invs = Invoice.query.all()
for i in invs:
print(i.id) # Should print 1,2,3
Invoice.query.filter(id=3).delete() # no 3 now
db.session.commit()
i4 = Invoice()
db.session.add(i4)
db.session.commit()
invs = Invoice.query.all()
for i in invs:
print(i.id) # Should print 1,2,4
Other, solutions said to use autoincrement=False. Okay, but then how do I determine what the number to set the id to is? Is there a way to save a variable in the class without it being a column:
class Test2(db.Model)
__tablename__ = 'test2'
id = ...
last_id = 3
# code to set last_id when a record is deleted
Edit:
So I could (although I do not think I should) use Python to do this. I think this more clearly tries to illustrate what I am trying to do.
class Test1(db.Model):
__tablename__ = "test1"
# id = ... this is what needs to change
id = db.Column(db.Integer, primary_key=True)
last_used_id = 30
def __init__(self):
self.id = last_used_id + 1
self.last_used_id +=1
# Not sure if this somehow messes with SQLAlchemy / the db making the id first.
This will make any new record not touch an id that was already used.
However, with this I approach, I do encounter the class variable issue behavior of Python. See this SO question
Future self checking: See UUID per #net comment here:
You should use autoincrement=True. This will automatically increment the id everytime you add a new row.
class Test1(db.Model):
__tablename__ = "test1"
id = db.Column(db.Integer, primary_key=True, autoincrement=True, unique=True, nullable=False)
....
By default Postgres will not reuse ids due to performance issues. Attempting to avoid gaps or to re-use deleted IDs creates horrible performance problems. See the PostgreSQL wiki FAQ.
You don't need to keep track of the id. When you call db.session.add(i4) and db.session.commit() it will automatically insert with the incremented id.
I have some models with a relationship defined between them like so:
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True, nullable=False)
children = Relationship(Child, lazy='joined')
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, primary_key=True, nullable=False)
father_id = Column(Integer, ForeignKey('parent.id'), nullable=False)
If I add a child within the session (using session.add(Child(...))), I would expect its father's children relationship to update to include this child after flushing the session. However, I'm not seeing that.
parent = session.query(Parent).get(parent_id)
num_children = len(parent.children)
# num_children == 3, for example
session.add(Child(father_id=parent_id))
session.flush()
new_num_children = len(parent.children)
# num_children == 3, it should be 4!
Any help would be much appreciated!
I can add the new child to the parent.children list directly, and flush the session, but I'm due to other existing code, I want to add it using session.add.
I can also commit after adding the child, which does correctly update the parent.children relationship, but I don't want to commit the transaction at the point.
I've tried adding a backref to the children relationship, but that doesn't seem to make any difference.
I've just run into this problem myself. SQLAlchemy does some internal memoisation to prevent it emitting a new SQL query every time you access a relationship. The problem is that it doesn't seem to realise that updating the foreign key directly could have an effect on the relationship. While SQLAlchemy probably could be patched to deal with this for simple joins, it would be very difficult for complex joins and I presume this is why it behaves the way it does.
When you do session.flush(), you're sending the changes back to the database, but SQLAlchemy doesn't realise it needs to query the database to update the relationship.
If you call session.expire_all() after the flush, then you force SQLAlchemy to reload every model instance and relationship when they're next accessed - this solves the problem.
You can also use session.expire(obj) to do this more selectively or session.refresh(obj) to do it selectively and immediately re-query the database.
For more information about these methods and how they differ, I found a helpful blog post: https://www.michaelcho.me/article/sqlalchemy-commit-flush-expire-refresh-merge-whats-the-difference
Official docs: https://docs.sqlalchemy.org/en/13/orm/session_api.html
I have a large number of .create() calls that rely on a ForeignKey in another table (Users). However, there is no point in the code where I actually create users.
Is there a way for there to be a Users entry created for each foreign key is specified on another table in SQLAlchemy?
For example:
class Rr(db.Model):
__tablename__ = 'rr'
id = db.Column(db.Integer, primary_key=True)
submitter = db.Column(db.String(50), db.ForeignKey('user.username'))
class User(db.Model):
__tablename__ = 'user'
username = db.Column(db.String, primary_key=True)
so If I call Rr(id, submitter=John) is there a way for a John entry to be created in the user table if it does not already exist?
I understand that I can create a wrapper around the .create() method such that it checks the submitter and creates one if it doesn't exist but this seems excess as there are a large number of models that want Users to be automatically created.
I can't think of any orm or sql implementation that does what you ask but there is something that effectively accomplishes what you seek to do described in this SO answer: Does SQLAlchemy have an equivalent of Django's get_or_create?
basically get the User from the db if it exists, if it doesn't create it.
The only down side to this method is that you would need to do 2 queries instead of one but I don't think there is a way to do what you seek in one query