We Keep Coding

Delete rows in database where id not found in a filtered query in Flask-SQLAlchemy

I'd like to remove rows in my database where an ID is not found amongst ID's from a filtered query. In layman's term, "Delete rows if the row's ID is not amongst ID's sourced from a query that kept unique users by name".
For example, I would like to clear out users in my database with duplicate names:
from project.models import db, User
query = User.query.group_by(User.name)
User.query.filter(** filter out User.id not found in `query.id` **).delete()
db.session.commit()

You can call filter multiple times on same query object.
query = User.query.group_by(User.name).all()
delete_query = db.session.query()
for user in query:
delete_query = delete_query.filter(User.id != user.id)
delete_query.delete()
db.session.commit()

Ok I came to a working solution after some digging:
from project.models import db, User
# Filter id's where `User.name` is unique
query = User.query.with_entities(User.id).group_by(User.name)
del_query = User.__table__.delete().where(User.id.not_in(query))
# Delete posts with non-unique names
db.session.execute(del_query)
db.session.commit()
This solution does not filter iteratively using User.query.group_by(User.name).all(), which could introduce time and memory issues if your database is large.

django db_table not changing

Hi i have created a dynamic table model and it is not changing whenever i select another table from the list i ve created in the template...
if i select the first table to see the data of it and then go back to the list and select the second table i get this error
InternalError: (1054, u"Unknown column 'table_name1.id' in 'field list'")
So, when I change from http://127.0.0.1:8000/tables/id=1 to http://127.0.0.1:8000/tables/id=2 it gives me the error.
But if I restart the server and go straight to
http://127.0.0.1:8000/tables/id=2 it works.
But now http://127.0.0.1:8000/tables/id=1 it doesn't work.
this is the party of views.py
def addview(request, pk):
table_name = Crawledtables.objects.get(id=pk)
print table_name
AllTables._meta.db_table = table_name.name
print AllTables._meta.db_table
tbl_detail = AllTables.objects.all()
print tbl_detail
return render(request, 'tables/table_list.html', {'details': tbl_detail})
The prints are just for testing and they work.
It prints out the correct table I select.
But the tbl_detail it doesn't print when i go to the 2nd table that i selected. only when I restart the server.
CrawledTables holds all the table names inside that DB with the date of creation of each table. I get the table name of the CrawledTables and put it in the AllTables._meta.db_table.
AllTables access all the tables in the DB ( same structure, different table names with different data inside of each table)
Do I need to clear the db_table everytime i go to the selected table?
I have done all the migrations correctly.
Please help me. Thank you in advance.

Does PeeWee support interaction with MySQL Views?

I am trying to access pre-created MySQL View in the database via. peewee treating it as a table [peewee.model], however I am still prompted with Operational Error 1054 unknown column.
Does PeeWee Supports interactions with database view ?

Peewee has been able to query against views when I've tried it, but while typing up a simple proof-of-concept I ran into two potential gotcha's.
First, the code:
from peewee import *
db = SqliteDatabase(':memory:')
class Foo(Model):
name = TextField()
class Meta: database = db
db.create_tables([Foo])
for name in ('huey', 'mickey', 'zaizee'):
Foo.create(name=name)
OK -- nothing exciting, just loaded three names into a table. Then I made a view that corresponds to the upper-case conversion of the name:
db.execute_sql('CREATE VIEW foo_view AS SELECT UPPER(name) FROM foo')
I then tried the following, which failed:
class FooView(Foo):
class Meta:
db_table = 'foo_view'
print [fv.name for fv in FooView.select()]
Then I ran into the first issue.
When I subclassed "Foo", I brought along a primary key column named "id". Since I used a bare select() (FooView.select()), peewee assumed i wasnted both the "id" and the "name". Since the view has no "id", I got an error.
I tried again, specifying only the name:
print [fv.name for fv in FooView.select(FooView.name)]
This also failed.
The reason this second query fails can be found by looking at the cursor description on a bare select:
curs = db.execute_sql('select * from foo_view')
print curs.description[0][0] # Print the first column's name.
# prints UPPER(name)
SQLite named the view's column "UPPER(name)". To fix this, I redefined the view:
db.execute_sql('CREATE VIEW foo_view AS SELECT UPPER(name) AS name FROM foo')
Now, when I query the view it works just fine:
print [x.name for x in FooView.select(FooView.name)]
# prints ['HUEY', 'MICKEY', 'ZAIZEE']
Hope that helps.

ProgrammingError, Flask with postgres and sqlalchemy

I am trying to get this setup to work, the database is created correctly, but trying to insert data I get the following error:
On sqlite:
sqlalchemy.exc.OperationalError
OperationalError: (sqlite3.OperationalError) no such column: Author [SQL: u'SELECT count(*) AS count_1 \nFROM (SELECT Author) AS anon_1']
On postgres:
sqlalchemy.exc.ProgrammingError
ProgrammingError: (psycopg2.ProgrammingError) column "author" does not exist
LINE 2: FROM (SELECT Author) AS anon_1
^
[SQL: 'SELECT count(*) AS count_1 \nFROM (SELECT Author) AS anon_1']
edit: Perhaps this has to do with it: I don't understand why it says "anon_1", as I am using credentials clearly?
I have inspected postgres and sqlite and the tables are created correctly. It seems to be an ORM configuration error, as it only seems to happend on inspecting or creating entries, any suggestion would be welcome!
class Author(CommonColumns):
__tablename__ = 'author'
author = Column(String(200))
author_url = Column(String(2000))
author_icon = Column(String(2000))
comment = Column(String(5000))
registerSchema('author')(Author)
SETTINGS = {
'SQLALCHEMY_TRACK_MODIFICATIONS': True,
'SQLALCHEMY_DATABASE_URI': 'sqlite:////tmp/test.db',
# 'SQLALCHEMY_DATABASE_URI': 'postgresql://xxx:xxx#localhost/test',
}
application = Flask(__name__)
# bind SQLAlchemy
db = application.data.driver
Base.metadata.bind = db.engine
db.Model = Base
db.create_all()
if __name__ == "__main__":
application.run(debug=True)

What is the query you're using to insert data?
I think the error messages may be a bit more opaque than they need to be because you're using Author/author in three very similar contexts:
the Class name
the table name
the column name
For easier debugging, the first thing I'd do is temporarily make each one unique (AuthorClass, author_table, author_column) so you can check which 'Author' is actually being referred to by the error message.
Since you're using the ORM, I suspect the underlying issue is that your insert statement uses Author (the object) when it should actually be using Author.author (the attribute/column name). The SELECT statements are complaining that they can't find the column 'author', but because you use author for both the table and column name, it's unclear what's actually being passed into the SQL statement.

Django - Following a foreign key relationship (i.e JOIN in SQL)

Busy playing with django, but one thing seems to be tripping me up is following a foreign key relationship. Now, I have a ton of experience in writing SQL, so i could prob. return the result if the ORM was not there.
Basically this is the SQL query i want returned
Select
table1.id
table1.text
table1.user
table2.user_name
table2.url
from table1, table2
where table1.user_id = table2.id
My model classes have been defined as:
class Table1(models.Model):
#other fields
text = models.TextField()
user = models.ForeignKey('table2')
class Table2(models.Model):
# other fields
user_name = models.CharField(max_length=50)
url = models.URLField(blank=True, null=True)
I have been through the documentation and reference for querysets, models and views on the django website. But its still not clear on how to do this.
I have also setup the url with a generic list view, but would like to access the user_name field from the second table in the template. I tried select_related in urls.py and also via the shell but it does not seem to work. See examples below.
config in urls
url(r'^$','django.views.generic.list_detail.object_list', { 'queryset': Table1.objects.select_related() }),
At the shell
>>> a = Table1.objects.select_related().get(id=1)
>>> a.id
1
>>> a.user_name
Traceback (most recent call last):
File "<console>", line 1, in <module>
AttributeError: 'Table1' object has no attribute 'user_name'
So basically,
What am i doing wrong?
Am i missing something?
What's the best way to pass fields from two tables in the same queryset to your template (So fields from both tables can be accessed)
Can this be done with generic views?

Something like this should work:
u = Table1.objects.get(id=1)
print u.id
print u.user.user_name
If you want to follow a foreign key, you must do it explicitly. You don't get an automatic join, when you retrieve an object from Table1. You will only get an object from Table2, when you access a foreign key field, which is user in this example

select_related() do not add second table result directly to the query, it just "Loads" them, insead of giving to you lazily. You should only use it, when you are sure it can boost your site performance. To get second table you need to write
a.user.username
In Django to get related table you need to follow them through foreign-keys that you have designed. Query itself do not directly translate to SQL query, because it is "lazy". It will execute only SQL that you need and only when you need.
If you have select_related SQL would have been executed at the moment when you do original query for a. But if you didn't have select_related then it would load DB only when you actually execute a.user.username.