We Keep Coding

How to rename a column in a table with Flask + SQLAlchemy

I created two tables for my truck scheduling application:
class appts_db(db.Model):
id = db.Column(db.Integer, primary_key=True)
carrier = db.Column(db.String(100))
material = db.Column(db.String(10))
pickup_date = db.Column(db.String(10))
class carriers_db(db.Model):
carrier_id = db.Column(db.Integer, primary_key=True)
carrier = db.Column(db.String(100))
phone_number = db.Column(db.String(15))
How can I rename the column carrier to carrier_name in both tables to make it more clear what the columns contain. I tried using the command prompt
>python3
>db.create_all()
But the column name doesn't update. Is there some command that I'm missing that can update the column name in the db?

(1.) This seems to be a question about "how to migrate a single table?", twice. That is, whatever answer works for appts_db will also need to be applied to carriers_db -- I don't see a FK relation so I think most technical solutions would need to be manually told about that 2nd rename.
(2.) There are many nice "version my schema!" approaches, including the usual ruby-on-rails approach. Here, I recommend alembic. It takes some getting used to, but once implemented it lets you roll forward / roll back in time, and table schemas will match the currently-checked-out source code's expectations. It is specifically very good at column renames.
(3.) The simplest possible thing you could do here is a pair of DROP TABLE and then re-run the db.create_all(). The existing table is preventing create_all from having any effect, but after the DROP it will do just what you want. Of course, if you care about the existing rows you will want to tuck them away somewhere before you get too adventurous.

I ended up using DB Browser for SQLite (I had downloaded it previously) and ran this code in the "Execute SQL" tab:
ALTER TABLE carriers_db
RENAME COLUMN carrier TO carrier_name;

Array type in SQlite

I'm in the middle of developing a small site in Python. I use flask and venv.
I am currently in the middle of writing the data base and here is one of my tables:
class Message(db.Model):
message_id = db.Column(db.Integer, primary_key=True)
session_id = db.Column(db.String(30), unique=True)
application_id = db.Column(db.Integer)
participants = db.Column(db.Array())
content = db.Column(db.String(200))
The problem is in line 5:
"Array".
There is no such variable type.
I want to create a list of message recipients. Is there an Array or List variable type in SQlite?
If so, what is and how is it used?
And if not, how can I make a list of recipients anyway?
Anyone know?
Thank you very much!

SQLite does not support arrays directly. It only does Ints, Floats and Text. See here the type it supports.
To accomplish what you need, you have to use a custom encoding, or use an FK, i.e. create another table, where each item in the array is stored as a row. This would get tedious in my opinion.
Alternatively, it can be done in SQLAlchemy and you will want to have a look at the PickleType:
array = db.Column(db.PickleType(mutable=True))
Please note that you will have to use the mutable=True parameter to be able to edit the column. SQLAlchemy will detect changes automatically and they will be saved as soon as you commit them.
Also, have a look at the ScalarListType in SQLAlchemy for saving multiple values in column.
Update:
In SqlAlchemy You can use array column.
For example:
class Example(db.Model):
id = db.Column(db.Integer, primary_key=True)
my_array = db.Column(db.ARRAY(db.Integer())
# You can easily find records:
# Example.my_array.contains([1, 2, 3]).all()
# You can use text items of array
# db.Column(db.ARRAY(db.Text())
Update: This doesn't work in SQLite, SQLAlchemy's ARRAY type is for Postgres databases only. The best alternative for most people would be something involving JSON or switching to Postgres if possible. I'll be attempting JSON myself. credit to the replier in the comments.

Assigning identifiable unique IDs to rows when importing data from XML

I'm designing a database in which I'll be importing a large amount of data from XML daily to create or update existing rows.
Item data spans dozens of tables all related to the item_id in the main item table
For every item in the XML file, I need to check if it already exists in the database and update or create if it's not there.
Every XML belongs to a source_id and every item in the XML contains a unique alphanumeric ID up to 50 chars (but those IDs are not unique across all XMLs), so source_id:xml_item_id would be unique here
What I need is a way of finding if the item already exists in the database. Ideally, I will search by pk and use the same pk to join other tables
Attempt 1
I've tried encoding source_id:xml_item_id into a bigint for the pk as well as decode the bigint back to the original source_id:xml_item_id, but most of the times this is overflowing
So this is not going to work
Attempt 2
Use a UUID for the pk and source_id:xml_item_id as unique_id (string) for which to search by, but join related tables to UUID
While I don't see anything wrong here (IMO), JOINs might be affected, and I would prefer numeric pk for use in URLs
Attempt 3
Use source_id:xml_item_id as pk (string)
Same worries as with Attempt 2
The reason I've avoided AI PKs in all attempts is that there is a high possibility to shard this data in the future and I'd like this to have a relatively low impact on how PKs are being generated when this happens
What would be the best approach to handle this?
To identify if items already exist in the database
Have a user-friendly pk for URLs
Try not to impact JOIN performance too much

You can use unique together
class Data(models.Model):
source_id = models.CharField()
xml_item_id = models.CharField()
# ... other fields
class Meta:
unique_together = ("source_id", "xml_item_id")
Then in your import function just:
scid = your_xml_source_id
xmlid = your_xml_id
obj, created = Data.objects.get_or_create(source_id=scid, xml_item_id=xmlid)
if created:
# it's the new object, populate obj with the rest of the data
obj.other_field = your_xml_other_field
else:
# it's existing object, update object with a new value
obj.other_field = new_value
obj.save()

Flask-Admin deleting secondary mapping on update

I'm using Flask-Admin and when I update my content object, it deletes all the items in its associated lookup table (content_products_table).
So if I have a Content object and 7 products mapped in its lookup table (content_products_table) all the mapping from the content to the product is deleted when its updated.
I believe this is happening because my products list is empty while its updating, but in reality my product list could be 1000 items long so I wouldn't want to load it before updating, just simply not update it at all. I feel like there is a way to configure Flask-Admin or sqlalchemy to ignore the field on update.
Example my object below:
content_products_table = db.Table('content_products', db.Model.metadata,
db.Column('content_id', db.Integer, db.ForeignKey('content.id')),
db.Column('product_id', db.Integer, db.ForeignKey('product.id'))
)
class Content(db.Model):
id = db.Column(db.Integer, primary_key=True)
products = db.relationship('Product', secondary=content_products_table)
Example of the code thats updating my object:
form.populate_obj(model)
self._on_model_change(form, model, False)
self.session.commit()
Any information would help since I'm new to both of these platforms. Thanks!
EDIT
I wanted to update that when my objects are updated, it deletes all relationships. So if I have a table User who has a One To Many to a table Content. The user_id will be set to null in the Content table when I update the User.

form_excluded_columns = ('products')
Need to explicitly exclude object properties. This seems very dangerous.

Mongoengine integer id, and User creating

MongoDB is using string(hash) _id field instead of integer; so, how to get classic id primary key? Increment some variable each time I create my class instance?
class Post(Document):
authors_id = ListField(IntField(required=True), required=True)
content = StringField(max_length=100000, required=True)
id = IntField(required=True, primary_key=True)
def __init__(self):
//what next?
Trying to create new user raises exception:
mongoengine.queryset.OperationError: Tried to save duplicate unique keys
(E11000 duplicate key error index: test.user.$_types_1_username_1
dup key: { : "User", : "admin" })
Code:
user = User.create_user(username='admin', email='example#mail.com',
password='pass')
user.is_superuser = True
user.save()
Why?

There is the SequenceField which you could use to provide this. But as stated incrementing id's dont scale well and are they really needed? Can't you use ObjectId or a slug instead?

If you want to use an incrementing integer ID, the method to do it is described here:
http://www.mongodb.org/display/DOCS/How+to+Make+an+Auto+Incrementing+Field
This won't scale for a vary large DB/app but it works well for small or moderate application.

1) If you really want to do it you have to override the mongoengine method saving your documents, to make it look for one document with the highest value for your id and save the document using that id+1. This will create overhead (and one additional read every write), therefore I discourage you to follow this path. You could also have issues of duplicated IDs (if you save two records at the exactly same time, you'll read twice the last id - say 1 and save twice the id 1+1 = 2 => that's really bad - to avoid this issue you'd need to lock the entire collection at every insert, by losing performances).
2) Simply you can't save more than one user with the same username (as the error message is telling you) - and you already have a user called "admin".