Im trying to migrate my flask db using heroku. I performed the migrations in my local app, then committed the changes to github and deployed to heroku. I then executed
heroku run flask db migrate and heroku run flask db upgrade and based on the logs everything seems to have worked fine and without any errors:
INFO [alembic.autogenerate.compare] Detected added column 'users.active'
INFO [alembic.autogenerate.compare] Detected added column 'users.password'
Generating /app/migrations/versions/a92ff10fdb60_.py ... done
C:\Users\A>heroku run flask db upgrade -a certifit
Running flask db upgrade on ⬢ certifit... up, run.6994 (Free)
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
As seen from logs, the migration should have added two new columns to the users table:
INFO [alembic.autogenerate.compare] Detected added column 'users.active'
INFO [alembic.autogenerate.compare] Detected added column 'users.password'
However, when I run a SQL query the result is the same as prior migrataion:
id | email | username | password_hash | acquirer_id
----+--------------------+----------+-----------------------------------------------------------------------------------------------+-------------
4 | pooostgre#mail.com | fffff | a3990046bdc7d7a861363eab41f5f4ac8a7f574fe314ea | 11
Any ideas what could be the problem?
Thanks
This article helped a lot https://gist.github.com/mayukh18/2223bc8fc152631205abd7cbf1efdd41/
In short:
Changed SQLALCHEMY_DATABASE_URI from os.environ.get('DATABASE_URL') to heroku DATABASE_URL value (actual link). Im not sure if this made any difference during migrating database, but still I'll mention it here;
I ran flask db migrate and flask db upgrade in my local flask app.py (previously I did it in my heroku environment);
Changed SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL') (because of step 1);
Pushed changes to github and deployed new version to heroku
I have problem with running automated migrations with alembic library (I use raw alembic library).
So this is setup of the application:
I have scheduler (python script which calculates something and then
stores it in database)
and flask REST API (which uses data stored in database by scheduler
to return adequate response)
I then deploy the app with script which runs these three commands:
alembic revision --autogenereate
alembic upgrade head
python run_scheduler.py
After initial deployment, alembic_version table is created in PostgreSQL database with identifier value under version_num column, and migration script (lets call this script xx.py) is created in alembic/versions/
When I redeploy the app (with running migrations and scheduler): I get the
" Can't locate revision identified by 'xxxxxxx'
Why?
Because there is no xx.py script anymore (docker is built from source control repo) and xx is the value under version_num column in alembic_version table.
How to approach to and solve this problem?
Quick fix of author: delete alembic_version table with code below (inside alembic/env.py script)
target_metadata = Base.metadata # for context
sql.execute('DROP TABLE IF EXISTS alembic_version', engine)
I'm working to modify a cookiecutter Flask app.
locally I have deleted the migration folder and sqllite db a couple of times during development. I've pushed my chnages to heroku.
When trying to migrate the heroku postgresdb :
$ heroku run python manage.py db upgrade
.....
alembic.util.CommandError: Multiple head revisions are present for given argument 'head'; please specify a specific target revision, '<branchname>#head' to narrow to a specific head, or 'heads' for all heads
following http://alembic.readthedocs.org/en/latest/branches.html I tried:
$ heroku run python manage.py db merge heads
Running python manage.py db merge heads on myapp... up, run.9635
Generating /app/migrations/versions/35888775_.py ... done
Then I tried:
$ heroku run python manage.py db upgrade
Running python manage.py db upgrade on myapp... up, run.7021
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
....
"%s#head" % branch_label if branch_label else "head")
alembic.util.CommandError: Multiple head revisions are present for given argument 'head'; please specify a specific target revision, '<branchname>#head' to narrow to a specific head
, or 'heads' for all heads
How can I merge the revision heads into one and make sure this is synced with my development version?
I contacted heroku support and got the following back (which worked for me):
Hi,
To remove a folder from your local repository, git rm needs to be run. Could you please try something like below?
$ git rm -r migrations
$ git commit -m 'Remove migrations directory'
$ git push heroku master
To see differences between actual files and what are registered onto your local repository, git status may be useful.
Please let us know if you have any difficulty here.
I dropped some table related to an app. and again tried the syncdb command
python manage.py syncdb
It shows error like
django.db.utils.ProgrammingError: (1146, "Table 'someapp.feed' doesn't exist")
models.py
class feed(models.Model):
user = models.ForeignKey(User,null=True,blank=True)
feed_text = models.CharField(max_length=2000)
date = models.CharField(max_length=30)
upvote = models.IntegerField(default=0)
downvote = models.IntegerField(default=0)
def __str__(self):
return feed.content
What I can do to get the tables for that app ?
drop tables (you already did),
comment-out the model in model.py,
and ..
if django version >= 1.7:
python manage.py makemigrations
python manage.py migrate --fake
else
python manage.py schemamigration someapp --auto
python manage.py migrate someapp --fake
comment-in your model in models.py
go to step 3. BUT this time without --fake
For those that may still be having trouble (like me), try this out:
Comment out all the URL's in the main app's urls.py
Then go ahead and run migrations:
$ ./manage.py makemigrations
$ ./manage.py migrate
The problem was alleviated by removing the ()'s
solved_time = models.DateTimeField('solved time', default=timezone.now())
to
solved_time = models.DateTimeField('solved time', default=timezone.now)
I got this answer from reddit
What solved my problem in situation when manage.py setmigration and then migrate to the SQL database is not working properly did is following:
Run command : python manage.py migrate --fake MyApp zero
After that: python manage.py migrate MyApp
And the problem that rises with connections.py and and after running correctly the migration command is solved! At least for me.
I'm using Python (3.x), MySQL DB, Django (3.x), and I was in situation when I needed after some time of successfully creating tables in my database, that some error regarding connections.py raises. But, above commands helped. I hope they will help all those who are having these type of problems.
I just ran migrations with the name of the app attached, for all the apps I had provisioned and that worked.
e.g. python3 manage.py makemigrations my_custom_app
After running for all of them I ran a migrate command to seal the deal.
python3 manage.py migrate. That was it. I'm still wondering why django behaves this way sometimes though.
none of the above solutions worked for me, I finally solved by
sudo systemctl stop mysql.service
sudo apt-get purge mysql-server
sudo apt-get install mysql-server
sudo systemctl stop mysql.service
In my case the code that I pulled had managed = False and I wanted the tables to be maintained by Django.
But when I did makemigrations the custom tables were not being detected or I was getting the error that the app_name.Table_name does not exist
I tried the following:
delete all the migration files inside the migrations folder (except init.py file) and then makemigrations then finally migrate
above 2 answers
this
PS: This solution is only feasible if backup is present or data is not important or you are just started creating the tables, as purging mysql will lead to loss of data
This is linked to the migration data in the scripts inside the project not matching with the migration scripts in the database as far as I could tell.
I solved this by the following steps :
Delete all the migration scripts under migration folder except __ini__
Make sure that the model.py contains the same structure as the table in the database and managed=True
Remove all Django Created tables like auth_user,... etc
Run the following code
$ ./manage.py makemigrations
$ ./manage.py migrate
This will create the migration scripts again and will apply it to your database.
I had this issue where I was playing with same database structure in production vs development. While dropping and recreating tables will probably resolve the issue, its worth checking your database itself and see if the model is actually correct. For myself I created the development database incorrectly with the table names all in lowercase while in production the first letter of tables were capitalized. I used the python manage.py inspectdb command on production db, and compared it to the model and realized that in the model it was trying to insert data to table 'test' instead of 'Test' for example. Hope that helps some of you in future.
I had a similar issue.
I had another python (with a class) file which need access to DB.
For some reasons, when running 'makemigrations' this file was processed (I guess this is linked to some import chains).
In this class, I had a method containing a default arg method(defaultModel=Model.get_default()) in the signature which was accessing to the default object in the DB (static method included in the Model class).
A the import time, this default arg was evaluated and as the table is not populated yet, it gives this error.
So I just set None for the default args and asks for the default model object inside the method. This solved the issue.
I have to face same issue and there are a couple of approaches, but the one I think is the most probable one.
Maybe you are loading views or queries to database but you haven´t granted enough time for Django to migrate the models to DB. That's why the "table doesn't exist".
Make sure you use this sort of initialization in you view's code:
Class RegisterForm(forms.Form):
def __init__(self, *args, **kwargs):
super(RegisterForm, self).__init__(*args, **kwargs)
A second approach is you clean previous migrations, delete the database and start over the migration process.
I tried all of the above tricks and never worked for me.
I commented on all imports and URLs that called a particular Table
In this solution, your data will be removed. I removed the app and created the app again. I copied the app folder somewhere and delete the app folder from my project. I commented on all lines in urls.py and files similar views.py and admin.py that use this app. also app name in settings.py.
In mysql:
truncate django_migrations;
truncate django_admin_log;
Do this for all models in your app and change n.
n is app id.
delete from auth_permission where content_type_id=n
delete from django_content_type where app_label='appname'
python manage.py startapp your_app_name
Then uncomment previous lines and restore files and run
python manage.py makemigrations
python manage.pt migrate
I faced the same problem, some of the above mentioned answers seemed not to work for me, but here's a simple 4 step solution:
1) Delete the migrations files below __init__.py (don't delete __init__.py) in your specific app.
2) python manage.py makemigrations AppName
3) python manage.py migrate --fake AppName zero
4) python manage.py migrate AppName
Hope these works for you.
I faced the same issue earlier when I accidentally deleted my migrations folder in an app. I was able to fix it by running manual makemigrations for that specific app.
Here's the fix for Windows,
py manage.py makemigrations <your_app_name>
py manage.py migrate
For other OS you need to replace py with python3 or python
I hope this helped fix your issue!
if the python manage.py migrate still doesn't work I mean when you do this it nothing do anything you can delete the app's migrations from django_migrations table after then do
python manage.py migrate
I'm bulding an app with Flask and Angular hosted on heroku. I have a problem with migrating heroku postgresql. I'm using flask-migrate which is a tiny wrapper around alembic. Locally everything is fine. I got an exception when I run heroku run upgrade which runs alembic upgrade command.
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
INFO [alembic.migration] Running upgrade None -> 19aeffe4063d, empty message
Traceback (most recent call last):
File "manage.py", line 13, in <module>
manager.run()
...
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (ProgrammingError) relation "users" already exists
'\nCREATE TABLE users (\n\tid SERIAL NOT NULL, \n\tusername VARCHAR(32), \n\tpassword_hash VARCHAR(128), \n\tPRIMARY KEY (id)\n)\n\n' {}
Simply, alembic is trying to run from the first migration which creates the db. I tried to set explicitly the proper revision with heroku run python manage.py db upgrade +2 or revision number but the exception is the same.
lukas$ heroku run python manage.py db current
Running `python manage.py db current` attached to terminal... up, run.1401
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
Current revision for postgres://...: None
My guess was that due to ephemeral filesystem of Heroku revision in not stored, but that shouldn't be a problem if I explicitly set the revision, right? :)
How can I set the current head of revision?
Here is relevant code:
Procfile:
web: gunicorn server:app
init: python manage.py db init
upgrade: python manage.py db upgrade
models.py
db = SQLAlchemy()
ROLE_USER = 0
ROLE_ADMIN = 1
class User(db.Model):
__tablename__ = "users"
id = db.Column(db.Integer, primary_key = True)
username = db.Column(db.String(32), unique = True, index = True)
email = db.Column(db.String(255), unique = True)
password_hash = db.Column(db.String(128))
role = db.Column(db.SmallInteger, default = ROLE_USER)
The init command that you put in the Procfile creates a brand new Alembic repository, which is something you only do once on your development machine. When you deploy to a new machine all you need to do is run the upgrade command to get the database created and updated to the last revision.
Alembic and Flask-Migrate have a command called stamp that can help you fix this problem. With stamp you can tell Alembic to write the revision of your choice to the database, without touching the database itself.
For example, creating a database from scratch when there are a lot of migrations can take a long time if Alembic has to go through all the migrations one by one. Instead, you can create the database with db.create_all() and then run:
$ ./manage.py db stamp HEAD
and with this the database is marked as updated.
Also, at some point I favored the idea of putting maintenance commands in the Procfile, but these days I only put services there, so I would only leave the web line there. To upgrade the database I think it is more predictable to run the command explicitly:
$ heroku run python manage.py db upgrade