I have a worker running python script every 2 hour on Heroku.
The problem is each time I 'pull' the changes from git.
There is no changes at all for the sqlite3 database.
But I am sure the program is running and the database has changed by looking at the log file.
heroku log
How to retrieve the .db file then ?
It sounds like you have a little misconception. Heroku's git support is effectively one-way; you can use it to push new code to be run on the server, but you can't use it to copy files from Heroku back to your local tree.
Unfortunately it looks like there's not a good easy way to copy a file from your app to your local machine; you can use heroku run console to get a bash shell, and then scp a file out, but you're "pushing" it out of Heroku, and thus run can only copy to things with valid IP addresses.
If you're really using sqlite for your app's storage, though, you're going to run into a bigger problem. The filesystem for your app on Heroku is ephemeral, in that changes you make can be wiped out at any time. Heroku will delete your app's local storage and start over fresh whenever it wants to.
The right way to do it is use Heroku's built-in Postgres support and store your application's data there. Not only will it persist, but you'll be able to access it directly using the Postgres command-line tools.
Accessing the heroku console can now be done with:
heroku run bash
then i downloaded the linux gdrive application and ran in locally in the folder to upload my file to google drive. https://olivermarshall.net/how-to-upload-a-file-to-google-drive-from-the-command-line/ (skip step 4 and run with ./ like this ./gdrive upload my_file.txt
the other suggestion of heroku run console did not work for me (running a python flask app)
Related
I'm dockerizing a Flask / React application. I've started the process by installing gunicorn so that I could make this the default web server, and to that end created a wsgi.py file to load the application from there, and then from the command line I ran:
gunicorn --bind 0.0.0.0:5000 wsgi:app
And it worked without any issues.
Now I'm attempting to dockerize the app for my first time. I didn't use a gunicorn.config file yet but I've seen it referred to in a couple of articles.
Do I need to add a gunicorn.config file to my application in order for it to run smoothly once inside a Docker container? I'm leaving it out for now but I'd appreciate any advice on what is good practice or downright necessary.
It is generally a good idea to export any modifiable configuration to a config file when running your app in production. You don't have to use a config file, but hard coding all the configuration means you'll need to rebuild and redeploy the image if you want to make a change, which is not ideal.
When using Docker, you can use the -v option to mount specific files/directories from the host to the container, which is what you'd want to use when mounting an external config file.
I've recently deployed an application to Heroku. Due to the constraints of the file system I'm looking to replace two css files within a Flask Package upon the application starting.
My main goal is to take a file from the app directory (the file is part of the git repo) and use it to replace a python package file located in the site packages directory.
I've tried to run the following from the Heroku CLI but nothing seems to happen.
heroku run mv ./bootstrap.css ./.heroku/python/lib/python3.6/site-packages/flask_bootstrap/static/css/bootstrap.css
I've also tried to remove the files from the site packages directory using RM but again nothing happens.
Could you please let me know if standard unix commands work on Heroku?
Fork and edit flask_bootstrap and add that to requirements.txt file like this.
Heroku using ephemeral filesystem. Whatever change you made to your filesystem only last until the dyno is restarted.
So my opinion is to change the file in your local repo and push to heroku again.
Source : https://help.heroku.com/K1PPS2WM/why-are-my-file-uploads-missing-deleted
I have deployed an app in Heroku using Django. The Django program uses a SQLite database db.sqlite3 on root directory to populate its page. Separately, there is also a Node.js scraper program that inserts to that database.
The problem is that the hard-refreshed webpage shows the same data even after the content of the database changed. Curiously, this does not happen when it is tested locally with python manage.py runserver. How can I fix this problem?
Thank you in advance!
For reference, here is my requirements.txt file:
Django==1.10.6
gunicorn==19.7.1
Pillow==4.0.0
selenium==3.3.1
whitenoise==3.3.0
You cannot use sqlite on Heroku.
An sqlite db is stored as a file on the local filesystem. But in Heroku the filesystem is ephemeral and is not shared between dynos. Every time you redeploy your app, or scale your process, or in your case launch a worker, you get a new filesystem with a different copy of the db file.
Use the proper Postgres support via the add-ons.
I have a project on Heroku. This project allows me to update a JSON file. If update something on Heroku via the project's web interface I see the update. I can close the browser, open it again and the update persists.
Now, I want to push something to this project. If I do, the JSON file will be overwritten. So, I pulled first to get the current project state from heroku. Heroku's prompt says that everything is up to date, which is not true since I changed the JSON file.
Using heroku run ls static/json/ shows the changed file. I need to get this file before pushing to heroku to avoid destroying updates done via the web interface.
I don't think this is possible, for one reason: A Heroku dyno has it's own ephemeral filesystem with a git checkout of the most recent code. It cannot go the other way around however, it's not possible to check file changes in the dyno in into the git repo. This is why you get the response 'Already up-to-date.' when trying to pull, because the heroku git remote is inn sync with your local repo.
You should maybe store your file on Amazon's S3 or in a database or even write your data to a database, so that it is persisted between heroku deployments.
You can run heroku run bash and backup the file then deploy your app run heroku run bash again and remove the new file and replace it with the old one you backed up. Hope it helps
If you want to check your files in your last deploy, you can run heroku run bash --app your-app-name.
However if you upload files programmatically to your app like submitting a form with a file within it, these files will disappear after a while as herok doc mentions
The Heroku filesystem is ephemeral - that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted. Each dyno boots with a clean copy of the filesystem from the most recent deploy. This is similar to how many container based systems, such as Docker, operate.
In addition, under normal operations dynos will restart every day in a process known as "Cycling".
I would like to be able to log the number of words in certain files in a Github repo whenever there is a new push to the repo. I have set up a hook on Github to hit a Django Heroku app url after each push, but I don't know how to run a git pull in python from a Django app running on Heroku. Is it possible to write to the local file system in Heroku?
Check out github repo from Heroku?
from the command line you can pull from heroku easily: git pull heroku master
have set up a hook on Github to hit a Django Heroku app url after each push, but I don't know how to run a git pull in python from a Django app running on Heroku?
Is it a different heroku App (from the one that was deployed) that will be doing the pull?
Yes? then you are going to have issues. Because the pull app needs permission (heroku login) to pull... and it wont have it. Also, b/c of the ephemeral filesystem, even if you login (via heroku run bash or the like) to it, the pull app will eventually lose its logged in session (see why below)
No? then don't pull. just use the os filesystem libraries to look into the application directory...
Is it even possible to write to the local file system in Heroku?
Yes and No. You can write to the local filesystem, but its going to get nuked. See: https://devcenter.heroku.com/articles/dynos#ephemeral-filesystem
Also, with the EFS, each dyno is going to have a different EFS - so each web process is in a way sandboxed.