My site writes new .html files into /tmp after the dyno is created.
The cherrypy app is in /app due to the Heroku's structure.
This prevents me from routing the .html files created with Cherrypy. Any idea on how to do this?
Heroku's filesystem is ephemeral:
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code. During the dyno’s lifetime its running processes can use the filesystem as a temporary scratchpad, but no files that are written are visible to processes in any other dyno and any files written will be discarded the moment the dyno is stopped or restarted. For example, this occurs any time a dyno is replaced due to application deployment and approximately once a day as part of normal dyno management.
It's not meant for permanent storage, and anything you write out to disk can disappear at any moment.
If you need to write data out persistently you can use something like Amazon S3 or store it in a database.
Will it be possible to serve the code directly from db then? Assuming that i write the code into db?
Yes.
Heroku itself provides a PostgreSQL service and many others are available from the addons marketplace.
Related
I have a Flask application deployed on GKE, duplicated in several pods.
For performance issues, I want this app to read local files from the app repository.
I need to update these files frequently with another Python script.
I could implement an endpoint to fetch the refreshed files from a server but if I ping this endpoint only one pod will update the local files, and I need all app instances to read from latest data files.
Do you have an idea on how to solve this issue?
There are multiple solutions, the best one is to set up a shared NFS volume for those files: The Flask Pods mount it read-only, the Python script mounts it read and write, and updates the files.
An NFS volume allows an existing NFS (Network File System) share to be mounted into a Pod. Unlike emptyDir, which is erased when a Pod is removed, the contents of an NFS volume are preserved and the volume is merely unmounted. This means that an NFS volume can be pre-populated with data, and that data can be shared between pods.
Reference link to create kubernetes NFS volume on GKE and advantages.
I have a small Node.js / Express app deployed to Heroku.
I'd like to use a lightweight database like NeDB to persist some data. Is it possible to periodically backup / copy a file from Heroku if I used this approach?
File-based databases aren't a good fit for Heroku due to its ephemeral filesystem (bold added):
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code. During the dyno’s lifetime its running processes can use the filesystem as a temporary scratchpad, but no files that are written are visible to processes in any other dyno and any files written will be discarded the moment the dyno is stopped or restarted. For example, this occurs any time a dyno is replaced due to application deployment and approximately once a day as part of normal dyno management.
Depending on your use case I recommend using a client-server database (this looks like a good fit here) or something like Amazon S3 for file storage.
Heroku restarts every 24 hours, is there any way I can program it to not do this, or update my files that are inside it every 23h or 24h?
I use python, and use heroku to host my discord bot.
The Heroku filesystem is ephemeral that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted.
If you are looking to work with PaaS (Platform as a Service) you must to change your application design and your mine.
Heroku
12 FACTOR
You can't store information on Heroku file system,
you need to store your data in some database, for example, Postgres or MongoDB.
The can be added easily as an addon on Resources tab.
I have a worker running python script every 2 hour on Heroku.
The problem is each time I 'pull' the changes from git.
There is no changes at all for the sqlite3 database.
But I am sure the program is running and the database has changed by looking at the log file.
heroku log
How to retrieve the .db file then ?
It sounds like you have a little misconception. Heroku's git support is effectively one-way; you can use it to push new code to be run on the server, but you can't use it to copy files from Heroku back to your local tree.
Unfortunately it looks like there's not a good easy way to copy a file from your app to your local machine; you can use heroku run console to get a bash shell, and then scp a file out, but you're "pushing" it out of Heroku, and thus run can only copy to things with valid IP addresses.
If you're really using sqlite for your app's storage, though, you're going to run into a bigger problem. The filesystem for your app on Heroku is ephemeral, in that changes you make can be wiped out at any time. Heroku will delete your app's local storage and start over fresh whenever it wants to.
The right way to do it is use Heroku's built-in Postgres support and store your application's data there. Not only will it persist, but you'll be able to access it directly using the Postgres command-line tools.
Accessing the heroku console can now be done with:
heroku run bash
then i downloaded the linux gdrive application and ran in locally in the folder to upload my file to google drive. https://olivermarshall.net/how-to-upload-a-file-to-google-drive-from-the-command-line/ (skip step 4 and run with ./ like this ./gdrive upload my_file.txt
the other suggestion of heroku run console did not work for me (running a python flask app)
I have a project on Heroku. This project allows me to update a JSON file. If update something on Heroku via the project's web interface I see the update. I can close the browser, open it again and the update persists.
Now, I want to push something to this project. If I do, the JSON file will be overwritten. So, I pulled first to get the current project state from heroku. Heroku's prompt says that everything is up to date, which is not true since I changed the JSON file.
Using heroku run ls static/json/ shows the changed file. I need to get this file before pushing to heroku to avoid destroying updates done via the web interface.
I don't think this is possible, for one reason: A Heroku dyno has it's own ephemeral filesystem with a git checkout of the most recent code. It cannot go the other way around however, it's not possible to check file changes in the dyno in into the git repo. This is why you get the response 'Already up-to-date.' when trying to pull, because the heroku git remote is inn sync with your local repo.
You should maybe store your file on Amazon's S3 or in a database or even write your data to a database, so that it is persisted between heroku deployments.
You can run heroku run bash and backup the file then deploy your app run heroku run bash again and remove the new file and replace it with the old one you backed up. Hope it helps
If you want to check your files in your last deploy, you can run heroku run bash --app your-app-name.
However if you upload files programmatically to your app like submitting a form with a file within it, these files will disappear after a while as herok doc mentions
The Heroku filesystem is ephemeral - that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted. Each dyno boots with a clean copy of the filesystem from the most recent deploy. This is similar to how many container based systems, such as Docker, operate.
In addition, under normal operations dynos will restart every day in a process known as "Cycling".