Heroku restarts every 24 hours, is there any way I can program it to not do this, or update my files that are inside it every 23h or 24h?
I use python, and use heroku to host my discord bot.
The Heroku filesystem is ephemeral that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted.
If you are looking to work with PaaS (Platform as a Service) you must to change your application design and your mine.
Heroku
12 FACTOR
You can't store information on Heroku file system,
you need to store your data in some database, for example, Postgres or MongoDB.
The can be added easily as an addon on Resources tab.
Related
I have a small Node.js / Express app deployed to Heroku.
I'd like to use a lightweight database like NeDB to persist some data. Is it possible to periodically backup / copy a file from Heroku if I used this approach?
File-based databases aren't a good fit for Heroku due to its ephemeral filesystem (bold added):
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code. During the dyno’s lifetime its running processes can use the filesystem as a temporary scratchpad, but no files that are written are visible to processes in any other dyno and any files written will be discarded the moment the dyno is stopped or restarted. For example, this occurs any time a dyno is replaced due to application deployment and approximately once a day as part of normal dyno management.
Depending on your use case I recommend using a client-server database (this looks like a good fit here) or something like Amazon S3 for file storage.
apologies upfront. I am an extreme newbie and this is probably a very easy question. After much trial and error, I set up an app on Heroku that runs a python script that scrapes data off of a website and stores it in a text file. (I may switch the output to a .csv file). The script and app are running on a Heroku Scheduler, so the scraping takes place on a schedule and the data automatically gets written to the file that is on the Heroku platform. I simply want to download the particular output file occasionally so that I can look at it. (Part of the data that is scraped is being tweeted on a twitter bot that is part of the script.)
(Not sure that this is relevant but I uploaded everything through Git.)
Many thank in advance.
You can run this command heroku run cat path/to/file.txt, but keep in mind that Heroku uses ephemeral storage, so you don't have any guarantee that your file will be there.
For example, Heroku restarts your dynos every 24 hours or so. After that you won't have that file anymore. The general practice is to store files on some external storage provider like Amazon S3.
Not just ephemeral, but immutable, which means you can't write to the file system. You'll have to put the file in something like S3, or just put the data into a database like Postgres.
My site writes new .html files into /tmp after the dyno is created.
The cherrypy app is in /app due to the Heroku's structure.
This prevents me from routing the .html files created with Cherrypy. Any idea on how to do this?
Heroku's filesystem is ephemeral:
Each dyno gets its own ephemeral filesystem, with a fresh copy of the most recently deployed code. During the dyno’s lifetime its running processes can use the filesystem as a temporary scratchpad, but no files that are written are visible to processes in any other dyno and any files written will be discarded the moment the dyno is stopped or restarted. For example, this occurs any time a dyno is replaced due to application deployment and approximately once a day as part of normal dyno management.
It's not meant for permanent storage, and anything you write out to disk can disappear at any moment.
If you need to write data out persistently you can use something like Amazon S3 or store it in a database.
Will it be possible to serve the code directly from db then? Assuming that i write the code into db?
Yes.
Heroku itself provides a PostgreSQL service and many others are available from the addons marketplace.
I have a worker running python script every 2 hour on Heroku.
The problem is each time I 'pull' the changes from git.
There is no changes at all for the sqlite3 database.
But I am sure the program is running and the database has changed by looking at the log file.
heroku log
How to retrieve the .db file then ?
It sounds like you have a little misconception. Heroku's git support is effectively one-way; you can use it to push new code to be run on the server, but you can't use it to copy files from Heroku back to your local tree.
Unfortunately it looks like there's not a good easy way to copy a file from your app to your local machine; you can use heroku run console to get a bash shell, and then scp a file out, but you're "pushing" it out of Heroku, and thus run can only copy to things with valid IP addresses.
If you're really using sqlite for your app's storage, though, you're going to run into a bigger problem. The filesystem for your app on Heroku is ephemeral, in that changes you make can be wiped out at any time. Heroku will delete your app's local storage and start over fresh whenever it wants to.
The right way to do it is use Heroku's built-in Postgres support and store your application's data there. Not only will it persist, but you'll be able to access it directly using the Postgres command-line tools.
Accessing the heroku console can now be done with:
heroku run bash
then i downloaded the linux gdrive application and ran in locally in the folder to upload my file to google drive. https://olivermarshall.net/how-to-upload-a-file-to-google-drive-from-the-command-line/ (skip step 4 and run with ./ like this ./gdrive upload my_file.txt
the other suggestion of heroku run console did not work for me (running a python flask app)
I have a project on Heroku. This project allows me to update a JSON file. If update something on Heroku via the project's web interface I see the update. I can close the browser, open it again and the update persists.
Now, I want to push something to this project. If I do, the JSON file will be overwritten. So, I pulled first to get the current project state from heroku. Heroku's prompt says that everything is up to date, which is not true since I changed the JSON file.
Using heroku run ls static/json/ shows the changed file. I need to get this file before pushing to heroku to avoid destroying updates done via the web interface.
I don't think this is possible, for one reason: A Heroku dyno has it's own ephemeral filesystem with a git checkout of the most recent code. It cannot go the other way around however, it's not possible to check file changes in the dyno in into the git repo. This is why you get the response 'Already up-to-date.' when trying to pull, because the heroku git remote is inn sync with your local repo.
You should maybe store your file on Amazon's S3 or in a database or even write your data to a database, so that it is persisted between heroku deployments.
You can run heroku run bash and backup the file then deploy your app run heroku run bash again and remove the new file and replace it with the old one you backed up. Hope it helps
If you want to check your files in your last deploy, you can run heroku run bash --app your-app-name.
However if you upload files programmatically to your app like submitting a form with a file within it, these files will disappear after a while as herok doc mentions
The Heroku filesystem is ephemeral - that means that any changes to the filesystem whilst the dyno is running only last until that dyno is shut down or restarted. Each dyno boots with a clean copy of the filesystem from the most recent deploy. This is similar to how many container based systems, such as Docker, operate.
In addition, under normal operations dynos will restart every day in a process known as "Cycling".