We Keep Coding

Heroku still accessing old jobs in the queue and not using the updated code

I'm trying to build a python app on Heroku. I want to upload an excel file to via botocore to a bucket hosted on AWS, but I get the following error:
2020-11-19T20:21:20.616363+00:00 app[web.1]: botocore.exceptions.NoCredentialsError: Unable to locate credentials
I modified the code to include the credentials, cleared the cache, restarted the dynos and then started the app again. Now, I get the following error (along with the credentials error described above):
2020-11-19T20:21:20.616193+00:00 app[web.1]: Started at: 2020-11-10 15:35:54.303420
2020-11-19T20:21:20.616295+00:00 app[web.1]: Ended at: 2020-11-10 15:35:57.484733
2020-11-19T20:21:20.616331+00:00 app[web.1]: Exception status: Traceback (most recent call last):
I had 2 questions here:
Why is heroku not using the updated code?
Why is it still accessing an old job even when I've cleared the cache, restarted the dynos, etc.?
Any help would be much appreciated -Thanks!

So, the way around this problem was to remove the redistogo addon and re-add it using the following commands:
heroku addons:remove redistogo --confirm <app-name>
heroku addons:add redistogo
Heroku was now reading the correct code. I had to do this every time there was an error in my code. Once, all the errors in the code were sorted out, there was no need to do this.

Heroku running old deploy even after pushing a new deploy

Old deploy had the code at line 34 -
for _ in range(HEX[0][0]):
but HEX[0][0] was a string, realizing this mistake I changed it to
for _ in range(len(HEX[0][0])):
and redeployed
but Heroku is still referring to the old code and giving the error
2020-07-12T09:02:32.927382+00:00 app[web.1]: Traceback (most recent call last):
2020-07-12T09:02:32.927384+00:00 app[web.1]: File "/app/.heroku/python/lib/pytho
n3.6/site-packages/apscheduler/executors/base.py", line 125, in run_job
2020-07-12T09:02:32.927384+00:00 app[web.1]: retval = job.func(*job.args, **job.
kwargs)
2020-07-12T09:02:32.927385+00:00 app[web.1]: File "/app/finalDeploy.py", line 3
4, in timed_job_garbage
2020-07-12T09:02:32.927385+00:00 app[web.1]: for _ in range(HEX[0][0]):
2020-07-12T09:02:32.940876+00:00 app[web.1]: TypeError: 'str' object cannot be i
nterpreted as an integer
The code is in file clock.py which runs a custom clock for scheduling tasks but the error is showing on finalDeploy.py(which is my app file ) but the code doesn't even exist on finalDeploy.py.

This was a problem with heroku. I switched off the dynos from the dashboard and used heroku ps:scale web=0 and heroku ps:scale clock=0. The app stopped 15 minutes after doing this! Then I simply changed the app name. Reinitialised the git and heroku repositories using git init . and heroku git:remote -a <new app name> and then redeployed. The app worked fine after that.

Endpoints quickstart-app-engine for Python deploy error

I'm trying to following the step-by-step tutorial found here, but the deploy just does not work, always returning Error Response: [13] An internal error occurred.
I did not change anything in the example code itself. As I said, I just followed the linked tutorial carefully. It fails and return the error above when I try gcloud app deploy.
Using gcloud app deploy --verbosity debug it brings some stacktrace, but without any usefull meaning. I'm copying below for completeness:
Updating service [default] (this may take several minutes)...failed.
DEBUG: (gcloud.app.deploy) Error Response: [13] An internal error occurred
Traceback (most recent call last):
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 791, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 756, in Run
resources = command_instance.Run(args)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/surface/app/deploy.py", line 65, in Run
parallel_build=False)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 587, in RunDeploy
flex_image_build_option=flex_image_build_option)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 395, in Deploy
extra_config_settings)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 188, in DeployService
message=message)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 244, in WaitForOperation
sleep_ms=retry_interval)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 266, in WaitFor
sleep_ms=sleep_ms)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/core/util/retry.py", line 222, in RetryOnResult
if not should_retry(result, state):
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 260, in _IsNotDone
return not poller.IsDone(operation)
File "/Users/jamesmiranda/Documents/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 169, in IsDone
encoding.MessageToPyValue(operation.error)))
OperationError: Error Response: [13] An internal error occurred
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred
Below is the app.yaml (exactly tha same of example git, except for the APPID):
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
python_version: 3
# [START configuration]
endpoints_api_service:
# The following values are to be replaced by information from the output of
# 'gcloud endpoints services deploy openapi-appengine.yaml' command.
name: echo-api.endpoints.MYAPPID.cloud.goog
config_id: [2018-01-09r1]
# [END configuration]
What I tried until now:
Change python version to python 2 (python_version: 2);
Include some files to skip in the app.yaml (copied from endpoint framework in standard env tutorial);
skip_files:
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?\..*$
- ^(.*/)?setuptools/script \(dev\).tmpl$
Tried the workaround from here;
gcloud config set app/use_deprecated_preparation True
NOthing worked. What I'm doing wrong?
Notes:
It works fine locally following the README info;
Everything works in GAE standard env folowing this another tutorial;
I did not found any problem in the Endpoint itself (I can see it deployed in API explorer), but the app deploy did not work in any way.

If the app.yaml file you are using is exactly the same as the one you copied in your question then there seems to be an error in the name and config-id you are entering. With the info provided in your question your app.yaml should look like:
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
python_version: 3
# [START configuration]
endpoints_api_service:
# The following values are to be replaced by information from the output of
# 'gcloud endpoints services deploy openapi-appengine.yaml' command.
name: "echo-api.endpoints.MYAPPID.cloud.goog"
config_id: "2018-01-09r1"
# [END configuration]
See that you were not entering the values for name and config_id inside quotation marks, and also you should not enter your config_id inside brackets.
I have tried this myself and works alright. If it still doesn't work for you maybe your name is not correct, as the format seems to be:
"example-project.appspot.com"
So if your project id is i.e. my-project-id, then your name would look like:
"my-project-id.appspot.com"

How to get Uwsgi working with wagtail (django)

I am trying to get up to speed with wagtail. I am running it on a remote server. I have installed a virtual environment, then I switched to the virtual environment and installed wagtail as per the steps here: http://docs.wagtail.io/en/v1.9/getting_started/tutorial.html
pip install wagtail
wagtail start rocker
cd rocker
pip install -r requirements.txt
python manage.py migrate
python manage.py createsuperuser
The next step in the guide is to test the installation has worked by running:
python manage.py runserver
I can't do this as working with a remote server, and also there is already a Django app running on the server (using uwsgi).
So I am now trying to connect to this wagtail app, via uwsgi.
Using the string that starts the existing app as a template, I have modified it to bind a socket to the wagtail app: -
uwsgi --chdir=/opt/rocker/rocker --module=rocker.wsgi:application --env DJANGO_SETTINGS_MODULE=rocker.settings --master --pidfile=/tmp/rocker.pid --socket=/opt/rocker/core.sock --processes=5 --uid=www-data --gid=www-data --harakiri=20 --max-requests=5000 --vacuum --home=/opt/rocker --daemonize=/var/log/uwsgi/rocker.logroot#caspium:/etc/init.d#
However the app isn't starting... the error in the uwsgi log says the following: -
*** Operational MODE: preforking ***
Traceback (most recent call last):
File "./rocker/wsgi.py", line 18, in <module>
application = get_wsgi_application()
File "/opt/rocker/local/lib/python2.7/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
django.setup(set_prefix=False)
File "/opt/rocker/local/lib/python2.7/site-packages/django/__init__.py", line 22, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/opt/rocker/local/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/opt/rocker/local/lib/python2.7/site-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/opt/rocker/local/lib/python2.7/site-packages/django/conf/__init__.py", line 116, in __init__
raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
**django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.**
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 4617)
spawned uWSGI worker 1 (pid: 4622, cores: 1)
spawned uWSGI worker 2 (pid: 4623, cores: 1)
spawned uWSGI worker 3 (pid: 4624, cores: 1)
spawned uWSGI worker 4 (pid: 4625, cores: 1)
spawned uWSGI worker 5 (pid: 4626, cores: 1)
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting
must not be empty.
I researched this and found this Django: ImproperlyConfigured: The SECRET_KEY setting must not be empty which suggests adding
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project_name.settings.local")
in manage.py which I did. This did not fix anything.
I also found this django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty which says you need a secret key specified in settings.py
However in the wagtail project there is no settings.py
I created one and added a secret key, and am still getting the error.
Can someone advise how I can fix this so I can run uwsgi to connect to the wagtail, and test if it's working.
Thanks

In the Wagtail base project structure, settings.py is split into several files in the settings directory - see http://docs.wagtail.io/en/v1.9/reference/project_template.html#django-settings for details. The best place to put SECRET_KEY is settings/local.py, since you want to keep it out of version control.

The problem was in the following file:
/opt/rocker/local/lib/python2.7/site-packages/django/conf/global_settings.py
There is a Secret Key = '' specified here.
I added a secret key, and the app ran ok with Uwsgi

What worked for me after so many tries, was to delete the whole settings/ folder and create a proper settings.py file instead. If you guys from #Wagtail want to separate settings... you can do this https://simpleisbetterthancomplex.com/tips/2016/11/01/django-tip-19-protecting-sensitive-information.html

Heroku worker crashing immediately

I deployed a python worker in Heroku cloud with an app. My Procfile is as follows.
worker: python worker.py
Worker will read a text file, do some changes and writes to another text file. Both text files are in app root directory. This is happening line by line. Approximately, this will take around 6hrs. But once I deploy it, within 1min or so it will crash. Error is logged as
State changed from up to crashed
Process exited with status 0
I have no clue what is causing this.