i have developed a Django web application for a chatbot that acts as s store's customers service.
At first i was using Sqlite3 foe my databases, i was working fine but some times it causes the (Data Base is Locked error).So, i switch to Mysql for the application backend and the other databases.
to be clear i have 5 DB all are under the user eman_saad:
1.db.
2.chating_log.
3.female_brain.
4.male_brain.
5.orderss.
But now i have this error:
django.db.utils.OperationalError: (1203, "User eman_saad already has more than 'max_user_connections' active connections")
i'ev tired increasing the max_connection using the terminal and login mysql>.
but still the error remain the same.
PS: I am using django10, Python3.5,webfaction shared server,Mysql
Related
Edit:
After doing some further investigation, the delay seems to be more Django than the Cloud SQL Proxy.
I added a couple of print statements at the start and end of a view, and they print instantly when the request is made, but it takes a further 60 seconds for the page to load.
I've stripped back the template files to include only the bare bones, removing most scripts and static resources and it's still pretty similar.
Changing my view to return a simple HttpResponse('Done') cuts the time drastically.
Whilst developing locally I am using Django to serve the static files as described in the docs. Again, I don't have this issue with other projects though.
Original Post:
I've recently noticed my Django application is incredibly slow to connect to my Google Cloud SQL database when using the Cloud SQL Proxy in my local development environment.
The initial connection takes 2-3 minutes, then 60 seconds per request thereafter. This applies when performing migrations or running the development server. Eventually the request completes.
I've tried scaling up the database but to no effect (it's relatively small anyway). Database version is MySQL 5.7 with machine type db-n1-standard-1. Previously I've used Django Channels but have since removed all references to this.
The Middleware and settings.py are relatively standard and identical to another Django app that connects in an instant.
The live site also connects very fast without any issues.
Python version is 3.6 w/ Django 2.1.4 and mysqlclient 1.3.14.
My database settings are defined as:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': os.getenv('DB_NAME'),
'USER': os.getenv('DB_USER'),
'PASSWORD': os.getenv('DB_PASSWORD'),
'PORT': '3306',
}
}
DATABASES['default']['HOST'] = os.getenv('DB_HOST')
if os.getenv('GAE_INSTANCE'):
pass
else:
DATABASES['default']['HOST'] = '127.0.0.1'
Using environment variables or not doesn't seem to make a difference.
I'm starting the Cloud SQL Proxy via ./cloud_sql_proxy -instances="my-project:europe-west1:my-project-instance"=tcp:3306.
After invoking the proxy via the command line I see Ready for new connections. Running python manage.py runserver shows New connection for "my-project:europe-west1:my-project-instance" but then takes an age before I see Starting development server at http://127.0.0.1:8000/.
I'm also noticing several errors in Stackdriver:
_mysql_exceptions.OperationalError: (2006, "Lost connection to MySQL server at 'reading initial communication packet', system error: 95")
django.db.utils.OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 95")
AttributeError: 'SessionStore' object has no attribute '_session_cache'
These appear - or don't - from time to time without changing any settings.
I've read they may be an access rights issue but the connection is eventually made, it's just incredibly slow. I'm authorising via the Google Cloud SDK, which seems to work fine.
Eventually I found that the main source of the delay was a recursive function being called in one of my admin forms (which delayed the initial startup) and context processors (which delayed each load). After removing it, the pages loaded without issue. It worked fine when deployed to App Engine or when using a test/local SQLite database, though, which is what made debugging a little harder.
I'm completely new to Django. I have developed a simple web application in Django and hosted it on external server. And that web application uses default mysql database. Now, I want to switch to a secondary server if my primary server goes down.
Copying and running the same code is not the option.
Can anyone explain how do I do it along with an example ?
not a beginner topic or answer but, in a prototypical production-esque deployment with nginx/apache/django, you can use the nginx upstream module
it'll detect when a node is down and won't route requests to the down node, so you'll get the failover behavior you desire out of the box
http://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream
I googled "nginx upstream module failover" and the second result had some examples: https://serverfault.com/questions/140990/nginx-automatic-failover-load-balancing
i'ev developed a chatbot web application using Django and python3 on web faction server. Basically the chatbot interacts with a store's users as a customer server.It uses the REST API to POST the user input and GET to display the chatbot output and a python file to process the input and find the output.
How the chatting works:
1. chatboy.js: it first POST the user input in the API then run the python file chabot.py.
2. chabot.py: a python file that connects to the Django backend db.sqlite3 and the conversation.sqlite3. So it select the user input from db.sqlite3 then select the matching output in the conversation.sqlite3. Finally the file will update the chatbot output in the db.sqlite3.
3. chatboy.js: will GET the last chatbot output and display it.
In development stage when i was testing the application in my local server everything works fine but problems came we i deployed the Django project in the internet. Mostly and the main ERORR that stops the application form working is :
The database is locked
I did many research and found out that:
sqlite3 is not for production and only for small or stand alone application.
sqlite3 have a multiple threads problem.
not good for realtime chatting application.
Some suggestion said either
switch all the sqlite3 to Mysql ( but how? and how i can change the sql queries in the python file to fit Mysql)
use a fast key value store like Reddis ( does it mean to make it the backend db form my project or juts the conversation.sqlite3?)
please help me because it is very frustrating and i had this error for a long time and i couldn't found and solution for the problem.
thank you,
PS: i'm using Django 10 , python3 , sqlite3, web faction server
You only need to update your settings file to change the DATABASES setting from sqlite3 to mysql or postgresql. But before you do that you will need to create a database in your Webfaction Control Panel (more information here).
I'm currently working on a django application. I can't add an element to my database on the admin view. I fill all the information but when I click on save button but the operation doesn't finish and I get a timeout. I use sqlite3 as database.
My question is there any one that know the origin of this problem. If not how could I investigate the problem. When I worked with other language (Java, C ...etc) when I have a problem I can use a debugger. What are the options I have?
This Problem can occur because of following reasons:
(Less Probable) You computation code is too Slow: Which is a rarity because the Timeout is set to about 1 minute or so, and code doesn't take that time to Execute
Your app is waiting on some external resource but it is not Responding. For this you will have to check for the Django logs and check if some external resource error is there
(Most Probable) Database taking too much time: This can occur either because:
App can't connect to Database: For this you have to check database logs OR try and connect manually with database through python manage.py dbshell
DB Query Taking so much time to execute: You can test this by checking database logs for how much time a query is taking OR you can connect manually via dbshell and make the same query there
Your can also use tools Like django-profiler , Django debug toolbar etc for debugging purposes. and for native python code python debugger
I am developing an app on Google App Engine (SDK version 1.7.0 on Windows) that I need to test often, and this testing involves a lot of GQL queries on the datastore.
You can run GQL queries at the browser in App Engine's admin interface online:
But this seems to be not possible on the SDK admin console on localhost. There is no such option:
So it seems I can run GQL queries only through python scripts on my localhost.
It's not convenient to write a python script every time to query the results for short tasks here and there. Also, if you do it at the online admin interface, you can see the query results immediately in a nice table below. If you want to visualize all your query results in a browser window through a python script you'd have to write a lot more code, basically trying to rewrite what Google has already done very well at their server admin interface.
So I am stuck having to do this at the browser on the App Engine admin interface after deployment, which keeps adding to my allowed operations count until I have soon hit my free daily quota limit.
So isn't there any way to run GQL queries (non-programmatically, on a browser) on my localhost?
You have the Interactive Console for that, you get run GQL query there.
For example you can run following code in the Interactive Console:
from google.appengine.ext import db
q = db.GqlQuery("SELECT * FROM Song WHERE composer = 'Lennon, John'")