I am noticing locking (OperationalError) when using the SQLite 3.7 for python application . I am using SQLalchemy for ORM.This issue occurs on windows. I using the DB on the same machine as the application.
It seems to be working fine on ubuntu. Wondering what might be the issue? Did some older version of sqlite had this issue that got fixed?
Some thoughts :
-There is locking issue with windows NFS for SQLlite , but since i am using local filesystem this possibility is discarded.
Can a antivirus running on the machine cause a lock , since it reads file as and when updated.
The lock on the database is never released ( i waited for like 5 min , while accessing the DB via command line). This make me think whether some hanging process was holding on to lock or the process got killed. But seems that was not the case as the only process(python bottle server) that was accessing the DB was running. It is running under single process Apache.
The way locking happens with SQLite is documented at http://www.sqlite.org/lockingv3.html - read it. On Windows you can use Sysinternal's Process Monitor to monitor who is accessing the file which will zero down on the culprit.
Related
I am running datapoint in Pycharm on a Mac. When I am connecting to the Database via the Pycharm Terminal, I can execute the code. Also, using the Pycharm Python Console, I am able to connect to the database. However, here, when I want to execute code or have a look at tables, it raises the error "Datajoint Lost Connection Error (Server connection lost due to an interface error)". Does anyone know, why this might be/ How to solve this problem?
I have tried to connect to other datajoint databanks, but it raises the same error.
While I'm less familiar with PyCharm, it is the case with Jupyter/IPython terminals that the connection will lapse after a timeout. By restarting the kernel, you can re-establish this connection.
If this is a consistent issue with certain time-intensive autopopulate tables, I would recommend breaking down make functions into separate pieces across multiple tables.
I am on mac with homebrew mysql, making a django app. I also use jupyter. I don't know which of these things is the root cause of the issue.
I used to get this error message coming up every now and again, and I know how to fix it: just type brew services restart mysql and wait a bit.
The problem is that it's now happening with increasing frequency, several times an hour. Is there a known fix for this, when working on a local machine?
I thought, since I'm ctrl-c ing programs frequently, that might be exhausting the connection pool so I increased the connection pool size using SET GLOBAL max_connections = 5000;, but that has not helped.
Just a suggestion - I would give Docker a shot (since you keep getting these interruptions).
Create a new container which will run your MySQL server exposing a port to the DB server then connect to that from your mac.
Here's a good resource to get you started - https://docs.docker.com/samples/library/mysql/#-via-docker-stack-deploy-or-docker-compose
Alternatively, if you don't have to use MySQL you could try using PostgreSQL. Here's a good article for that one - https://medium.com/agatha-codes/painless-postgresql-django-d4f03364989 . Many django projects are using that in production as it has a lot more features to offer.
I have a Python script that uses sqlalchemy to create a MySQL schema.
Today I launched an Amazon RDS DB and successfully logged in, staring at a blinking cursor
mysql>
At this point, I realized I had no schema. The schema I want to use was written by someone else and it lives in a Python script.
Python is not installed; nor do I believe it should be. I simply have a MySQL database, yet no way to implement the tables.
How can I execute a Python script while on my AWS RDB, or is there some other way to go about this?
I suspect your issue is easily and intuitively solved with a slight conceptual shift on your part.
Ask yourself this:
Just now, which computer was I connected to when I saw the $ or C:\> or some other shell prompt? Which computer is the one where I was connected immediately before I saw the mysql> prompt?
Now, contrast that to the answer you would have given to the same question, if you were not using RDS? (I'll call this a "conventional" setup.)
In the latter question, the answer might be "the MySQL server," but in the former question, whichever machine it may be, the answer can't be "the MySQL server." You had to be somewhere else.
So, it seems like you are dealing with two different scenarios, but that isn't true. In actuality, in the conventional setup, the fact that you may be "on" the server is not really important, because programs and scripts that interact with MySQL may be running on the same machine, but they are still making a connection to the MySQL server process and operating at arm's length. People often run scripts (or their web server, etc.) on the server itself, assuming this has meaning, but it doesn't.
The place where you run the script is the computer you're using before connecting to MySQL... whether it's your laptop or an EC2 instance, or whatever. The script just needs to be connected to the MySQL server with the appropriate hostname, username, and password, when it runs from wherever it is running, whatever machine that happens to be.
EDIT 2:
Going to a MySQL shell prompt and launching a Python script from there is, AFAIK, not possible.
I'm working on a simple Flask web application. I use Eclipse/Pydev. When I'm working on the app, I have to restart this app very often because of code changes. And that's the problem. When I run the app, I can see the frame on my localhost, which is good. But when I want to close this app, just click on the red square which should stop applications in Eclipse, sometimes (often), the old version of application keeps running so I can't test the new version. In this case the only thing which helps is to force close every process in Windows Task Manager.
Will you give me any advice how to manage this problem? Thank you in advance.
EDIT: This maybe helps: Many times, I have to run the app twice. Otherwise I can't connect.
I've faced the same problem and solved it. I think it may help.
When we run a flask based site locally it is assigned to a TCP port: 5000 and the Default IP: 127.0.0.1:5000
Sometimes TCP connection remains even after closing the program or terminating the code. So, The idea is kill the TCP connection. You can do it from command-prompt(cmd)
Two Steps to Follow:
1. See the Process ID(PID) for the running TCP connection.
Go to cmd and type:
netstat -ano
Kill The Process By PID. Command for this: taskkill /f /im [PID]. Example is showed bellow.
taskkill /f /im 7332
I've had a very similar thing happen to me. I was using CherryPy rather than Flask, but my solution might still work for you. Oftentimes browsers save webpages locally so that they don't have to re-download them every time the website is visited. This is called caching, and although it's very useful for the average web user, it can be a real pain to app developers. If you're frequently generating new versions of the application, it's possible that your browser is displaying an old version of the app that it has cached instead of the most up to date version. I recommend clearing that cache every time you restart your application, or disabling the cache altogether.
This actually shouldn't happen with the latest versions of PyDev (i.e.: since PyDev 3.4.1: http://pydev.org/history_pydev.html, PyDev should kill all the subprocesses of the main process).
So, can you check which PyDev version are you using?
If you're in the latest version of PyDev, you can use Ctrl+Shift+F9 to terminate/relaunch by default.
But as you're dealing with flask, you should be able to use it to reload automatically on code-changes without doing anything by setting use_reloader=True.
I.e.: I haven't actually tested, but its documentation says that you can set the reload flag for that run(use_reloader=True) -- and PyDev should even be able to debug it (I'll take a better look and improve the PyDev docs on that area later on).
I'm trying to convert a django / cPython app to run as a Tomcat WAR using Jython. We are using a mysql database (MySQLdb in cPython / mysql-connector-java-5.1.15-bin.jar in jython). I have changed the DATABASE_ENGINE between the two configurations. The app runs great with cPython (after I disabled SELinux limitations on http connections to a database). The front page, images, and static content work in the jython. However, when I request a page that hits the database, it seems to hang for a while, then returns with:
Communications link failure
The last packet sent successfully to
the server was 0 milliseconds ago. The
driver has not received any packets
from the server. [SQLCode: 0],
[SQLState: 08S01]
Has anyone seen this before? I don't see any recent discussions of this type or problem. I'm using django 1.2.5, jython 2.5.2. I've tried running the Java app under Fedora (jre-1.6.0-openjdk.x86_64) and Windows (Java(TM) SE Runtime Environment (build 1.6.0_23-b05)). Same behavior. I've also tried both mysql-connector-java-5.1.15-bin.jar (the latest version) and mysql-connector-java-5.1.10-bin.jar ("extensively tested" according to http://packages.python.org/django-jython/database-backends.html). None of the various configurations affect the behavior.
How can I solve it or where should I look next?
I found the mistake. In hindsight, it's obvious. I made a type-o in the IP address of the database server, but only in the configuration file on the jython branch. As a result, the Jython instance hung while the cPython instance didn't. Found the error when I merged the two into a single settings.py (using "if os.name=='java'" to make the necessary jython changes.
So . . . What's the correct protocol here? Should I delete the question or leave it up in case someone else makes the same type of mistake I did?