I've caught a problem I can't explain using pygresql for python. This is a simple SELECT query and as far as I can tell the only query that has the problem in the whole script.
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "run_daily_script.py", line 122, in DelegateAPINodes
vm_api_group = Execute(db_conn, fetch_api_group, (cust_id, nodes["node_id"]))
File ".../PreparedQueries.py", line 14, in Execute
result = db.execute(command)
File "/usr/local/lib/python2.7/site-packages/pgdb.py", line 1030, in execute
return self.executemany(operation, [parameters])
File "/usr/local/lib/python2.7/site-packages/pgdb.py", line 1065, in executemany
raise _op_error("Internal error in '%s': %s" % (sql, err))
OperationalError: Internal error in 'SELECT * FROM vm_api_group WHERE cust_id=3 AND node_id=18': insufficient data in "T" message
I ran the query via the console and it works just fine. I monitored psql logs and there were no errors from the database. It seems to be an internal operation with pygresql.
What does "T" mean and how do I fix this?
Related
tl;dr: An app that had been working fine is suddenly throwing a "Bad file descriptor" error with no other changes; I need advice for how to evaluate this.
I inherited an app that had been untouched for years, after the server crashed and I needed to move it to another machine. It's built with Flask, and uses Peewee to talk to a Postgres database over pyscopg2. It has a bunch of other stuff--an Elasticsearch engine for searching, a lot of heavy JS on the front end--but that doesn't seem to be the problem here. The code is moderately complex, and I am not very knowledgeable about all of its pieces.
It took me a while to get it set up using the sketchy deployment instructions that had been left behind, but eventually I got it running, and was able to get a test version running on a clean VM and then deploy it on an actual server, using gunicorn and nginx. It's been working fine in production for a week. I'm using Debian Buster for all versions. I'm using the most recent versions of all software.
I then decided to do some basic code cleanup, and ran the entire app through a linter, before looking at some other changes to make, that the end user had requested. Unfortunately, after this, the app consistently fails at the same point with a "Bad file descriptor" error. This is in a pre-run section, which parses a large XML file and saves the info to the database and to Elasticsearch; the app receives an XML upload, forks a few processes, and runs the parse/index process in the background.
I am subsequently unable to get past this error by any means. I have launched a clean VM and installed everything from scratch; I've reverted the git repo to before I linted the code. Same problem. I don't see how it can be a code issue, as it's now at the same point it was when I started. But I'm at a loss for what to do, and terrified that the production machine will fail.
The errors I get (trimming the first few lines that refer to places in the app itself) are:
[2021-03-14 14:40:11.699837] self.execute()
[2021-03-14 14:40:11.699878] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 1906, in inner
[2021-03-14 14:40:11.699907] return method(self, database, *args, **kwargs)
[2021-03-14 14:40:11.699946] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 1977, in execute
[2021-03-14 14:40:11.699976] return self._execute(database)
[2021-03-14 14:40:11.700004] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 2149, in _execute
[2021-03-14 14:40:11.700032] cursor = database.execute(self)
[2021-03-14 14:40:11.700060] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 3156, in execute
[2021-03-14 14:40:11.700088] return self.execute_sql(sql, params, commit=commit)
[2021-03-14 14:40:11.700115] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 3150, in execute_sql
[2021-03-14 14:40:11.700143] self.commit()
[2021-03-14 14:40:11.700171] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 2916, in __exit__
[2021-03-14 14:40:11.700198] reraise(new_type, new_type(exc_value, *exc_args), traceback)
[2021-03-14 14:40:11.700226] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 190, in reraise
[2021-03-14 14:40:11.700254] raise value.with_traceback(tb)
[2021-03-14 14:40:11.700282] File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/peewee.py", line 3143, in execute_sql
[2021-03-14 14:40:11.700309] cursor.execute(sql, params or ())
[2021-03-14 14:40:11.700339] OperationalError('SSL SYSCALL error: Bad file descriptor\n')
127.0.0.1 - - [14/Mar/2021 10:40:11] "POST /manage/versions/upload HTTP/1.1" 500 -
Error on request:
Traceback (most recent call last):
File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/werkzeug/serving.py", line 323, in run_wsgi
execute(self.server.app)
File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/werkzeug/serving.py", line 315, in execute
write(data)
File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/werkzeug/serving.py", line 273, in write
self.send_response(code, msg)
File "/home/deploy/git/myapp/venv/lib/python3.7/site-packages/werkzeug/serving.py", line 388, in send_response
self.wfile.write(hdr.encode("ascii"))
File "/usr/lib/python3.7/socketserver.py", line 799, in write
self._sock.sendall(b)
OSError: [Errno 9] Bad file descriptor
Exception in thread Thread-22:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.7/socketserver.py", line 654, in process_request_thread
self.shutdown_request(request)
File "/usr/lib/python3.7/socketserver.py", line 509, in shutdown_request
self.close_request(request)
File "/usr/lib/python3.7/socketserver.py", line 513, in close_request
request.close()
File "/usr/lib/python3.7/socket.py", line 420, in close
self._real_close()
File "/usr/lib/python3.7/socket.py", line 414, in _real_close
_ss.close(self)
OSError: [Errno 9] Bad file descriptor
I note that the final section ("Exception in thread Thread-22") is showing the system Python, rather than my virtual environment; I don't know if that's relevant, or if that's just what's running some overall process. I didn't get to this point doing anything different, though--the app is running in the virtual environment.
I'd be very grateful for any thoughts here--I'm obviously hoping it's some kind of stupid permission error or something, as I can't easily go into the code because of its complexity.
So I'm using MySQL connector and I'm getting an error which is weird because I didn't get any errors with this line before. Please let me know what the issue is.
def update(Sno,Passengers,Distance,Cargo,Aircraft):
myconn=mys.connect(host='localhost',user='root',passwd='adis',database='adis')
if myconn.is_connected():
print('Connection successful')
mycur=myconn.cursor()
query="update AMAS set Passengers={},Distance={},Cargo={},Aircraft={} where Sno={}".format(Passengers,Distance,Cargo,Aircraft,Sno)
mycur.execute(query)
myconn.commit()
print("AMAS Aircraft Database updated")
My error:
Connection successful
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\vinayak\AppData\Local\Programs\Thonny\lib\tkinter\__init__.py", line 1705, in __call__
return self.func(*args)
File "C:\Users\vinayak\AppData\Local\Programs\Thonny\lib\tkinter\__init__.py", line 3442, in __call__
self.__callback(self.__value, *args)
File "C:\Users\vinayak\Documents\Programs 12th\Project.py", line 146, in exe
submit()
File "C:\Users\vinayak\Documents\Programs 12th\Project.py", line 127, in submit
update(Sno,Passengers,Distance,Cargo,Aircraft)
File "C:\Users\vinayak\Documents\Programs 12th\Project.py", line 135, in update
mycur.execute(query)
File "C:\Users\vinayak\AppData\Roaming\Python\Python37\site-packages\mysql\connector\cursor.py", line 551, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "C:\Users\vinayak\AppData\Roaming\Python\Python37\site-packages\mysql\connector\connection.py", line 490, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "C:\Users\vinayak\AppData\Roaming\Python\Python37\site-packages\mysql\connector\connection.py", line 395, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Distance=,Cargo=,Aircraft= where Sno=' at line 1
Also, I'm new to Stack Overflow so please mention how I can improve asking questions here withput frustrating others.
I tried to do a simple query to my LAB instrument by:
>>> import visa
>>> rm = visa.ResourceManager()
>>> viavi = rm.open_resource("TCPIP0::10.0.2.76::5001::SOCKET")
>>> print(viavi.query("*IDN?"))
The result was:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files\Python35\lib\site-packages\pyvisa\resources\messagebase
d.py", line 407, in query
return self.read()
File "C:\Program Files\Python35\lib\site-packages\pyvisa\resources\messagebase
d.py", line 332, in read
message = self.read_raw().decode(enco)
File "C:\Program Files\Python35\lib\site-packages\pyvisa\resources\messagebase
d.py", line 306, in read_raw
chunk, status = self.visalib.read(self.session, size)
File "C:\Program Files\Python35\lib\site-packages\pyvisa\ctwrapper\functions.p
y", line 1582, in read
ret = library.viRead(session, buffer, count, byref(return_count))
File "C:\Program Files\Python35\lib\site-packages\pyvisa\ctwrapper\highlevel.p
y", line 188, in _return_handler
raise errors.VisaIOError(ret_value)
pyvisa.errors.VisaIOError: VI_ERROR_TMO (-1073807339): Timeout expired before op
eration completed.
According to what I learned till now (from the experience of others). This timeout error is somehow related to line termination ("\n"). How can I solve this problem ?
I found out that it was all related to the read_termination. My LAB instrument simply terminated its response by a '\n'.
While my script was looking for a '\r' all that time.
I am trying to fetch 150 million rows with messages data from data warehouse into my EC2 instance. I have 90GB available space on my disk. When I put my query for running, following is the error I get:
Traceback (most recent call last):
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 964, in fetchall
l = self.process_rows(self._fetchall_impl())
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 915, in _fetchall_impl
return self.cursor.fetchall()
pyodbc.Error: ('08S01', '[08S01] [FreeTDS][SQL Server]Read from the server failed (20004) (SQLFetch)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "en_str.py", line 45, in <module>
INNER JOIN notified.Message m ON c.MessageID = m.MessageID where c.CreateDate>'2016-01-01 00:00:00.0';''',db007_cnxn)
File "/usr/local/anaconda/lib/python3.5/site-packages/pandas/io/sql.py", line 331, in read_sql_query
parse_dates=parse_dates, chunksize=chunksize)
File "/usr/local/anaconda/lib/python3.5/site-packages/pandas/io/sql.py", line 1092, in read_query
data = result.fetchall()
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 970, in fetchall
self.cursor, self.context)
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
exc_info
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 185, in reraise
raise value.with_traceback(tb)
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 964, in fetchall
l = self.process_rows(self._fetchall_impl())
File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 915, in _fetchall_impl
return self.cursor.fetchall()
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('08S01', '[08S01] [FreeTDS][SQL Server]Read from the server failed (20004) (SQLFetch)')
My disk space gets full and odbc.log file in tmp folder is taking 90GB space. I have another mounted disk with 100TB space but odbc.log is generated only in tmp folder. What is the solution for this? Any thoughts on how this problem can be solved? Alternatively, how can I change odbc.log file location from 100 GB space disk to the disk with 100 TB space?
Also, instead of giving ec2 instance location, if I give s3 bucket location, will that avoid this odbc.log file?
Any help would be appreciated!
It looks like you may have verbose FreeTDS logging on. Most of the time, the config file is located in /etc/freetds.conf or /etc/freetds/freetds.conf.
You'll want to look for dump file in the FreeTDS log. Also, check odbcinst.ini, as logging can be turned on there.
You'll probably find one of the various logging facilities pointing to /tmp/odbc.log and that'll be your culprit. Good luck!
I use:
MongoDB 1.6.5
Pymongo 1.9
Python 2.6.6
I have 3 types of daemons. 1st load data from web, 2nd analyze it and save result, and 3rd group result. All of them working with Mongodb.
At some time 3rd daemon throws many exceptions like this(mostly when there are big amount of data in DB):
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/gevent-0.13.1-py2.6-linux-x86_64.egg/gevent/greenlet.py", line 405, in run
result = self._run(*self.args, **self.kwargs)
File "/data/www/spider/daemon/scripts/mainconverter.py", line 72, in work
for item in res:
File "/usr/local/lib/python2.6/dist-packages/pymongo-1.9_-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 601, in next
if len(self.__data) or self._refresh():
File "/usr/local/lib/python2.6/dist-packages/pymongo-1.9_-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 564, in _refresh
self.__query_spec(), self.__fields))
File "/usr/local/lib/python2.6/dist-packages/pymongo-1.9_-py2.6-linux-x86_64.egg/pymongo/cursor.py", line 521, in __send_message
**kwargs)
File "/usr/local/lib/python2.6/dist-packages/pymongo-1.9_-py2.6-linux-x86_64.egg/pymongo/connection.py", line 743, in _send_message_with_response
return self.__send_and_receive(message, sock)
File "/usr/local/lib/python2.6/dist-packages/pymongo-1.9_-py2.6-linux-x86_64.egg/pymongo/connection.py", line 724, in __send_and_receive
return self.__receive_message_on_socket(1, request_id, sock)
File "/usr/local/lib/python2.6/dist-packages/pymongo-1.9_-py2.6-linux-x86_64.egg/pymongo/connection.py", line 714, in __receive_message_on_socket
struct.unpack("<i", header[8:12])[0])
AssertionError: ids don't match -561338340 0
<Greenlet at 0x2baa628: <bound method Worker.work of <scripts.mainconverter.Worker object at 0x2ba8450>>> failed with AssertionError
Can anyone tell what cause this exeption and how to fix this.
Thanks.
This is likely a threading problem related to how you are using worker threads with gevent coroutines. It seems like the pymongo connection object is reading a response for a request it didn't make.