I have database with a single table Person which has a name(str) and age(int) columns. So, I create simple validate function for my sqlite3.connection
def adult(age):
return age > 18
And with following code, it works fine
connection = sqlite3.connect(r'C:\Dev\Garbage\database.db')
with connection:
connection.create_function('adult', 1, adult)
cursor = connection.cursor()
persons = cursor.execute('select "p"."name", "p"."age" from "Person" "p" where adult("p"."age")').fetchall()
for person in persons:
print(person)
But if I change adult like this
def adult(age):
return 1 / 0
I will get sqlite3.OperationalError: user-defined function raised exception.
In my project, it might be a huge amount of functions and I'd like to know - is there any way to know which function raised an exception? Or get ZeroDivisionError: division by zero instead of this.
Python's sqlite3 module throws away any error information from the exception, and replaces it with the constant message you've seen:
void _pysqlite_func_callback(sqlite3_context* context, int argc, sqlite3_value** argv)
{
...
py_func = (PyObject*)sqlite3_user_data(context);
args = _pysqlite_build_py_params(context, argc, argv);
if (args) {
py_retval = PyObject_CallObject(py_func, args);
Py_DECREF(args);
}
ok = 0;
if (py_retval) {
ok = _pysqlite_set_result(context, py_retval) == 0;
Py_DECREF(py_retval);
}
if (!ok) {
if (_enable_callback_tracebacks) {
PyErr_Print();
} else {
PyErr_Clear();
}
sqlite3_result_error(context, "user-defined function raised exception", -1);
}
...
}
I don't know what prevents it from appending the exception message to the returned error message.
Anyway, it is possible to print out the inner stack trace by calling enable_callback_tracebacks:
import sqlite3
db = sqlite3.connect(':memory:')
def error():
raise Exception('hello')
db.create_function('error', 0, error)
sqlite3.enable_callback_tracebacks(True) # <-- !
db.execute('select error()')
Traceback (most recent call last):
File "<stdin>", line 1, in error
Exception: hello
Traceback (most recent call last):
File "<stdin>", line 1, in
sqlite3.OperationalError: user-defined function raised exception
Related
I was not able to access the (nested) enums in a trivial protocol buffer message. I tried either way, nested or separated from DataNodeManagement!:
syntax = "proto3";
message DataNodeManagement {
string name = 1;
string id = 2;
string origin = 3;
ConnectionType con_type = 4;
enum ConnectionType {
UNKNOWN = 0;
MQTT = 1;
}
}
I am using this code to fill data in my message:
config = data_node_pb2.DataNodeManagement()
config.name = "Scanner1"
config.id = key
config.origin = "PC1"
config.con_type = data_node_pb2.ConnectionType.MQTT
# or
# config.con_type = data_node_pb2.DataNodeManagement.ConnectionType.MQTT
datasource.advertise_data_node(config.SerializeToString())
And it complains:
Traceback (most recent call last):
File "scanner-connector.py", line 144, in <module>
config.con_type = data_node_pb2.ConnectionType.MQTT
AttributeError: 'EnumTypeWrapper' object has no attribute 'MQTT'
respective:
Traceback (most recent call last):
File "scanner-connector.py", line 144, in <module>
config.con_type = data_node_pb2.DataNodeManagement.ConnectionType.MQTT
AttributeError: type object 'DataNodeManagement' has no attribute 'ConnectionType'
I am using these versions:
python --version
Python 3.6.6 :: Anaconda custom (64-bit)
protoc --version
libprotoc 3.6.1
Is there any special thing I overlooked as a beginner?
You have to skip the enum-name to access the values from your enum.
As can be seen in the protocol-buffers python-tutorial the enum is defined within a message
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
repeated PhoneNumber phones = 4;
}
In the section Reading a Message the enum is accessed by
import addressbook_pb2
addressbook_pb2.Person.MOBILE
So in your example it should be data_node_pb2.DataNodeManagement.MQTT
I am working on embedding python interpreter in a larger system and one of the features is to run a python script on the fly.
Testing with the following code snippet works great, but with recursive function only the first call is executed then crashes
//C++ code
int main()
{
Py_Initialize();
PyObject* m_pMainModule = PyImport_AddModule("__main__");
PyObject* m_pGlobalDict = PyModule_GetDict(m_pMainModule);
PyObject* m_pLocalDict = PyDict_New();
PyObject* fd = PyFile_FromString("script.py", "r");
if (fd == NULL)
{
PyErr_SetString(PyExc_IOError, "File not found");
}
PyObject * s = PyRun_File(PyFile_AsFile(fd), "script.py", Py_file_input, m_pGlobalDict, m_pLocalDict);
Py_XDECREF(fd);
Py_XDECREF(s);
if (PyErr_Occurred())
{
std::string result;
PyObject* ptype;
PyObject* pvalue;
PyObject* ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
PyErr_NormalizeException(&ptype, &pvalue, &ptraceback); // in order to convert pvalue from tuples to real objects
//Attach exception name first
PyObject* objectStr = PyObject_GetAttrString(ptype, "__name__");
result = PyString_AS_STRING(objectStr);
result = "Exception: " + result;;
Py_XDECREF(objectStr);
objectStr = PyObject_Str(pvalue);
if (objectStr != NULL) {
result = result + " was raised with error message : " + PyString_AS_STRING(objectStr);
Py_XDECREF(objectStr);
}
std::cout << result;
}
return 0;
}
Here is the python script I use
def fac(i):
print "Call to FAC(",i,") !"
if i <= 1:
return 1
else:
return i*fac(i-1)
print "Hello world"
print fac(4)
And here is the output
Hello world
Call to FAC( 4 ) !
Exception: NameError was raised with error message : global name 'fac' is not defined
While the expected output ( when run directly by invoking the script )
Hello world
Call to FAC( 4 ) !
Call to FAC( 3 ) !
Call to FAC( 2 ) !
Call to FAC( 1 ) !
24
Any ideas on how to proceed ?
Edit:
Platform: Windows 10 x64
Python: 2.7.14
Explanation:
In the following line, PyRun_File uses two different dictionaries; one for globals and one for locals.
PyObject * s = PyRun_File(PyFile_AsFile(fd), "script.py", Py_file_input, m_pGlobalDict, m_pLocalDict);
The strange part that fac function name is added to the local one of file.
I have no idea Why, But I really wanaa know.
So the recursive call fails as there is no fac inside locals of the function or the global.
Solution:
pass the same dictionary for both local and global
PyObject * s = PyRun_File(PyFile_AsFile(fd), "script.py", Py_file_input, m_pGlobalDict, m_pGlobalDict);
I am trying to make a Python iterator in boost::python. So I have a function
PyObject *my_iterator_next(MyIterator *iter) {
if (!iter->is_end()) {
return *(*iter)++;
}
else {
PyErr_SetNone(PyExc_StopIteration);
// this doesn't work either
// PyErr_SetString(PyExc_StopIteration, "end of collection");
return NULL;
}
}
In Python:
// x is MyContainer([1, 2, 3])
for x in my_container:
print(x)
And I get:
1
2
3
NoneTraceback (most recent call last):
File "main.py", line 6, in <module>
print(x)
StopIteration: end of collection
Also
it = my_collection.__iter__()
try:
it.__next__();
it.__next__();
it.__next__();
it.__next__();
except:
print("caught exception")
This code doesn't print anything, so no type of exception is caught.
Why?
After setting the Python exception, you must notify Boost.Python like this:
throw_error_already_set();
See http://www.boost.org/doc/libs/1_35_0/libs/python/doc/v2/errors.html
I am wrapping a C lib to python module using SWIG. But exception does not seem to be raised at the right place, I have a simple demo for this,
except_test.i
%module except_test
%{
#include "except_test.h"
#include <stdio.h>
%}
%{
static int flagged_exception = 0;
void throw_except()
{
flagged_exception = 1;
printf("flag set \n");
}
%}
%exception {
$action
printf("exception block\n");
if (flagged_exception) {
printf("before setstring\n");
PyErr_SetString(PyExc_RuntimeError, "test except");
printf("after setstring\n");
flagged_exception = 0;
}
}
%include "except_test.h"
except_test.c
#include "except_test.h"
int except_test(int a) {
if (a < 0) {
throw_except();
return 0;
} else{
return -1;
}
}
run_except.py
from except_test import *
import time
def test():
b = except_test(-1)
print 'b=', b
try:
test()
except:
print "caught exception"
for i in range(10):
print i
time.sleep(1)
Now if I run run_except.py
$python run_except.py
flag set
exception block
before setstring
after setstring
b= 0
0
Traceback (most recent call last):
File "run_except.py", line 15, in <module>
time.sleep(1)
RuntimeError: test except
as the output shows, the try/catch block didn't catch the exception.
Why is this? and How to avoid this?
Thanks,
You have to return NULL from a Python extension to have it notice the error immediately:
if (flagged_exception) {
PyErr_SetString(PyExc_RuntimeError, "test except");
flagged_exception = 0;
return NULL;
}
But using the generic SWIG macros will make the SWIG interface more portable to other languages.
You need to put SWIG_fail; right after PyErr_SetString. Alternatively, there is a convenient (and what's more important language-independent) macro SWIG_exception(SWIG_RuntimeError, "error message") wrapping PyErr_SetString and SWIG_fail.
I have a C dll that exports this function:
DECLDIR int runTest (char *historyPath, unsigned int candleNumber, void (*testUpdate)(double percentageOfTestCompleted), void (*testFinished)(void), char **error);
I'm trying to use the function inside my python script this way:
historyFilePath_c = c_char_p(historyFilePath)
candleNumber_c = c_int(1000)
error_c = c_char_p(300);
TEST_UPDATE = CFUNCTYPE(c_int, POINTER(c_double))
testUpdate_c = TEST_UPDATE(testUpdate)
TEST_FINISHED = CFUNCTYPE(c_int)
testFinished_c = TEST_FINISHED(testFinished)
astdll.runTest (historyFilePath_c, candleNumber_c, testUpdate_c, testFinished_c, byref(error_c))
def testUpdate(percentageOfTestCompleted):
print 'Test running ' , percentageOfTestCompleted[0]
return True
def testFinished():
print 'Test finished!!'
return True
I'm getting this error (several times because the callback function is running many times. I'll leave just the last error. All of them are the same)
Test running
Traceback (most recent call last):
File "_ctypes/callbacks.c", line 314, in 'calling callback function'
File "ast.py", line 67, in testUpdate
print 'Test running ' , percentageOfTestCompleted[0]
ValueError: NULL pointer access
Test finished!!
The testUpdate callback takes a double by value, not a pointer, and both callbacks return void, i.e. None.
candleNumber_c is unnecessary, especially if you declare argtypes. The same applies to historyFilePath_c. A Python string already contains a pointer to a null-terminated string, which is what the c_char_p constructor uses.
Why do you initialize error_c to the address 300?
Here's an example that should help:
tmp.py:
from ctypes import *
lib = CDLL('./tmp.so')
c_test_update_t = CFUNCTYPE(None, c_double)
c_test_finished_t = CFUNCTYPE(None)
run_test = lib.runTest
run_test.argtypes = [
c_char_p, c_uint, c_test_update_t, c_test_finished_t,
POINTER(c_char_p)]
def test_update(percent_completed):
print 'percent completed:', percent_completed
test_update_c = c_test_update_t(test_update)
def test_finished():
print 'test finished'
test_finished_c = c_test_finished_t(test_finished)
error = c_char_p()
result = run_test(
'path/to/history', 1000, test_update_c, test_finished_c,
byref(error))
print result
print error.value
tmp.c:
#include <stdio.h>
char error_string[] = "error string";
int runTest(
char *historyPath,
unsigned int candleNumber,
void (*testUpdate)(double percentageOfTestCompleted),
void (*testFinished)(void),
char **error)
{
printf("historyPath: %s\n", historyPath);
printf("candleNumber: %d\n", candleNumber);
testUpdate(0.0);
testFinished();
*error = error_string;
return 0;
}
(Just a simple test lib, compiled with gcc on Linux)
Output:
historyPath: path/to/history
candleNumber: 1000
percent completed: 0.0
test finished
0
error string