I'm writing following python script:
import sqlite3
import sys
if len(sys.argv) < 2:
print("Error: You must supply at least SQL script.")
print("Usage: %s table.db ./sql-dump.sql" % (sys.argv[0]))
sys.exit(1)
script_path = sys.argv[1]
if len(sys.argv) == 3:
db = sys.argv[2]
else:
db = ":memory:"
try:
con = sqlite3.connect(db)
with con:
cur = con.cursor()
with open(script_path,'rb') as f:
cur.executescript(f.read())
except sqlite3.Error as err:
print("Error occured: %s" % err)
I saved this program as sqlite_import.py.
I have a database file named test.db and a SQL file world.sql.
Now I tried to run program as:
sqlite_import.py test.db world.sql
But it shows me error like following:
Traceback (most recent call last):
File "C:\Users\Jarvis\OneDrive\Documents\Python\Python Data Visualization Cookbook, 2E\2 Knowing Your Data\sqlite_import.py", line 21, in <module>
cur.executescript(f.read())
ValueError: script argument must be unicode.
Help me to fix this.
You opened the script file as binary:
with open(script_path,'rb') as f:
This produces a b'...' bytes value, not a Unicode str object. Remove the b, and perhaps add an encoding argument to specific what codec to use to decode the data with.
Related
Running some code with Python3 reports the below error:
$ python3 report.py --regions ap-southeast-2 --file file.csv
File "report.py", line 51
except Exception, e:
^
SyntaxError: invalid syntax
Research indicates that this deprecated syntax. I have found conflicting information on how I may fix this.
I have tried to engage python3 syntax, which I believe would be to switch
try:
f = file(filepath, 'wt')
except Exception, e:
f = None
sys.stderr.write ('Could not open file %s. reason: %s\n' % (filepath, e))
To:
try:
f = file(filepath, 'wt')
except:
f = None
sys.stderr.write ('Could not open file %s. reason: %s\n' % (filepath, e))
What happens then is that I get errors relating to the "e" being missing... so I'm unsure how best and easiest to resolve the syntax issues between the two versions. Can you help or advise? Thanks!
If you really want a method that is compatible for both Python2 and Python3,
Then try something like this:
import sys
try:
### your filepath code goes here or any other code
except Exception:
tb, err = sys.exc_info()[:2]
print(err)
Using exc_info() here is good as it provides you with a tuple of information on what the error and the traceback to that faulting code, specifically the (type, value, traceback). In this case, you are getting back the traceback and the error (value).
Currently I call an existing existing LibreOffice macro with this:
def OnLOtimestamp(self):
try:
pid= Popen(['lowriter '"'"'vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user'"'"],shell=True).pid
except OSError, e:
self.notify_show("Timestamp Error",str(e))
self.ma2.SetLabel("Macro timestamp")
self.database['Time_stamp'] = self.database['Time_stamp'] + 1
The key bit being the Popen call, where the macro name is fs2TimeStamp.py and the function is fs2_TimeStamp but this feels like a cop out and I would rather perform a direct call via Uno.
My research suggests that I may well need to use MasterScriptProvider, XscriptProvider and XscriptInvocation but trying to decipher the Uno API is like swimming through treacle.
Has anybody got a code sample of calling an existing macro in Libreoffice, using Uno?
Edit:
So far the answer appears to be No!
This is the current state of play.
#!/usr/bin/python3
# -*- coding: utf-8 -*-
##
# a python script to run a libreoffice python macro externally
#
import uno
from com.sun.star.connection import NoConnectException
from com.sun.star.uno import RuntimeException
from com.sun.star.uno import Exception
from com.sun.star.lang import IllegalArgumentException
def test2(*args):
localContext = uno.getComponentContext()
localsmgr = localContext.ServiceManager
resolver = localsmgr.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext )
try:
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
except NoConnectException as e:
print ("LibreOffice is not running or not listening on the port given - ("+e.Message+")")
return
except IllegalArgumentException as e:
print ("Invalid argument given - ( "+ e.Message+ ")")
return
except RuntimeException as e:
print ("An unknown error occurred: " + e.Message)
return
servmgr = ctx.ServiceManager
desktop = servmgr.createInstanceWithContext( "com.sun.star.frame.Desktop",ctx)
model = desktop.getCurrentComponent()
# scriptP = model.getScriptProvider()
# print("scriptP", scriptP)
scriptx = model.getScriptProvider().getScript('vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user')
print("scriptx", scriptx)
try:
scriptx.invoke("",0,0)
except IllegalArgumentException as e:
print ("The command given is invalid ( "+ e.Message+ ")")
return
except RuntimeException as e:
print("An unknown error occurred: " + e.Message)
return
except Exception as e:
print ("Script error ( "+ e.Message+ ")")
print(e)
return
except:
print("Error")
return(None)
test2()
This code works happily when invoked as a macro within Libreoffice and scriptx prints out as:
scriptx <pythonscript.PythonScript object at 0x7fa2879c42e8>
however when run from the command line the script does nothing and scriptx prints out as:
scriptx pyuno object (com.sun.star.script.provider.XScript)0x1e749d8{, supportedInterfaces={com.sun.star.lang.XTypeProvider,com.sun.star.script.provider.XScript}}
So either getScriptProvider or getScript are not being provided with something that they require. I am currently at a loss as to what is missing and yet, I feel in my bones that I'm close to a solution.
Can anyone see where I have made a mistake?
Finally, I have a working solution. Ding! Dong!
#!/usr/bin/python3
# -*- coding: utf-8 -*-
##
# a python script to run a libreoffice python macro externally
#
import uno
from com.sun.star.connection import NoConnectException
from com.sun.star.uno import RuntimeException
from com.sun.star.uno import Exception
from com.sun.star.lang import IllegalArgumentException
def test2(*args):
localContext = uno.getComponentContext()
localsmgr = localContext.ServiceManager
resolver = localsmgr.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext )
try:
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
except NoConnectException as e:
print ("LibreOffice is not running or not listening on the port given - ("+e.Message+")")
return
msp = ctx.getValueByName("/singletons/com.sun.star.script.provider.theMasterScriptProviderFactory")
sp = msp.createScriptProvider("")
scriptx = sp.getScript('vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user')
try:
scriptx.invoke((), (), ())
except IllegalArgumentException as e:
print ("The command given is invalid ( "+ e.Message+ ")")
return
except RuntimeException as e:
print("An unknown error occurred: " + e.Message)
return
except Exception as e:
print ("Script error ( "+ e.Message+ ")")
return(None)
test2()
Note: For clarity the existing python script is called fs2TimeStamp.py, it contains 1 (one) function defined as def fs2_TimeStamp(*args):
See the line:
scriptx = sp.getScript('vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user')
and it is stored in $HOME/.config/libreoffice/4/user/Scripts/python
For this solution to work, libreoffice must be running in listening mode, so start libreoffice with a command like:
soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore
OR
nohup soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore &
Alternatively you could use the more direct method (for writer in this example):
lowriter "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp"
OR
nohup lowriter "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp" &
Also note that you must run the script with python3
This is a simpler and more generic version of the first answer, which was very much slanted to my specific problem, which called a macro, which then talked to another program via TCP before inserting text sent via the TCP connection.
This version will run standalone and should be instantly replicable.
As before you must start LibreOffice writer in listening mode (see previous answer for options):
lowriter "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp"
This is the external python routine, which must be run with python3:
#!/usr/bin/python3
# -*- coding: utf-8 -*-
##
# a python script to run a libreoffice python macro externally
# NOTE: for this to run start libreoffice in the following manner
# soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore
# OR
# nohup soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore &
#
import uno
from com.sun.star.connection import NoConnectException
from com.sun.star.uno import RuntimeException
from com.sun.star.uno import Exception
from com.sun.star.lang import IllegalArgumentException
def uno_directmacro(*args):
localContext = uno.getComponentContext()
localsmgr = localContext.ServiceManager
resolver = localsmgr.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext )
try:
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
except NoConnectException as e:
print ("LibreOffice is not running or not listening on the port given - ("+e.Message+")")
return
msp = ctx.getValueByName("/singletons/com.sun.star.script.provider.theMasterScriptProviderFactory")
sp = msp.createScriptProvider("")
scriptx = sp.getScript('vnd.sun.star.script:directmacro.py$directmacro?language=Python&location=user')
try:
scriptx.invoke((), (), ())
except IllegalArgumentException as e:
print ("The command given is invalid ( "+ e.Message+ ")")
return
except RuntimeException as e:
print("An unknown error occurred: " + e.Message)
return
except Exception as e:
print ("Script error ( "+ e.Message+ ")")
print(e)
return
return(None)
uno_directmacro()
This is the LibreOffice python macro directmacro.py which should be located in:
$HOME/.config/libreoffice/4/user/Scripts/python
(assuming use of LibreOffice version 4 here)
The directmacro.py macro:
#!/usr/bin/python
class FontSlant():
from com.sun.star.awt.FontSlant import (NONE, ITALIC,)
def directmacro(*args):
#get the doc from the scripting context which is made available to all scripts
desktop = XSCRIPTCONTEXT.getDesktop()
model = desktop.getCurrentComponent()
text = model.Text
tRange = text.End
cursor = desktop.getCurrentComponent().getCurrentController().getViewCursor()
doc = XSCRIPTCONTEXT.getDocument()
parentwindow = doc.CurrentController.Frame.ContainerWindow
# your cannot insert simple text and text into a table with the same method
# so we have to know if we are in a table or not.
# oTable and oCurCell will be null if we are not in a table
oTable = cursor.TextTable
oCurCell = cursor.Cell
insert_text = "This is text inserted into a LibreOffice Document\ndirectly from a macro called externally"
Text_Italic = FontSlant.ITALIC
Text_None = FontSlant.NONE
cursor.CharPosture=Text_Italic
if oCurCell == None: # Are we inserting into a table or not?
text.insertString(cursor, insert_text, 0)
else:
cell = oTable.getCellByName(oCurCell.CellName)
cell.insertString(cursor, insert_text, False)
cursor.CharPosture=Text_None
return None
I have custom python library installed on my NAS which has this code:
try:
i = open("/volume1/web/python/session.txt")
try:
a = str(i.read())
except Exception as err:
a = "Exception: {0}".format(err)
finally:
i.close()
except Exception as err:
a = "Exception: {0}".format(err)
return a
This code is invoked by app.py which saves returned text to file.
When app.py is run from SSH, it saves the content of session.txt as intended.
But when app.py is run as task (run.sh), it saves exception "'ascii' codec can't decode byte 0xc3 in position 12: ordinal not in range(128)". Yeah, the file contains "รก" in position 12, so I add encoding="utf-8" to open() function. And this fails (does not save anything, so I don't even know if and what error it threw.)
QUESTION
Any ideas how to safely open and read text file by python library (when by regular script, runs correctly even if it is run as task)?
FILES involved
library:
class API():
def __init__(self, dir):
self.DIR = dir
def CheckTokens(self):
print("\nChecking all saved tokens")
try:
i = open(os.path.join(self.DIR, "session.txt"), encoding='utf-8')
try:
a = i.read()
except Exception as err:
a = "Exception: {0}".format(err)
finally:
i.close()
except Exception as err:
a = "Exception: {0}".format(err)
return a
app.py:
import os.path
from time import localtime, strftime
from APIfile import API
if __name__ == "__main__":
LOCAL_DIR = os.path.realpath(os.path.join(os.getcwd(), os.path.dirname(__file__)))
def write(text, type):
time = localtime()
time = "%s#%s" % (strftime("%Y%m%d", time), strftime("%H%M%S", time))
filename = "%s%s.txt" % (time, ("_%s" % type if type != "JSON" else ""))
f = open(os.path.join(LOCAL_DIR, "logs", filename), "a")
f.write(str(text))
f.close()
INSTANCE = API(LOCAL_DIR)
tokens = INSTANCE.CheckTokens()
write(tokens, "DEBUG")
run.sh:
python3 /volume1/web/python/app.py
session.txt and logs folder is located in the same directory as app.py
OUTCOMES
Run from SSH python3 /volume1/web/python/app.py -> saves content of session.txt to log file
Run from SSH /volume1/web/python/run.sh -> does the same as 1.
Run run.sh file as task (something like cron job) -> does not create log file
When I remove encoding='utf-8' statement from open function, in SSH run correctly, in task throws an exception ('ascii' codec can't decode byte 0xc3 in position 12: ordinal not in range(128))
I'm a little puzzled, but here's my guess: open() defaults to the shell's preferred encoding, and your NAS's locale is somehow set differently than your local shell's. I think that encoding=utf-8 is the fix for your current issue.
Now, you say that doesn't work either. I'm not sure what you mean by "does not save anything" -- my guess here is that your script needs to save its output to a file, and the NAS daemon doesn't have permissions to write to that file. You don't say what the call responsible for the saving to disk actually looks like, but I believe that's a separate issue from the problem with your encoding.
Based on an example for forking, I build up this little script:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sqlanydb
import os
def child():
conn = sqlanydb.connect(uid='dba', pwd='sql', eng='somedb_IQ', dbn='somedb')
curs = conn.cursor()
curs.execute("""SELECT * FROM foobaa;""")
os.exit(0)
def parent():
while True:
newpid = os.fork()
if newpid == 0:
child()
else:
pids = (os.getpid(), newpid)
print "parent: %d, child: %d" % pids
if raw_input( ) == 'q': break
parent()
The intention is to do the database action inside a seperate process (big goal later is to run a huge number of queries at the same time).
But when running the script, I'm getting:
parent: 20580, child: 20587
Traceback (most recent call last):
File "connectiontest.py", line 25, in <module>
parent()
File "connectiontest.py", line 19, in parent
child()
File "connectiontest.py", line 8, in child
conn = sqlanydb.connect(uid='dba', pwd='sql', eng='somedb_IQ', dbn='somedb')
File "/usr/local/lib/python2.6/dist-packages/sqlanydb.py", line 461, in connect
return Connection(args, kwargs)
File "/usr/local/lib/python2.6/dist-packages/sqlanydb.py", line 510, in __init__
self.handleerror(*error)
File "/usr/local/lib/python2.6/dist-packages/sqlanydb.py", line 520, in handleerror
eh(self, None, errorclass, errorvalue)
File "/usr/local/lib/python2.6/dist-packages/sqlanydb.py", line 342, in standardErrorHandler
raise errorclass(errorvalue)
sqlanydb.OperationalError: Failed to initialize connection object
What did I might miss?
Since Sybase IQ is based on Sybase ASA, are you sure that you're using the proper keys for the credentials? This (albeit, old) documentation looks like it wants DSN and DSF instead of ENG and DBN.
http://dcx.sybase.com/1101/en/dbprogramming_en11/python-writing-open.html
The issue seems to don't happen by moving import sqlanydb into child()-methode. So it would look something like:
def child():
import sqlanydb
conn = sqlanydb.connect(uid='dba', pwd='sql', dsn='some_db')
curs = conn.cursor()
curs.execute("""SELECT * FROM SA100_1_1;""")
curs.close()
conn.close()
You need to hack the sqlanydb source to print out the actual error being seen. Whatever the problem is is being masked by a generic OperationalError which is not giving enough information to fix the problem. Line 510 is where you need to add a couple prints to figure out what is (not) going on.
I was trying out a few python test scripts with sqlite3.
Here is the script that I wrote
#!/usr/bin/env python
from sqlite3 import dbapi2 as sqlite
from sys import argv,exit
db_name = "filenames.db"
def define_db():
try:
conn = sqlite.connect(db_name)
except IOError as e:
print "problem while creating/connecting the db:",e.args[0]
exit(1)
return conn
def write_db(conn,cursor,fni):
conn.execute("CREATE TABLE IF NOT EXISTS file (filenames TEXT UNIQUE)")
query = "INSERT OR REPLACE INTO file VALUES($filenames)"
cursor.execute(query,[fni])
cursor.close()
conn.commit()
conn.close()
print fni,"should now be in the db"
exit(0)
if __name__ == "__main__":
if len(argv) == 2:
etag = argv[1]
else:
print "no argument given - stopping now"
exit(1)
conn = define_db()
cursor = conn.cursor()
write_db(conn,cursor,fni)
I keep getting this error and was not able to solve it.
Traceback (most recent call last):
File "blah.py", line 37, in <module>
write_db(conn,cursor,fni)
NameError: name 'fni' is not defined
Any idea what the problem is.
At this moment I use python 2.7.3
The last line of your script refers to a name fni that is not defined.
You have not defined the variable "fni", but you are using it.
Static analysis tools like pyflakes or pylint can be useful to catch silly errors like this
If you wrote the bulk of the code in a function (so it doesn't assume blub is a global variable, which don't make pyflakes/pylint complain):
def main():
if len(argv) == 2:
blub = argv[1]
else:
print "no argument given - stopping now"
exit(1)
conn = define_db()
cursor = conn.cursor()
write_db(conn,cursor,fni)
if __name__ == "__main__":
main()
...then you would get a pair of errors, which points out exactly what the error is (you stored the argument in blub, but tried to access it with fni):
$ pip install pyflakes
$ pyflakes example.py
example.py:30: local variable 'blub' is assigned to but never used
example.py:37: undefined name 'fni'