I'm relatively new in python, but I'm trying to learn hard. For the last couple of days tried to solve an error on this script :
import requests
import subprocess
import json
import sys
import multiprocessing
import time
import random
channel_url = "gaming.youtube.com/game/"
processes = [5]
def get_channel():
# Reading the channel name - passed as an argument to this script
if len(sys.argv) >= 2:
global channel_url
channel_url += sys.argv[1]
else:
print "An error has occurred while trying to read arguments. Did you specify the channel?"
sys.exit(1)
def get_proxies():
# Reading the list of proxies
try:
lines = [line.rstrip("\n") for line in open("proxylist.txt")]
except IOError as e:
print "An error has occurred while trying to read the list of proxies: %s" % e.strerror
sys.exit(1)
return lines
def get_url():
# Getting the json with all data regarding the stream
try:
response = subprocess.Popen(
["livestreamer.exe", "--http-header", "Client-ID=ewvlchtxgqq88ru9gmfp1gmyt6h2b93",
channel_url, "-j"], stdout=subprocess.PIPE).communicate()[0]
except subprocess.CalledProcessError:
print "An error has occurred while trying to get the stream data. Is the channel online? Is the channel name correct?"
sys.exit(1)
except OSError:
print "An error has occurred while trying to use livestreamer package. Is it installed? Do you have Python in your PATH variable?"
# Decoding the url to the worst quality of the stream
try:
url = json.loads(response)['streams']['audio_only']['url']
except:
try:
url = json.loads(response)['streams']['worst']['url']
except (ValueError, KeyError):
print "An error has occurred while trying to get the stream data. Is the channel online? Is the channel name correct?"
sys.exit(1)
return url
def open_url(url, proxy):
# Sending HEAD requests
while True:
try:
with requests.Session() as s:
response = s.head(url, proxies=proxy)
print "Sent HEAD request with %s" % proxy["http"]
time.sleep(20)
except requests.exceptions.Timeout:
print " Timeout error for %s" % proxy["http"]
except requests.exceptions.ConnectionError:
print " Connection error for %s" % proxy["http"]
def prepare_processes():
global processes
proxies = get_proxies()
n = 0
if len(proxies) < 1:
print "An error has occurred while preparing the process: Not enough proxy servers. Need at least 1 to function."
sys.exit(1)
for proxy in proxies:
# Preparing the process and giving it its own proxy
processes.append(
multiprocessing.Process(
target=open_url, kwargs={
"url": get_url(), "proxy": {
"http": proxy}}))
print '.',
print ''
if __name__ == "__main__":
print "Obtaining the channel..."
get_channel()
print "Obtained the channel"
print "Preparing the processes..."
prepare_processes()
print "Prepared the processes"
print "Booting up the processes..."
# Timer multiplier
n = 8
# Starting up the processes
for process in processes:
time.sleep(random.randint(1, 5) * n)
process.daemon = True
process.start()
if n > 1:
n -= 1
# Running infinitely
while True:
time.sleep(1)
ERROR :
python test999.py UCbadKBJT1bE14AtfBnsw27g
Obtaining the channel...
Obtained the channel
Preparing the processes...
An error has occurred while trying to use livestreamer package. Is it installed? Do you have Python in your PATH variable?
Traceback (most recent call last):
File "test999.py", line 115, in <module>
prepare_processes()
File "test999.py", line 103, in prepare_processes
"url": get_url(), "proxy": {
File "test999.py", line 67, in get_url
url = json.loads(response)['streams']['worst']['url']
UnboundLocalError: local variable 'response' referenced before assignment
I had tried on windows, installed all modules (and updated them ) livestreamer, rtmdump, dlls and other needed binary files.
On linux : installed livestreamer, requests, json and all neede modules. Still can't resolve it. Please help !
response = subprocess.Popen() in try clause. When it fails, no response lkocal variable. You should assign response before try,and process None that subporcess.Popen() may return.
Related
Would anyone know when using python-shell if you can keep the python script running when its called from Javascript? Maybe call a method from a Python class from Javascript using python-shell?
For example this code works but it kills the Python script every time its called.
let {PythonShell} = require("python-shell");
let options = {
mode: "text",
args: ["read", "12345:2", "analogInput", "2"],
};
PythonShell.run("bac0Worker.py", options, function (err, results) {
if (err){
console.log(err)
console.log("An error happened")
}else{
// results is an array consisting of messages collected during execution
console.log("results: ", results);
console.log("Python Script Finished");
}
})
On the Python side I am experimenting with a package called BAC0 for BACnet systems. What I am trying to do is figure out if I can keep the BAC0 script running, I think under the hood of BAC0 there's alot of process going on where it may create lots of unnecessary traffic on a BACnet network if the script is started/stopped a lot.
bac0Worker.py
import sys
import BAC0
BAC0.log_level('silence')
bacnet = BAC0.lite()
def do_things(address,object_type,object_instance):
try:
read_vals = f'{address} {object_type} {object_instance} presentValue'
read_result = bacnet.read(read_vals)
if isinstance(read_result, str):
pass
else:
read_result = round(read_result,2)
print(read_result)
except Exception as error:
print("read error")
def main():
# args from Javascript
first = sys.argv[1]
second = sys.argv[2]
third = sys.argv[3]
fourth = sys.argv[4]
# get sensor data
do_things(second, third, fourth)
# all done
bacnet.disconnect()
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("bacnet app error")
bacnet.disconnect()
Like I mentioned the code works if I run from Node I can see the sensor value from the BACnet network 71.7 degrees.
C:\Program Files\nodejs\node.exe .\start.js
results: (1) ['71.7']
Python Script Finished
Hopefully this makes sense, sorry for the odd question but curious if anyone would have any tips for how to keep the Python script running where I could then just pass values (sensor & device addressing info) to BAC0 script to request a BACnet read from the network. In the main function of the Python file if I replace this:
# all done
bacnet.disconnect()
With:
while True:
pass
This would keep the Python file alive I just don't know how to pass values to a live Python script using python-shell. Thanks for any tips not a lot of wisdom here best practices. Curious if I should change my do_things function into a Python class with a method called do_things where then this class could get called from Javascript?
.js
const { PythonShell } = require("python-shell");
let pyshell = new PythonShell("my_script.py");
pyshell.send("hello");
pyshell.on("message", function (message) {
console.log(message);
});
setTimeout(() => {
pyshell.send("Another Hello");
}, 3000);
const end = () => {
pyshell.end(function (err, code, signal) {
if (err) throw err;
console.log("finished");
});
};
// end();
.py
import sys
def log():
print('hello from python')
while True:
inp = input(" ")
if inp=="hello":
log()
this will keep the process running unless you call the end method from the nodejs or some error occurs. This may incur performance issues because python thingy will be sharing same resources as your nodejs. A better way to do this would be to use a microservice like deploy some Rest Api which handles python specific tasks.
Just for fun Ill post what I was able to learn from #sandeep (thanks a million btw), this works. I can print BACnet sensor values which are gathered from Python from running a javascript file on 60 second setInterval(pythonDo, 60000); with keeping the Python file alive:
C:\Program Files\nodejs\node.exe .\start.js
BAC0 start success
68.47
68.43
68.29
68.25
start.js
const { PythonShell } = require("python-shell");
let pyshell = new PythonShell("bac0Worker.py");
function pythonDo() {
pyshell.send("read 12345:2 analogInput 2");
}
pyshell.on("message", function (message) {
console.log(message);
});
var data = setInterval(pythonDo, 60000);
bac0Worker.py
import sys
import BAC0
BAC0.log_level('silence')
bacnet = BAC0.lite()
print("BAC0 start success")
# available BACnet actions
SUBSTRING_READ = "read"
SUBSTRING_WRITE = "write"
SUBSTRING_RELEASE = "release"
def bac0_worker(action, BACnet_request, **kwargs):
value = kwargs.get('value', None)
priority = kwargs.get('priority', None)
if action == "read":
try:
read_vals = f'{BACnet_request} presentValue'
read_result = bacnet.read(read_vals)
if isinstance(read_result, str):
pass
else:
read_result = round(read_result,2)
print(read_result)
except Exception as error:
print("read error")
elif action == "write":
try:
write_vals = f'{BACnet_request} presentValue {value} - {priority}'
bacnet.write(write_vals)
print(write_vals)
except Exception as error:
print("write error")
elif action == "release":
try:
release_vals = f'{BACnet_request} presentValue null - {priority}'
bacnet.write(release_vals)
print(release_vals)
except Exception as error:
print("release error")
else:
return "server error on BACnet opts"
def main():
while True:
# from javascript
inp = input(" ")
for word in inp.split():
if SUBSTRING_READ == word:
stripped_string = inp.replace(SUBSTRING_READ + " ", "")
# do a BACnet READ request with BAC0
bac0_worker(SUBSTRING_READ,stripped_string)
if SUBSTRING_WRITE == word:
print("WRITE Match Found")
if SUBSTRING_RELEASE == word:
print("RELEASE Match Found")
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("bacnet app error")
bacnet.disconnect()
I am trying to use the anaconda prompt to run my python script. It runs smoothly on the first call but stops there. I tried on Spyder, it works but I would like it to work on anaconda prompt or command line. Any reason why?
from decompress import decompress
from reddit import reddit
from clean import clean
from wikipedia import wikipedia
def main():
dir_of_file = r"D:\Users\Jonathan\Desktop\Reddit Data\Demo\\"
print('0. Path: ' + dir_of_file)
reddit_repo = reddit()
wikipedia_repo = wikipedia()
pattern_filter = "*2007*&*2008*"
print('1. Creating data lake')
reddit_repo.download_files(pattern_filter,"https://files.pushshift.io/reddit/submissions/",dir_of_file,'s')
reddit_repo.download_files(pattern_filter,"https://files.pushshift.io/reddit/comments/",dir_of_file,'c')
if __name__ == "__main__":
main()
The RS Downloaded is this line of code being ran:
reddit_repo.download_files(pattern_filter,"https://files.pushshift.io/reddit/submissions/",dir_of_file,'s')
Update:
Added the class/function
class reddit:
def multithread_download_files_func(self,list_of_file):
filename = list_of_file[list_of_file.rfind("/")+1:]
path_to_save_filename = self.ptsf_download_files + filename
if not os.path.exists(path_to_save_filename):
data_content = None
try:
request = urllib.request.Request(list_of_file)
response = urllib.request.urlopen(request)
data_content = response.read()
except urllib.error.HTTPError:
print('HTTP Error')
except Exception as e:
print(e)
if data_content:
with open(path_to_save_filename, 'wb') as wf:
wf.write(data_content)
print(self.present_download_files + filename)
def download_files(self,filter_files_df,url_to_download_df,path_to_save_file_df,prefix):
#do some processing
matching_fnmatch_list.sort()
p = ThreadPool(200)
p.map(self.multithread_download_files_func, matching_fnmatch_list)
It was the download which was taking a lot of time. I have changed network and it worked as expected. So there is no issue with cmd or anaconda prompt
Currently I call an existing existing LibreOffice macro with this:
def OnLOtimestamp(self):
try:
pid= Popen(['lowriter '"'"'vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user'"'"],shell=True).pid
except OSError, e:
self.notify_show("Timestamp Error",str(e))
self.ma2.SetLabel("Macro timestamp")
self.database['Time_stamp'] = self.database['Time_stamp'] + 1
The key bit being the Popen call, where the macro name is fs2TimeStamp.py and the function is fs2_TimeStamp but this feels like a cop out and I would rather perform a direct call via Uno.
My research suggests that I may well need to use MasterScriptProvider, XscriptProvider and XscriptInvocation but trying to decipher the Uno API is like swimming through treacle.
Has anybody got a code sample of calling an existing macro in Libreoffice, using Uno?
Edit:
So far the answer appears to be No!
This is the current state of play.
#!/usr/bin/python3
# -*- coding: utf-8 -*-
##
# a python script to run a libreoffice python macro externally
#
import uno
from com.sun.star.connection import NoConnectException
from com.sun.star.uno import RuntimeException
from com.sun.star.uno import Exception
from com.sun.star.lang import IllegalArgumentException
def test2(*args):
localContext = uno.getComponentContext()
localsmgr = localContext.ServiceManager
resolver = localsmgr.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext )
try:
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
except NoConnectException as e:
print ("LibreOffice is not running or not listening on the port given - ("+e.Message+")")
return
except IllegalArgumentException as e:
print ("Invalid argument given - ( "+ e.Message+ ")")
return
except RuntimeException as e:
print ("An unknown error occurred: " + e.Message)
return
servmgr = ctx.ServiceManager
desktop = servmgr.createInstanceWithContext( "com.sun.star.frame.Desktop",ctx)
model = desktop.getCurrentComponent()
# scriptP = model.getScriptProvider()
# print("scriptP", scriptP)
scriptx = model.getScriptProvider().getScript('vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user')
print("scriptx", scriptx)
try:
scriptx.invoke("",0,0)
except IllegalArgumentException as e:
print ("The command given is invalid ( "+ e.Message+ ")")
return
except RuntimeException as e:
print("An unknown error occurred: " + e.Message)
return
except Exception as e:
print ("Script error ( "+ e.Message+ ")")
print(e)
return
except:
print("Error")
return(None)
test2()
This code works happily when invoked as a macro within Libreoffice and scriptx prints out as:
scriptx <pythonscript.PythonScript object at 0x7fa2879c42e8>
however when run from the command line the script does nothing and scriptx prints out as:
scriptx pyuno object (com.sun.star.script.provider.XScript)0x1e749d8{, supportedInterfaces={com.sun.star.lang.XTypeProvider,com.sun.star.script.provider.XScript}}
So either getScriptProvider or getScript are not being provided with something that they require. I am currently at a loss as to what is missing and yet, I feel in my bones that I'm close to a solution.
Can anyone see where I have made a mistake?
Finally, I have a working solution. Ding! Dong!
#!/usr/bin/python3
# -*- coding: utf-8 -*-
##
# a python script to run a libreoffice python macro externally
#
import uno
from com.sun.star.connection import NoConnectException
from com.sun.star.uno import RuntimeException
from com.sun.star.uno import Exception
from com.sun.star.lang import IllegalArgumentException
def test2(*args):
localContext = uno.getComponentContext()
localsmgr = localContext.ServiceManager
resolver = localsmgr.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext )
try:
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
except NoConnectException as e:
print ("LibreOffice is not running or not listening on the port given - ("+e.Message+")")
return
msp = ctx.getValueByName("/singletons/com.sun.star.script.provider.theMasterScriptProviderFactory")
sp = msp.createScriptProvider("")
scriptx = sp.getScript('vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user')
try:
scriptx.invoke((), (), ())
except IllegalArgumentException as e:
print ("The command given is invalid ( "+ e.Message+ ")")
return
except RuntimeException as e:
print("An unknown error occurred: " + e.Message)
return
except Exception as e:
print ("Script error ( "+ e.Message+ ")")
return(None)
test2()
Note: For clarity the existing python script is called fs2TimeStamp.py, it contains 1 (one) function defined as def fs2_TimeStamp(*args):
See the line:
scriptx = sp.getScript('vnd.sun.star.script:fs2TimeStamp.py$fs2_TimeStamp?language=Python&location=user')
and it is stored in $HOME/.config/libreoffice/4/user/Scripts/python
For this solution to work, libreoffice must be running in listening mode, so start libreoffice with a command like:
soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore
OR
nohup soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore &
Alternatively you could use the more direct method (for writer in this example):
lowriter "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp"
OR
nohup lowriter "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp" &
Also note that you must run the script with python3
This is a simpler and more generic version of the first answer, which was very much slanted to my specific problem, which called a macro, which then talked to another program via TCP before inserting text sent via the TCP connection.
This version will run standalone and should be instantly replicable.
As before you must start LibreOffice writer in listening mode (see previous answer for options):
lowriter "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp"
This is the external python routine, which must be run with python3:
#!/usr/bin/python3
# -*- coding: utf-8 -*-
##
# a python script to run a libreoffice python macro externally
# NOTE: for this to run start libreoffice in the following manner
# soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore
# OR
# nohup soffice "--accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;" --writer --norestore &
#
import uno
from com.sun.star.connection import NoConnectException
from com.sun.star.uno import RuntimeException
from com.sun.star.uno import Exception
from com.sun.star.lang import IllegalArgumentException
def uno_directmacro(*args):
localContext = uno.getComponentContext()
localsmgr = localContext.ServiceManager
resolver = localsmgr.createInstanceWithContext("com.sun.star.bridge.UnoUrlResolver", localContext )
try:
ctx = resolver.resolve("uno:socket,host=localhost,port=2002;urp;StarOffice.ComponentContext")
except NoConnectException as e:
print ("LibreOffice is not running or not listening on the port given - ("+e.Message+")")
return
msp = ctx.getValueByName("/singletons/com.sun.star.script.provider.theMasterScriptProviderFactory")
sp = msp.createScriptProvider("")
scriptx = sp.getScript('vnd.sun.star.script:directmacro.py$directmacro?language=Python&location=user')
try:
scriptx.invoke((), (), ())
except IllegalArgumentException as e:
print ("The command given is invalid ( "+ e.Message+ ")")
return
except RuntimeException as e:
print("An unknown error occurred: " + e.Message)
return
except Exception as e:
print ("Script error ( "+ e.Message+ ")")
print(e)
return
return(None)
uno_directmacro()
This is the LibreOffice python macro directmacro.py which should be located in:
$HOME/.config/libreoffice/4/user/Scripts/python
(assuming use of LibreOffice version 4 here)
The directmacro.py macro:
#!/usr/bin/python
class FontSlant():
from com.sun.star.awt.FontSlant import (NONE, ITALIC,)
def directmacro(*args):
#get the doc from the scripting context which is made available to all scripts
desktop = XSCRIPTCONTEXT.getDesktop()
model = desktop.getCurrentComponent()
text = model.Text
tRange = text.End
cursor = desktop.getCurrentComponent().getCurrentController().getViewCursor()
doc = XSCRIPTCONTEXT.getDocument()
parentwindow = doc.CurrentController.Frame.ContainerWindow
# your cannot insert simple text and text into a table with the same method
# so we have to know if we are in a table or not.
# oTable and oCurCell will be null if we are not in a table
oTable = cursor.TextTable
oCurCell = cursor.Cell
insert_text = "This is text inserted into a LibreOffice Document\ndirectly from a macro called externally"
Text_Italic = FontSlant.ITALIC
Text_None = FontSlant.NONE
cursor.CharPosture=Text_Italic
if oCurCell == None: # Are we inserting into a table or not?
text.insertString(cursor, insert_text, 0)
else:
cell = oTable.getCellByName(oCurCell.CellName)
cell.insertString(cursor, insert_text, False)
cursor.CharPosture=Text_None
return None
I am building or trying to build a python script which check's a list of ip addresses (ips.txt) for a specific program using the wmi python module. However, no matter how I handle the exceptions on assets with no RPC service running the script stops running on an error. I am using python 2.7.5
Can I catch and pass the error's to proceed?
Can I catch the error and print or return a note that the ip was not alive or rpc was not running?
Thank you in advance
Here is my code:
import wmi
list = open("ips.txt")
for line in list.readlines():
asset = line.strip('\n')
c = wmi.WMI(asset)
try:
for process in c.Win32_Process (name="SbClientManager.exe"):
print asset, process.ProcessId, process.Name
except Exception:
pass
I have tried handling the exceptions in multiple way's to continue parsing my list, but the script continues to error out with the following:
Traceback (most recent call last):
File ".\check_service.py", line 12, in <module>
c = wmi.WMI(asset)
File "C:\Python27\lib\site-packages\wmi.py", line 1290, in connect
handle_com_error ()
File "C:\Python27\lib\site-packages\wmi.py", line 241, in handle_com_error
raise klass (com_error=err)
wmi.x_wmi: <x_wmi: Unexpected COM Error (-2147023174, 'The RPC server is unavailable.', None, None)>
Ultimately, I am just trying to continue the script and catch the error. Maybe a note stating that IP was not responsive would be helpful. Here are the exceptions samples that I have tried:
except Exception:
sys.exc_clear()
except:
pass
except wmi.x_wmi, x:
pass
The traceback you pasted says that the error is in the c = wmi.WMI(asset) line. You need to put that line inside the try block.
Like so:
import wmi
list = open("ips.txt")
bad_assets = []
for line in list.readlines():
asset = line.strip('\n')
try:
c = wmi.WMI(asset)
for process in c.Win32_Process (name="SbClientManager.exe"):
print asset, process.ProcessId, process.Name
except Exception:
bad_assets.append(asset)
Also, trying to catch the right exception is recommended.
Help me figure out how the script will keep on running when WMI can't connect to host and go to next computer in list. Should I use continue after except?
import wmi
MachineList = ["Computer1","Computer2","Computer3"]
try:
for machines in MachineList:
c = wmi.WMI(machines) # <---- Go to next in the for loop when connection fail???
for os in c.Win32_OperatingSystem():
print os.Caption
except:
pass
import wmi
MachineList = ["Computer1","Computer2","Computer3"]
for machines in MachineList:
try:
c = wmi.WMI(machines) # <---- Go to next in the for loop when connection fail???
for os in c.Win32_OperatingSystem():
print os.Caption
except Exception: #Intended Exception should be mentioned here
print "Cannot Connect to {}".format(machines)
Generally speaking unless you are using exception for control flow, it should be caught as soon as plausible to prevent mixing with other exceptions. Also you should be specific what exception you want to catch rather than catching something generic.
for machine in MachineList:
try:
c = wmi.WMI(machine)
for os in c.Win32_OperatingSystem():
print os.caption
except Exception:
print "Failed on machine %s" % machine