I have a running loop script in Python, which I want interact with a HTML-page. For example:
(HTML)Clicking button -> magic -> (Python) Do something function in
script.py
This response is not appropriate for this.
Probably you can use Selenium python binding for the purpose of interecting with Web page with your python script.
Selenium link
Example:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.send_keys("pycon")
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
driver.close()
I resole this problem by PIPE in python. Python script reads file p1.txt by line and server.js sends text. For example, here server.js sends string 'COMMAND' and we are getting script.py response.
script.py
import os,sys
fd = "./p1.txt"
try:
os.mkfifo(fd)
except OSError:
pass
rp = os.open(fd, os.O_RDONLY)
#Need to create new thread, for background reading file
while True:
response = os.read(rp, 54)
if response == "COMMAND":
#do
print 'ICATCHYOU'
server.js
var fs = require('fs');
var path = '/home/pi/develop/p1.txt';
var streamOptions = { flags: 'w',
encoding: 'utf-8',
mode: 0666 };
var afterOpen = function(err, fd) {
var writeStream = fs.createWriteStream(path, streamOptions);
writeStream.write("COMMAND");
};
fs.open(path, 'w', afterOpen);
Related
Would anyone know when using python-shell if you can keep the python script running when its called from Javascript? Maybe call a method from a Python class from Javascript using python-shell?
For example this code works but it kills the Python script every time its called.
let {PythonShell} = require("python-shell");
let options = {
mode: "text",
args: ["read", "12345:2", "analogInput", "2"],
};
PythonShell.run("bac0Worker.py", options, function (err, results) {
if (err){
console.log(err)
console.log("An error happened")
}else{
// results is an array consisting of messages collected during execution
console.log("results: ", results);
console.log("Python Script Finished");
}
})
On the Python side I am experimenting with a package called BAC0 for BACnet systems. What I am trying to do is figure out if I can keep the BAC0 script running, I think under the hood of BAC0 there's alot of process going on where it may create lots of unnecessary traffic on a BACnet network if the script is started/stopped a lot.
bac0Worker.py
import sys
import BAC0
BAC0.log_level('silence')
bacnet = BAC0.lite()
def do_things(address,object_type,object_instance):
try:
read_vals = f'{address} {object_type} {object_instance} presentValue'
read_result = bacnet.read(read_vals)
if isinstance(read_result, str):
pass
else:
read_result = round(read_result,2)
print(read_result)
except Exception as error:
print("read error")
def main():
# args from Javascript
first = sys.argv[1]
second = sys.argv[2]
third = sys.argv[3]
fourth = sys.argv[4]
# get sensor data
do_things(second, third, fourth)
# all done
bacnet.disconnect()
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("bacnet app error")
bacnet.disconnect()
Like I mentioned the code works if I run from Node I can see the sensor value from the BACnet network 71.7 degrees.
C:\Program Files\nodejs\node.exe .\start.js
results: (1) ['71.7']
Python Script Finished
Hopefully this makes sense, sorry for the odd question but curious if anyone would have any tips for how to keep the Python script running where I could then just pass values (sensor & device addressing info) to BAC0 script to request a BACnet read from the network. In the main function of the Python file if I replace this:
# all done
bacnet.disconnect()
With:
while True:
pass
This would keep the Python file alive I just don't know how to pass values to a live Python script using python-shell. Thanks for any tips not a lot of wisdom here best practices. Curious if I should change my do_things function into a Python class with a method called do_things where then this class could get called from Javascript?
.js
const { PythonShell } = require("python-shell");
let pyshell = new PythonShell("my_script.py");
pyshell.send("hello");
pyshell.on("message", function (message) {
console.log(message);
});
setTimeout(() => {
pyshell.send("Another Hello");
}, 3000);
const end = () => {
pyshell.end(function (err, code, signal) {
if (err) throw err;
console.log("finished");
});
};
// end();
.py
import sys
def log():
print('hello from python')
while True:
inp = input(" ")
if inp=="hello":
log()
this will keep the process running unless you call the end method from the nodejs or some error occurs. This may incur performance issues because python thingy will be sharing same resources as your nodejs. A better way to do this would be to use a microservice like deploy some Rest Api which handles python specific tasks.
Just for fun Ill post what I was able to learn from #sandeep (thanks a million btw), this works. I can print BACnet sensor values which are gathered from Python from running a javascript file on 60 second setInterval(pythonDo, 60000); with keeping the Python file alive:
C:\Program Files\nodejs\node.exe .\start.js
BAC0 start success
68.47
68.43
68.29
68.25
start.js
const { PythonShell } = require("python-shell");
let pyshell = new PythonShell("bac0Worker.py");
function pythonDo() {
pyshell.send("read 12345:2 analogInput 2");
}
pyshell.on("message", function (message) {
console.log(message);
});
var data = setInterval(pythonDo, 60000);
bac0Worker.py
import sys
import BAC0
BAC0.log_level('silence')
bacnet = BAC0.lite()
print("BAC0 start success")
# available BACnet actions
SUBSTRING_READ = "read"
SUBSTRING_WRITE = "write"
SUBSTRING_RELEASE = "release"
def bac0_worker(action, BACnet_request, **kwargs):
value = kwargs.get('value', None)
priority = kwargs.get('priority', None)
if action == "read":
try:
read_vals = f'{BACnet_request} presentValue'
read_result = bacnet.read(read_vals)
if isinstance(read_result, str):
pass
else:
read_result = round(read_result,2)
print(read_result)
except Exception as error:
print("read error")
elif action == "write":
try:
write_vals = f'{BACnet_request} presentValue {value} - {priority}'
bacnet.write(write_vals)
print(write_vals)
except Exception as error:
print("write error")
elif action == "release":
try:
release_vals = f'{BACnet_request} presentValue null - {priority}'
bacnet.write(release_vals)
print(release_vals)
except Exception as error:
print("release error")
else:
return "server error on BACnet opts"
def main():
while True:
# from javascript
inp = input(" ")
for word in inp.split():
if SUBSTRING_READ == word:
stripped_string = inp.replace(SUBSTRING_READ + " ", "")
# do a BACnet READ request with BAC0
bac0_worker(SUBSTRING_READ,stripped_string)
if SUBSTRING_WRITE == word:
print("WRITE Match Found")
if SUBSTRING_RELEASE == word:
print("RELEASE Match Found")
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("bacnet app error")
bacnet.disconnect()
I want to use Cron to execute my python script every hour of the day. Therefore I created a cronjob that looks like: #hourly /home/pi/Desktop/repository/auslastung_download/auslastung.py
The cronjob should execute the following script:
from bs4 import BeautifulSoup
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium import webdriver
from datetime import datetime, date
def get_auslastung_lichtenberg():
try:
url = "https://www.mcfit.com/de/fitnessstudios/studiosuche/studiodetails/studio/berlin-lichtenberg/"
options = FirefoxOptions()
options.add_argument("--headless")
driver = webdriver.Firefox(options=options)
driver.get(url)
html_content = driver.page_source
soup = BeautifulSoup(html_content, 'html.parser')
elems = soup.find_all('div', {'class': 'sc-iJCRLp eDJvQP'})
#print(elems)
auslastung = str(elems).split("<span>")[1]
#print(auslastung)
auslastung = auslastung[:auslastung.rfind('</span>')]
#print(auslastung)
auslastung = str(auslastung).split("Auslastung ")[1]
#print(auslastung)
auslastung = auslastung[:auslastung.rfind('%')]
print(auslastung)
now = datetime.now()
current_time = now.strftime("%H:%M:%S")
#print("Current Time =", current_time)
today = date.today()
print(today)
ergebnis = {'date': today, 'time': current_time,'studio': "Berlin Lichtenberg", 'auslastung': auslastung}
return ergebnis
finally:
try:
driver.close()
except:
pass
"""
import json
with open('database.json', 'w') as f:
json.dump(get_auslastung_lichtenberg(), f)
"""
import csv
with open('/home/pi/Desktop/repository/auslastung_download/data.csv', mode='a') as file:
fieldnames = ['date', 'time', 'studio', 'auslastung']
writer = csv.DictWriter(file, fieldnames=fieldnames)
writer.writerow(get_auslastung_lichtenberg())
When executed via python3 auslastung.pyeverything works fine and the script writes into the data.csv file.
Maybe someone can help me :)
First of all you must ensure that your script runs.
If interactively you run python3 auslastung.py why do you invoke your python script differently on your cron.
have you tried to run just /home/pi/Desktop/repository/auslastung_download/auslastung.py interactively? without initial python3, does it run?
If your script runs with python3 auslastung.py on your crontab you should include full path to both interpreter and script:
#hourly /paht/to/python3 /full/path/to/script.py
If you made your script to run directly without need to indicate interpreter, just /full/path/to/script.py then on your crontab you should include full path to script:
#hourly /full/path/to/script.py
You may include a shebang: a very first line of your script indicate which interpreter is used to execute it. So your first line should be #!/path/to/your/interpreter
An you have to ensure then that script has execute permision with chmod +x auslastung.py.
i want following script as an executable... it become build and everything... but it doesnt make a token.txt when I run it (also as admin). I dont know if it dont work at all or just the file creation part.
But think its dont work at all... run the exe in cmd dont show the print ...
Also there is opening a geckodriver.exe window.
When I dont drive as admin the exe ask for firewall permission.
When I start the exe every seound time (realy only every secound time)there is comming up an error that say:
"Failed to execute script Etherscrape" (Ethersscrape is the name of the .exe)
Also is there a geckodriver.log what shows an error:
*** You are running in headless mode.
JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
1591714269803 Marionette INFO Listening on port 54219
1591714270054 Marionette WARN TLS certificate errors will be ignored for this session
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from collections import defaultdict
counted = defaultdict(int)
tokenliste = []
options = Options()
options.headless = True
driver = webdriver.Firefox(firefox_options=options)
for x in range(1,10):
my_url = "https://etherscan.io/tokentxns?ps=100&p="+str(x)
driver.get(my_url)
for i in range(1,100):
xpath = "/html/body/div[1]/main/div[2]/div/div/div[2]/table/tbody/tr["+str(i)+"]/td[9]/a"
p_element = driver.find_element_by_xpath(xpath)
tokenliste.append(p_element.text)
for x in tokenliste:
counted[x] += 1
print(counted)
with open("token.txt","w",encoding="utf-8") as f:
for key, value in sorted(counted.items(), key=lambda item: item[1]):
stri = str(key)+ ": " + str(value)+ "\n"
f.write(stri)
My python script has two main steps:
Open two webbrowser tabs (default browser). And let a LED blink.
When i am executing the python script via shell with "python netflix.py" everything works fine.
But when i am trying to start it via my (see below) NodeJS script. Only the LED will be blink. The webbrowser tabs won't come up.
Does anyone know where the issue is?
#!/usr/bin/env node
process.env.UV_THREADPOOL_SIZE = 1208;
var http = require('http');
const server = http.createServer(function(request, response) {
try{
console.log('Request incoming...');
var spawn = require('child_process').spawn;
if(request.url == '/abort'){
console.log('Calling abort.py ...');
var process = spawn('python',["./abort.py"]);
}else{
console.log('Calling netflix.py');
var process = spawn('python',["./netflix.py"]);
}
}catch(e){
console.log('Error');
console.log(e);
}
});
server.on('error', function(){
console.log('error');
});
const port = 8000;
server.listen(port);
server.timeout = 10000;
console.log(`Listening at http://leitner-pi:${port}`);
import sys
import webbrowser
from gpiozero import LED
from time import sleep
try:
webbrowser.open_new_tab("http://netflix.com");
finally:
print("")
print("Lasse PIN 7 blinken..")
led = LED(17)
while True:
led.on()
sleep(0.3)
led.off()
sleep(0.3)
Fixed: Well, I made a workaround.
First, I split the python script into two scripts. Webbrowser and LED script, to avoid any kind of interruptions or something else. Second, I changed the webbrowser to:
os.system(‘sudo -upi chromium-browser URL1 URL2’).
At last, I am calling both new
scripts from my node webserver, like I used before.
Now it works fine.
I want to use syslog-ng to receive netgear log, and use python script process.
But syslog-ng didn't run the python script.
syslog-ng.config
#version:3.2
options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (no);
use_fqdn (no);
create_dirs (no);
keep_hostname (yes);
};
source s_sys {
udp(ip(0.0.0.0) port(514));
};
destination d_python{
program("/usr/local/bin/python /opt/syslog.py");
#program("/bin/echo 'haha' >> /tmp/test");
};
log { source(s_sys); destination(d_python);};
and python script like this
#!/usr/local/bin/python
#coding:utf8
import os
import sys
import datetime
f = open('/var/log/pnet.log', 'a')
f.write('\nstart\n')
f.write('args\n')
f.write('%s\n' % sys.argv)
if not sys.stdin.isatty():
f.write('stdin\n')
f.write('%s\n' % date.date.now().isoformat() )
tty_read = sys.stdin.read()
f.write("'''\n")
f.write(tty_read)
f.write("\n'''\n")
f.write('end\n')
f.close()
The script is already 777
Even I change my config to use 'echo' directly pipe into a file, didn't write a word too...
So...why?
silly question, but do you have incoming logs? If you use a simple file destination instead of the program, do you receive logs? If not, the problem is not in the program destination.
Also, try changing the flush_lines (0); option to 1 to see if it helps.
Regards,
Robert Fekete
I could show you my code for reference:
my syslog-ng.conf :
source s_test{
file("/home/test/in.log" follow-freq(1) flags(no-parse));
};
destination d_test{
program ("/home/test/out.py" flush_lines(1) flags(no_multi_line));
};
log {
source(s_test);
destination(d_test);
flags(flow-control);
};
my out.py:
#!/usr/bin/env python
# encoding: utf-8
import sys
while True:
line = sys.stdin.readline()
file = open("/home/test/out_from_python.log","a")
file.write(line)
file.close()
when you type echo "something" >> /home/test/in.log , there would be a new log lie in /home/test/out_from_python.log