I'm currently working on a Linux script on my Raspberry Pi, which should call another script with a time delay. I chose the "at command" for this and let python execute the code directly in the console.
import shlex
import subprocess
subprocess.call(shlex.split("at now + 5 minutes -f /home/raspberry/Desktop/Filename.py"))
When I run the "at command" directly in the console, I get the following feedback:
warning: commands will be executed using /bin/sh
job 13 at Sun Feb 19 23:17:00 2023
Unfortunately, nothing happens at the specified time and the script is not executed. The Filename.py Code is:
#!/usr/bin/env python3
print("Hallo Welt")
Does anyone have any ideas what I forgot or need to do differently?
Reply of /var/spool/mail/raspberry
From raspberry#raspberrypi Sun Feb 19 23:17:00 2023
Return-path: <raspberry#raspberrypi>
Envelope-to: raspberry#raspberrypi
Delivery-date: Sun, 19 Feb 2023 23:17:00 +0100
Received: from raspberry by raspberrypi.fritz.box with local (Exim 4.94.2)
(envelope-from <raspberry#raspberrypi>)
id 1pTrzQ-0000we-Fr
for raspberry#raspberrypi; Sun, 19 Feb 2023 23:17:00 +0100
Subject: Output from your job 13
To: raspberry#raspberrypi
Message-Id: <E1pTrzQ-0000we-Fr#raspberrypi.fritz.box>
From: raspberry#raspberrypi
Date: Sun, 19 Feb 2023 23:17:00 +0100
sh: 47: Syntax error: word unexpected (expecting ")")
Related
I'm trying to send a zip file to a remote server via sftp but it keeps holding up the script. On windows with a decent connection the file is sent without a problem but on a linux it doesn't work. I've tried raising an exception when the upload finishes but with no luck.
These are the log messages:
Jul 28 10:24:40 igt22-dev mqtt.sh[1353]: 46.21875 46.258296966552734
Jul 28 10:24:40 igt22-dev mqtt.sh[1353]: 46.25 46.258296966552734
Jul 28 10:24:40 igt22-dev mqtt.sh[1353]: 46.258296966552734 46.258296966552734
Jul 28 10:24:40 igt22-dev mqtt.sh[1353]: finished
and this is the code:
ftp_client = server_client.open_sftp()
source = "/usr/local/bin/log_history.zip"
destination = "/home/ubuntu/log_history.zip"
def callback_fun(current, maximum):
print(current / 1048576, maximum / 1048576)
sys.stdout.flush()
if current == maximum:
print("finished")
sys.stdout.flush()
raise Exception("finished")
ftp_client.put(source, destination, callback=callback_fun)
ftp_client.close()
print("exited")
sys.stdout.flush()
Not a fix but I managed to find a workaround. I run the code in a separate thread and when it finishes i just kill it
I'm new to Python and I'm asking for help to understand the problem with unescaping/print etc. the special HTML character in a string variable. As an example. String contains a special character \x2d to be replaced with a -.
Sample script that works:
import re
from html import unescape # python 3.4+
def processLine(index, row):
print(index, unescape(row))
testList = ["Aug 11 20:34:56 uls2204-release snapd[1259]: overlord.go:268: Acquired state lock file", "Aug 11 20:34:56 uls2204-release systemd[1]: tmp-syscheck\x2dmountpoint\x2d297125419.mount: Deactivated successfully.", "Aug 11 20:38:27 uls2204-release systemd[1]: Started User Manager for UID 1000."]
testString = "Aug 11 20:34:56 uls2204-release systemd[1]: tmp-syscheck\x2dmountpoint\x2d297125419.mount: Deactivated successfully."
for idx, element in enumerate(testList, start = 1):
if element == testString:
print("True")
processLine(idx, element)
In this example, the element[1] of the testList contains a character - as a HTML character \x2d twice. The script prints all the elements of the list as I need:
1 Aug 11 20:34:56 uls2204-release snapd[1259]: overlord.go:268: Acquired state lock file
True
2 Aug 11 20:34:56 uls2204-release systemd[1]: tmp-syscheck-mountpoint-297125419.mount: Deactivated successfully.
3 Aug 11 20:38:27 uls2204-release systemd[1]: Started User Manager for UID 1000.
But if i change the script and create a string variable from the lines of the text file (line by line read) the replacement is not done. Modified script:
import re
from html import unescape # python 3.4+
def processLine(index, row):
print(index, unescape(row))
testList = ["Aug 11 20:34:56 uls2204-release snapd[1259]: overlord.go:268: Acquired state lock file", "Aug 11 20:34:56 uls2204-release systemd[1]: tmp-syscheck\x2dmountpoint\x2d297125419.mount: Deactivated successfully.", "Aug 11 20:38:27 uls2204-release systemd[1]: Started User Manager for UID 1000."]
testString = "Aug 11 20:34:56 uls2204-release systemd[1]: tmp-syscheck\x2dmountpoint\x2d297125419.mount: Deactivated successfully."
logSource = 'file.in'
idx = 1
with open(logSource, 'rt', encoding = 'utf-8') as logInput:
while (line := logInput.readline().strip()):
if line == testString:
print("True")
processLine(idx, line)
idx += 1
The modified script prints the following output:
1 Aug 11 20:34:56 uls2204-release snapd[1259]: overlord.go:268: Acquired state lock file
2 Aug 11 20:34:56 uls2204-release systemd[1]: tmp-syscheck\x2dmountpoint\x2d297125419.mount: Deactivated successfully.
3 Aug 11 20:38:27 uls2204-release systemd[1]: Started User Manager for UID 1000.
Overwriting HTML code \x2d to character - is not performed in the output line #2.
Input file file.in content is:
$ cat file.in
Aug 11 20:34:56 uls2204-release snapd[1259]: overlord.go:268: Acquired state lock file
Aug 11 20:34:56 uls2204-release systemd[1]: tmp-syscheck\x2dmountpoint\x2d297125419.mount: Deactivated successfully.
Aug 11 20:38:27 uls2204-release systemd[1]: Started User Manager for UID 1000.
Python version is:
$ python3 --version
Python 3.10.4
Please help me understand what is the reason and what I am doing wrong. I need to rewrite all HTML codes to characters before final processing of the file lines.
Many thanks!
Most likely it just because on the end of line in your file.in you have special character of line break (\n)
I'm playing with re module and I have a difference of results I can't explain.
I have an input file containing the following text
Sybase Adaptive Server Enterprise System Performance Report
Server Version: Adaptive Server Enterprise/15.7.0/EBF 22639 SMP SP52 /P/
Sun_svr4/OS 5.10/ase157sp5x/3293/64-bit/FBO/Wed May 21 0
2:49:14 2014
Run Date: Mar 24, 2017
Sampling Started at: Mar 24, 2017 12:16:01
Sampling Ended at: Mar 24, 2017 12:31:01
Sample Interval: 00:15:00
Sample Mode: No Clear
Counters Last Cleared: Mar 21, 2017 14:49:21
Server Name: SYB01
I have the following python code where I want to get the line starting with 'Run Date:' (later I want to extract block of lines but let's proceed step by step)
import re
filename = 'D:/sp_sysmon/sp_sysmon_SYB01_170324_121544.txt'
file_r = open(filename,'r')
file_content = file_r.read()
print (file_content)
pattern_date = re.compile(r"Run Date:", re.MULTILINE)
bingo = pattern_date.search(file_content)
bingo.group()
If I execute this code directly in a python console I have the following
C:\Users\simon>python
Python 3.6.0 |Anaconda 4.3.1 (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC >v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>>
>>> filename = 'D:/sp_sysmon/sp_sysmon_SYB01_170324_121544.txt'
>>>
>>> file_r = open(filename,'r')
>>> file_content = file_r.read()
>>> print (file_content)
Sybase Adaptive Server Enterprise System Performance Report
Server Version: Adaptive Server Enterprise/15.7.0/EBF 22639 SMP SP52 /P/
Sun_svr4/OS 5.10/ase157sp5x/3293/64-bit/FBO/Wed May 21 0
2:49:14 2014
Run Date: Mar 24, 2017
Sampling Started at: Mar 24, 2017 12:16:01
Sampling Ended at: Mar 24, 2017 12:31:01
Sample Interval: 00:15:00
Sample Mode: No Clear
Counters Last Cleared: Mar 21, 2017 14:49:21
Server Name: SYB01
>>>
>>> pattern_date = re.compile(r"Run Date:", re.MULTILINE)
>>> bingo = pattern_date.search(file_content)
>>> bingo.group()
'Run Date:'
>>>
This is the output I expect
Now If copy this code in a script.py file and execute it, I have a different result where the call the bingo.group() doesn't show anything
D:\Perso\My_code\python>python script.py
Sybase Adaptive Server Enterprise System Performance Report
Server Version: Adaptive Server Enterprise/15.7.0/EBF 22639 SMP SP52 /P/
Sun_svr4/OS 5.10/ase157sp5x/3293/64-bit/FBO/Wed May 21 0
2:49:14 2014
Run Date: Mar 24, 2017
Sampling Started at: Mar 24, 2017 12:16:01
Sampling Ended at: Mar 24, 2017 12:31:01
Sample Interval: 00:15:00
Sample Mode: No Clear
Counters Last Cleared: Mar 21, 2017 14:49:21
Server Name: SYB01 D:\Perso\My_code\python>
Does anybody have an ID of this difference?
You need to print the result of that call.
You forgot
print(bingo.group())
In the interactive console, after each command, a repr() of its result is printed for your convenience.
This doesn't happen when executing a script, so you need to explicitly print whatever you want to be printed.
Currently I am experience issues with the script automatic run after wifi adapter connects to a network.
After ridiculously extended research, I've made several attempts to add script to a /etc/network/if-up.d/. Manually my script works; however it does not automatically.
User permissions:
ls -al /etc/network/if-up.d/*
-rwxr-xr-x 1 root root 703 Jul 25 2011 /etc/network/if-up.d/000resolvconf
-rwxr-xr-x 1 root root 484 Apr 13 2015 /etc/network/if-up.d/avahi-daemon
-rwxr-xr-x 1 root root 4958 Apr 6 2015 /etc/network/if-up.d/mountnfs
-rwxr-xr-x 1 root root 945 Apr 14 2016 /etc/network/if-up.d/openssh-server
-rwxr-xr-x 1 root root 48 Apr 26 03:21 /etc/network/if-up.d/sendemail
-rwxr-xr-x 1 root root 1483 Jan 6 2013 /etc/network/if-up.d/upstart
lrwxrwxrwx 1 root root 32 Sep 17 2016 /etc/network/if-up.d/wpasupplicant -> ../../wpa_supplicant/ifupdown.sh
Also, I've tried to push the command directly in /etc/network/interfaces
by adding a row
post-up /home/pi/r/sendemail.sh
Contents of sendemail.sh:
#!/bin/sh
python /home/pi/r/pip.py
After the reboot, nothing actually happen. I've even tried sudo in front
I assume that wpasupplicant is the thing which causes that, but I cannot get how to run my script in ifupdown.sh script under /etc/wpa_supplicant.
Appreciate your help!
If you have no connectivity prior to initializing the wifi interface, I would suggest adding a cron job of a bash or python script that checks for connectivity every X minutes.
Ping (host);
If host is up then run python commands or external command.
This is rather ambiguous but hopefully is of some help.
Here is an example of a script that will check if a host is alive;
import re,commands
class CheckAlive:
def __init__(self):
myCommand = commands.getstatusoutput('ping ' + 'google.com)
searchString = r'ping: unknown host'
match = re.search(searchString,str(myCommand))
if match:
# host is not alive
print 'no alive, don't do stuff';
else:
# host is alive
print 'alive, time do stuff';
as i am not really aware of the underlying strategies or protocols used by Ladon, Webservices and Apache (i am using Ladon and Python with mod_wsgi.so on a Windows Apache server - switched to Ubuntu system)
i wonder if this can be possible to load some ressources for python once, so that exposed methods use these ressources from python code without having to load these ressources again when considering /serving new queries to the web services?
do you have any clue on how to achieve this if possible, or any work around if not ?
typically i am loading some huge dictionaries from files that take too much time to load (I/O) and as it is loaded when receiving each new ladon query, the WS is too slow, i would have like to tell Ladon : "load this when apache start, and made that available to all my python web services/codes as a dictionary during all the time that Apache is running". I will not modify these datas, so i just need to able to read/access them.
best regards
first EDIT : if this could help, looks like on my Ubuntu (i have switched to Ubuntu from my Win config to be more "standard", hope i was right doing this), Apache2 is set in prefork mode rather than MPM, (as suggested by Jakob Simon-Gaarde) readed from :
#: sudo /usr/sbin/apache2 -l
Compiled in modules:
core.c
mod_log_config.c
mod_logio.c
prefork.c
http_core.c
mod_so.c
#: sudo /usr/sbin/apache2 -l | grep MPM
#:
i'm going to check how this can be done, maybe i am also putting some simplified code here, because for now i'm in a noway even with your helpful answers (i can make anything work here :/)
when installing MPM mode, found how to do here: $ sudo apt-get install apache2-mpm-worker
last EDIT:
here is the skeleton of my WS code :
MODEL_DIR = "/home/mydata.file"
import sys
import codecs
import glob
import os
import re
import numpy
from ladon.ladonizer import ladonize
from ladon.types.ladontype import LadonType
from ladon.compat import PORTABLE_STRING
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
return cls._instances[cls]
class LDtest(object):
__metaclass__ = Singleton
modeldir = MODEL_DIR
def __init__(self):
self.load()
def load(self):
modeldir = LDtest.modeldir
self.data = mywrapperfordata.mywrapperfordata(modeldir)
b = datetime.datetime.now()
self.features = self.mywrapperfordata.load() # loading is wrapped here
c = datetime.datetime.now()
print("loading: %s done." % (c-b))
def letsdoit(self, myinput):
return [] # actually main logic ie complex stuff involving accessing to self.features
#ladonize(PORTABLE_STRING, [ PORTABLE_STRING ], rtype = [ PORTABLE_STRING ] )
def ws(self, myinput):
result = self.letsdoit(myinput)
return result
import datetime
a = datetime.datetime.now()
myLDtest = LDtest()
b = datetime.datetime.now()
print("LDtest: %s" % (b-a))
about loading time: from my apache2 log: -notice that module 1 is required and imported by module 2 and also providing as a lonely webservice. It looks like the singleton is not built or not quickly enough?
[Tue Jul 09 11:09:11 2013] [notice] caught SIGTERM, shutting down
[Tue Jul 09 11:09:12 2013] [notice] Apache/2.2.16 (Debian) mod_wsgi/3.3 Python/2.6.6 configured -- resuming normal operations
[Tue Jul 09 11:09:50 2013] [error] Module 4: 0:00:02.885693.
[Tue Jul 09 11:09:51 2013] [error] Module 0: 0:00:03.061020
[Tue Jul 09 11:09:51 2013] [error] Module 1: 0:00:00.026059.
[Tue Jul 09 11:09:51 2013] [error] Module 1: 0:00:00.012517.
[Tue Jul 09 11:09:51 2013] [error] Module 2: 0:00:00.012678.
[Tue Jul 09 11:09:51 2013] [error] Module (dbload): 0:00:00.402387 (22030)
[Tue Jul 09 11:09:54 2013] [error] Module 3: 0:00:00.000036.
[Tue Jul 09 11:13:00 2013] [error] Module 0: 0:00:03.055841
[Tue Jul 09 11:13:01 2013] [error] Module 1: 0:00:00.026215.
[Tue Jul 09 11:13:01 2013] [error] Module 1: 0:00:00.012600.
[Tue Jul 09 11:13:01 2013] [error] Module 2: 0:00:00.012643.
[Tue Jul 09 11:13:01 2013] [error] Module (dbload): 0:00:00.322444 (22030)
[Tue Jul 09 11:13:03 2013] [error] Module 3: 0:00:00.000035.
mod_wsgi launches one or more Python processes upon startup and leaves them running to handle requests. If you load a module or set a global variable, they'll still be there when you handle the next request - however, each Python process has its own separate block of memory, so if you configure mod_wsgi to launch 8 processes and load a 1G dataset, eventually you'll be using 8G of memory. Maybe you should consider using a database?
edit: Thanks Graham :-) So with only one process and multiple threads, you can share one copy of your huge dictionary between all worker threads.
We use Ladon extensively at my work with all our web projects, and I have the priviledge of being able to develop my private project (I am the Ladon developer) and getting payed for it ;-)
Some of our services have very heavy resource consumptions, for instance we have a text-to-speach service that loads around 1Gb of data into memory per supported language, and a wordprediction service that loads around 100Mb per supported language.
mod_wsgi is fine - we use that aswell - What you need to do is make sure that your apache server is compiled as mpm-worker (http://httpd.apache.org/docs/2.2/mod/worker.html). In this configuration your service runs in a multi-threaded environment instead of a multi-process environment. The effect is that you only fire up one interpreter per server process which then runs your service in several underlying threads that share resources. The caveeat is that you have to make sure that your service does not step on it's own toes, meaning you will have to protect global variables and class-static variables shared between service class instances with mutex.acquire()/mutex.release().
Other than that Ladon as a framework is build for multi-threaded environments.
Best regards Jakob Simon-Gaarde