crawler stopped when running in the background in ubuntu [closed] - python

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Closed 8 years ago.
Improve this question
I made a simple crawler using python that has infinite loop so it can't be stop.
With random delay 17 ~ 30, this crawler crawl same one page and find 'href' links that is updated periodically and store to Mysql.
I used Ubuntu server.
Because i used Linux command that
$ nohup python crawer.py &
so this crawler was running in Ubuntu server background.
and it has ran about 4 hours i think.
but suddenly crawler stopped.
and next day i try again. and it works well!
what is the problem? is this about web page's block? or does nohup command has limit time????
thanks a lot.

No, nohup will do what it's designed to to. That is:
The nohup utility invokes utility with its arguments and at this time
sets the signal SIGHUP to be ignored. If the standard output is a termi-
nal, the standard output is appended to the file nohup.out in the current
directory. If standard error is a terminal, it is directed to the same
place as the standard output.
Some shells may provide a builtin nohup command which is similar or iden-
tical to this utility. Consult the builtin(1) manual page.
Bash's (and other shells) & will background the task. nohup with & effectively lets the process run in the background even whilst you terminate your tty/pty session.
I believe the problem is your Python program here is crashing. You should invest some time in some logging and find out. e.g:
nohup my_app.py &> myapp.log &

Related

how to test if a python script is running successfuly through a batch file [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 3 years ago.
Improve this question
I have 2 python scripts that do some geoprocessing, one is depending on the other and i run them via a batch file. At the end of the execution, i send email using powershell script for feedback.
I just want to receive emails when there is an error and not everytime the script is running.
I want a way to test if the batch file has run successfuly the python scripts.
Any Ideas?
You have to control the output of your python script depending on the success of the script. For example via exit(1).
I would probably start the python scripts via powershell. You are able to get the exit code of applications via $LASTEXITCODE.
python code (script does not run as intended):
exit(1)
PS code:
#run python script here
if ($lastexitcode -gt 0){
#send mail
}

How to make a windows scheduler Start and Stop a python task [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 5 years ago.
Improve this question
I have a python script that I typically kick off manually. It then runs for several days collecting information until it gets disconnected.
What I want to do is have the Windows scheduler start the job 12:01 AM and then terminate the job at 11:58 pm that same day.
Typically I would be able to do this using CRON in the linux world but am unsure how to do this in Windows Scheduler.
Thank you for your help!
Open Task Scheduler in Windows. Control Panel -> Administrative Tools -> Task Scheduler
Click 'Create Basic Task' (to start the script)
Set the trigger time
Set Program/Script = [full path to python.exe]
Add Arguments = [full path to your scheduled python script]
Click 'Create Basic Task' (to end the script)
Set the trigger time
Program/Script = taskkill /f /im [full path to python.exe]
This guide goes into more detail.

Cron tab pattern to run a script once in a year/ once in lifetime [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am looking for crontab pattern that should run my python script once in a year. Below is the date and time I am looking to trigger my script through crontab.
Date: Aug-16-2017, Time: 5:00PM (should trigger the script only once this year at this specified time)
Can someone please help me with this?
Note: Im actually looking for something that should trigger my script in the background for once and leave it running (coz my script has infinite loop so once triggered and keep the process alive in the background should do good). For this I used "at" command (from googling I see "at" jobs are triggered once at a specific time and will kept alive until/unless the server/system reboots which is fine with me). But looks like "at" job didnt work as expected. I started "at" job at 12:20 PM PST which should keep my script running and my script is expected to send an output and 1:15 PM PST. Just to test this, I closed SSH session at 1:00 PM. Before closing I see output "atq" showing jobs triggered by at. But later when I re-sshed into the server again, I dont see any jobs running under "at".
Can some one please help me one of these problems? Ultimately I m looking for a solution to keep the process alive for ever in the background.
Cron isn't suited for this, because it runs programs based on a certain interval.
A clean solution would be to use Supervisor: http://supervisord.org/
Install supervisor through your package manager, make your script executable, and add this to /etc/supervisor/conf.d/$YOUR_SCRIPT_NAME.conf, then restart supervisor.
[program:$YOUR_SCRIPT_NAME]
command=$PATH_TO_YOUR_SCRIPT
autostart=true
autorestart=true
You can set the locations for the log and error output files in the config file too: http://supervisord.org/configuration.html
lets start to create the system you need :
first of all you need to create a runner script for your python code :
/usr/local/bin/my_runner :
#!/bin/bash
python /path/to/my/script
$sudo chmod +x /usr/local/bin/my_runner
then you need to create /usr/local/bin/startup_runner
#!/bin/bash
echo "#reboot /usr/local/bin/my_runner" >> /var/spool/cron/crontabs/root
crontab -u root -l | grep -v '/usr/local/bin/startup_runner' | crontab -u root -
$sudo chmod +x /usr/local/bin/startup_runner
at the end of this , you must go for the first caller at Aug-16-2017, Time: 5:00PM
$sudo crontab -e
add the following to the end of crontab file
0 17 16 8 * /usr/local/bin/startup_runner
this will work fine for you , any questions about the scripts ?

Linux: Warn Python Script of Restart [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Improve this question
I have a Raspberry Pi which uses a USB wireless adapter and wicd-curses. A Python script runs in the background which uses a WebSocket. When I call sudo reboot, my Python script gets the signal to restart (with SIGTERM) about 20 seconds later. (I don't know why the computer takes 20 seconds to restart anyway. I don't remember it being this way before installing wicd-curses.)
By the time 20 seconds has passed, wicd-curses has already disconnected from the wireless network, meaning my Python script cannot properly close the WebSocket connection. So the core of my question is this: what Python commands are available to me to ensure that my script is notified of the system shutdown earlier than it is now? Is there any sort of event for which I can listen? Preferably, I want to be able to run the script on demand (python myscript.py) without the use of a daemon or service or whatever it might be called in the Linux world. Thank you.

Run python in terminal and don't terminate when terminal is closed [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Improve this question
I need to make static website. So I connected via ssh to some local server, where I want to make a static website. Then I used python to make it work:
$ python -m http.server 55550
But if I close terminal, then python program is terminated. I want to shut down my computer, but I want to let this process running on that local server, so other people could still access that website.
How can I do this? After that, how should I terminate that process later?
Thanks for any help
Use the nohup shell builtin:
nohup python -m http.server 55550
To terminate the process, simply kill it using the kill command, just like any other process.
you can also launch it in background
python -m http.server 55550 &
then enter
disown
to detach the process to the current term
screen
python -m SimpleHTTPServer 55550 &
press ctrl+a, then press d
exit
shutdown your computer
...
start your computer
ssh your server
screen -r

Categories