I am running an Instagram bot using python and selenium. I use a bash script to run a python script with the accounts credentials(Username, password, hashtags, etc...) I run multiple Instagrams so I have made multiple copies of this file. Is there a way to put this in a single file that I can click on and run?
To open multiple terminals running their assigned account?
I've already tried just to add them to one big file but the scripts wont run until the previous one finishes.
Also since I'm using selenium, trying multi threading in python is somewhat difficult but would not mind going that route if someone could point me to where I could start with that.
#!/bin/sh
cd PycharmProjects/InstaBot
python3 W.py
I highly recommend that everyone read about Bash Job Control.
Getting into multithreading is ridiculous overkill if your bottleneck has nothing to do with the CPU.
for script in PycharmProjects/InstaBot/*.py; do
python3 "$script" &
done
jobs
Only one process in shell can run in foreground mode. So the command gets executed only when the previous completes.
Adding "&" symbol at the end of the command line tells shell that the command should be executed in the background. This way the shell will start python and continue without waiting.
This will execute two instances simultaneously, but they will all output to the same terminal:
#!/bin/sh
cd PycharmProjects/InstaBot
python3 W.py first_credentials &
python3 W.py second_credentials &
You can use the same technique to start a new terminal process for each python script:
#!/bin/sh
cd PycharmProjects/InstaBot
gnome-terminal -e "python3 W.py first_credentials" &
gnome-terminal -e "python3 W.py second_credentials" &
How to get an sh script for starting a new terminal, execute a python script and keep it running? The python script is supposed to run continuously in a perpetual loop, spitting out results as they pop in. Whenever trying with sh-script for gnome-terminal just getting: child process exited normally with status 2
Manually it would just be: python home/ubuntu/pyscript.py
Could someone give an idea how to do this?
I have a list of scripts to run, so resorting to the manual solution is tedious.
You can use gnome-terminal with the -x flag.
Suppose you have a spam.py script; then the following command will spawn a new terminal, run spam.py in it, and close the terminal once the script has ended.
gnome-terminal -x python spam.py
Try with this script:
# spam.py
import time
for _ in range(5):
print("eggs")
time.sleep(1)
Then the previous command will spawn a terminal, that will be printed eggs five times, and then will be closed.
If you want to leave the terminal open with the Python interpret still running after the script ended, then Python's -i flag (doc then CTRL+F -> -i) is what you want:
gnome-terminal -x python -i spam.py
To run the Python script in a new instance of your favourite terminal, write:
x-terminal-emulator -e python -i home/ubuntu/pyscript.py
This will start the Python script and run it until it ends, then display a Python prompt to stop the terminal emulator from closing.
This will work with x-terminal-emulator substituted with any of the many, many terminals installed on my computer, so will work with little modification across all POSIX-compatible systems with the standard terminals installed. This won't work on a Mac, however. For a properly cross-platform Python implementation of something slightly different, see here. Most of the techniques should be transferable.
To run the Python script in the same terminal whilst carrying on with the rest of the shell script, write:
python home/ubuntu/pyscript.py &
Note the &, which runs the program as a new process (but still connects the output to the virtual terminal).
I have a python program that uses the ThreadPool for multithreading. The program is one step in a shell script. When I execute the shell script manually on the command line, the entire flow works as expected. However, when I execute the shell script as a cronjob, it appears that the flow goes to the next steps before the python multithreading steps are completely finished.
Inside the python program, I do call AsyncResult.get(timeout) to wait for all the results to come back before moving on.
Run your program via batch(1) (see the output of the command man batch) as well. If that works OK, but the cron version does not, then it is almost certainly a problem with your environment variable setup. To verify that, run printenv from your interactive shell to inspect your environment there. Then do the same thing inside the crontab (you will just need to temporarily set up an extra cron entry for it). Try setting the variables in your shell script before invoking Python.
On the other hand, if it doesn't work via batch(1) either, it could be something to do with the files that your code has open. Try running your shell script with input redirected from /dev/null and output going to a file:
$ /usr/local/bin/myscript </dev/null >|/tmp/outfile.txt 2>&1
Try setting "TERM=xterm" (or whatever env variable you have, figure out by command 'env' on your terminal) in your crontab.
I'm running a python script on a remote server with the time command as follows:
time python myscript.py
SSH timeout occurs on the server after some time, so i also need to run it with nohup.So, i have the following two questions:
Is nohup time myscript.py & the right command to execute my python script ?
If the script runs in the background, how will i see the output of the time command ?
Please Help
Thank You
nohup will usually write STDOUT and STDERR to a file called "nohup.out" in the current directory. You'll be able to see the output of time at the end of that file.
Another way of solving this redirection of the output like this:
nohup time bla.py >myoutput &
To 1.: Yes
To 2.: You can redirect the output to a file
nohup time myscript.py > ~/time_output &
I have Python script bgservice.py and I want it to run all the time, because it is part of the web service I build. How can I make it run continuously even after I logout SSH?
Run nohup python bgservice.py & to get the script to ignore the hangup signal and keep running. Output will be put in nohup.out.
Ideally, you'd run your script with something like supervise so that it can be restarted if (when) it dies.
If you've already started the process, and don't want to kill it and restart under nohup, you can send it to the background, then disown it.
Ctrl+Z (suspend the process)
bg (restart the process in the background
disown %1 (assuming this is job #1, use jobs to determine)
Running a Python Script in the Background
First, you need to add a shebang line in the Python script which looks like the following:
#!/usr/bin/env python3
This path is necessary if you have multiple versions of Python installed and /usr/bin/env will ensure that the first Python interpreter in your $$PATH environment variable is taken. You can also hardcode the path of your Python interpreter (e.g. #!/usr/bin/python3), but this is not flexible and not portable on other machines. Next, you’ll need to set the permissions of the file to allow execution:
chmod +x test.py
Now you can run the script with nohup which ignores the hangup signal. This means that you can close the terminal without stopping the execution. Also, don’t forget to add & so the script runs in the background:
nohup /path/to/test.py &
If you did not add a shebang to the file you can instead run the script with this command:
nohup python /path/to/test.py &
The output will be saved in the nohup.out file, unless you specify the output file like here:
nohup /path/to/test.py > output.log &
nohup python /path/to/test.py > output.log &
If you have redirected the output of the command somewhere else - including /dev/null - that's where it goes instead.
# doesn't create nohup.out
nohup command >/dev/null 2>&1
If you're using nohup, that probably means you want to run the command in the background by putting another & on the end of the whole thing:
# runs in background, still doesn't create nohup.out
nohup command >/dev/null 2>&1 &
You can find the process and its process ID with this command:
ps ax | grep test.py
# or
# list of running processes Python
ps -fA | grep python
ps stands for process status
If you want to stop the execution, you can kill it with the kill command:
kill PID
You could also use GNU screen which just about every Linux/Unix system should have.
If you are on Ubuntu/Debian, its enhanced variant byobu is rather nice too.
You might consider turning your python script into a proper python daemon, as described here.
python-daemon is a good tool that can be used to run python scripts as a background daemon process rather than a forever running script. You will need to modify existing code a bit but its plain and simple.
If you are facing problems with python-daemon, there is another utility supervisor that will do the same for you, but in this case you wont have to write any code (or modify existing) as this is a out of the box solution for daemonizing processes.
Alternate answer: tmux
ssh into the remote machine
type tmux into cmd
start the process you want inside the tmux e.g. python3 main.py
leaving the tmux session by Ctrl+b then d
It is now safe to exit the remote machine. When you come back use tmux attach to re-enter tmux session.
If you want to start multiple sessions, name each session using Ctrl+b then $. then type your session name.
to list all session use tmux list-sessions
to attach a running session use tmux attach-session -t <session-name>.
You can nohup it, but I prefer screen.
Here is a simple solution inside python using a decorator:
import os, time
def daemon(func):
def wrapper(*args, **kwargs):
if os.fork(): return
func(*args, **kwargs)
os._exit(os.EX_OK)
return wrapper
#daemon
def my_func(count=10):
for i in range(0,count):
print('parent pid: %d' % os.getppid())
time.sleep(1)
my_func(count=10)
#still in parent thread
time.sleep(2)
#after 2 seconds the function my_func lives on is own
You can of course replace the content of your bgservice.py file in place of my_func.
Try this:
nohup python -u <your file name>.py >> <your log file>.log &
You can run above command in screen and come out of screen.
Now you can tail logs of your python script by: tail -f <your log file>.log
To kill you script, you can use ps -aux and kill commands.
The zsh shell has an option to make all background processes run with nohup.
In ~/.zshrc add the lines:
setopt nocheckjobs #don't warn about bg processes on exit
setopt nohup #don't kill bg processes on exit
Then you just need to run a process like so: python bgservice.py &, and you no longer need to use the nohup command.
I know not many people use zsh, but it's a really cool shell which I would recommend.
If what you need is that the process should run forever no matter whether you are logged in or not, consider running the process as a daemon.
supervisord is a great out of the box solution that can be used to daemonize any process. It has another controlling utility supervisorctl that can be used to monitor processes that are being run by supervisor.
You don't have to write any extra code or modify existing scripts to make this work. Moreover, verbose documentation makes this process much simpler.
After scratching my head for hours around python-daemon, supervisor is the solution that worked for me in minutes.
Hope this helps someone trying to make python-daemon work
You can also use Yapdi:
Basic usage:
import yapdi
daemon = yapdi.Daemon()
retcode = daemon.daemonize()
# This would run in daemon mode; output is not visible
if retcode == yapdi.OPERATION_SUCCESSFUL:
print('Hello Daemon')