We Keep Coding

Auto run from rc.local not working when forked (&)

I've been having a lot of problems trying to get my python script to run at boot. I've essentially narrowed it down to a problem with forking.
I'm running on a RPi3.
In rc.local if I have:
python /home/pi/script.py
It seems to run, however as soon as I add
python /home/pi/script.py &
I get zero results.
If I run rc.local manually after boot the fork appears to work as expected.
I've also tried to point rc.local to a .sh file in /home/pi with exactly the same results. This even happens with basic echo commands:
echo "Hello world" > /tmp/log.txt
vs
echo "Hello world" > /tmp/log/txt &
Any help would be greatly appreciated.

I'm guessing that this has something to do with the fact that all child processes of the rc.local script get killed as soon as it reaches the end of the script, which is just about instantaneous if the only command in the file is running a python script as a background process.
The fork will get killed before it can do anything useful.
If you want the process to keep running after rc.local has ended, you should run the process as a daemon.
Some examples on how to do this can be found in this question: Run bash script as daemon

nohup multiple sequential commands not logging output

I have a python script (which takes a lot of time to complete the execution) that I need to run several times varying the parameters. And it's executed in a remote machine. For instance and test purposes, take the following script test.py:
import time
print("\nStart time: {}".format(time.ctime()))
time.sleep(10)
print("End time: {}".format(time.ctime()))
For this I normaly use nohup. It's work just fine whith one execution using the following command:
nohup test.py &
The outputs are correctly saved in nohup.out file. To run in sequence I've done some research and found [this question][1] and I came up with the following command:
nohup $(python test.py; python test.py) &
Which works fine. I run the command, quickly logged out and in again and saw through htop the first execution running, then finishing and then the second one starting. But the problem is that the output isn't been saved into nohup.out file. If I wait in terminal for both executions to finish, the following error is showed:
nohup: failed to run command 'Start': No such file or directory
What am I doing wrong here?
PS.:
I need to log the outputs because I need to see the current progress of the script and know which error happened if it doesn't finish properly. So if there is some other command to use instead of nohup which could log python print's it will be welcomed too.

The command you have:
nohup $(python test.py; python test.py) &
Will attempt to execute the output of the script. It's likely not what you wanted.
What you wanted here is to have nohup start off a command that will execute the two commands in sequence. The most straight forward program that can do this is to use run a child shell:
nohup bash -c "python one.py; python two.py" &
As for a better way to do this, you might want to investigate tmux or screen. If you start off a command in a tmux/screen, not only you can detach the session from the currently running shell, you'd also be able to reconnect to the session later on to resume and interact with the program.

The nohup command is passed a utility and arguments for that utility.
If you like to run your script in sequence via nohup, perhaps using bash as the utility isn't a bad idea
nohup bash -c "python ./test.py; python ./test.py"
However, I recommend to look into using python's logging package as I consider nohup to be a workaround (nohup only appends to nohup.out if the standard output is the terminal.)
Also, there is the approach of using a queue to manage the running of your tasks sequentially.
Here you needn't have to be verbose to run the same script twice. Then again between what you have and writing a worker to consume the queue, I think what you've is simpler ¯\_(ツ)_/¯

Open terminal, run python script and keep it open for results?

How to get an sh script for starting a new terminal, execute a python script and keep it running? The python script is supposed to run continuously in a perpetual loop, spitting out results as they pop in. Whenever trying with sh-script for gnome-terminal just getting: child process exited normally with status 2
Manually it would just be: python home/ubuntu/pyscript.py
Could someone give an idea how to do this?
I have a list of scripts to run, so resorting to the manual solution is tedious.

You can use gnome-terminal with the -x flag.
Suppose you have a spam.py script; then the following command will spawn a new terminal, run spam.py in it, and close the terminal once the script has ended.
gnome-terminal -x python spam.py
Try with this script:
# spam.py
import time
for _ in range(5):
print("eggs")
time.sleep(1)
Then the previous command will spawn a terminal, that will be printed eggs five times, and then will be closed.
If you want to leave the terminal open with the Python interpret still running after the script ended, then Python's -i flag (doc then CTRL+F -> -i) is what you want:
gnome-terminal -x python -i spam.py

To run the Python script in a new instance of your favourite terminal, write:
x-terminal-emulator -e python -i home/ubuntu/pyscript.py
This will start the Python script and run it until it ends, then display a Python prompt to stop the terminal emulator from closing.
This will work with x-terminal-emulator substituted with any of the many, many terminals installed on my computer, so will work with little modification across all POSIX-compatible systems with the standard terminals installed. This won't work on a Mac, however. For a properly cross-platform Python implementation of something slightly different, see here. Most of the techniques should be transferable.
To run the Python script in the same terminal whilst carrying on with the rest of the shell script, write:
python home/ubuntu/pyscript.py &
Note the &, which runs the program as a new process (but still connects the output to the virtual terminal).

Run python script from rc.local does not write file

I want to run a python script at boot of Lubuntu 15.04. This python script writes some string into a text file placed in /home/aUser/aFile.txt
My /etc/rc.local file is:
#!/bin/sh -e
python /home/aUser/theScript.py &
exit 0
And the script /home/aUser/theScript.py is:
#!/usr/bin/python
f = open('/home/aUser/aFile.txt','w');
f.write("Some string...");
f.close();
Actually the python script does more, and run an infinite loop, this is why I run the script in background with &. Of course I have python installed:
~$ python --version
Python 2.7.9
I checked if /etc/rc.local is called at boot, and it is, proof of that: I added a test into the /etc/rc.local in this way:
#!/bin/sh -e
python /home/aUser/theScript.py &
exit 0
echo "Test" >> /home/aUser/aTest.txt
and the file /home/aUser/aTest.txt is written/created at boot.
So everything looks correct, proof of that:
if I run manually ~$ /etc/rc.local the file aFile.txt is correctly written.
Instead if I start (or reboot) the OS, the file is not written at boot.
I suspect that could be a problem of permissions/user: I know that /etc/rc.local is run as root, but even if I set root or aUser as owner of the file, the situation is the same. Also run the python script in the /etc/rc.local as user aUser (with su command) does not solve the problem.

Ok I found the problem and fix it, thanks to the #Zac comment.
Actually the python script try to open a network connection before writing the file: at boot time, when the python script is run from /etc/rc.local (so, it is run), the network is still not ready (probably because it is a wireless network) and therefore an exception is raised and the entire script stops. Capturing the exception solves the problem.
So at the end it was my fault, (not) helped by the rc.local that does not provide an easy way to debug.

python multithreading issue in cronjob

I have a python program that uses the ThreadPool for multithreading. The program is one step in a shell script. When I execute the shell script manually on the command line, the entire flow works as expected. However, when I execute the shell script as a cronjob, it appears that the flow goes to the next steps before the python multithreading steps are completely finished.
Inside the python program, I do call AsyncResult.get(timeout) to wait for all the results to come back before moving on.

Run your program via batch(1) (see the output of the command man batch) as well. If that works OK, but the cron version does not, then it is almost certainly a problem with your environment variable setup. To verify that, run printenv from your interactive shell to inspect your environment there. Then do the same thing inside the crontab (you will just need to temporarily set up an extra cron entry for it). Try setting the variables in your shell script before invoking Python.
On the other hand, if it doesn't work via batch(1) either, it could be something to do with the files that your code has open. Try running your shell script with input redirected from /dev/null and output going to a file:
$ /usr/local/bin/myscript </dev/null >|/tmp/outfile.txt 2>&1

Try setting "TERM=xterm" (or whatever env variable you have, figure out by command 'env' on your terminal) in your crontab.