I have developed a Python web server using Flask, and some of the endpoints make use of the subprocess module to call different executables. On development, using the Flask debug server, everything works fine. However, when running the server along with nginx+WSGI (on the exact same machine), some subprocess calls fail.
For example, one of the tools I'm using is Microsoft's dotnet, which I installed from my user as sudo apt-get install -y aspnetcore-runtime-5.0 and is then called from Python with the subprocess module. When I run the server with python3 server.py, it works like a charm. However, when using nginx and WSGI, the subprocess call fails with an exception that says: /bin/sh: 1: dotnet: not found.
I suspect this is due to the command not being accessible to the user and group running the server. I have used this guide as a reference to deploy the app, and on the wsgi .ini file, I have set uid = javierd and gid = www-data, while on the systemd .service file I have User=javierd, Group=www-data.
I have tried to add the executables' paths to /etc/profile, but it didn't work, and I don't know any other way to fix it. I find also very surprising that this happens to some executables, but not to all, and that it happes to dotnet, for example, which is located at /usr/bin/dotnet and therefore should be accessible to every user. Any idea on how to solve this problem? Furthermore, if somebody could explain me why this is happening, I would really appreciate the effort.
Thanks a lot!
Ok, finally after having a big headache, I noticed the error, and it was really simple.
On the tutorial I linked, when creating the system service file, the following line was included: Environment="PATH=/home/myuser/myfolder/enviroment/bin".
Of course, as this was overriding the path, there was no way of executing the commands. Once I notices it I just removed that line, restarted the service, and it was fixed.
Related
I have been attempting to run my python script as a service, and have followed the advice contained in several previous forum posts. However, these have not helped me thus far. Here is what I have attempted up until now:
Used an SMWinservice class that allowed me to install my Python script as a service. This solution, however, would not launch. If I would try to start it, I would receive an error message.
I have tried using NSSM with : nssm install myService pathToInterpreter PathToScript. Note, all files etc. can be found in the local directory of the PathToScript. For what its worth, I also tried using the GUI version of NSSM. When I rebooted my computer, it showed as "Paused" in task manager. I stopped it, and then tried to run it again, and I received an error. I have tried NSSM with both python.exe and pythonw.exe.
I suspect that a possible source of error is the fact that my program uses a text file as a configuration file. This configuration file has been coded as being in my local working directory. However, I wouldn't think this would be an issue with NSSM. I know for a fact that my program will completely exit, using the exit command, if the configuration file is not found.
I was thinking of doing a batch file and starting the batch file with the script and running it like that, but I prefer a service since it can monitor the process, can restart it, or windows can notify me via email if there is an issue with my service.
For completeness, I should also mention that the program runs without issue outside of a service. For the reason that the program runs as expected, I decided to not post the code, unless someone would like to see it for whatever reason.
I have a situation where if I run Apache with wsgi (now uninstalled), a test website works, but running the same server with runserver 0.0.0.0:8080 gives ERR_CONNECTION_REFUSED from local or remote (even with the apache2 service stopped).
Edit: I don't think it's Apache, I've reproduced the problem on a clean server with no Apache installed, so unless Apache somehow modified something under source control it's not that
My knowledge of web details is hazy, I don't even know where to troubleshoot this problem - the devserver runs (runserver prints as expected and doesn't give any errors) but never receives a request, I have nothing in iptables.
Sorry for anyone who read this, it would probably have been impossible to solve given my supplied information.
What had actually happened was that I'd been having to modify my wsgi.py script in order to make it happy inside the Apache server, and I'd added a line which said "os.system('/bin/bash --rcfile )" to try and make sure that when running inside apache it got the virtualenv activated.
This line must have been causing some strange problem, another symptom was that I realised when I was running "runserver", it wasn't crashing the the python process was backgrounding itself, where normally it runs inside that console window.
Thanks everyone who asked questions helping me debug!
I have set up a service, and when I run it, I get the following error:
ImportError: No module named httplib2
I have httplib2 installed with pip and
my systemd ExecStart command is like this:
ExecStart=/usr/bin/python /home/orionas/Desktop/quickstart.py
The same script runs perfect from command line.
Hmm, I think you probably have installed httplib2 under your user but systemd uses another user to run the quickstart script.
Under [Service] include a line "User=" The python script will then inherit the permissions and paths of that user AFAIK.
Note: It is probably Not recommended to run a systemd service with userid the same as yours. Potential security risk. Another possible solution would be to run the python script within a [virtualenv] http://docs.python-guide.org/en/latest/dev/virtualenvs/ Many people do this and as far as I know it's the recommended practise
Some of my Python shell scripts are newly throwing security errors under Apple OSX 10.11, El Capitan. It seems the new App Transport Security doesn't like how the scripts are calling HTTP resources in plain text, rather than using HTTPS.
Fetching http://blahblah.com
Python[5553:5648168] App Transport Security has blocked a cleartext HTTP (http://)
resource load since it is insecure. Temporary exceptions can be configured
via your app's Info.plist file.
How might I go about fixing this? There is no HTTPS resource I can call, so I'm stuck with HTTP. The advice from Apple is to make an exception in the app's info.plist file, but this is a Python script invoked from a shell script, so there is no info.plist file to be edited.
Ideas? The root problem seems to be with webkit2png, which is in Python. Its non-HTTPS requests are being blocked by ATS, and there is no info.plist to modify.
I found a solution here that worked for me: https://apple.stackexchange.com/questions/210588/how-does-one-configure-a-temporary-exception-to-ats-on-el-capitan-and-fix-webkit
First make sure you have a version of webkit2png that is new enough to have the --ignore-ssl-check option. Version 0.5 does NOT have this option.
Second, you need to edit the source file and add a couple lines of code as shown here: https://github.com/bendalton/webkit2png/commit/9a96ac8977c386a84edb674ca1518e90452cee88
Finally use option as indicated in the solution linked above (copied here for convenience):
webkit2png --ignore-ssl-check [options] [http://example/]
Thanks for Arthur Hebert
At first I'm confused for the code then I figure it out
so I summarise the steps as follows for reference
import AppKit
Add the following code in your py script
AppKit.NSBundle.mainBundle().infoDictionary()['NSAppTransportSecurity'] = dict(NSAllowsArbitraryLoads = True)
Assume you are sure to use no HTTPS resource in MAC OSX higher than 10.11
The installation of webkit2png is not necessary in my case
I have tried using both subprocess() and os.system()
import os
def whatever(request):
open file
code write to file
close file
os.system(command theFileIwroteto argument)
Now the code is fine. I have opened python and literally copied and pasted the exact command in and it works fine.
python:
import os
os.system(command theFileIwroteto argument)
Why would it work using the python shell but refuse to execute from the django views file?
You should look at the error log in order to make a proper diagnostic.
Without more information, I can think of two reasons:
The Apache mod_wsgi restricts access to stdout, and the command you are running outputs to stdout. If you are using mod_wsgi, look for errors like IOError: sys.stdout access restricted by mod_wsgi in the logs;
You are testing your command while logged in as your user or root, but the webserver is running as other user with less privileges (like www-data in Ubuntu). Find out under what user the webserver is running, su to the same user and try the same commands. Any error will hint you about the permissions lacking.
It is almost always a bad idea to run a shell from a Django view, specially if the command take a few seconds to run. It has bad scalability, makes easier for someone to DDoS your site and makes responses slow. The proper way to do it is to place the job on a queue and have one or more background processes consuming this queue. Take a look at celery, its popular for this kind of task.