I have a simple python web application made with flask. It compiles and runs the given c++ files on given input. Upon receiving the inputs from front-end, I use this function to compile and run the given file.
exec_id = get_unique_identifier(data["user_id"])
code = data["code"]
input_data = data["input"]
code_path = file_ops.write_code(code, data["lang"], exec_id)
logging.info("Code written to "+code_path)
#compile the code
logging.info("Compiling the code")
cmd = ['g++',code_path,'-o',exec_id]
logging.info("Command is : "+" ".join(cmd))
try:
compile_code=subprocess.run(cmd, stderr=PIPE)
has_compiled = compile_code.returncode
except Exception as e:
return {"status": "server Error", "output" : e}
compiled_file_path = './'+exec_id
file_ops.write_code is a utility function which takes input code, extension as input and writes the code in a file in the same directory. The code runs alright in local environment, However, In production server with nginx, The code never gets compiled and I get the following error(After a big traceback)
[Errno 2] No such file or directory: 'g++'
I tried the following things
Giving Absolute Path in code
passing shell=True in subprocess.run
Does this have something to do with Configuration with Nginx ? I'm running this application with gunicorn and nginx on Ubuntu VPS. If I run the code in development environment on vps, it runs without any error.
I am having a problem with python code. In main thread i am creating a new threading holding a flask api with:
thread = Thread(target=app.run, kwargs={'host':'127.0.0.1', 'port':5000, 'debug':False, 'use_reloader':False})
then in main thread i have i while loop waiting for commands from terminal with:
while True:
command = input("> ")
the problem is that after some commands i get:
File "run.py", line 44, in cli_app
command = input("> ")
ValueError: I/O operation on closed file.
although my cli is still on and i can communicate from other terminals, that client can no longer create a command.
all i found was about opening csv files, thats why i am asking.
thanks in advace.
Are you trying to create terminal commands in flask, right? so flask have a decorators to make this easy :
import click
from flask import Flask
app = Flask(__name__)
#app.cli.command("create-user")
#click.argument("name")
def create_user(name):
#logic
return
And to Run this command you only need to write this command in terminal flask create-user admin, the first arg is the function and the second is the value(dont forget to use in this, you need to set FLASK_APP), for a complete doc look this flask doc
The other way, if you are using multiple server in same host, create a route like commands, and to call this command access the url.
#app.route("/user")
def create_user():
name= request.args.get('name')
#logic
return
And call it in url localhost:5001/user?name=Thomas
And create a script to run every server in background(i dont now if you are using gunicorn, so i use vallina thread pool`, and i using linux terminal)
nohup python server1/main.py &
nohup python server2/main.py &
nohup python server3/main.py &
nohup python server4/main.py &
nohup python server5/main.py &
Objective:
I am using Ubuntu 16.04 and am using WMI-CLient-Wrapper module to connect to a remote Windows Machine and send an executable to it(eg. Process Explorer) and further execute it and collect the logs it creates and fetch them back to my Linux Machine for further processing. Using WMI CLient Wrapper is the only option available as WMI Module doesn't work with Linux.
Problem:
I am able to send the file to the remote Windows machine, by establishing a connection using WMI-Client-Wrapper and SMB File Transfer Mechanism. After that when I try to create a Process for the same and try to execute that process it gives me an error stating that some of the attributes that WMI actually has, are not supported by WMI client Wrapper.
What I tried
Python Code:
import os
import wmi_client_wrapper as wmic
from socket import *
import time
wmic = wmic.WmiClientWrapper(
host ="192.168.115.128",
username = "LegalWrongDoer",
password = "sasuke14"
)
SW_SHOWNORMAL = 1
str = "smbclient //192.168.115.128/C$ -U LegalWrongDoer%sasuke14 -c \'put \"procexp64.exe\"\'"
os.system(str)
print("Folder sent")
process_startup = wmic.Win32_ProcessStartup.new()
process_startup.ShowWindow = SW_SHOWNORMAL
process_id, result = wmic.Win32_Process.Create(CommandLine="C:/procexp64.exe", ProcessStartupInformation=process_startup)
process_startup.ShowWindow = SW_SHOWNORMAL
if result == 0:
print("Process started successfully")
else:
print("Sorry, but can't execute Process!")
When I run this python file, it gives me the output to the initial query I make. But the Process_StartUp fails.
Further Traceback Calls:
Traceback (most recent call last):
File "WMIClient.py", line 22, in <module>
process_startup = wmic.Win32_ProcessStartup.new()
AttributeError: 'WmiClientWrapper' object has no attribute 'Win32_ProcessStartup'
I'd be extremely grateful if anyone of you can be able to help me through this. Thanks in advance :)
Well I finally managed to get a work-around for this whole scenario, and it might look a little messy but it sure does work for me.
Firstly I use smbclient to transfer the executable to the end-point where I want to execute it. Inside my code I use os.system() calls to make this happen.
import os
str1 = "smbclient //'<HostMachineIP>'/admin$ -U '<domain>\\<username>%<password>' -c \'lcd /usr/local/acpl/bin/endPoint/; put \"EndPointForeignsics.exe\"\'"
os.system(str1)
This helps me put the executable in desired shared folder that the user has access(Admin in my case) to and then use WMI-query through a tool called Winexe to get access to the console/command prompt of the end-point. I use another os.system() call to execute this again.
str2 = r'/usr/local/bin/winexe -U "<domain>\\<username>%<password>" //<HostMachineIP> "cmd /c c:\windows\EndPointForeignsics.exe '
os.system(str2)
P.S:-- Winexe is a tool that you'll have to download off the internet and compile it. It may take some time and effort to do that, but is quite achievable. You'll get a lot of help on the same from StackOverflow and Documentation of the tool.
I am trying to create a secure (e.g., SSL/HTTPS) XML-RPC Client Server. The client-server part works perfectly when the required certificates are present on my system; however, when I try to create the certificates during execution, I receive a FileNotFoundError when opening the ssl-wrapped socket even though the certificates are clearly present (because the preceding function created them.)
Why is the FileNotFoundError given when the files are present? (If I simply close and restart the python script no error is produced when opening the socket and everything works with no issue whatsoever.)
I've searched elsewhere for solutions, but the best/closest answer I've found is, perhaps, "race conditions" between creating the certificates and opening them. However, I've tried adding "sleep" to alleviate the possibility of race conditions (as well as running each function individually via a user input menu) with the same error every time.
What I am missing?
Here is a snippet of my code:
import os
import threading
import ssl
from xmlrpc.server import SimpleXMLRPCServer
import certs.gencert as gencert # <---- My python module for generating certs
...
rootDomain = "mydomain"
CERTFILE = "certs/mydomain.cert"
KEYFILE = "certs/mydomain.key"
...
def listenNow(ipAdd, portNum, serverCert, serverKey):
# Create XMLRPC Server, based on ipAdd/port received
server = SimpleXMLRPCServer((ipAdd, portNum))
# **THIS** is what causes the FileNotFoundError ONLY if
# the certificates are created during THE SAME execution
# of the program.
server.socket = ssl.wrap_socket(server.socket,
certfile=serverCert,
keyfile=serverKey,
do_handshake_on_connect=True,
server_side=True)
...
# Start server listening [forever]
server.serve_forever()
...
# Verify Certificates are present; if not present,
# create new certificates
def verifyCerts():
# If cert or key file not present, create new certs
if not os.path.isfile(CERTFILE) or not os.path.isfile(KEYFILE):
# NOTE: This [genert] will create certificates matching
# the file names listed in CERTFILE and KEYFILE at the top
gencert.gencert(rootDomain)
print("Certfile(s) NOT present; new certs created.")
else:
print("Certfiles Verified Present")
# Start a thread to run server connection as a daemon
def startServer(hostIP, serverPort):
# Verify certificates present prior to starting server
verifyCerts()
# Now, start thread
t = threading.Thread(name="ServerDaemon",
target=listenNow,
args=(hostIP,
serverPort,
CERTFILE,
KEYFILE
)
)
t.daemon = True
t.start()
if __name__ == '__main__':
startServer("127.0.0.1", 12345)
time.sleep(60) # <--To allow me to connect w/client before closing
When I run the above, with NO certificates present, this is the error I receive:
$ python3 test.py
Certfile(s) NOT present; new certs created.
Exception in thread ServerDaemon:
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "test.py", line 41, in listenNow
server_side=True)
File "/usr/lib/python3.5/ssl.py", line 1069, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python3.5/ssl.py", line 691, in __init__
self._context.load_cert_chain(certfile, keyfile)
FileNotFoundError: [Errno 2] No such file or directory
When I simply re-run the script a second time (i.e., the cert files are already present when it starts, everything runs as expected with NO errors, and I can connect my client just fine.
$ python3 test.py
Certfiles Verified Present
What is preventing the ssl.wrap_socket function from seeing/accessing the files that were just created (and thus producing the FileNotFoundError exception)?
EDIT 1:
Thanks for the comments John Gordon. Here is a copy of gencert.py, courtesy of Atul Varm, found here https://gist.github.com/toolness/3073310
import os
import sys
import hashlib
import subprocess
import datetime
OPENSSL_CONFIG_TEMPLATE = """
prompt = no
distinguished_name = req_distinguished_name
req_extensions = v3_req
[ req_distinguished_name ]
C = US
ST = IL
L = Chicago
O = Toolness
OU = Experimental Software Authority
CN = %(domain)s
emailAddress = varmaa#toolness.com
[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = #alt_names
[ alt_names ]
DNS.1 = %(domain)s
DNS.2 = *.%(domain)s
"""
MYDIR = os.path.abspath(os.path.dirname(__file__))
OPENSSL = '/usr/bin/openssl'
KEY_SIZE = 1024
DAYS = 3650
CA_CERT = 'ca.cert'
CA_KEY = 'ca.key'
# Extra X509 args. Consider using e.g. ('-passin', 'pass:blah') if your
# CA password is 'blah'. For more information, see:
#
# http://www.openssl.org/docs/apps/openssl.html#PASS_PHRASE_ARGUMENTS
X509_EXTRA_ARGS = ()
def openssl(*args):
cmdline = [OPENSSL] + list(args)
subprocess.check_call(cmdline)
def gencert(domain, rootdir=MYDIR, keysize=KEY_SIZE, days=DAYS,
ca_cert=CA_CERT, ca_key=CA_KEY):
def dfile(ext):
return os.path.join('domains', '%s.%s' % (domain, ext))
os.chdir(rootdir)
if not os.path.exists('domains'):
os.mkdir('domains')
if not os.path.exists(dfile('key')):
openssl('genrsa', '-out', dfile('key'), str(keysize))
# EDIT 3: mydomain.key gets output here during execution
config = open(dfile('config'), 'w')
config.write(OPENSSL_CONFIG_TEMPLATE % {'domain': domain})
config.close()
# EDIT 3: mydomain.config gets output here during execution
openssl('req', '-new', '-key', dfile('key'), '-out', dfile('request'),
'-config', dfile('config'))
# EDIT 3: mydomain.request gets output here during execution
openssl('x509', '-req', '-days', str(days), '-in', dfile('request'),
'-CA', ca_cert, '-CAkey', ca_key,
'-set_serial',
'0x%s' % hashlib.md5(domain +
str(datetime.datetime.now())).hexdigest(),
'-out', dfile('cert'),
'-extensions', 'v3_req', '-extfile', dfile('config'),
*X509_EXTRA_ARGS)
# EDIT 3: mydomain.cert gets output here during execution
print "Done. The private key is at %s, the cert is at %s, and the " \
"CA cert is at %s." % (dfile('key'), dfile('cert'), ca_cert)
if __name__ == "__main__":
if len(sys.argv) < 2:
print "usage: %s <domain-name>" % sys.argv[0]
sys.exit(1)
gencert(sys.argv[1])
EDIT 2:
Regarding John's comment, "this might mean that those files are being created, but not in the directory [I] expect":
When I have the directory open in another window, I see the files pop up in the correct location during execution. In addition, when running the test.py script a second time with no changes, the files are identified as present in the correct (the same) location. This leads me to believe that file location is not the problem. Thanks for the suggestion. I'll keep looking.
EDIT 3:
I stepped through the gencert.py program, and each of the files are correctly output at the right time during execution. I indicated when exactly they were output within the above file, labeled with "EDIT 3"
While gencert is paused awaiting my input (raw_input), I can open/view/edit the mentioned files in another program with no problem.
In addition, with the first test.py instance running (paused, waiting for user input, just after mydomain.cert appears), I can run a second instance of test.py in another terminal and it sees/uses the files just fine.
Within the first instance, however, if I continue the program it outputs "FileNotFoundError."
The problem contained in the above stems from the use of os.chdir(rootdir) as suggested by John; however, the specifics are slightly different than the created files being in the wrong location. The problem is the current working directory (cwd) of the running program being changed by gencert(). Here are the specifics:
The program is started with test.py, which calls verifyCerts(). At this point the program is running in the current directory of whichever folder test.py is running inside of. Use os.getcwd() to find the current directory at this point. In this case (as an example), it is running in:
/home/name/testfolder/
Next, os.path.isfile() looks for the files "certs/mydomain.cert" and "certs/mydomain.key"; based on the file path above [e.g., the cwd], it is looking for the following files:
/home/name/testfolder/certs/mydomain.cert
/home/name/testfolder/certs/mydomain.key
The above files are not present, so the program executes gencert.gencert(rootDomain) which correctly creates both files as expected in the exact locations mentioned above in number 2.
The problem is indeed the os.chdir() call: When gencert() executes, it uses os.chdir() to change the cwd to "rootdir," which is os.path.abspath(os.path.dirname(__file__)), which is the directory of the current file (gencert.py). Since this file is located a folder deeper, the new cwd becomes:
/home/name/testfolder/certs/
When gencert() finishes execution and the control returns to test.py, the cwd never changes again; the cwd remains as /home/name/testfolder/certs/ even when execution returns to test.py.
Later, when ssl.wrap_socket() tries to find the serverCert and serverKey, it looks for "certs/mydomain.cert" and "certs/mydomain.key" in the cwd, so here is the full path of what it is looking for:
/home/name/testfolder/certs/certs/mydomain.cert
/home/name/testfolder/certs/certs/mydomain.key
These two files are NOT present, so the program correctly returns "FileNotFoundError".
Solution
A) Move the "gencert.py" file to the same directory as "test.py"
B) At the beginning of "gencert.py" add cwd = os.getcwd() to record the original cwd of the program; then, at the very end, add os.chdir(cwd) to change back to the original cwd before ending and giving control back to the calling program.
I went with option 'B', and my program now works flawlessly. I appreciate the assistance from John Gordon to point me toward finding the source of my problem.
I wrote the following script, which sends an email to a specific email address, and saved it inside the .openshift/cron/minutely directory:
import smtplib
g = smtplib.SMTP('smtp.gmail.com:587')
g.ehlo()
g.starttls()
g.ehlo()
g.login('myusername','mypassword')
g.sendmail('myemail','otheremail','message')
I then pushed the script to the server.
I expected the program to run once every minute, and receive an email every minute. However, there is no evidence indicating that my code is being run. Any idea what might be causing the problem? Did I forget a step while setting up my application?
Note: I've checked that the email address and password I provided were correct, and that cron is installed.
EDIT: It seems that the problem is originating from the server:
I deleted the original contents of the file, created 'testfile.txt', and wrote this code instead:
a = open('testfile.txt','r+')
if not a.read():
a.write('Test writing')
a.close()
after waiting for the code to run and ssh-ing into the server, I changed to the directory named app-root/logs and displayed the contents of cron.log, which looked something like this:
Sat Nov 8 11:01:11 EST 2014: START minutely cron run
__________________________________________________________________________
/var/lib/openshift/545a6ac550044652510001d3/app-root/runtime/repo//.openshift/cron/minutely/test_openshift.py:
/var/lib/openshift/545a6ac550044652510001d3/app-root/runtime/repo//.openshift/cron/minutely/test_openshift.py: line 1: syntax error near unexpected token `('
/var/lib/openshift/545a6ac550044652510001d3/app-root/runtime/repo//.openshift/cron/minutely/test_openshift.py: line 1: `a = open('testfile.txt','r+')'
__________________________________________________________________________
Sat Nov 8 11:01:11 EST 2014: END minutely cron run - status=0
__________________________________________________________________________
Could it be that the server is not interpreting the code in my file as python code? Any suggestions welcome.
connect to openshift console
rhc ssh app_name
Change to a directory to have permission to create script:
cd $OPENSHIFT_DATA_DIR
create test01.py script
touch test01.py
Give executing permission to test01.py
chmod +x test01.py
Edit script
nano test01.py
Add a simple code like
print("Hello")
run script:
./test01.py
Error:
./test01.py: line 1: syntax error near unexpected token `"Hello"'
./test01.py: line 1: `print("Hello")'
Now check python path
which python
Output
/var/lib/openshift/your-sesseion-id/python/virtenv/venv/bin/python
Now add a she bang to test01.py
#!/var/lib/openshift/your-sesseion-id/python/virtenv/venv/bin/python
print("Hello")
Now Execute it
./test01.py
Output:
Hello
Conclusion:
Your script should know how to run and where is python path, so add it at the first line of your script