Running python cgi script Interpreter results differ to browser - python

I was having difficulty converting a program I made to a cgi script. I suspected it was to do with os.walk so I made a smaller test script to test this.
(I noticed the single \ before the D in the variable loc and tried changing that to a double \ still no change)
Produces no errors cant tell why it doesn't run the for loop with os.walk in the browser.
I tried adding some data into s and run for loop printing of contents of it and that worked fine, but trying to do it on os.walk I can't seem to get it to work. I can't find anything relating to the issue on google or stackoverflow.
Below is the code:
import cgi,cgitb,os
loc = "C:\\Users\\wen\Desktop\\sample data\\old py stuff\\"
cgitb.enable(display=1,logdir=loc)
s = []
print("Content-type:text/html\r\n\r\n")
print("<html>")
print("<body>")
print("<p>"+loc+"</p>")
for r,ds,fs in os.walk(loc):
print("<p>omgwtf</p>")
for f in fs:
s.append(f)
for i in s:
print("<p>"+i+"</p>")
print("</body>")
print("</html>")
Took a screenshot, the output in interpreter on the left and browser on right
i.imgur.com/136y1Yq.jpg
webserver is running iis7

I'm pretty sure I've solved the problem, I needed to give the folders permissions for 'Authenticated users'.

Related

"with open as f" won't open and It doesn't even say there's a problem

import requests
res = requests.get("https://google.com")
res.raise_for_status()
print(len(res.text))
print(res.text)
with open('mygoogle.html',"w",encoding=('utf-8')) as f :
f.write(res.text)
The with open as f part doesn't work.
If it does, it has to open new file, but it doesn't.
I have tested this code and it works properly - mygoogle.html gets written and contains the expected content. The issue is likely outside of the script then. Since it is using a relative path, the issue could be that the file is getting written somewhere where you don't expect it due to the working directory not being set how you expect - Windows often defaults to C:\Windows\system32 or your user profile directory (%userprofile% aka C:\users\<username>) for new cmd instances.

Finding file in windows path

I'm trying to get someone else's Python script to run on my computer and part of the script finds a file in the USERPROFILE. Here is that code:
for w in os.walk(os.getenv('USERPROFILE')):
if 'FilenName' in w[1]:
path = str(w[0]) + r'\FilenName\UsrData\Directory\Data'
However, in the above code, the program tries to search in the following directory:
C:\Users\User\AppData\Roaming\
When, in fact, the program should be looking in
C:\Users\User\AppData\Local\
If I replace the problem code with the following, it works, but I need it to run for all USERPROFILEs, not just mine:
path = r'C:\Users\Bill\AppData\Local\FilenName\UsrData\Directory\Data'
What is the solution to this?
Thanks.
I'm not on a windows machine, so this is a bit tricky, but could you find all user profiles using the env var ALLUSERSPROFILE?
Another option may be to replace "Roaming" with "Local" in the string. It's a bit hacky, but can be done:
for w in os.walk(os.getenv('USERPROFILE')):
if 'FilenName' in w[1]:
path = (str(w[0]) + r'\FilenName\UsrData\Directory\Data').replace('Roaming', 'Local')

Executing R files from inside Python using PypeR

My current work project is on writing a Python program that must at various points rely on R. Since I don't know R so well, and the person helping me doesn't know Python so well, the actual R code is not in my program. Instead, he opened Notepad, put the R code in there, and saved it as (name).r. When executed, the output is written into a txt file, which Python can then read.
All I have to do is ask Python to ask R to run (name).r
I've tried using subprocess.run. That worked for awhile, and then for some unknown reason stopped working and now does nothing. Then I tried using rpy2, which also worked for awhile; but now it looks like the installation is broken and I'm having trouble getting it reinstalled.
I'd like to give a 3rd option a try now: PypeR. I used pip install pyper. Looked like it was successful.
To keep things simple, I opened Notepad and typed in the following, and saved it as hello.r:
message <- 'goodbye'
write.table(message,'C:/Users/(my name)/Desktop/(folder)/goodbye.txt',row.names=FALSE,col.names=FALSE)
Manually opening R and copy-pasting the lines in one-at-a-time does indeed work. But I'm having trouble getting it to work from Python. Here are some things I've tried (I always put import pyper at the top):
pyper.runR("source('C:/Users/(muy name)/Desktop/(folder)/hello.r')")
This gives NameError: name 'dump_stdout' is not defined
pyper.R("source('C:/Users/(my name)/Desktop/(folder)/hello.r')")
This gives FileNotFoundError: [WinError 2] The system cannot find the file specified
r=pyper.R("C:/Program Files/R/R-3.4.1/bin/i386/Rgui.exe")
r("source('C:/Users/(my name)/Desktop/(folder)/hello.r')")
This causes RGui to open up with a blank R Console. And then nothing happens. When I click back to Python, the console shows Python is busy until I click the halt button, whereupon I get "OSError: [Errno 22] Invalid argument
What is the correct way to execute hello.r?
Thank you
Looks like I got it. This works:
r=pyper.R(RCMD="C:/Program Files/R/R-3.4.1/bin/R")
r.run("source('C:/Users/(my name)/Desktop/(folder)/hello.r')")

Executing subprocess cannot find specified file on Windows

I'm working inside a system that has Jython2.5 but I need to be able to call some of Google's apis so I wrote an offline script that I wanted to call from my Jython environment and return to me small pieces of data. Like a JobID or a sheet URL or something from Google.
I've tried a number of things but I always get an error back from Windows, saying that it cannot find the file specified.
Path is done in two ways.
The first way using a string
stringPath = r"‪C:\GooglePipes\Scripts\filetobq.py C:\GooglePipes\Keys\DEV-BigQueryKey.json nofile C:\GooglePipes\BQ_Downtime\TESTFILE.CSV dataset1 table1"
And the second way, as a sequence (per the docs, using shell=false supply a sequence)
seqPath = [r"‪C:\GooglePipes\Scripts\filetobq.py",r"C:\GooglePipes\Keys\DEV-BigQueryKey.json","nofile",r"C:\GooglePipes\BQ_Downtime\TESTFILE.CSV","dataset1","table1"]
Called with
data, err = Popen(seqPath, shell=True, stderr=PIPE, stdout=PIPE).communicate()
#Read values back in
print data
print err
Replacing seqPath with stringPath to try it either way.
I've been at this all weekend, every time I run it I get from Windows
The system cannot find the path specified.
from the err print. I've been unable to debug much further than this. I'm not really sure what's happening. When I paste the stringPath variable directly into my computer's command window it executes.
I've also called subprocess.list2cmdline(seqPath) to see what it's outputting. It's giving me a ? in front of the string, but I haven't been able to figure out what that means. I can paste the rest of the string, starting after the question mark into the command window and it executes.
?C:\GooglePipes\Scripts\filetobq.py C:\GooglePipes...
I've tried a number of different combinations of true and false on shell, passing different args into Popen, double slashes, and I have no less than 30 tabs open from stack overflow and other help forums. I just have no idea what to do at this point and any help is appreciated.
Edit
The ? at the start of the sting is actually a NULL character when I did some additional logging. This seems to be the root of my problem. I can't figure out why it shows up, but it was present in my copy pastes. I started manually typing, and I got it working. When I feed the path with my Jython program it is present again.
Ultimately the error was the ?/NULL character.
I went back to the source value where the program was grabbing the path and it was present there. After I hand-re keyed it in, everything started working.
If you copy and paste what I put in the question, you can see the NULL character in the string if you run it through a string->ASCII converter.
>C:
>NULL 67 58
What a bunch of bullsh***.

Printing to NamedTemporaryFile on Windows

The original Problem - MCVE
The following script should use chrome headless, to print to pdf (I am running windows 10, and python 3.6):
import subprocess
from tempfile import NamedTemporaryFile
output = NamedTemporaryFile()
CHROME_PATH=r'"C:\Program Files (x86)\Google\Chrome\Application\chrome"'
chrome_args=[CHROME_PATH,
'--headless',
r'--print-to-pdf="{}"'.format(output.name),
'--disable-gpu',
'https://www.google.com/',]
subprocess.call(chrome_args,shell=True)
However the generated file, is just empty.
Attempt at debugging
To try and figure out what is going, on I adapted the script to the following:
import subprocess
CHROME_PATH=r'"C:\Program Files (x86)\Google\Chrome\Application\chrome"'
chrome_args=[CHROME_PATH,
'--headless',
r'--print-to-pdf="c:\Users\timmc\Documents\output.pdf"',
'--disable-gpu',
'https://www.google.com/',]
print(r" ".join(chrome_args)) #For debuging
subprocess.call(chrome_args,shell=True)
In this case, there is just no file generated at the expected location. The result of the print is:
"C:\Program Files (x86)\Google\Chrome\Application\chrome" --headless --print-to-pdf="c:\Users\timmc\Documents\output.pdf" --disable-gpu https://www.google.com/
if I run the following (creating a raw string literal), everything works as expected and the file is produced.
subprocess.call(r'"C:\Program Files (x86)\Google\Chrome\Application\chrome" --headless --print-to-pdf="c:\Users\timmc\Documents\output.pdf" --disable-gpu https://www.google.com/', shell=True)
Having searched around on stack-overflow, and tried a few things, I still can’t get the original script to work. Any ideas?
Part of the problem is that I can't seem to get any meaningful debug from the subprocess call. Any help with that would also be much appreciated.
I'll try to answer instead of commenting again and again, but obviously I cannot test this.
The issue is mainly the forcing of the double quotes & shell=True. Leaving the quoting to subprocess (also in CHROME_PATH) and splitting arguments properly usually work. I've solved a lot of questions here with this technique.
Since your comments state that it does not, and that you found a workaround, let me suggest an improvement of this workaround: injecting the output filename in the command line that works:
subprocess.call(r'"C:\Program Files (x86)\Google\Chrome\Application\chrome" --headless --print-to-pdf="{}" --disable-gpu https://www.google.com/'.format(output.name), shell=True)
not satisfactory to me but it has a good chance to work.
It turns out that the reason the subprocess wasn't running properly, is that when python creates a NamedTemporaryFile in windows, it does so with a FILE_SHARE_DELETE tag which prevents any other process accessing it unless it also has this tag. There is more discussion of this here.
Fortunately, Django comes with its own NamedTemporaryFile which was made to partially address this problem, and does so well enough for these purposes.

Categories