I have a standalone python script that I currently run from the command line, that I would like to use from a static webpage. I've currently deployed it to heroku, and can run it from console in a dyno, but that's as far as I've gotten.
My script takes .txt files as input, and generates a .csv file as output. I would like to use the tag to request the input file from the user on a form page, pass that file to the python script and execute the script, then make the output file available for user download.
I realize this is a broad question, so I'm not really asking for a step by step solution, but rather an overview of how this needs to be handled, and maybe some links to appropriate docs/tutorials. I've been searching google for hours, but most of the results are framework based webapp tutorials, which is not what I need at this time.
Cheers
Related
I’m working on a project that will need to load python code files dynamically from GitHub on launch. Here’s what it needs to look like:
User asks us to launch an instance for them and provides us with a GitHub url
We have an existing docker with our own python code (a server) that will be using those files from GitHub
We need to launch the docker with our own code, but subbing in parts that we got from the users GitHub, basically creating a server with half our code, half user code
In other words, we need to launch a docker that has some pre planned code from us, and some dynamic code from the user.
Any ideas how to do this? I’ve seen many examples of docker files that load code from GitHub, but I’m having a hard time figuring out how to make it half our code, and half code dynamically from GitHub on run.
I have a scraping script made with Python (beautiful Soup) that scraps data from few websites and saves the data in CSV files.
I need to run this script every day at a certain times.
I am currently running it locally with Schedule Task.
I was wondering if it is possible to run such scripts on a third party hosting server and what kind of characterstic I should look into this web server for it to work.
I would like to avoid the necessity to automatically switch on my pc every day and instead to let everything happen automatically on-line
I am aware of Django and that you can make dynamic HTML pages with it but mime is a different thing and I can not find information regarding it.
apologies upfront. I am an extreme newbie and this is probably a very easy question. After much trial and error, I set up an app on Heroku that runs a python script that scrapes data off of a website and stores it in a text file. (I may switch the output to a .csv file). The script and app are running on a Heroku Scheduler, so the scraping takes place on a schedule and the data automatically gets written to the file that is on the Heroku platform. I simply want to download the particular output file occasionally so that I can look at it. (Part of the data that is scraped is being tweeted on a twitter bot that is part of the script.)
(Not sure that this is relevant but I uploaded everything through Git.)
Many thank in advance.
You can run this command heroku run cat path/to/file.txt, but keep in mind that Heroku uses ephemeral storage, so you don't have any guarantee that your file will be there.
For example, Heroku restarts your dynos every 24 hours or so. After that you won't have that file anymore. The general practice is to store files on some external storage provider like Amazon S3.
Not just ephemeral, but immutable, which means you can't write to the file system. You'll have to put the file in something like S3, or just put the data into a database like Postgres.
I finished up some python tutorials and would like to go a bit further. I can open the IDLE and execute the code just fine by pressing f5 (save and run) on my desktop but that is the limit of my abilities. I would like to be able to execute the programs on a webpage
I tried simply uploading the file to my server, then browsing to it in chrome. I'm sure you know what happened: the url displayed text on the screen.
Since I am brand new to python, I am not sure where to start or even what questions to ask. Basically I would like to run the program in the browswer as if the browswer was the IDLE, or better yet, create an html/css button that runs the program when clicked.
I'd advise you to look into something like flask. It's a micro-framework that includes a basic web server. The documentation should get you most of the way that you want to go.
Running python on the web usually means that when you hit an URL, the server runs some kind of python script, and returns a string - the HTML content of the page you're requesting. You can't use python to 'script' the webpage as you'd use javascript.
You can run python in an interactive interpreter running within a webpage though - just check out try ipython.
Flask is good.
Cherrypy - http://www.cherrypy.org/ - is also a great choice for a really simple way to run python on the server.
Fundamentally, you just configure your web server to execute the file instead of display it. Typically you set this for *.py files but you could restrict it, say, to files in a particular directory. Apparently, your server already has such a setting for PHP files.
Wrapping Python with PHP (obviously) adds neither speed nor security or utility.
Down the line you might want to look at frameworks, mod_python, WSGI, etc, but for your immediate problem, those are severe overkill.
This is limited to static server-side code; JavaScript runs interactively in the visitor's browser, allowing for much richer user interaction. A server-side script runs when the browser attempts to load the page, and the page load finishes when the script is done. If you want something like IDLE in your browser, that's a JavaScript challenge rather than a Python task (and that particular wheel has already been invented, productized, marketed, and sold to the Americans: http://pythonfiddle.com/)
like this:
python -c "import urllib2; exec urllib2.urlopen("http://localhost:8000/test.py").read()"
Can I use os.system() or subprocess.call() to execute a Python program on a webserver?
I mean can I write these functions in a .py script and run it from a web browser and expect the program to be executed?
Thanks a lot.
EDIT:
Sorry for all the confusion, I am giving you more background to my problem.
The reason I am trying to do is this.
I have a Python program that accepts an XML file and returns me TTF file.
I run that program in terminal like this:
ttx somefile.xml
After which it does all the work and generates a ttf file.
Now when I deploy this script as a module on web server. I use a to allow user to browse and select the XML file.
Then I read the file data to temporary file and then pass the file to the module script to be executed like this:
ttx.main([temp_filename])
Is this right way to do it? Because at this point, I don't get any error in the log or in browser. I get blank screen.
When this didn't work, I was going to try os.system or subprocess.call
You do not use os.system or subprocess.call to execute something as a cgi process.
Maybe you should read the Python cgi tutorial here:
http://www.cs.virginia.edu/~lab2q/
If you want your cgi process to communicate with another process on your local machine, you might want to look at "REST frameworks" for Python.
So long as your server is configured to run CGI scripts (Apache's documentation for that is here, for example), yes, you can execute a python script from a webserver. Simply make sure the script is in the appropriate cgi-bin/ directory and that the file has executable permission on the server.
With regards to your comments:
You can, if you really want, explicitly allow other folders on the server to run executable code. I don't know what server you're using, but on Apache this is done by setting Option +ExecCGI for the folder you want. Again, see the docs I linked to.
You need to give an absolute path with respect to the server. As an example, a site I develop has the layout: /public_html/cgi-bin/ When I want to access .cgi or .py files, the url for the site is something like http://chess.narnia.homeunix.com/cgi-bin/index.cgi. You can also set up re-directs to certain files if you want.
One way to pass parameters through your browser is to append them to the URL like an HTTP POST method. Here's a good example of doing that.
Is that what you were looking for with your question, or did you want to actually invoke the python script with os.system()?
Yes, I do it all the time. Import as you would do normally, stick your .py in your cgi-bin folder and make sure the server is capable of handling python.
Another option would be to simply create an application on Google's App Engine. That gives you oodles of resources and APIs for Python execution.
http://code.google.com/appengine
I've done it quite a bit in classic ASP on IIS 5 and above. I would have the ASP engine execute python code (instead of, e.g., vbscript (hearkening back to the old days, here)). Behind those asp pages would be python modules written in straight python that could be imported and could execute pretty much arbitrary code. As others have mentioned, the effective user needs to have execute permission on the thing you're trying to execute.