I'm trying to understand what paste script and paster are. The website is far from clear.
I used paster to generate pre-made layouts for projects, but I don't get the big picture.
As far as I understand, and from the wikipedia entry, it says it's a framework for web frameworks, but that seems reductive. paster create seems to be able to create pre-made layouts for setuptools/distutils enabled packages.
What is the problem (or set of problems) it's trying to solve?
Paste got several components:
Paste Core: various modules to aid in creating wsgi web apps or frameworks (module index). Includes stuff like request and response objects. From the web site: "The future of these pieces is to split them into independent packages, and refactor the internal Paste dependencies to rely instead on WebOb". If you're considering using components from paste core, I suggest you look at the spin-offs instead, like WebOb.
Paste Deploy: a system for loading and configuring WSGI applications and servers (module index). Basically some stuff to read a config file and create a WSGI app as specified in the file.
Paste Script: A framework for defining commands. It comes with a few commands out of the box, like paster serve (loads and serves a WSGI application defined in a Paste Deploy config file) and paster create (creates directory layout for packages etc). The best intro to paste script I found is http://pythonpaste.org/script/developer.html
Here's the source for the paster serve command: serve.py.
And paster create: create_distro.py.
PasteScript (and its companion PasteDeploy) are tools for running Python code using 'entry points'. Basically, a python library can specify in metadata that it knows how to create a certain kind of Python project, or perform certain operations on those projects. paster is a commandline tool that looks up the appropriate code for the operation you requested. It's a very general kind of problem; if you're familiar with Ruby at all, the equivalent might be 'rake'.
In particular, PasteDeploy is a configuration format to serve Python webapps using paster. Both PasteScript and PasteDeploy are important for the Pylons web framework.
Related
I have developed a Flask project, that uses Waitress for production. I have followed this link
The thing is that I don't want to send all the project to the production server. I mean, as an example production in Java, I make .war of the project and run on the Tomcat server.
How can I do this in Python with Flask?
Python is an interpreted language, so there is no need to "build" your code. The reason Java includes a build step is that Java requires compilation. At high level, all you do is put your Python code on a server, installing any pip requirements, and telling Apache/Nginx where to find your code and how to run it.
If you have complex Javascript as part of your code, you may very well have a build step for the Javascript, such as using babel and webpack. But that should be entirely separate from getting the Python side to work.
If you want to protect your code for some reason, you can do that with a code obfuscation or by just deploying the pyc files, but all of these are complex techniques that you should only do if deploying to untrusted servers. See this article for some examples.
As an aside, Flask has a guide for how to configure it with mod_wsgi and Apache: https://flask.palletsprojects.com/en/1.1.x/deploying/mod_wsgi/
I have a Python web application that I want to wrap in Electron. The web application backend is a very slim Flask app that forwarded calls to a Python package that does the processing, and formats the results. We have a React frontend that talks to this backend. We also have a pip based installation, that runs the Flask backend and serves the frontend, so you can pip install run the server and use it from your browser. This is similar to how Pgadmin 4 works.
Since this application is only used by people on their own computers, and never installed on a server, I want to convert it into an Electron app. However, I couldn't figure out how to distribute this application in one setup for Windows, MacOS and Linux. I don't want the users to have to install Python on their computers.
How can I do that?
There is a couple of clues on how to do that, even though I'm still unsure whether all necessary python modules can be bundled easily.
I have a similar case, even though I just want to bundle a prototype in an electron application so I can send it to collaborators for evaluation, without any intent of shipping it to final users.
My list of hints:
https://github.com/matbloch/electron-flask
https://efficientcoder.net/connect-python-3-electron-nodejs-build-desktop-apps/
https://www.techiediaries.com/flask-electron-tutorial/
I really don't see why you need to throw electron in the mix, instead of just using your browser. I reckon that a barebone electron app that serves your page in a single window is going to be 50Mb. The key benefits of electron is that it lets you do system calls (access local files / devices), but if you are running flask you already have this ability.
Your main obstacle is how to distribute the flask app, specifically without installing python - and electron is not going to make things any easier to that respect. You should probably look at pyinstaller which lets you create executables that embed python.
Now, if you're talking of getting rid of python altogether, then indeed you could do that, nodejs has a rich set of libraries for everything os / db-related, even image processing, but it will lack in data science and processing. YMMV.
I have a django project which I want to deploy on apache2 Http Server.
However, I want to automatically copy all the python files from some directory to apache srv directory which is /srv/www/myproject. Is there any automatic python tool which could solve the purpose.
I have looked into DistUtils and setup.py but I am unsure about how I would copy all .py files (along with directory structure) to the apache directory.
Any help will be appreciated!!!
Take a look to fabric, it is a great tool to do automatic deployments
From the django tutorial:
If your background is in PHP, you're probably used to putting code
under the Web server's document root (in a place such as /var/www).
With Django, you don't do that. It's not a good idea to put any of
this Python code within your Web server's document root, because it
risks the possibility that people may be able to view your code over
the Web. That's not good for security.
Put your code in some directory outside of the document root, such as
/home/mycode.
For copying files, you can use something as simple as FTP/SCP which you can automate; or you can use more full blown deployment options like fabric (see this blog entry for a step-by-step guide on fabric + virtualenv + apache mod_wsgi).
You can use any tool to automate the task; but please put the code in the appropriate non-web browsable directory.
Is there any lightweight mvc webframework which is not necessary to install to the server?
I need something simple, that i could just copy to the shared hosting. And it must handle urls other that localhost/test.py, something like this localhost/Blog/test
You should probably check out Flask or Bottle, two nice Python microframeworks. With an appropriate "main" Python script (to initialize your app and dispatch requests to it) and mod_rewrite rules in place, you can probably get pretty close to your goal of "just copy[ing] to the shared hosting" with nice URLs.
Flask has good documentation on deploying via CGI, which is what you might have to use on your shared host. (If your host supports FastCGI or mod_wsgi, those deployment options would be preferable.)
Checkout web2py. Seems to be about the simplest python based webserver I can think of.
Django might do, it's hefty, but it comes with it's own development server.
web2py includes everything (ssl-enabled web server, sqlite sql based transaction safe database, web based Integrated Development Enviroment, web based database interface) in one package. The web2py binaries for windows and mac also include Python itself. web2py does not require configuration or installation and can run off a usb drive. It was originally developed as a teaching tool for MVC.
checkout https://github.com/salimane/bottle-mvc or https://github.com/salimane/flask-mvc . They are boilerplates that could get you started with controllers, models in separate folders. They are based on bottle and flask micro frameworks, no useless features, they give you the flexibility to plugin whatever modules you want.
I am deploying a WSGI application. There are many ways to skin this cat. I am currently using apache2 with mod-wsgi, but I can see some potential problems with this.
So how can it be done?
Apache Mod-wsgi (the other mod-wsgi's seem to not be worth it)
Pure Python web server eg paste, cherrypy, Spawning, Twisted.web
as 2 but with reverse proxy from nginx, apache2 etc, with good static file handling
Conversion to other protocol such as FCGI with a bridge (eg Flup) and running in a conventional web server.
More?
I want to know how you do it, and why it is the best way to do it. I would absolutely love you to bore me with details about the whats and the whys, application specific stuff, etc.
As always: It depends ;-)
When I don't need any apache features I am going with a pure python webserver like paste etc. Which one exactly depends on your application I guess and can be decided by doing some benchmarks. I always wanted to do some but never came to it. I guess Spawning might have some advantages in using non blocking IO out of the box but I had sometimes problems with it because of the patching it's doing.
You are always free to put a varnish in front as well of course.
If an Apache is required I am usually going with solution 3 so that I can keep processes separate. You can also more easily move processes to other servers etc. I simply like to keep things separate.
For static files I am using right now a separate server for a project which just serves static images/css/js. I am using lighttpd as webserver which has great performance (in this case I don't have a varnish in front anymore).
Another useful tool is supervisord for controlling and monitoring these services.
I am additionally using buildout for managing my deployments and development sandboxes (together with virtualenv).
I would absolutely love you to bore me with details about the whats and the whys, application specific stuff, etc
Ho. Well you asked for it!
Like Daniel I personally use Apache with mod_wsgi. It is still new enough that deploying it in some environments can be a struggle, but if you're compiling everything yourself anyway it's pretty easy. I've found it very reliable, even the early versions. Props to Graham Dumpleton for keeping control of it pretty much by himself.
However for me it's essential that WSGI applications work across all possible servers. There is a bit of a hole at the moment in this area: you have the WSGI standard telling you what a WSGI callable (application) does, but there's no standardisation of deployment; no single way to tell the web server where to find the application. There's also no standardised way to make the server reload the application when you've updated it.
The approach I've adopted is to put:
all application logic in modules/packages, preferably in classes
all website-specific customisations to be done by subclassing the main Application and overriding members
all server-specific deployment settings (eg. database connection factory, mail relay settings) as class __init__() parameters
one top-level ‘application.py’ script that initialises the Application class with the correct deployment settings for the current server, then runs the application in such a way that it can work deployed as a CGI script, a mod_wsgi WSGIScriptAlias (or Passenger, which apparently works the same way), or can be interacted with from the command line
a helper module that takes care of above deployment issues and allows the application to be reloaded when the modules the application is relying on change
So what the application.py looks like in the end is something like:
#!/usr/bin/env python
import os.path
basedir= os.path.dirname(__file__)
import MySQLdb
def dbfactory():
return MySQLdb.connect(db= 'myappdb', unix_socket= '/var/mysql/socket', user= 'u', passwd= 'p')
def appfactory():
import myapplication
return myapplication.Application(basedir, dbfactory, debug= False)
import wsgiwrap
ismain= __name__=='__main__'
libdir= os.path.join(basedir, 'system', 'lib')
application= wsgiwrap.Wrapper(appfactory, libdir, 10, ismain)
The wsgiwrap.Wrapper checks every 10 seconds to see if any of the application modules in libdir have been updated, and if so does some nasty sys.modules magic to unload them all reliably. Then appfactory() will be called again to get a new instance of the updated application.
(You can also use command line tools such as
./application.py setup
./application.py daemon
to run any setup and background-tasks hooks provided by the application callable — a bit like how distutils works. It also responds to start/stop/restart like an init script.)
Another trick I use is to put the deployment settings for multiple servers (development/testing/production) in the same application.py script, and sniff ‘socket.gethostname()’ to decide which server-specific bunch of settings to use.
At some point I might package wsgiwrap up and release it properly (possibly under a different name). In the meantime if you're interested, you can see a dogfood-development version at http://www.doxdesk.com/file/software/py/v/wsgiwrap-0.5.py.
The absolute easiest thing to deploy is CherryPy. Your web application can also become a standalone webserver. CherryPy is also a fairly fast server considering that it's written in pure Python. With that said, it's not Apache. Thus, I find that CherryPy is a good choice for lower volume webapps.
Other than that, I don't think there's any right or wrong answer to this question. Lots of high-volume websites have been built on the technologies you talk about, and I don't think you can go too wrong any of those ways (although I will say that I agree with mod-wsgi not being up to snuff on every non-apache server).
Also, I've been using isapi_wsgi to deploy python apps under IIS. It's a less than ideal setup, but it works and you don't always get to choose otherwise when you live in a windows-centric world.
Nginx reverse proxy and static file sharing + XSendfile + uploadprogress_module. Nothing beats it for the purpose.
On the WSGI side either Apache + mod_wsgi or cherrypy server. I like to use cherrypy wsgi server for applications on servers with less memory and less requests.
Reasoning:
I've done benchmarks with different tools for different popular solutions.
I have more experience with lower level TCP/IP than web development, especially http implementations. I'm more confident that I can recognize a good http server than I can recognize a good web framework.
I know Twisted much more than Django or Pylons. The http stack in Twisted is still not up to this but it will be there.
I'm using Google App Engine for an application I'm developing. It runs WSGI applications.
Here's a couple bits of info on it.
This is the first web-app I've ever really worked on, so I don't have a basis for comparison, but if you're a Google fan, you might want to look into it. I've had a lot of fun using it as my framework for learning.
TurboGears (2.0)
TurboGears 2.0 is leaving Beta within the next month (has been in it for plenty of time). 2.0 improves upon 1.0 series and attempts to give you best-of-breed WSGI stack, so it makes some default choices for you, if you want the least fuss.
it has the tg* tools for testing and deployment in 1.x series, but now transformed to paster equivalents in 2.0 series, which shoud seem familiar if you've expermiented with pylons.
tg-admin quickstart —> paster quickstart
tg-admin info —> paster tginfo
tg-admin toolbox –> paster toolbox
tg-admin shell –> paster shell
tg-admin sql create –> paster setup-app development.ini
Pylons
It you'd like to be more flexible in your WSGI stack (choice of ORM, choice of templater, choice of form-ing), Pylons is becoming the consolidated choice. This would be my recommended choice, since it offers excellent documentation and allows you to experiment with different components.
It is a pleasure to work with as a result, and works on under Apache (production deployment) or stand-alone (helpful for testing and experimenting stage).
so it follows, you can do both with Pylons:
2 option for testing stage (python standalone)
4 for scalable production purposes (FastCGI, assuming the database you choose can keep up)
The Pylons admin interface is very similar to TurboGears. Here's a toy standalone example:
$ paster create -t pylons helloworld
$ cd helloworld
$ paster serve --reload development.ini
for production-class deployment, you could refer to the setup guide of Apache + FastCGI + mod_rewrite is available here. this would scale up to most needs.
Apache httpd + mod_fcgid using web.py (which is a wsgi application).
Works like a charm.
We are using pure Paste for some of our web services. It is easy to deploy (with our internal deployment mechanism; we're not using Paste Deploy or anything like that) and it is nice to minimize the difference between production systems and what's running on developers' workstations. Caveat: we don't expect low latency out of Paste itself because of the heavyweight nature of our requests. In some crude benchmarking we did we weren't getting fantastic results; it just ended up being moot due to the expense of our typical request handler. So far it has worked fine.
Static data has been handled by completely separate (and somewhat "organically" grown) stacks, including the use of S3, Akamai, Apache and IIS, in various ways.
Apache+mod_wsgi,
Simple, clean. (only four lines of webserver config), easy for other sysadimns to get their head around.