python _+ django, is it compiled code? - python

Just looking into python from a .net background.
Is python compiled like .net?
If yes, can it be obfuscated and is it more or less secure than .net compiled code that is obfuscated?
does pretty much every web host (unix) support django and python?

There are many implementations of the Python language; the three that are certainly solid, mature and complete enough for production use are CPython, IronPython, and Jython. All of them are typically compiled to some form of bytecode, also known as intermediate code. The compilation from source to bytecode may take place as and when needed, but you can also do it in advance if you prefer; however Google App Engine, which lets you run small Python web apps, including Django, for free, as one of its limitations requires you to upload source and not compiled code (I know of no other host imposing the same limitation, but then I know of none giving you so many resources for free in exchange;-).
You might be most at home with IronPython, which is a Microsoft product (albeit, I believe, the first Microsoft product to be entirely open-source): in that case you can be certain that it is "compiled like .net", because it is (part of) .net (more precisely, .net and silverlight). Therefore it cannot be neither more nor less obfuscated and/or secure than .net (meaning, any other .net language).
Jython works on JVM, the Java Virtual Machine, much like IronPython works on Microsoft's Common Language Runtime, aka CLR. CPython has its own dedicated virtual machine.
For completeness, other implementations (not yet recommended for production use) include pypy, a highly flexible implementation that supports many possible back-ends (including but not limited to .net); unladen swallow, focused on evolving CPython to make it faster; pynie, a Python compiler to the Parrot virtual machine; wpython, a reimplementation based on "wordcode" instead of "bytecode"; and no doubt many, many others.
CPython, IronPython, Jython and pypy can all run Django (other implementations might also be complete enough for that, but I'm not certain).

I don't know about the security part but.
Python is interpreted. Like PHP. (It's turned into bytecode which CPython reads)
Django is just a framework on top of Python.
Python can be compiled.
And no not all hosts support python + django.

You shouldn't have to worry about obfuscating your code, specially since it's going to run on your server.
You are not supposed to put your code in a public directory anyway. The right thing to do with django (as oposed to PHP) is to make the code accessible by the webserver, but not by the public.
And if your server's security has been breached, then you have other things to worry about...

Obfuscation is false security. And the only thing worse than no security is false security. Why would you obfuscate a web app anyways?
Python is compiled to bytecode and run on a virtual machine, but usually distributed as source code.
Unless you really plan to run your webapp on "pretty much every web host" that question doesn't matter. There are many good hosts that support python and django.

Code obfuscation in .NET are mostly a question of changing variable names to make it harder to understand the disassembled code. Yes, you can do those techniques with CPython too.
Now, why ever you would want to is another question completely. It doesn't actually provide you with any security, and does not prevent anybody from stealing your software.

The Python is interpreted language. But you can compile the python program into a Unix executable using Freeze.

Related

Python vs C++ for CUDA web server?

I am planning on writing some software for a web server that uses machine learning to process large amounts of data. This will be real estate data from a MySQL server. I will be using the CUDA framework from Nvidia with python/caffe or the c++ library. I will be using a Tesla P100. Although python is more widely used for machine learning I presume it is hard to write a server app in python without sacrificing performance. Is this true? Is c++ well supported for machine learning? Will anything be sacrificed by writing a professional server app in python (ex: connecting to MySQL database)?
Python is a language that performs worse than c++ in terms of runtime for several reasons:
First and foremost, Python is a scripting language that runs with an interpreter as opposed to c++ which compiled into machine code before running.
Secondly: python runs in the background a garbage collector system while in c++ the memory management is done manually by the programmer.
In your case, I recommend that you work with Python for several reasons:
Writing in Python in CUDA allows you to compile the code even though it is Python (CUDA provides JIT - Just In Time compiler, as well as a compiler and other effective tools), Which greatly improves performance
Python provides many, rich varied libraries that will help you a lot in the project, especially in the field of machine learning.
The development time and code length will be significantly shorter in Python.
From my experience with working at CUDA in the Python language I recommend you use numba and numbapro libraries, they are comfortable to work with and provide support for many libraries like numpy.
Best of luck.
Both C++ and Python are perfectly reasonable languages for implementing web servers. Python is actually quite common, and there are many frameworks for writing web servers in Python such as flask, bottle, django, etc. Architecturally, I wonder whether you really need the machine learning (which I imagine would be a data processing pipeline) and the web server to be the same process / binary; however, even if you do them in the same server, I suspect that either language would be perfectly reasonable; moreover, if you ever came to the point where you needed to run a piece of computation in C++ for performance, using SWIG to call C++ from Python or using some form of message passing from Python to a C++ helper process (such as via gRPC) are options.

Which Python version should I use with Raspberry Pi running web applications?

I'm totally new in Python world.
I want to create a web application with some Python code behind. I want to use Python to control Raspberry Pi inputs and outputs etc.
There are Python 2 and Python 3 available. I've read some about these version, but I'm still not sure which version I should use.
Unless you have a specific reason to stick with Python 2 (e.g. a framework that has not been ported yet), you should really be using Python 3.
Quoting Python wiki, "Python 2.x is legacy, Python 3.x is the present and future of the language".
Most of the books on the topic of Python and Raspberry Pi refer to Python 3.x. I'm finding a lot of online courses and books are focusing more on 3.x than 2.7. Unless you're working at a company that's on Python 2.x and don't plan on going to 3.x, you're better off learning Python 3.x.
It depends, what web framework you are going to use.
Some of them might have a bit limited functionality on Python 3 but still can be worth to use.
This could be case of Flask, which is very lightweight, provides all what you need, but according to heavy users lack in few small details complete support for Python 3. This situation is likely to be resolved in near future, but if you want to have it developed now, it is better to use the version of Python, which fits your web framework.
Comments on few (not all) web frameworks
Django
Very popular, but will force you to do things in Django style.
Final solution can become a bit heavier, then really necessary, this could be a problem on Raspberry Pi, which has very limited resources available.
Flask
Also rather popular (even though not as much as Django).
Gives you freedom to use only what you need.
Very good tutorials.
Most of the applications run under Python 2 and Python 3, few (supporting) libraries are told to be not ported completely yet (I cannot serve exactly which ones).
CherryPy
Minimalistic web framework, but with very good builtin HTTP and WSGI server.
Not so easy to find good tutorials, best is using (now a bit old) book about programming in CherryPy.
Note: By default, the applications are developed in debug mode and code is autoreloaded from disk. This disk activity can slow down on RPi and consume some energy, so if you have troubles with that, set the app to production mode.
Conclusions
My current choice is using Flask on Python 2.7, but this is partially due to a lot of legacy code I have developed in Python 2.7.
You shall make your decision about what framework you are going to use and check, what is status of Python 3 support.

Can I call external *python* functions from google refine?

I'm investigating Google refine to speed up some of my data work -- never used it before this week, but I like a lot of what I see.
My biggest question so far is whether it's possible to call external python functions from Refine. I know you can call jython internally, but that doesn't provide access to C-based python libraries (e.g. lxml), and I have scripts elsewhere that I'd like to integrate, without lots of copy-paste or rewrite hassle.
What options are there for doing this in Refine? I'm willing to get creative -- I just want a stable, re-usable solution.
As Google Refine Wiki says:
lxml will NOT work in Jython, since lxml has C bindings for CPython (regular Python), hence will not work in Refine which is Jython / Java only, and has no CPython interpreter built-in
But you can try Google Refine Python Client Library to create projects and manipulate your data programmatically.
I'm going to mark reclosedev's answer as accepted, but there's still a litle more to the story.
The other answer to this question is that you can set up your own python-based API. For this project, I was able to set up a django app running on a local server. It only took an hour or so to build the API to my existing library.
More hassle than I'd have liked, but it fit the bill for this project without soaking up too much time.

I need a beginners guide to setting up windows for python development

I currently work with .NET exclusively and would like to have a go at python. To this end I need to set up a python development environment. I guide to this would be handy. I guess I would be doing web development so will need a web server and probably a database. I also need pointers to popular ORM's, an MVC framework, and a testing library.
One of my main criteria with all this is that I want to understand how it works, and I want it to be as isolated as possible. This is important as i am wary of polluting what is a working .NET environment with 3rd party web and database servers. I am perfectly happy using SQLite to start with if this is possible.
If I get on well with this I am also likely to want to set up automated build and ci server (On a virtual machine, probably ubuntu). Any suggestions for these would be useful.
My ultimate aim if i like python is to have similar sorts of tools that i have available with .NET and to really understand the build and deployment of it all. To start with I will settle for a simple development environment that is as isolated as possible and will be easy to remove if I don't like it. I don't want to use IronPython as I want the full experience of developing a python solution using the tools and frameworks that are generally used.
It's not that hard to set up a Python environment, and I've never had it muck up my .NET work. Basically, install Python --- I'd use 2.6 rather than 3.0, which is not yet broadly accepted --- and add it to your PATH, and you're ready to go with the language. I wouldn't recommend using a Ubuntu VM as your development environment; if you're working on Windows, you might as well develop on Windows, and I've had no significant problems doing so. I go back and forth from Windows to Linux with no trouble.
If you have an editor that you're comfortable with that has basic support for Python, I'd stick with it. If not, I've found Geany to be a nice, light, easy-to-use editor with good Python support, though I use Emacs myself because I know it; other people like SCITE, NotePad++, or any of a slew of others. I'd avoid fancy IDEs for Python, because they don't match the character of the language, and I wouldn't bother with IDLE (included with Python), because it's a royal pain to use.
Suggestions for libraries and frameworks:
Django is the standard web framework, but it's big and you have to work django's way; I prefer CherryPy, which is also actively supported, but is light, gives you great freedom, and contains a nice, solid webserver that can be replaced easily with httpd.
Django includes its own ORM, which is nice enough; there's a standalone one for Python, though, which is even nicer: SQL Alchemy
As far as a testing library goes, pyunit seems to me to be the obvious choice
Good luck, and welcome to a really fun language!
EDIT summary: I originally recommended Karrigell, but can't any more: since the 3.0 release, it's been continuously broken, and the community is not large enough to solve the problems. CherryPy is a good substitute if you like a light, simple framework that doesn't get in your way, so I've changed the above to suggest it instead.
Well, if you're thinking of setting up an Ubuntu VM anyway, you might as well make that your development environment. Then you can install Apache and MySQL or Postgres on that VM just via the standard packaging tools (apt-get install), and there's no danger of polluting your Windows environment.
You can either do the actual development on your Windows machine via your favourite IDE, using the VM as a networked drive and saving the code there, or you can just use the VM as a full desktop environment and do everything there, which is what I would recommend.
Install the pre-configured ActivePython release from activestate.
Among other features, it includes the PythonWin IDE (Windows only) which makes it easy to explore Python interactively.
The recommended reference is Dive Into Python, mentioned many times on similar SO discussions.
You should install python 2.4, python 2.5, python 2.6 and python 3.0, and add to your path the one you use more often (Add c:\Pythonxx\ and c:\Pythonxx\Scripts).
For every python 2.x, install easy_install; Download ez_setup.py and then from the cmd:
c:\Python2x\python.exe x:\path\to\ez_setup.py
c:\Python2x\Scripts\easy_install virtualenv
Then each time you start a new project create a new virtual environment to isolate the specific package you needs for your project:
mkdir <project name>
cd <project name>
c:\Python2x\Scripts\virtualenv --no-site-packages .\v
It creates a copy of python and its libraries in .v\Scripts and .\v\Lib. Every third party packages you install in that environment will be put into .\v\Lib\site-packages. The -no-site-packages don't give access to the global site package, so you can be sure all your dependencies are in .\v\Lib\site-packages.
To activate the virtual environment:
.\v\Scripts\activate
For the frameworks, there are many. Django is great and very well documented but you should probably look at Pylons first for its documentions on unicode, packaging, deployment and testing, and for its better WSGI support.
For the IDE, Python comes with IDLE which is enough for learning, however you might want to look at Eclipse+PyDev, Komodo or Wingware Python IDE. Netbean 6.5 has beta support for python that looks promising (See top 5 python IDE).
For the webserver, you don't need any; Python has its own and all web framework come with their own. You might want to install MySql or ProgreSql; it's often better to develop on the same DB you will use for production.
Also, when you have learnt Python, look at Foundations of Agile Python Development or Expert Python Programming.
Using Python on Windows
SO: Python tutorial for total beginners?
Take a look at Pylons, read about WSGI and Paste.
There's nice introductory Google tech talk about them: ReUsable Web Components with Python and Future Python Web Development.
Here's my answer to similar question:
Django vs other Python web frameworks?
NOTE: I included a lot of links to frameworks, projects and what-not, but as a new user I was limited to 1 link per answer. If someone else with enough reputation to edit wants/can edit them into this answer instead of the footnotes, I'd be grateful.
There are some Python IDE's such as Wing IDE[1], I believe some people use Eclipse[2] with a python plugin[3] as well. A lot of people in the #python channel of FreeNode seem to prefer vim, emacs, nano and similar text editors in favor of IDE's. My personal preffered editor is Vim, but if you've mostly done .NET development on windows, presumably with the usual Visual X IDE's, vim and emacs will probably cause you culture shock and you'd be better of using an IDE.
Nearly all python web frameworks* support the WSGI standard[4], most of the large web servers have some sort of plugin to support WSGI, the others support WSGI via fast cgi or plain cgi.
The Zope[5] and Django[6] frameworks have their own ORM's, of other ORM's the two most well known appear to be SQL Alchemy[7] and SQL Object[8]. I only have experience with the former, but both support all possible sane database choices, including SQLite which is installed together with Python and hence perfectly suited to testing and experimenting without polluting your .NET environment with 3rd part web servers and database servers.
The builtin unittest[9] and pyunit[10] frameworks seem to be the preffered solutions for unit testing, but I don't have much experience with these.
bpython[11] and ipython[12] offer enhanced interactive python shells which can greatly help speed up and testing small bits of code and hence worth looking in to.
As for a list of well known and often used web frameworks, look into the following frameworks**:
Twisted[13] is a generic networking framework, which supports almost every single protocol under the sun.
Pylons[14] is light-weight framework aimed at being as flexible as possible and leaving all the choices about what ORM, templating language and what-not to you.
CherryPy[15] tries to provide an interface to expose Python objects to the web.
Django[6] attempts to be an all-in-one solution, builtin template system, ORM, admin pages and internationalization. While the previous frameworks have more DIY wiring together various frameworks work involved with them.
Zope[5] is aimed to be suitable for large enterprise applications, I've heard nothing but good things about it, but consensus seems to be that for smaller you're probably better off with one of the simpler and smaller frameworks.
TurboGears[16] is the framework I know the least about, but it seems to be mostly competition for Django.
This is everything I can think of right now, I'll edit and add stuff if I can think of it. I hope this helps you some in the wonderful world of python.
* - The main exception would be Apache's mod_python, which you should avoid for exactly that reason, use mod_wsgi instead.
** - Word of warning, I have not personally used these frameworks this is just a very short impression I have gotten from talking to other people about each framework, it may be wildly inaccurate. (If anyone has any corrections, do comment and I'll try to edit and fix this answer).
(The http:// is missing since they're recognized as links otherwise)
[1] www.wingware.com/
[2] www.eclipse.org/
[3] pydev.sourceforge.net/
[4] wsgi.org/wsgi/
[5] www.zope.org/
[6] www.djangoproject.com/
[7] www.sqlalchemy.org/
[8] www.sqlobject.org/
[9] docs.python.org/library/unittest.html
[10] pyunit.sourceforge.net/pyunit.html
[11] www.bpython-interpreter.org/
[12] ipython.scipy.org/
[13] twistedmatrix.com/trac/
[14] pylonshq.com/
[15] www.cherrypy.org/
[16] turbogears.org/
Environment?
Here is the simplest solution:
Install Active Python 2.6. Its the Python itself, but comes with some extra handy useful stuff, like DiveintoPython chm.
Use Komodo Edit 5. It is among the good free editor you can use for Python.
Use IDLE. Its the best simplest short snippet editor, with syntax highlighting and auto complete unmatched by most other IDEs. It comes bundled with python.
Use Ipython. Its a shell that does syntax highlighting and auto complete, bash functions, pretty print, logging, history and many such things.
Install easy_install and/or pip for installing various 3rd party apps easily.
Coming from Visual Studio and .Net it will sound a lot different, but its an entirely different world.
For the framework, django works the best. Walk thro the tutorial and you will be impressed enough. The documentation rocks. The community, you have to see for yourself, to know how wonderful it is!!
Python has build in SQL like database and web server, so you wouldn't need to install any third party apps. Remember Python comes with batteries included.
If you've worked with Eclipse before you could give Pydev a try

Django -vs- Grails -vs-?

I'm wondering if there's such a thing as Django-like ease of web app development combined with good deployment, debugging and other tools?
Django is a very productive framework for building content-heavy sites; the best I've tried and a breath of fresh air compared to some of the Java monstrosities out there. However it's written in Python which means there's little real support in the way of deployment/packaging, debugging, profilers and other tools that make building and maintaining applications much easier.
Ruby has similar issues and although I do like Ruby much better than I like Python, I get the impression that Rails is roughly in the same boat at Django when it comes to managing/supporting the app.
Has anyone here tried both Django and Grails (or other web frameworks) for non-trivial projects? How did they compare?
You asked for someone who used both Grails and Django. I've done work on both for big projects. Here's my Thoughts:
IDE's:
Django works really well in Eclipse, Grails works really well in IntelliJ Idea.
Debugging:
Practically the same (assuming you use IntelliJ for Grails, and Eclipse for Python). Step debugging, inspecting variables, etc... never need a print statement for either. Sometimes django error messages can be useless but Grails error messages are usually pretty lengthy and hard to parse through.
Time to run a unit test:
django: 2 seconds.
Grails: 20 seconds (the tests themselves both run in a fraction of a second, it's the part about loading the framework to run them that takes the rest... as you can see, Grails is frustratingly slow to load).
Deployment:
Django: copy & paste one file into an apache config, and to redeploy, just change the code and reload apache.
Grails: create a .war file, deploy it on tomcat, rinse and repeat to redeploy.
Programming languages:
Groovy is TOTALLY awesome. I love it, more so than Python. But I certainly have no complaints.
Plugins:
Grails: lots of broken plugins (and can use every java lib ever).
Django: a few stable plugins, but enough to do most of what you need.
Database:
Django: schema migrations using South, and generally intuitive relations.
Grails: no schema migrations, and by default it deletes the database on startup... WTF
Usage:
Django: startups (especially in the Gov 2.0 space), independent web dev shops.
Grails: enterprise
Hope that helps!
However it's written in Python which
means there's little real support in
the way of deployment/packaging,
debugging, profilers and other tools
that make building and maintaining
applications much easier.
Python has:
a great interactive debugger, which makes very good use of Python REPL.
easy_install anv virtualenv for dependency management, packaging and deployment.
profiling features comparable to other languages
So IMHO you shouldn't worry about this things, use Python and Django and live happily :-)
Lucky for you, newest version of Django runs on Jython, so you don't need to leave your whole Java ecosystem behind.
Speaking of frameworks, I evaluated this year:
Pylons (Python)
webpy (Python)
Symfony (PHP)
CakePHP (PHP)
None of this frameworks comes close to the power of Django or Ruby on Rails. Based on my collegue opinion I could recommend you kohana framework. The downside is, it's written in PHP and, as far as I know, PHP doesn't have superb tools for debugging, profiling and packaging of apps.
Edit: Here is a very good article about packaging and deployment of Python apps (specifically Django apps). It's a hot topic in Django community now.
The statement that grails deletes the database on start-up is completely wrong. It's behavior on start-up is completely configurable and easy to configure. I generally use create-drop when running an app in dev mode. I use update when I run in test and production.
I also love the bootstrap processing that lets me pre-configure test users, data, etc by environment in Grails.
I'd love to see someone who has really built and deployed some commercial projects comment on the pros / cons. Be a really interesting read.
Grails.
Grails just looks like Rails (Ruby),but it uses groovy which is simpler than java. It uses java technology and you can use any java lib without any trouble.
I also choose Grails over simplicity and there are lots of java lib (such as jasper report, jawr etc) and I am glad that now they join with SpringSource which makes their base solid.
I have two friends who originally started writing an application using Ruby on Rails, but ran into a number of issues and limitations. After about 8 weeks of working on it, they decided to investigate other alternatives.
They settled on the Catalyst Framework, and Perl. That was about 4 months ago now, and they've repeatedly talked about how much better the application is going, and how much more flexibility they have.
With Perl, you have all of CPAN available to you, along with the large quantity of tools included. I'd suggest taking a look at it, at least.
The "good deployment" issue -- for Python -- doesn't have the Deep Significance that it has for Java.
Python deployment for Django is basically "move the files". You can run straight out of the subversion trunk directory if you want to.
You can, without breaking much of a sweat, using the Python distutils and build yourself a distribution kit that puts your Django apps into Python's site-packages. I'm not a big fan of it, but it's really easy to do.
Since my stuff runs in Linux, I have simple "install.py" scripts that move stuff out of the Subversion directories into /opt/this and /opt/that directories. I use an explicit path settings in my Apache configuration to name those directories where the applications live.
Patching can be done by editing the files in place. (A bad policy.) I prefer to edit in the SVN location and rerun my little install to be sure I actually have all the files under control.
cakephp.org
Cakephp is really good, really close to ruby on rails (1.2). It is in php, works very well on shared hosts and is easy to implement.
The only downside is that the documentation is somewhat lacking, but you quickly get it and quickly start doing cool stuff.
I totally recommend cakephp.
Personally I made some rather big projects with Django, but I can compare only with said "montrosities" (Spring, EJB) and really low-level stuff like Twisted.
Web frameworks using interpreted languages are mostly in its infancy and all of them (actively maintained, that is) are getting better with every day.
By "good deployment" are you comparing it with Java's EAR files, which allow you to deploy web applications by uploading a single file to a J2EE server? (And, to a lesser extent, WAR files; EAR files can have WAR files for dependent projects)
I don't think Django or Rails have gotten quite to that point yet, but I could be wrong... zuber pointed out an article with more details on the Python side.
Capistrano may help out on the Ruby side.
Unfortunately, I haven't really worked with either Python or Ruby that much, so I can't help out on profilers or debuggers.

Categories