During developing one of my applications, I've come to a point where I'd like to give the users a more powerful filter. Therefore, I'd like to provide a simple scripting interface to the users. The scripting language would be Python.
For obvious reasons, I'd like to tighten the scope of the language to match my particular purposes (I don't want the users to touch the server's HDD files etc.). I also don't want to write a Python interpreter myself (which would be reinventing the wheel and the "new" wheel would be rectangular in the end). However, I haven't found any suitable library or module for this purpose.
Groovy's approach with its Compilation Customizers and Compiler Configuration would be exactly what I want, does something similar exist for Python?
What you're looking for is called a "sandbox" or "restricted execution." This wiki page discusses some of the details.
In a nutshell, there have been several efforts by Python geeks and gurus to build a sandbox on top of Python but they all failed eventually.
The main reason is that Python offers so many paths to do something that the sandbox would either have to forbid common use cases (rendering a lot of the library and Python code useless) or it would have to have holes in the sandbox which would make the concept useless.
So while it looks like a good and simple idea, so far, there is no solution. AFAIK, there are no hooks in Python to tweak the byte code compiler to achieve something like Groovy Sandbox.
Related:
How can I sandbox Python in pure Python?
Is there a "safe" subset of Python for use as an embedded scripting language?
Related
Is there any good way to call python from clojure as a means of doing data science with scipy, numpy, scikit-learn, etc.
I know about implementations of clojure which run on python instead of java, but this doeesn't work for me, as I also need to call java libraries in my project. I also know about Jython, but I don't know of a clean way to use this with Clojure.
I want to use Clojure in my projects because I prefer it as a language, but I can't deny that Python has an incredible community, and some of the most beautiful, well-designed libraries around.
Instead of trying to get Jython to play well with both Clojure and numpy/scipy, you can use Hy. It is hosted on Python and it somewhat resembles Clojure.
If I really wanted to use numpy/scipy, I would write the backend in Python (or Hy), run it as a separate service. And if I really like ring for instance, or can't live without Instaparse, I would write a frontend in Clojure.
As an aside Python has EDN libs. It would be an interesting project to integrate one of them in Hy, or write one from scratch.
Give the toolz library a try, it's a functional standard library for Python that was designed to generally adhere to the API of the Clojure standard library.
Apart from that, I'd encourage you to find the seams between your computations, and write individual tools in the Unix way in either Clojure or Python depending on which seems to fit that use case best. Serialize data between the tools, either as text/JSON through pipes or using a binary serialization format like Protobuf, which has standard APIs for both Java and Python.
If you had a gun to my head and told me to build Clojure/Python interop, I'd start with py4j and bridge the two languages through Java interfaces, using implements members in a Python class and reify on the Clojure side.
You could use Graal VM now. Although some large companies are using it in production, it's still early days. Here's an example of using Python from Clojure:
(.eval context "python" "
import time;
time.clock()
")
http://gigasquidsoftware.com/blog/2017/10/22/embedded-interop-between-clojure-r-and-python-with-graalvm/
I'm building a rendering engine in Python for fun. I need to load 3D scenes. Any standard modern format like DAE, 3DS, or MAX would work: I can convert my files easily between standard formats.
OpenSceneGraph seems to be the most comprehensive and well-maintained solution. It would be ideal to be able to use it in Python without much hassle. Are there working Python bindings for OSG that are easy to install, work on Mac OS X (I'm on 10.8), and are compatible with the latest versions of OSG?
I searched around and came across osgswig (http://code.google.com/p/osgswig/) and PyOSG (http://sourceforge.net/projects/pyosg/), but they don't seem to be actively maintained. I don't see any recent activity related to these packages, and it seems that people had trouble running osgswig on OSX. Ideally, I'd like to find something that "just works", without major compilation hassles. I'd like to just install a package and be able to import a module that will let me load COLLADA or 3DS files.
I also came across pycollada (https://github.com/pycollada/pycollada). It seems active, but fairly early-stage. Ideally, I'd like a reasonably comprehensive package that supports specular maps, normal maps, and other reasonably advanced features. Animation would be nice as well.
In summary, I need to load 3D scenes in Python. Bindings for OSG would probably be ideal, because OSG is so comprehensive. But I need something that works on OSX. I would also prefer something that can be installed reasonably easily. Does something like this exist?
Thanks!
Take a look at Open Asset Import Library (short name: Assimp). It is a portable Open Source library to import various well-known 3D model formats in a uniform manner. http://www.assimp.org/
You should loot at panda3D (http://www.panda3d.org/), it's a game engine with extensive python bindings. It has the features you want : http://www.panda3d.org/manual/index.php/Features
I used it for a few years and it was a solid tool.
I made my own fork of a mirror of a clone of the osgswig project for a similar purpose. I have it working with OpenSceneGraph version 3.2.1 on Windows and Mac; and it's likely I will eventually polish it for linux too. I'm already delivering one product to customers based on my version of osgswig, and I'm considering making others. Find my fork here:
https://github.com/cmbruns/osgswig
If others show enough interest, I might be coaxed into creating binary installers for my version of the osgswig module, to make installation easier.
If you just want the easiest OpenSceneGraph bindings for OSG 3.2.1, you can stop reading this answer here. Read on for more of my thoughts for the future.
Though I am maintaining a fork of osgswig (as stated above), I sort of hate SWIG, and I would prefer to use bindings based on Boost.Python, rather than on SWIG. For large, complex C++ APIs, like OpenSceneGraph, Boost.Python can be much more elegant than SWIG, both for the API consumer, and for the binding maintainer (me, and me). I found one project using Boost.Python to wrap OSG, at https://code.google.com/p/osgboostpython/, but the developer is lovingly wrapping each part of the interface by hand, and has thus only completed a tiny fraction of the large OpenSceneGraph API.
Taking that Boost.Python based project as inspiration, I created yet another OpenSceneGraph Python binding project, at https://github.com/JaneliaSciComp/osgpyplusplus. Eventually, I want to use this osgpyplusplus project for all my python osg needs. And I would appreciate help in making it ready. Right now, osgpyplusplus suffers from the following weaknesses, compared to osgswig:
osgpyplusplus is not yet used in any working product
The build environment is tricky to set up, requiring both Boost.Python and Pyplusplus
I haven't paid much attention to osgpyplusplus recently, so it might rust away if I continue to ignore it.
Though osgpyplusplus probably wraps most of the OpenSceneGraph API, there are probably some important missing pieces that won't be identified until someone tries to develop a significant project with it.
It would be a lot of work for me to create a binary module installer for osgpyplusplus at this point, so please don't ask me to.
I want to add scripting capabilities to my application and I just came across python4delphi http://code.google.com/p/python4delphi/, which seems to be stable.
At first sight it looks very easy to use on the developers side, but I couldn't find a way to debug a script. Would be great if I could embed the IDLE IDE in my application, I just have no idea how to do it.
Is there a easy way to add debugging capabilities with python4delphi? Should I use other script engines like pascalScript (seems to have been discontinued)?
Edit: After having a look at PySctipt and pdb I realized that what I am looking is more like a visual editor with embedded debugging and the capability to read python4delphi custom modules (the ones where I export my delphi objects. Being a python noob I have no idea how p4d does this). I am probably asking too much, but would be great to find an implementation of that and not having to code it all from scratch.
Python is really a great language.
But if you need to embed script, with debug, in a Delphi application you should considere Pascal scripting, which is more available in Delphi.
I use the excellent PascalScipt, in the exemples you will see editor, debugger and a lot of other functions.
Is it possible to deploy python applications such that you don't release the source code and you don't have to be sure the customer has python installed?
I'm thinking maybe there is some installation process that can run a python app from just the .pyc files and a shared library containing the interpreter or something like that?
Basically I'm keen to get the development benefits of a language like Python - high productivity etc. but can't quite see how you could deploy it professionally to a customer where you don't know how there machine is set up and you definitely can't deliver the source.
How do professional software houses developing in python do it (or maybe the answer is that they don't) ?
You protect your source code legally, not technologically. Distributing py files really isn't a big deal. The only technological solution here is not to ship your program (which is really becoming more popular these days, as software is provided over the internet rather than fully installed locally more often.)
If you don't want the user to have to have Python installed but want to run Python programs, you'll have to bundle Python. Your resistance to doing so seems quite odd to me. Java programs have to either bundle or anticipate the JVM's presence. C programs have to either bundle or anticipate libc's presence (usually the latter), etc. There's nothing hacky about using what you need.
Professional Python desktop software bundles Python, either through something like py2exe/cx_Freeze/some in-house thing that does the same thing or through embedding Python (in which case Python comes along as a library rather than an executable). The former approach is usually a lot more powerful and robust.
Yes, it is possible to make installation packages. Look for py2exe, cx_freeze and others.
No, it is not possible to keep the source code completely safe. There are always ways to decompile.
Original source code can trivially be obtained from .pyc files if someone wants to do it. Code obfuscation would make it more difficult to do something with the code.
I am surprised no one mentioned this before now, but Cython seems like a viable solution to this problem. It will take your Python code and transpile it into CPython compatible C code. You also get a small speed boost (~25% last I checked) since it will be compiled to native machine code instead of just Python byte code. You still need to be sure the user has Python installed (either by making it a pre-requisite pushed off onto the user to deal with, or bundling it as part of the installer process). Also, you do need to have at least one small part of your application in pure Python: the hook into the main function.
So you would need something basic like this:
import cython_compiled_module
if __name__ == '__main__':
cython_compiled_module.main()
But this effectively leaks no implementation details. I think using Cython should meet the criteria in the question, but it also introduces the added complexity of compiling in C, which loses some of Python's easy cross-platform nature. Whether that is worth it or not is up to you.
As others stated, even the resulting compiled C code could be decompiled with a little effort, but it is likely much more close to the type of obfuscation you were initially hoping for.
Well, it depends what you want to do. If by "not releasing the source code" you mean "the customer should not be able to access the source code in any way", well, you're fighting a losing battle. Even programs written in C can be reverse engineered, after all. If you're afraid someone will steal from you, make them sign a contract and sue them if there's trouble.
But if you mean "the customer should not care about python files, and not be able to casually access them", you can use a solution like cx_Freeze to turn your Python application into an executable.
Build a web application in python. Then the world can use it via a browser with zero install.
I recently started learning Python. Not yet ventured into coding.
During one of my learning sessions, i came accross the term Jython.
I googled it & got some information.
I would like to know if anyone has implemented any real-world program using Jython.
Most of the time, Jython isn't used directly to write full read-world programs, but a lot of programs actually embed Jython to use it as a scripting language.
The official Jython website gives a list of projects, some written in Jython, others using Jython for scripting:
http://wiki.python.org/jython/JythonUsers
I am writing a full application in Jython at the moment, and would highly recommend it. Having all of the Java libraries at your disposal is very handy, and the Python syntax and language features actually make using some of them easier than it is in Java (I'm mostly talking about Swing here).
Check out the chapter on GUI Applications from the Jython book. It does a lot of comparisons like 'Look at all this Java code, and now look at it reduced to Python code of half the length!'.
The only caveats I've found are:
Jython development tends to run slightly behind Python, which can be annoying if you find a cool way of doing something in Python, only to discover it's not supported in the current Jython version.
Occasionally you might have hiccups with the interface between Python and Java (I have a couple of unsolved problems here and here, although there are always workarounds for this kind of thing).
Distribution is not as simple as it could be, although once you figure out how to do it, it's fairly painless. I recommend following the method here. It essentially consists of:
Exploding jython.jar and adding your own modules into it.
Writing and compiling a small Java class that creates a Python interpreter and loads up your Python modules.
Creating an executable .jar file consisting of the jython.jar modules, your own Python modules, and the Java class.
Jython really shines for dependency injection.
You know those pesky variables you have to give your program, like
file system paths
server names
ports
Jython provides a really nice way of injecting those variables by putting them in a script. It works equally well for injecting java dependencies, as well.
WebSphere and WebLogic use it as their default scripting engine for administrative purposes.
A lot of other Oracle products ship it as part of their "oracle_commons" module (Oracle Universal Installer, Oracle HTTP Server etc). It's mostly version 2.2 being deployed though, which is a bit old and clunky.
There is a list of application that uses jython at http://wiki.python.org/jython/JythonUsers