This question derives from a very specific use-case. Some clever person has put together a repl.it that allows you to use the Python version of the Processing libraries on the web:
https://repl.it/#templates/Python-Processing
The Python code looks like this:
from browser import document, window, alert
def sketch(p):
#this p is needed. it will be the processing sketch itself.
# to do things like background(0) instead do p.background(0)
def setup():
p.createCanvas(700, 410)
p.background(0)
p.rectMode(p.CENTER)
def draw():
#p.background(0)
p.fill(255,255,0,128)
p.ellipse(p.mouseX,p.mouseY,50,50)
def mousePressed(self):
p.background(0)
def keyPressed(self):
if p.key==" ":
print("Hallo")
p.setup = setup
p.draw = draw
p.mousePressed = mousePressed
p.keyPressed = keyPressed
myp5 = window.p5.new(sketch)
The issue is that I'd really like the user code to just look like this (as it would in a desktop version of the Python version of Processing):
def setup():
createCanvas(700, 410)
background(0)
rectMode(CENTER)
def draw():
fill(255,255,0,128)
ellipse(mouseX, mouseY, 50, 50)
def mousePressed(self):
background(0)
def keyPressed(self):
if key==" ":
print("Hallo")
...but as you can see in the repl, all of the calls have to be prefaced by "p." so that they are acting on the processing sketch, and wrapped in a function that takes that sketch as an argument.
I guess the question boils down to this: is there a way of wrapping things so that otherwise undefined global variables/functions (e.g. createCanvas) get bound to a particular object (in this case resolving to p.createCanvas)?
I recognize that it's kind of unpythonic, since it's about making implicit what was otherwise explicit, but in this case, it would serve to make the code more in the spirit of the Processing libraries.
Related
Maybe I am completely off track here (and above my paygrade for sure), but what I want to do is to give users of my app (That I am writing in Python since that's the language I know) a python interpreter to control some objects within my app. Something similar like many 3D and VFX softwares have (Maya, Blender, Nuke). This is the code I got so far:
#main.py
import code
import networkx as nx
class Env():
def __init__(self):
self.graph = nx.graph.DiGraph()
# load library with functions that will be availabel for user inside the app.
import mylib
functions = {f: getattr(mylib, f) for f in dir(mylib) if not f.startswith('__')}
self._interpreter = code.InteractiveInterpreter(locals=functions)
def execute_node(self, node=None):
# In IRL the main object to be pass1ed to users' interpreter will be the self.graph object
# but I made it more clear for this question.
self._interpreter.locals['var'] = 42
node = "print(var)\nprint(some_function())\nvar = 67" # Let's pretend node object contains this code.
self._interpreter.runcode(node)
if __name__ == '__main__':
e = Env()
# some code, node creation and so on...
e.execute_code()
print(e.locals['var'])
#mylib.py
var = None # I have to put this here because if there is no variable function fails at import
def some_function():
print(var)
Output:
42 # This prints as expected
None # The print within the function prints the value that was there when module was initialized
67 # The last print returns expected value
So, it is clear that python interprets the functions on first import and "bakes" the global variables that it had at the import time. Now the question is can I somehow easily make it use the globals passed from the code.InteractiveInterpreter() or I should look for a completely different solution (and which one) :)? Of course the idea is that the two python programs should communicate, the user should use a special library to operate the software and the backend code should not be exposed to them. Do I make any sense? Thanks :)
This is the one-ish instance where you do want to use the exec() function, but please remember that the user may be able to run any Python code, including stuff that could run forever, mess up your main program, write (or delete) files, etc.
def run_code(code, add_locals={}):
code_locals = {}
code_locals.update(add_locals) # Copy in the additional locals so that dict could be reused
exec(
code,
{}, # no globals (you may wish to replace this),
code_locals,
)
return code_locals # return updated locals
class Beeper: # define a toy object
def beep(self, times):
print("Beep! " * times)
beeper = Beeper() # instantiate the object to play with
# Some user code...
user_code = """
x = 5
beeper.beep(x)
x += 3
"""
new_locals = run_code(user_code, {"beeper": beeper})
print(new_locals)
This outputs
Beep! Beep! Beep! Beep! Beep!
{'beeper': <__main__.Beeper>, 'x': 8}
So you can see we can use the locals the user has modified if need be.
I am working on a django based web app that takes python file as input which contains some function, then in backend i have some lists that are passed as parameters through the user's function,which will generate a single value output.The result generated will be used for some further computation.
Here is how the function inside the user's file look like :
def somefunctionname(list):
''' some computation performed on list'''
return float value
At present the approach that i am using is taking user's file as normal file input. Then in my views.py i am executing the file as module and passing the parameters with eval function. Snippet is given below.
Here modulename is the python file name that i had taken from user and importing as module
exec("import "+modulename)
result = eval(f"{modulename}.{somefunctionname}(arguments)")
Which is working absolutely fine. But i know this is not the secured approach.
My question , Is there any other way through which i can run users file securely as the method that i am using is not secure ? I know the proposed solutions can't be full proof but what are the other ways in which i can run this (like if it can be solved with dockerization then what will be the approach or some external tools that i can use with API )?
Or if possible can somebody tell me how can i simply sandbox this or any tutorial that can help me..?
Any reference or resource will be helpful.
It is an important question. In python sandboxing is not trivial.
It is one of the few cases where the question which version of python interpreter you are using. For example, Jyton generates Java bytecode, and JVM has its own mechanism to run code securely.
For CPython, the default interpreter, originally there were some attempts to make a restricted execution mode, that were abandoned long time ago.
Currently, there is that unofficial project, RestrictedPython that might give you what you need. It is not a full sandbox, i.e. will not give you restricted filesystem access or something, but for you needs it may be just enough.
Basically the guys there just rewrote the python compilation in a more restricted way.
What it allows to do is to compile a piece of code and then execute, all in a restricted mode. For example:
from RestrictedPython import safe_builtins, compile_restricted
source_code = """
print('Hello world, but secure')
"""
byte_code = compile_restricted(
source_code,
filename='<string>',
mode='exec'
)
exec(byte_code, {__builtins__ = safe_builtins})
>>> Hello world, but secure
Running with builtins = safe_builtins disables the dangerous functions like open file, import or whatever. There are also other variations of builtins and other options, take some time to read the docs, they are pretty good.
EDIT:
Here is an example for you use case
from RestrictedPython import safe_builtins, compile_restricted
from RestrictedPython.Eval import default_guarded_getitem
def execute_user_code(user_code, user_func, *args, **kwargs):
""" Executed user code in restricted env
Args:
user_code(str) - String containing the unsafe code
user_func(str) - Function inside user_code to execute and return value
*args, **kwargs - arguments passed to the user function
Return:
Return value of the user_func
"""
def _apply(f, *a, **kw):
return f(*a, **kw)
try:
# This is the variables we allow user code to see. #result will contain return value.
restricted_locals = {
"result": None,
"args": args,
"kwargs": kwargs,
}
# If you want the user to be able to use some of your functions inside his code,
# you should add this function to this dictionary.
# By default many standard actions are disabled. Here I add _apply_ to be able to access
# args and kwargs and _getitem_ to be able to use arrays. Just think before you add
# something else. I am not saying you shouldn't do it. You should understand what you
# are doing thats all.
restricted_globals = {
"__builtins__": safe_builtins,
"_getitem_": default_guarded_getitem,
"_apply_": _apply,
}
# Add another line to user code that executes #user_func
user_code += "\nresult = {0}(*args, **kwargs)".format(user_func)
# Compile the user code
byte_code = compile_restricted(user_code, filename="<user_code>", mode="exec")
# Run it
exec(byte_code, restricted_globals, restricted_locals)
# User code has modified result inside restricted_locals. Return it.
return restricted_locals["result"]
except SyntaxError as e:
# Do whaever you want if the user has code that does not compile
raise
except Exception as e:
# The code did something that is not allowed. Add some nasty punishment to the user here.
raise
Now you have a function execute_user_code, that receives some unsafe code as a string, a name of a function from this code, arguments, and returns the return value of the function with the given arguments.
Here is a very stupid example of some user code:
example = """
def test(x, name="Johny"):
return name + " likes " + str(x*x)
"""
# Lets see how this works
print(execute_user_code(example, "test", 5))
# Result: Johny likes 25
But here is what happens when the user code tries to do something unsafe:
malicious_example = """
import sys
print("Now I have the access to your system, muhahahaha")
"""
# Lets see how this works
print(execute_user_code(malicious_example, "test", 5))
# Result - evil plan failed:
# Traceback (most recent call last):
# File "restr.py", line 69, in <module>
# print(execute_user_code(malitious_example, "test", 5))
# File "restr.py", line 45, in execute_user_code
# exec(byte_code, restricted_globals, restricted_locals)
# File "<user_code>", line 2, in <module>
#ImportError: __import__ not found
Possible extension:
Pay attention that the user code is compiled on each call to the function. However, it is possible that you would like to compile the user code once, then execute it with different parameters. So all you have to do is to save the byte_code somewhere, then to call exec with a different set of restricted_locals each time.
EDIT2:
If you want to use import, you can write your own import function that allows to use only modules that you consider safe. Example:
def _import(name, globals=None, locals=None, fromlist=(), level=0):
safe_modules = ["math"]
if name in safe_modules:
globals[name] = __import__(name, globals, locals, fromlist, level)
else:
raise Exception("Don't you even think about it {0}".format(name))
safe_builtins['__import__'] = _import # Must be a part of builtins
restricted_globals = {
"__builtins__": safe_builtins,
"_getitem_": default_guarded_getitem,
"_apply_": _apply,
}
....
i_example = """
import math
def myceil(x):
return math.ceil(x)
"""
print(execute_user_code(i_example, "myceil", 1.5))
Note that this sample import function is VERY primitive, it will not work with stuff like from x import y. You can look here for a more complex implementation.
EDIT3
Note, that lots of python built in functionality is not available out of the box in RestrictedPython, it does not mean it is not available at all. You may need to implement some function for it to become available.
Even some obvious things like sum or += operator are not obvious in the restricted environment.
For example, the for loop uses _getiter_ function that you must implement and provide yourself (in globals). Since you want to avoid infinite loops, you may want to put some limits on the number of iterations allowed. Here is a sample implementation that limits number of iterations to 100:
MAX_ITER_LEN = 100
class MaxCountIter:
def __init__(self, dataset, max_count):
self.i = iter(dataset)
self.left = max_count
def __iter__(self):
return self
def __next__(self):
if self.left > 0:
self.left -= 1
return next(self.i)
else:
raise StopIteration()
def _getiter(ob):
return MaxCountIter(ob, MAX_ITER_LEN)
....
restricted_globals = {
"_getiter_": _getiter,
....
for_ex = """
def sum(x):
y = 0
for i in range(x):
y = y + i
return y
"""
print(execute_user_code(for_ex, "sum", 6))
If you don't want to limit loop count, just use identity function as _getiter_:
restricted_globals = {
"_getiter_": labmda x: x,
Note that simply limiting the loop count does not guarantee security. First, loops can be nested. Second, you cannot limit the execution count of a while loop. To make it secure, you have to execute unsafe code under some timeout.
Please take a moment to read the docs.
Note that not everything is documented (although many things are). You have to learn to read the project's source code for more advanced things. Best way to learn is to try and run some code, and to see what kind function is missing, then to see the source code of the project to understand how to implement it.
EDIT4
There is still another problem - restricted code may have infinite loops. To avoid it, some kind of timeout is required on the code.
Unfortunately, since you are using django, that is multi threaded unless you explicitly specify otherwise, simple trick for timeouts using signeals will not work here, you have to use multiprocessing.
Easiest way in my opinion - use this library. Simply add a decorator to execute_user_code so it will look like this:
#timeout_decorator.timeout(5, use_signals=False)
def execute_user_code(user_code, user_func, *args, **kwargs):
And you are done. The code will never run more than 5 seconds.
Pay attention to use_signals=False, without this it may have some unexpected behavior in django.
Also note that this is relatively heavy on resources (and I don't really see a way to overcome this). I mean not really crazy heavy, but it is an extra process spawn. You should hold that in mind in your web server configuration - the api which allows to execute arbitrary user code is more vulnerable to ddos.
For sure with docker you can sandbox the execution if you are careful. You can restrict CPU cycles, max memory, close all network ports, run as a user with read only access to the file system and all).
Still,this would be extremely complex to get it right I think. For me you shall not allow a client to execute arbitrar code like that.
I would be to check if a production/solution isn't already done and use that. I was thinking that some sites allow you to submit some code (python, java, whatever) that is executed on the server.
I'm trying to add a custom IMessageFilter in a Winforms app using Python.net, but I'm getting a segfault.
Here's a minimal sample application:
import clr
clr.AddReference("System.Windows.Forms")
import System.Windows.Forms as WinForms
class MessageFilter(WinForms.IMessageFilter):
__namespace__ = 'System.Windows.Forms'
def PreFilterMessage(self, message):
print('filter', message)
return False
class HelloApp(WinForms.Form):
def __init__(self):
self.textbox = WinForms.TextBox()
self.textbox.Text = "Hello World"
self.Controls.Add(self.textbox)
def main():
form = HelloApp()
app = WinForms.Application
f = MessageFilter()
app.AddMessageFilter(f)
app.Run(form)
if __name__ == '__main__':
main()
If you run this code as presented, the application window displays, but you get a segfault immediately (I presume this is when the first message is passed to the filter). The segfault is entirely opaque. There's no stack trace or other helpful details - it's just the OS-level segfault handler.
If you comment out line 25 (app.AddMessageFilter(f), installing the actual filter), the code works fine.
If you modify MessageFilter so that it doesn't subclass Winforms.IMessageFilter, you get an error saying there's no AddMessageFilter method matching the given arguments.
If you rename or remove the PreFilterMessage() method, you get an error that the Python object doesn't have a PreFilterMessage method.
Any suggestions on what I'm doing wrong, and/or how to fix it? Or how to get more debugging information that could point at the source of the segfault?
It appears that this is a bug in Python.net itself, relating to a problem with marshaling byref arguments. Details can be found here.
Well... I was having a terrible time getting part of my code working, but I rearranged things and it suddenly started working correctly. Not sure what I did to be honest, so I guess that will be the subject of this question. I'm building a simple text-based card game that uses decks uploaded from two .txt files. It's aimed at Magic: the Gathering, but would probably work with others if people got creative with it. To provide a rough overview, here is how things are arranged:
import random
def shuffle(board1):
def game():
#board=[[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]]
#performs most of the actions relating to the game
board[0]=20
board[10]=20
def gameboard(board2):
#displays game board
def draw(board3, numcards, player):
#draws cards
def upload(deckname):
#uploads cards from file
def life(board4):
#asks about which player the life total is changing on, by how much, etc.
#and then does it
def maketoken(board5):
#creates tokens, counters, etc. based on user input
def move(board5):
#accepts user input and moves cards from zone to zone
def play(board6):
#handles casting spells, using abilities, triggered abilities, etc.
#main body of program is below function definitions
board=[[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]]
deckname1=input("\nWhat is the name of deck 1?")
deckname2=input("\nWhat is the name of deck 2?")
deck1=upload(deckname1)
deck2=uplaod(deckname2)
board[1]=deck1
board[11]=deck2
#this is where a lot of the other variables get set
game()
(note: most of the code has been removed for brevity and prettiness, as my code is pretty ugly)
I have a college-level C++ background, and just recently decided to pick up ye olde keyboard for the heck of it, so the assignment operator (=) not working the way I expect is driving me CRAZY. Therefore, I was also wondering if there was a way to get the functionality of the C++ '=' in Python, since I upload the decks from .txt files, and want to be through with the upload() function as soon as that's done (I use deck1=upload(deckname) (same for deck2). I want to use 'deck1' and 'deck2' to refill the decks after each game, but if I understand how '=' works in python, entering board[1]=deck1 means board[1] will point to the storage area of deck1 and changes to board[1] will change deck1, BUT I DON'T WANT THAT... GRRRR!!!!!!11). I'm sure there's a solution out there somewhere since it's making me nutty, but I haven't been able to find it. Thanks!!!
edit: This was the error I received when things were set up this way:
Traceback (most recent call last):
File "C:\Users\inventor487\Desktop\simplepy.py", line 444, in <module>
game()
File "C:\Users\inventor487\Desktop\simplepy.py", line 114, in game
board[1]=deck1
UnboundLocalError: local variable 'board' referenced before assignment
Summary:
Do I need to pass board to the game() function, even if it's set up as a global variable (or at least I thought it was)? Everything seems to work fine when I assign it inside the game() function (commented out to show this). (edit: nevermind... I'm an idiot.)
Does assigning part of board to a value inside game() make it a local variable (e.g. where I have board[0]=20)? (edit: yes, it does apparently...)
As you've discovered, the = operator in Python doesn't make a copy of an object like it does in C++ for example. If you want to make a copy to store in another variable you have to be explicit about it.
board[1] = deck1[:] # the slicing operator copies a subset or the whole list
A more general method is to use the copy module
import copy
board[1] = copy.copy(deck1)
board[1] = copy.deepcopy(deck1)
edit: a class solves all of this very easily, and much more cleanly. Having a separate variable for each zone, etc. makes everything a lot smoother. Thanks for the help, though guys. Much appreciated.
I'm working on a python application that runs on 2 different platforms, namely regular desktop linux and Maemo 4. We use PyGTK on both platforms but on Maemo there are a bunch of little tweaks to make it look nice which are implemented as follows:
if util.platform.MAEMO:
# do something fancy for maemo
else:
# regular pygtk
There are roughly 15 of these if statements needed to get the UI looking and working nice on Maemo 4.
This has been very manageable for all this time. The problem is that a while ago there was a new version of Maemo released (5, aka fremantle) and it has some big differences compared to Maemo 4. I don't want to add a bunch of checks throughout the GUI code in order to get all 3 platforms working nicely with the same codebase because that would get messy. I also don't want to create a copy of the original GUI code for each platform and simply modify it for the specific platform (I'd like to re-use as much code as possible).
So, what are ways to have slightly different UIs for different platforms which are based on the same core UI code? I don't think this is a python or Maemo-specific question, I'd just like to know how this is done.
You could wind up much of this in a factory:
def createSpec():
if util.platform.MAEMO: return Maemo4Spec()
elif util.platform.MAEMO5: return Maemo5Spec()
return StandardPyGTKSpec()
Then, somewhere early in your code, you just call that factory:
spec = createSpec()
Now, everywhere else you had conditions, you just call the necessary function:
spec.drawComboBox()
As long as drawComboBox(), handles anything specific to the platform, you should be in good shape.
You could isolate the platform specific stuff you need to do into small consistently named functions inside a platform module, create the right function name using the platform you're running on and then getattr the right one and call it. The if/else boilerplate would disappear then.
I've made a separate module to handle all of my specializing between normal Linux, Maemo 4.1, and Maemo 5. It detects what features are available and allows the program to gracefully degrade.
For example
def _fremantle_hildonize_window(app, window):
oldWindow = window
newWindow = hildon.StackableWindow()
oldWindow.get_child().reparent(newWindow)
app.add_window(newWindow)
return newWindow
def _hildon_hildonize_window(app, window):
oldWindow = window
newWindow = hildon.Window()
oldWindow.get_child().reparent(newWindow)
app.add_window(newWindow)
return newWindow
def _null_hildonize_window(app, window):
return window
try:
hildon.StackableWindow
hildonize_window = _fremantle_hildonize_window
except AttributeError:
try:
hildon.Window
hildonize_window = _hildon_hildonize_window
except AttributeError:
hildonize_window = _null_hildonize_window
For more, see
Dialcentral, Gonert, ejpi, or Quicknote's source code for a file called hildonize.py
https://garage.maemo.org/plugins/ggit/browse.php/?p=gc-dialer;a=blob;f=src/hildonize.py;
Another example from The One Ring's GObject Utils (go_utils.py)
def _old_timeout_add_seconds(timeout, callback):
return gobject.timeout_add(timeout * 1000, callback)
def _timeout_add_seconds(timeout, callback):
return gobject.timeout_add_seconds(timeout, callback)
try:
gobject.timeout_add_seconds
timeout_add_seconds = _timeout_add_seconds
except AttributeError:
timeout_add_seconds = _old_timeout_add_seconds