We Keep Coding

Is it possible to specify the pickle protocol when writing pandas to HDF5?

Is there a way to tell Pandas to use a specific pickle protocol (e.g. 4) when writing an HDF5 file?
Here is the situation (much simplified):
Client A is using python=3.8.1 (as well as pandas=1.0.0 and pytables=3.6.1). A writes some DataFrame using df.to_hdf(file, key).
Client B is using python=3.7.1 (and, as it happened, pandas=0.25.1 and pytables=3.5.2 --but that's irrelevant). B tries to read the data written by A using pd.read_hdf(file, key), and fails with ValueError: unsupported pickle protocol: 5.
Mind you, this doesn't happen with a purely numerical DataFrame (e.g. pd.DataFrame(np.random.normal(size=(10,10))). So here is a reproducible example:
(base) $ conda activate py38
(py38) $ python
Python 3.8.1 (default, Jan 8 2020, 22:29:32)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> df = pd.DataFrame(['hello', 'world']))
>>> df.to_hdf('foo', 'x')
>>> exit()
(py38) $ conda deactivate
(base) $ python
Python 3.7.4 (default, Aug 13 2019, 20:35:49)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> df = pd.read_hdf('foo', 'x')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/pytables.py", line 407, in read_hdf
return store.select(key, auto_close=auto_close, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/pytables.py", line 782, in select
return it.get_result()
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/pytables.py", line 1639, in get_result
results = self.func(self.start, self.stop, where)
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/pytables.py", line 766, in func
return s.read(start=_start, stop=_stop, where=_where, columns=columns)
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/pytables.py", line 3206, in read
"block{idx}_values".format(idx=i), start=_start, stop=_stop
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/pytables.py", line 2737, in read_array
ret = node[0][start:stop]
File "/opt/anaconda3/lib/python3.7/site-packages/tables/vlarray.py", line 681, in __getitem__
return self.read(start, stop, step)[0]
File "/opt/anaconda3/lib/python3.7/site-packages/tables/vlarray.py", line 825, in read
outlistarr = [atom.fromarray(arr) for arr in listarr]
File "/opt/anaconda3/lib/python3.7/site-packages/tables/vlarray.py", line 825, in <listcomp>
outlistarr = [atom.fromarray(arr) for arr in listarr]
File "/opt/anaconda3/lib/python3.7/site-packages/tables/atom.py", line 1227, in fromarray
return six.moves.cPickle.loads(array.tostring())
ValueError: unsupported pickle protocol: 5
>>>
Note: I tried also reading using pandas=1.0.0 (and pytables=3.6.1) in python=3.7.4. That fails too, so I believe it is simply the Python version (3.8 writer vs 3.7 reader) that causes the problem. This makes sense since pickle protocol 5 was introduced as PEP-574 for Python 3.8.

PyTable uses the highest protocol by default, which is hardcoded here: https://github.com/PyTables/PyTables/blob/50dc721ab50b56e494a5657e9c8da71776e9f358/tables/atom.py#L1216
As a workaround, you can monkey-patch the pickle module on the client A who writes a HDF file. You should do that before importing pandas:
import pickle
pickle.HIGHEST_PROTOCOL = 4
import pandas
df.to_hdf(file, key)
Now the HDF file has been created using pickle protocol version 4 instead version 5.

Update: I was wrong to assume this was not possible. In fact, based on the excellent "monkey-patch" suggestion of #PiotrJurkiewicz, here is a simple context manager that lets us temporarily change the highest pickle protocol. It:
Hides the monkey-patching, and
Has no side-effect outside of the context; it can be used at any time, whether pickle was previously imported or not, before or after pandas, no matter.
Here is the code (e.g. in a file pickle_prot.py):
import importlib
import pickle
class PickleProtocol:
def __init__(self, level):
self.previous = pickle.HIGHEST_PROTOCOL
self.level = level
def __enter__(self):
importlib.reload(pickle)
pickle.HIGHEST_PROTOCOL = self.level
def __exit__(self, *exc):
importlib.reload(pickle)
pickle.HIGHEST_PROTOCOL = self.previous
def pickle_protocol(level):
return PickleProtocol(level)
Usage example in a writer:
import pandas as pd
from pickle_prot import pickle_protocol
pd.DataFrame(['hello', 'world']).to_hdf('foo_0.h5', 'x')
with pickle_protocol(4):
pd.DataFrame(['hello', 'world']).to_hdf('foo_1.h5', 'x')
pd.DataFrame(['hello', 'world']).to_hdf('foo_2.h5', 'x')
And, using a simple test reader:
import pandas as pd
from glob import glob
for filename in sorted(glob('foo_*.h5')):
try:
df = pd.read_hdf(filename, 'x')
print(f'could read {filename}')
except Exception as e:
print(f'failed on {filename}: {e}')
Now, trying to read in py37 after having written in py38, we get:
failed on foo_0.h5: unsupported pickle protocol: 5
could read foo_1.h5
failed on foo_2.h5: unsupported pickle protocol: 5
But, using the same version (37 or 38) to read and write, we of course get no exception.
Note: the issue 33087 is still on Pandas issue tracker.

I'm (was) facing the same problem... I "know" how to solve it and I think you do too...
The solution is to reprocess the whole data to a pickle (or csv) and re-transform it in python3.7 to a hdf5 (which only knows protocol 4).
the flow is something like this:
python3.8 -> hdf5 -> python3.8 -> csv/pickle -> python3.7 -> hdf5 (compatible with both versions)
I avoided this route because I have chuncks of data of a dataframe being exported, creating a large number of files.
Are you actually limited to use python3.7 ? I was limited by tensorflow which as of now only supports up to 3.7 (officially) but you can install tensorflow- nightly-build and it works with python 3.8
Check if you can make the move to 3.8 that would surely solve your problem. :)

Python os.environ throws key error?

I'm accessing an environment variable in a script with os.environ.get and it's throwing a KeyError. It doesn't throw the error from the Python prompt. This is running on OS X 10.11.6, and is Python 2.7.10.
What is going on?
$ python score.py
Traceback (most recent call last):
File "score.py", line 4, in <module>
setup_logging()
File "/score/log.py", line 29, in setup_logging
config = get_config()
File "/score/log.py", line 11, in get_config
environment = os.environ.get('NODE_ENV')
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
KeyError: 'NODE_ENV'
$ python -c "import os; os.environ.get('NODE_ENV')"
$
As requested, here's the source code for score.py
from __future__ import print_function
from log import get_logger, setup_logging
setup_logging()
log = get_logger('score')
And here's log.py
import json
import os
import sys
from iron_worker import IronWorker
from logbook import Logger, Processor, NestedSetup, StderrHandler, SyslogHandler
IRON_IO_TASK_ID = IronWorker.task_id()
def get_config():
environment = os.environ.get('NODE_ENV')
if environment == 'production':
filename = '../config/config-production.json'
elif environment == 'integration':
filename = '../config/config-integration.json'
else:
filename = '../config/config-dev.json'
with open(filename) as f:
return json.load(f)
def setup_logging():
# This defines a remote Syslog handler
# This will include the TASK ID, if defined
app_name = 'scoreworker'
if IRON_IO_TASK_ID:
app_name += '-' + IRON_IO_TASK_ID
config = get_config()
default_log_handler = NestedSetup([
StderrHandler(),
SyslogHandler(
app_name,
address = (config['host'], config['port']),
level = 'ERROR',
bubble = True
)
])
default_log_handler.push_application()
def get_logger(name):
return Logger(name)

Try running:
find . -name \*.pyc -delete
To delete your .pyc files.
Researching your problem I came across this question, where a user was experiencing the same thing: .get() seemingly raising a KeyError. In that case, it was caused, according to this accepted answer, by a .pyc file which contained code where a dict value was being accessed by key (i.e., mydict['potentially_nonexistent_key']), while the traceback was showing the code from the updated .py file where .get() was used. I have never heard of this happening, where the traceback references current code from a .py file, but shows an error raised by an outdated .pyc file, but it seems to have happened at least once in the history of Python...
It is a long shot, but worth a try I thought.

I encountered a similar error when I set the environment variable without exporting it. So if you do this:
me#host:/# NODE_ENV=foo
You will get this:
me#host:/# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> node_env = os.environ['NODE_ENV']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'NODE_ENV'
>>>
But if you do this:
me#host:/# NODE_ENV=foo
me#host:/# export NODE_ENV
It works:
me#host:/# python3
Python 3.8.2 (default, Apr 27 2020, 15:53:34)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> node_env = os.environ['NODE_ENV']
>>> print(node_env)
foo
>>>

Command for windows to delete the .pyc files:
del /S *.pyc

I had the same problem. I solved that by making some corrections on the .env file:
Before:
Key = Value
After my correction:
Key=Value
without blank spaces and worked!

I was getting this error while trying to source from a .env file.
I didn't explicitly export the env vars so I had to change this.
ENVIRONMENT=DEV
to this
export ENVIRONMENT=DEV

Use export a=10 instead of a=10 while setting env variable. Add the same in ~./bashrc to reload the env var wherever you login.
Doing this resolved the issue

I'd recommend you start debugging os.py, for instance, on windows it's being used this implementation:
def get(self, key, failobj=None):
print self.data.__class__
print key
return self.data.get(key.upper(), failobj)
And if I test it with this:
import os
try:
os.environ.get('NODE_ENV')
except Exception as e:
print("-->{0}".format(e.__class__))
os.environ['NODE_ENV'] = "foobar"
try:
os.environ.get('NODE_ENV')
except Exception as e:
print("{0}".format(e.__class__))
The output will be:
<type 'dict'>
PYTHONUSERBASE
<type 'dict'>
APPDATA
<type 'dict'>
NODE_ENV
<type 'dict'>
NODE_ENV
So it makes sense the exception is not spawned reading dict.get docs.
In any case, if you don't want to mess up or debugging the python modules, try cleaning up the *.pyc files, try to set up properly NODE_ENV. And if all that don't work, restart your terminal to clear up.

`pickle`: yet another `ImportError: No module named my_module`

I have a class MyClass defined in my_module. MyClass has a method pickle_myself which pickles the instance of the class in question:
def pickle_myself(self, pkl_file_path):
with open(pkl_file_path, 'w+') as f:
pkl.dump(self, f, protocol=2)
I have made sure that my_module is in PYTHONPATH. In the interpreter, executing __import__('my_module') works fine:
>>> __import__('my_module')
<module 'my_module' from 'A:\my_stuff\my_module.pyc'>
However, when eventually loading the file, I get:
File "A:\Anaconda\lib\pickle.py", line 1128, in find_class
__import__(module)
ImportError: No module named my_module
Some things I have made sure of:
I have not changed the location of my_module.py (Python pickling after changing a module's directory)
I have tried to use dill instead, but still get the same error (More on python ImportError No module named)
EDIT -- A toy example that reproduces the error:
The example itself is spread over a bunch of files.
First, we have the module ball (stored in a file called ball.py):
class Ball():
def __init__(self, ball_radius):
self.ball_radius = ball_radius
def say_hello(self):
print "Hi, I'm a ball with radius {}!".format(self.ball_radius)
Then, we have the module test_environment:
import os
import ball
#import dill as pkl
import pickle as pkl
class Environment():
def __init__(self, store_dir, num_balls, default_ball_radius):
self.store_dir = store_dir
self.balls_in_environment = [ball.Ball(default_ball_radius) for x in range(num_balls)]
def persist(self):
pkl_file_path = os.path.join(self.store_dir, "test_stored_env.pkl")
with open(pkl_file_path, 'w+') as f:
pkl.dump(self, f, protocol=2)
Then, we have a module that has functions to make environments, persist them, and load them, called make_persist_load:
import os
import test_environment
#import pickle as pkl
import dill as pkl
def make_env_and_persist():
cwd = os.getcwd()
my_env = test_environment.Environment(cwd, 5, 5)
my_env.persist()
def load_env(store_path):
stored_env = None
with open(store_path, 'rb') as pkl_f:
stored_env = pkl.load(pkl_f)
return stored_env
Then we have a script to put it all together, in test_serialization.py:
import os
import make_persist_load
MAKE_AND_PERSIST = True
LOAD = (not MAKE_AND_PERSIST)
cwd = os.getcwd()
store_path = os.path.join(cwd, "test_stored_env.pkl")
if MAKE_AND_PERSIST == True:
make_persist_load.make_env_and_persist()
if LOAD == True:
loaded_env = make_persist_load.load_env(store_path)
In order to make it easy to use this toy example, I have put it all up on in a Github repository that simply needs to be cloned into your directory of choice.. Please see the README containing instructions, which I also reproduce here:
Instructions:
1) Clone repository into a directory.
2) Add repository directory to PYTHONPATH.
3) Open up test_serialization.py, and set the variable MAKE_AND_PERSIST to True. Run the script in an interpreter.
4) Close the previous interpreter instance, and start up a new one. In test_serialization.py, change MAKE_AND_PERSIST to False, and this will programmatically set LOAD to True. Run the script in an interpreter, causing ImportError: No module named test_environment.
5) By default, the test is set to use dill, instead of pickle. In order to change this, go into test_environment.py and make_persist_load.py, to change imports as required.
EDIT: after switching to dill '0.2.5.dev0', dill.detect.trace(True) output
C2: test_environment.Environment
# C2
D2: <dict object at 0x000000000A9BDAE8>
C2: ball.Ball
# C2
D2: <dict object at 0x000000000AA25048>
# D2
D2: <dict object at 0x000000000AA25268>
# D2
D2: <dict object at 0x000000000A9BD598>
# D2
D2: <dict object at 0x000000000A9BD9D8>
# D2
D2: <dict object at 0x000000000A9B0BF8>
# D2
# D2
EDIT: the toy example works perfectly well when run on Mac/Ubuntu (i.e. Unix-like systems?). It only fails on Windows.

I can tell from your question that you are probably doing something like this, with a class method that is attempting to pickle the instance of the class. It's ill-advised to do that, if you are doing that… it's much more sane to use pkl.dump external to the class instead (where pkl is pickle or dill etc). However, it can still work with this design, see below:
>>> class Thing(object):
... def pickle_myself(self, pkl_file_path):
... with open(pkl_file_path, 'w+') as f:
... pkl.dump(self, f, protocol=2)
...
>>> import dill as pkl
>>>
>>> t = Thing()
>>> t.pickle_myself('foo.pkl')
Then restarting...
Python 2.7.10 (default, Sep 2 2015, 17:36:25)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> f = open('foo.pkl', 'r')
>>> t = dill.load(f)
>>> t
<__main__.Thing object at 0x1060ff410>
If you have a much more complicated class, which I'm sure you do, then you are likely to run into trouble, especially if that class uses another file that is sitting in the same directory.
>>> import dill
>>> from bar import Zap
>>> print dill.source.getsource(Zap)
class Zap(object):
x = 1
def __init__(self, y):
self.y = y
>>>
>>> class Thing2(Zap):
... def pickle_myself(self, pkl_file_path):
... with open(pkl_file_path, 'w+') as f:
... dill.dump(self, f, protocol=2)
...
>>> t = Thing2(2)
>>> t.pickle_myself('foo2.pkl')
Then restarting…
Python 2.7.10 (default, Sep 2 2015, 17:36:25)
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> f = open('foo2.pkl', 'r')
>>> t = dill.load(f)
>>> t
<__main__.Thing2 object at 0x10eca8090>
>>> t.y
2
>>>
Well… shoot, that works too. You'll have to post your code, so we can see what pattern you are using that dill (and pickle) fails for. I know having one module import another that is not "installed" (i.e. in some local directory) and expecting the serialization to "just work" doesn't for all cases.
See dill issues:
https://github.com/uqfoundation/dill/issues/128
https://github.com/uqfoundation/dill/issues/129
and this SO question:
Why dill dumps external classes by reference, no matter what?
for some examples of failure and potential workarounds.
EDIT with regard to updated question:
I don't see your issue. Running from the command line, importing from the interpreter (import test_serialization), and running the script in the interpreter (as below, and indicated in your steps 3-5) all work. That leads me to think you might be using an older version of dill?
>>> import os
>>> import make_persist_load
>>>
>>> MAKE_AND_PERSIST = False #True
>>> LOAD = (not MAKE_AND_PERSIST)
>>>
>>> cwd = os.getcwd()
>>> store_path = os.path.join(cwd, "test_stored_env.pkl")
>>>
>>> if MAKE_AND_PERSIST == True:
... make_persist_load.make_env_and_persist()
...
>>> if LOAD == True:
... loaded_env = make_persist_load.load_env(store_path)
...
>>>
EDIT based on discussion in comments:
Looks like it's probably an issue with Windows, as that seems to be the only OS the error appears.
EDIT after some work (see: https://github.com/uqfoundation/dill/issues/140):
Using this minimal example, I can reproduce the same error on Windows, while on MacOSX it still works…
# test.py
class Environment():
def __init__(self):
pass
and
# doit.py
import test
import dill
env = test.Environment()
path = "test.pkl"
with open(path, 'w+') as f:
dill.dump(env, f)
with open(path, 'rb') as _f:
_env = dill.load(_f)
print _env
However, if you use open(path, 'r') as _f, it works on both Windows and MacOSX. So it looks like the __import__ on Windows is more sensitive to file type than on non-Windows systems. Still, throwing an ImportError is weird… but this one small change should make it work.

In case someone is having same problem, I had the same problem running Python 2.7 and the problem was the pickle file created on windows while I am running Linux, what I had to do is running dos2unix which has to be downloaded first using
sudo yum install dos2unix
And then you need to convert the pickle file example
dos2unix data.p

Does, With open() not works with python 2.6

I am trying to use "With open()" with python 2.6 and it is giving error(Syntax error) while it works fine with python 2.7.3
Am I missing something or some import to make my program work!
Any help would be appreciated.
Br
My code is here:
def compare_some_text_of_a_file(self, exportfileTransferFolder, exportfileCheckFilesFolder) :
flag = 0
error = ""
with open("check_files/"+exportfileCheckFilesFolder+".txt") as f1,open("transfer-out/"+exportfileTransferFolder) as f2:
if f1.read().strip() in f2.read():
print ""
else:
flag = 1
error = exportfileCheckFilesFolder
error = "Data of file " + error + " do not match with exported data\n"
if flag == 1:
raise AssertionError(error)

The with open() statement is supported in Python 2.6, you must have a different error.
See PEP 343 and the python File Objects documentation for the details.
Quick demo:
Python 2.6.8 (unknown, Apr 19 2012, 01:24:00)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> with open('/tmp/test/a.txt') as f:
... print f.readline()
...
foo
>>>
You are trying to use the with statement with multiple context managers though, which was only added in Python 2.7:
Changed in version 2.7: Support for multiple context expressions.
Use nested statements instead in 2.6:
with open("check_files/"+exportfileCheckFilesFolder+".txt") as f1:
with open("transfer-out/"+exportfileTransferFolder) as f2:
# f1 and f2 are now both open.

It is the "extended" with statement with multiple context expressions which causes your trouble.
In 2.6, instead of
with open(...) as f1, open(...) as f2:
do_stuff()
you should add a nesting level and write
with open(...) as f1:
with open(...) as f2:
do.stuff()
The docu says
Changed in version 2.7: Support for multiple context expressions.

The with open() syntax is supported by Python 2.6. On Python 2.4 it is not supported and gives a syntax error. If you need to support PYthon 2.4, I would suggest something like:
def readfile(filename, mode='r'):
f = open(filename, mode)
try:
for line in f:
yield f
except e:
f.close()
raise e
f.close()
for line in readfile(myfile):
print line

How to call a function stored in another file from a Python program?

If I have a text file that contains a python function definition, how can I make the function call from another Python program. Ps: The function will be defined in the Python program that does the call.
Ways in which can be done:
Consider the python function as a module and call it. Constraint here is that I have to convert a python bare function into a module which would give errors.
Insert the code(function code) into the program that calls the function.
Which would be the better way to go about it?
Edit: Thank you for all the replies. Have shed a lot of light on the initial confusion I myself had. Another doubt would be, what if the person(Obviously, not me) has written a os.system("rm -rf"). And I end up executing it. That would mean doomsday for me, right?
Edit2: As a lot of you have asked me to use exec, I would like to point to the this thread and most particularly the namespace problem. It gives user a lot of chances to "circumvent" python. Don't y'all think?

You are looking for the exec keyword.
>>> mycode = 'print "hello world"'
>>> exec mycode
Hello world
So if you read your text file as text (assuming that it only contains the function) like:
test.txt:
def a():
print "a()"
test.py:
mycode = open('test.txt').read()
exec mycode # this will execute the code in your textfile, thus define the a() function
a() # now you can call the function from your python file
Link to doc: http://docs.python.org/reference/simple_stmts.html#grammar-token-exec%5Fstmt
You may want to look at the compile statement too: here.

compile() and eval() can do the trick:
>>> code = compile('def foo(a): return a*2', '<string>', 'exec')
>>> eval(code)
>>> foo
52: <function foo at 0x01F65F70>
>>> foo(12)
53: 24
or with file:
with open(filename) as source:
eval(compile(source.read(), filename, 'exec'))

A way like Reflection in Java? If so, Python has a module named imp to provide it.
foo.py
def foo():
return "return from function foo in file foo.py"
some code anywhere
modes = imp.get_suffixes() #got modes Explained in link below
mode = modes[-2] # because I want load a py file
with open("foo.py") as file:
m = imp.load_module("name", file, "foo.py", mode)
print(m.foo())
above mode = modes[-2] because my imp.get_suffixes() is:
>>> imp.get_suffixes()
[('.cpython-32m.so', 'rb', 3), ('module.cpython-32m.so', 'rb', 3), ('.abi3.so', 'rb', 3), ('module.abi3.so', 'rb', 3), ('.so', 'rb', 3), ('module.so', 'rb', 3), ('.py', 'U', 1), ('.pyc', 'rb', 2)]
here is my output:
Python 3.2.1 (default, Aug 11 2011, 01:27:29)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp
>>> with open("foo.py") as file:
... m = imp.load_module("foo", file, "foo.py", ('.py', 'U', 1))
...
>>> m.foo()
'return from function foo in file foo.py'
Check it here: http://docs.python.org/py3k/library/imp.html
Both python 2.7 and python 3 works:
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import imp
>>> imp.get_suffixes()
[('.so', 'rb', 3), ('module.so', 'rb', 3), ('.py', 'U', 1), ('.pyc', 'rb', 2)]
>>> with open("foo.py") as file:
... m = imp.load_module("foo", file, "foo.py", ('.py', 'U', 1))
...
>>> m.foo()
'return from function foo in file foo.py'

You can use execfile:
execfile("path/example.py")
# example.py
# def example_func():
# return "Test"
#
print example_func()
# >Test
EDIT:
In case you want to execute some unsecure code, you can try to sandbox it this way,
although it is probably not very safe anyway:
def execfile_sandbox(filename):
from copy import copy
loc = globals()
bi = loc["__builtins__"]
if not isinstance(bi, dict): bi = bi.__dict__
bi = copy(bi)
# no files
del bi["file"]
# and definitely, no import
del bi["__import__"]
# you can delete other builtin functions you want to deny access to
new_locals = dict()
new_locals["__builtins__"] = bi
execfile(filename, new_locals, new_locals)
Usage:
try:
execfile_sandbox("path/example.py")
except:
# handle exception and errors here (like import error)
pass

I am not sure what is your purpose, but I suppose that you have function in one program and you do want that function run in another program. You can "marshal" function from first to second.
Example, first program:
# first program
def your_func():
return "your function"
import marshal
marshal.dump(your_func.func_code, file("path/function.bin","w"))
Second program:
# Second program
import marshal, types
code = marshal.load(file("path/function.bin"))
your_func = types.FunctionType(code, globals(), "your_func")
print your_func()
# >your function