We Keep Coding

How to define and use a python function across chunks in R Markdown?

Though I feel very stupid asking this question, I have not found a way to resolve this.
I am trying to define a simple python function in a code chunk in R Markdown.
def show(T):
print("shape of T:", T.shape)
print(T)
Then, I am trying to use it from another pyhton chunk in the same R Markdown.
import numpy as np
T = np.array([[...],[...], ...])
show(T)
But this throws an error saying that
NameError: name 'show' is not defined
I am using RStudio 1.2.907. Is there a way to resolve this?

How do you import a Python library within an R package using rPython?

The basic question is this: Let's say I was writing R functions which called python via rPython, and I want to integrate this into a package. That's simple---it's irrelevant that the R function wraps around Python, and you proceed as usual. e.g.
# trivial example
# library(rPython)
add <- function(x, y) {
python.assign("x", x)
python.assign("y", y)
python.exec("result = x+y")
result <- python.get("result")
return(result)
}
But what if the python code with R functions require users to import Python libraries first? e.g.
# python code, not R
import numpy as np
print(np.sin(np.deg2rad(90)))
# R function that call Python via rPython
# *this function will not run without first executing `import numpy as np`
print_sin <- function(degree){
python.assign("degree", degree)
python.exec('result = np.sin(np.deg2rad(degree))')
result <- python.get('result')
return(result)
}
If you run this without importing the library numpy, you will get an error.
How do you import a Python library in an R package? How do you comment it with roxygen2?
It appears the R standard is this:
# R function that call Python via rPython
# *this function will not run without first executing `import numpy as np`
print_sin <- function(degree){
python.assign("degree", degree)
python.exec('import numpy as np')
python.exec('result = np.sin(np.deg2rad(degree))')
result <- python.get('result')
return(result)
}
Each time you run an R function, you will import an entire Python library.

As #Spacedman and #DirkEddelbuettel suggest you could add a .onLoad/.onAttach function to your package that calls python.exec to import the modules that will typically always be required by users of your package.
You could also test whether the module has already been imported before importing it, but (a) that gets you into a bit of a regression problem because you need to import sys in order to perform the test, (b) the answers to that question suggest that at least in terms of performance, it shouldn't matter, e.g.
If you want to optimize by not importing things twice, save yourself the hassle because Python already takes care of this.
(although admittedly there is some quibblingdiscussion elsewhere on that page about possible scenarios where there could be a performance cost).
But maybe your concern is stylistic rather than performance-oriented ...

python: import numpy as np from outer code gets lost within my own user defined module

I'm doing simulations for scientific computing, and I'm almost always going to want to be in the interactive interpreter to poke around at the output of my simulations. I'm trying to write classes to define simulated objects (neural populations) and I'd like to formalize my testing of these classes by calling a script %run test_class_WC.py in ipython. Since the module/file containing the class is changing as I try to debug it/add features, I'm reloading it each time.
./test_class_WC.py:
import WC_class # make sure WC_class exists
reload(WC_class) # make sure it's the most current version
import numpy as np
from WC_class import WC_unit # put the class into my global namespace?
E1 = WC_unit(Iapp=100)
E1.update() # see if it works
print E1.r
So right off the bat I'm using reload to make sure I've got the most current version of the module loaded so I've got the freshest class definition-- I'm sure this is clunky as heck (and maybe more sinister?), but it saves me some trouble from doing %run WC_class.py and having to do a separate call to %run test_WC.py
and ./WC_class:
class WC_unit:
nUnits = 0
def __init__(self,**kwargs):
self.__dict__.update(dict( # a bunch of params
gee = .6, # i need to be able to change
ke=.1,the=.2, # in test_class_WC.py
tau=100.,dt=.1,r=0.,Iapp=1.), **kwargs)
WC_unit.nUnits +=1
def update(self):
def f(x,k=self.ke,th=self.the): # a function i define inside a method
return 1/(1+np.exp(-(x-th)/k)) # using some of those params
x = self.Iapp + self.gee * self.r
self.r += self.dt/self.tau * (-self.r + f(x))
WC_unit basically defines a bunch of default parameters and defines an ODE that updates using basic Euler integration. I expect that test_class_WC sets up a global namespace containing np (and WC_unit, and WC_class)
When I run it, I get the following error:
In [14]: %run test_class_WC.py
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
/Users/steeles/Desktop/science/WC_sequence/test_class_WC.py in <module>()
8
9 E1 = WC_unit(Iapp=100)
---> 10 E1.update()
11
12 # if bPlot:
/Users/steeles/Desktop/science/WC_sequence/WC_class.py in update(self)
19 return 1/(1+np.exp(-(x-th)/k))
20 x = self.Iapp + self.gee * self.r
---> 21 self.r += self.dt/self.tau * (-self.r + f(x))
22
23 # #class_method
/Users/steeles/Desktop/science/WC_sequence/WC_class.py in f(x, k, th)
17 def update(self):
18 def f(x,k=self.ke,th=self.the):
---> 19 return 1/(1+np.exp(-(x-th)/k))
20 x = self.Iapp + self.gee * self.r
21 self.r += self.dt/self.tau * (-self.r + f(x))
NameError: global name 'np' is not defined
Now I can get around this by just importing numpy as np in top of the WC_class module, or even by doing from numpy import exp in test_class_WC and change the update() method to contain exp() instead of np.exp()... but I'm not trying to do this because it's easy, I want to learn how all this namespace/module stuff works so I stop being a python idiot. Why is np getting lost in the WC_unit namespace? Is it because I'm dealing with two different files/modules? Does the call to np.exp inside a function have to do with it?
I'm also open to suggestions regarding improving my workflow and file structure, as it seems to be not particularly pythonic. My background is in MATLAB if that helps anyone understand. I'm editing my .py files in SublimeText2. Sorry the code is not very minimal, I've been having a hard time reproducing the problem.

The correct approach is to do an import numpy as np at the top of your sub-module as well. Here's why:
The key thing to note is that in Python, global actually means "shared at a module-level", and the namespaces for each module exist distinct from each other except when a module explicitly imports from another module. An imported module definitely cannot reach out to its 'parent' module's namespace, which is probably a good thing all things considered, otherwise you'll have modules whose behavior depends entirely on the variables defined in the module that imports it.
So when the stack trace says global name 'np' is not defined, it's talking about it at a module level. Python does not let the WC_Class module access objects in its parent module by default.
(As an aside, effbot has a quick note on how to do inter-module globals)
Another key thing to note is that even if you have multiple import numpy as np in various modules of your code, the module actually only gets loaded (i.e. executed) once. Once loaded, modules (being Python objects themselves) can be found in the dictionary sys.modules, and if a module already exists in this dictionary, any import module_to_import statement simply lets the importing module access names in the namespace of module_to_import. So having import numpy as np scattered across multiple modules in your codebase isn't wasteful.
Edit: On deeper digging, effbot has an even deeper (but still pretty quick and simple) exploration of what actually happens in module imports. For deeper exploration of the topic, you may want to check the import system discussion newly added in the Python 3 documentation.

It is normal in Python to import each module that is needed with in each. Don't count on any 'global' imports. In fact there isn't such a thing. With one exception. I discovered in
Do I have to specify import when Python script is being run in Ipython?
that %run -i myscript runs the script in the Ipython interactive namespace. So for quick test scripts this can save a bunch of imports.
I don't see the need for this triple import
import WC_class # make sure WC_class exists
reload(WC_class) # make sure it's the most current version
...
from WC_class import WC_unit
If all you are using from WC_class just use the last line.

Accessing Math functions from rootpy

I'm currently trying to access Math functions such as DeltaR from rootpy, but I'm not sure how this is done. I've seen nothing in the documentation or in any examples. The C++ equivalent would be something like:
double dR = ROOT::Math::VectorUtil::DeltaR((jets)[i],(partons)[i]);
But I'm unable to find a rootpy or even pyroot equivalent that'll work. If I try in pyroot with
import ROOT as r
r.Math.VectorUtil.DeltaR(jets[i],partons[i])
I get the error:
AttributeError: type object 'ROOT::Math' has no attribute 'VectorUtil'
When it quite clearly should, unless I don't understand correctly what it means by 'Attribute'. Anyway, I don't want to ask pyroot questions here :) I just put this down to a quirk in the way that pyroot handles such things, which is why I thought I'd give rootpy a try. I'm not sure if this is possible however.
Cheers,
Joseph

The functions from ROOT::Math::VectorUtil are in libGenVector which is loaded automatically in neither CINT nor PyROOT. Manually loading it (like you probably do in your root_logon.C) makes the functions available, e.g.
import ROOT as r
r.gSystem.Load('libGenVector')
# ...
r.Math.VectorUtil.DeltaR(jets[i],partons[i])

If jets and partons are TLorentzVectors then you should be able to do:
from ROOT import *
dR = jet.DeltaR(parton)

Jython: ImportError: No module named multiarray

When I try to call file and its method using Jython it shows the following error, while my Numpy, Python and NLTK is correctly installed and it works properly if I directly run directly from the Python shell
File "C:\Python26\Lib\site-packages\numpy\core\__init__.py", line 5, in <module>
import multiarray
ImportError: No module named multiarray
The code that I am using is simple one:
PyInstance hello = ie.createClass("PreProcessing", "None");
PyString str = new PyString("my name is abcd");
PyObject po = hello.invoke("preprocess", str);
System.out.println(po);
When I run only the file of python containing class PreProcessing and calling method preprocess it works fine, but with Jython it throws error.
Jython is unable to import all the libraries that have only compiled version kept in the folder not the class code itself. Like instead of multiarray.py it only has multiarray.pyd that is the compiled version so it is not getting detected in Jython.
Why is it showing this behaviour? How to resolve it?
Please help!

I know this is an old thread, but I recently ran into this same problem and was able to solve it and I figure the solution should be here in case anyone in the future runs into it. Like said above, Jython cannot deal with numpy's pre-compiled c files, but within nltk, the use of numpy is very limited and it's fairly straightforward to rewrite the affected bits of code. That's what I did, and I'm sure it's not the most computationally effective solution, but it works. This code is found in nltk.metrics.Segmentation, and I will only paste relevant code, but it will still be a little much.
def _init_mat(nrows, ncols, ins_cost, del_cost):
mat = [[4.97232652e-299 for x in xrange(ncols)] for x in xrange(nrows)]
for x in range(0,ncols):
mat[0][x] = x * ins_cost
for x in range(0, nrows):
mat[x][0] = x * del_cost
return mat
def _ghd_aux(mat, rowv, colv, ins_cost, del_cost, shift_cost_coeff):
for i, rowi in enumerate(rowv):
for j, colj in enumerate(colv):
shift_cost = shift_cost_coeff * abs(rowi - colj) + mat[i][j]
if rowi == colj:
# boundaries are at the same location, no transformation required
tcost = mat[i][j]
elif rowi > colj:
# boundary match through a deletion
tcost = del_cost + mat[i][j + 1]
else:
# boundary match through an insertion
tcost = ins_cost + mat[i + 1][j]
mat[i + 1][j + 1] = min(tcost, shift_cost)
Also at the end of ghd, change the return statement to
return mat[-1][-1]
I hope this helps someone! I don't know if there are other places where this is any issue, but this is the only one that I have encountered. If there are any other issues of this sort they can be solved in the same way(using a list of lists instead of a numpy array), again, you probably lose some efficiency, but it works.

jython is Java. Parts of Numpy are implemented as c extensions to Python (.pyd files). Some parts are implemented as .py files, which will work just fine in Jython. However, they cannot function with out access to the C level code. Currently, there is noway to use numpy in jython. See:
Using NumPy and Cpython with Jython
Or
Is there a good NumPy clone for Jython?
For recent discussions on alternatives.