We Keep Coding

How to automate pdb commands?

I am calling pdb on some function func i.e.,
def fun():
a = 10
c = fun2(a)
d = 40
return c+d
def fun2(a):
xyz ='str'
return a+10
Now I am running pdb using pdb.runcall(func,a,b) now It will open a pdb console for debugging, now suppose I press 2 time s(step) and q to quit in pdb console
but problem is I don't want to do this manually, I want to make some script which do something like this (automatic tell pdb first two command is s then third is q) , I am asking because there are many functions which needs atleast two time c (continue) to overall excecution of function so that it can yield/return some valid output (like say generators)
Any help will be a serious help for me.

Update after better understanding the question:
In general, I don't think this is the ideal way to test code; designing code for testability (e.g. using TDD) will often result in functions that are easier to test (e.g. using mocks/fake objects, dependency injection etc), and I would encourage you to consider refactoring the code if possible. The other issue with this approach is that the tests may become very tightly coupled to the code. However, I'll assume here that you know what you are doing, and that the above is not an option for whatever reason.
Scripting pdb
If you want to script pdb from code, this is actually possible by instantiating your own pdb.Pdb class and passing in the stdin and, at the time of writing, stdout argument (I'm not sure both should be required - I've filed https://bugs.python.org/issue33749).
Example (I just added the extra input argument to fun):
def fun(i):
a = 10 + i
c = fun2(a)
d = 40
return c+d
def fun2(a):
xyz ='str'
return a+10
import pdb
import io
output = io.StringIO()
# this contains the pdb commands we want to execute:
pdb_script = io.StringIO("p i;; i = 100;; n;; p a;; c;;")
mypdb = pdb.Pdb(stdin=pdb_script, stdout=output)
Normal result (no scripting):
In [40]: pdb.runcall(fun, 1)
...:
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb)
(Pdb) c
Out[40]: 61
Scripted pdb:
In [44]: mypdb = pdb.Pdb(stdin=pdb_script, stdout=output)
In [45]: mypdb.runcall(fun, 1)
Out[45]: 160
In [50]: print(output.getvalue())
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb) 1
> <ipython-input-1-28966c4f6e38>(3)fun()
-> c = fun2(a)
110
You may find using pdb_script.seek(0) helpful to reset the script.
Original answer - using conditional breakpoints
It sounds like what you really want is to only get into the debugger when your code is in a certain state. This can be done with conditional breakpoints (see pdb docs for details).
For example, let's say you want to break in fun2 if a > 10:
...:
In [2]: import pdb
In [3]: pdb.runcall(fun, 1)
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb) break fun2, a > 10
Breakpoint 1 at <ipython-input-1-28966c4f6e38>:6
(Pdb) c
> <ipython-input-1-28966c4f6e38>(7)fun2()
-> xyz ='str'
(Pdb) c
Out[3]: 61
In [4]: pdb.runcall(fun, -1)
> <ipython-input-1-28966c4f6e38>(2)fun()
-> a = 10 + i
(Pdb) c
Out[4]: 59
Notice in the first case you hit the breakpoint, in the second you didn't.
Original answer - using breakpoints and executing commands when hit
You could also try setting a breakpoint and using the commands facility.

extract number from exponent in Python

Hi everyone / Python Gurus
I would like to know how to accomplish the following task, which so far I've been unable to do so.
Here's what I have:
Q1 = 20e-6
Now this is an exponential number that if you print(Q1) as is it will show: 2e-5 which is fine. Mathematically speaking.
However, here's what I want to do with it:
I want Q1 to print only the number 20. And based on the whether this is e-6 then print uC or if this e-9 the print nC.
Here's an example for better understanding:
Q1=20e-6
When I run print(Q1) show: 20uC.
Q2=20e-9
When I run print(Q2) show: 20nC.
Can you please help me figure this out?

just replace the exponent using str.replace:
q1 = 'XXXXXX'
q1 = q1.replace('e-9', 'nC').replace('e-6', 'uC')
print(q1)

I recommend you using si-prefix.
You can install it using pip:
sudo pip install si-prefix
Then you can use something like this:
from si_prefix import si_format
# precision after the point
# char is the unity's char to be used
def get_format(a, char='C', precision=2):
temp = si_format(a, precision)
try:
num, prefix = temp.split()
except ValueError:
num, prefix = temp , ''
if '.' in num:
aa, bb = num.split('.')
if int(bb) == 0:
num = aa
if prefix:
return num + ' ' + prefix + char
else:
return num
tests = [20e-6, 21.46e05, 33.32e-10, 0.5e03, 0.33e-2, 112.044e-6]
for k in tests:
print get_format(k)
Output:
20 uC
2.15 MC
3.33 nC
500
3.30 mC
112.04 uC

You can try by splitting the string:
'20e-9'.split('e')
gives
['20', '-9']
From there on, you can insert whatever you want in between those values:
('u' if int(a[1]) > 0 else 'n').join(a)
(with a = '20e-9'.split('e'))

You can not. The behaviour you are looking for is called "monkey patching". And this is not allowed for int and float.
You can refer to this stackoverflow question
The only way I can think of is to create a class that extends float and then implement a __str__ method that shows as per your requirement.
------- More explanation -----
if you type
Q1 = 20e-6
in python shell and then
type(Q1)
your will get a
float
So basically your Q1 is considered as float by python type system
when you type print(Q1)
the _str__ method of float is called
The process of extending core class is one example of "monkey patch" and that is what I was refereing to.
Now the problem is that you can not "monkey patch" (or extend if you prefer that) core classes in python (which you can in some languages like in Ruby).
[int, float etc are core classes and written in C for your most common python distribution.]
So how do you solve it?
you need to create a new class like this
class Exponent(float):
def init(self, value):
self.value = value
def __str__(self):
return "ok"
x = Exponent(10.0)
print(x) ==> "ok"
hope this helps

Take error on wlst console when work with statements or loops

Not able to make loop and statement after it.
Example:
wls:/ADMIN_server/serverConfig> serverRuntime()
Location changed to serverRuntime tree. This is a read-only tree with ServerRuntimeMBean as the root.
For more help, use help(serverRuntime)
wls:/ADMIN_server/serverRuntime> dsMBeans = cmo.getJDBCServiceRuntime().getJDBCDataSourceRuntimeMBeans()
wls:/ADMIN_server/serverRuntime> ds_name = "ADMIN_DB"
wls:/ADMIN_server/serverRuntime> for ds in dsMBeans:
...
Traceback (innermost last):
(no code object) at line 0
File "<console>", line 2
SyntaxError: invalid syntax
Not sure is it need to import something before that options like and for that reason not able to make loop with statement:
import time
import sys

You shouldn't need to import anything specific to have access to the looping mechanism in WLST. For instance, try the following:
slist=range(1,4)
for i in slist: print 'i = ' + str(i);
The result should be:
i = 1
i = 2
i = 3
The syntax of your statements after the for loop are probably causing a problem in your python script OR if you are manually typing the commands in, you need to make sure you put a space after ... for it to interpret the next line like:
wls:/offline> j = 0
wls:/offline> while j<4:
... print 'j = ' + str(j)
... j = j + 1
...
jms 0
jms 1
jms 2
jms 3
Notice it is very important to type in a space character after ... or you will have the invalid syntax error.

call python with system() in R to run a python script emulating the python console

I want to pass a chunk of Python code to Python in R with something like system('python ...'), and I'm wondering if there is an easy way to emulate the python console in this case. For example, suppose the code is "print 'hello world'", how can I get the output like this in R?
>>> print 'hello world'
hello world
This only shows the output:
> system("python -c 'print \"hello world\"'")
hello world
Thanks!
BTW, I asked in r-help but have not got a response yet (if I do, I'll post the answer here).

Do you mean something like this?
export NUM=10
R -q -e "rnorm($NUM)"
You might also like to check out littler - http://dirk.eddelbuettel.com/code/littler.html
UPDATED
Following your comment below, I think I am beginning to understand your question better. You are asking about running python inside the R shell.
So here's an example:-
# code in a file named myfirstpythonfile.py
a = 1
b = 19
c = 3
mylist = [a, b, c]
for item in mylist:
print item
In your R shell, therefore, do this:
> system('python myfirstpythonfile.py')
1
19
3
Essentially, you can simply call python /path/to/your/python/file.py to execute a block of python code.
In my case, I can simply call python myfirstpythonfile.py assuming that I launched my R shell in the same directory (path) my python file resides.
FURTHER UPDATED
And if you really want to print out the source code, here's a brute force method that might be possible. In your R shell:-
> system('python -c "import sys; sys.stdout.write(file(\'myfirstpythonfile.py\', \'r\').read());"; python myfirstpythonfile.py')
a = 1
b = 19
c = 3
mylist = [a, b, c]
for item in mylist:
print item
1
19
3
AND FURTHER FURTHER UPDATED :-)
So if the purpose is to print the python code before the execution of a code, we can use the python trace module (reference: http://docs.python.org/library/trace.html). In command line, we use the -m option to call a python module and we specify the options for that python module following it.
So for my example above, it would be:-
$ python -m trace --trace myfirstpythonfile.py
--- modulename: myfirstpythonfile, funcname: <module>
myfirstpythonfile.py(1): a = 1
myfirstpythonfile.py(2): b = 19
myfirstpythonfile.py(3): c = 3
myfirstpythonfile.py(4): mylist = [a, b, c]
myfirstpythonfile.py(5): for item in mylist:
myfirstpythonfile.py(6): print item
1
myfirstpythonfile.py(5): for item in mylist:
myfirstpythonfile.py(6): print item
19
myfirstpythonfile.py(5): for item in mylist:
myfirstpythonfile.py(6): print item
3
myfirstpythonfile.py(5): for item in mylist:
--- modulename: trace, funcname: _unsettrace
trace.py(80): sys.settrace(None)
Which as we can see, traces the exact line of python code, executes the result immediately after and outputs it into stdout.

The system command has an option called intern = FALSE. Make this TRUE and Whatever output was just visible before, will be stored in a variable.
Now run your system command with this option and you should get your output directly in your variable. Like this
tmp <- system("python -c 'print \"hello world\"'",intern=T)

My work around for this problem is defining my own functions that paste in parameters, write out a temporary .py file, and them execute the python file via a system call. Here is an example that calls ArcGIS's Euclidean Distance function:
py.EucDistance = function(poly_path,poly_name,snap_raster,out_raster_path_name,maximum_distance,mask){
py_path = 'G:/Faculty/Mann/EucDistance_temp.py'
poly_path_name = paste(poly_path,poly_name, sep='')
fileConn<-file(paste(py_path))
writeLines(c(
paste('import arcpy'),
paste('from arcpy import env'),
paste('from arcpy.sa import *'),
paste('arcpy.CheckOutExtension("spatial")'),
paste('out_raster_path_name = "',out_raster_path_name,'"',sep=""),
paste('snap_raster = "',snap_raster,'"',sep=""),
paste('cellsize =arcpy.GetRasterProperties_management(snap_raster,"CELLSIZEX")'),
paste('mask = "',mask,'"',sep=""),
paste('maximum_distance = "',maximum_distance,'"',sep=""),
paste('sr = arcpy.Describe(snap_raster).spatialReference'),
paste('arcpy.env.overwriteOutput = True'),
paste('arcpy.env.snapRaster = "',snap_raster,'"',sep=""),
paste('arcpy.env.mask = mask'),
paste('arcpy.env.scratchWorkspace ="G:/Faculty/Mann/Historic_BCM/Aggregated1080/Scratch.gdb"'),
paste('arcpy.env.outputCoordinateSystem = sr'),
# get spatial reference for raster and force output to that
paste('sr = arcpy.Describe(snap_raster).spatialReference'),
paste('py_projection = sr.exportToString()'),
paste('arcpy.env.extent = snap_raster'),
paste('poly_name = "',poly_name,'"',sep=""),
paste('poly_path_name = "',poly_path_name,'"',sep=""),
paste('holder = EucDistance(poly_path_name, maximum_distance, cellsize, "")'),
paste('holder = SetNull(holder < -9999, holder)'),
paste('holder.save(out_raster_path_name) ')
), fileConn, sep = "\n")
close(fileConn)
system(paste('C:\\Python27\\ArcGIS10.1\\python.exe', py_path))
}

python doctest: expected result is the same as the "got" result but the test failed

I am on a learning stage of using python as a tool for software QA.
I wrote the next simple test in order to find the letter 'a' in a text file number matrix.
problem is that the test fails even though the expect equals to what i got.
Why is that? Can you tell me what am I doing wrong?
test script:
fin = open("abc.txt", "r")
arr_fin = []
for line in fin:
arr_fin.append(line.split())
print arr_fin
for row in arr_fin:
arr_fin_1 = " ".join('{0:4}'.format(i or " ") for i in row)
print arr_fin_1
def find_letter(x, arr_fin_1):
"""
>>> find_letter('a', arr_fin_1)
97
"""
t=ord(x) #exchange to letter's ASCII value
for i in arr_fin_1:
if i==x:
print t
return;
def _test():
import doctest
doctest.testmod()
if __name__ == "__main__":
_test()
error message:
Expected:
97
Got:
97
**********************************************************************
1 items had failures:
1 of 1 in __main__.find_letter
***Test Failed*** 1 failures.

You've got an extra space after the 97 - if you remove it, your test should run fine.

This:
return;
Makes your function return None.
Did you mean return t?
Besides that, IMHO doctest tests are meant to be self-contained. This is something the user should see in your documentation and understand without context. In your example, you're using a module-local arr_fin_1 object which is completely opaque to the user. It's better to define it in the doctest before the find_letter call to provide a self-contained example.