I've been using bpython for a while now for all of my Python interpreting needs. It's delightful, particularly when you're using unfamiliar new libraries, or libraries with a multitude of functions. In any case, it's nice to have a bpython interpreter running alongside what I'm doing, but it'd be even better if I had both the autocomplete-like feature, and the parameter description in the manner that bpython does while I'm editing code in Emacs. Am I completely crazy? Does anyone have an idea on how to do this?
Thanks,
Bradley Powers
You're not completely crazy.
python-mode can integrate with eldoc-mode to display the arg spec of the function you're calling at point. Just do M-x eldoc-mode while you're in a python file to turn it on and it should start working. It talks to an inferior python buffer to inspect the functions directly, so it should always be decently accurate. You can turn it on automatically for all new python-mode buffers with (add-hook 'python-mode-hook '(lambda () (eldoc-mode 1)) t) in your emacs startup file. Now, at this point I have to say that I don't do any regular python programming, and that when I tried it just now it didn't work. I spent a few minutes poking around in the source code and everything seems to be in place, but the code that it runs in the inferior process is just returning an empty string. Perhaps it's just my setup, or perhaps I'm reading the wrong source files; it's hard to say.
Emacs provides several different types of expansion/autocompletion. By default you have access to dabbrev-expand by hitting M-/. This is a fairly simple form of completion; it's just meant to work on any old file you happen to edit. More sophisticated is hippie-expand, but even that doesn't do anything python-specific. The documentation says that it can integrate with hippie-expand for exact completions, but this might be a lie; I couldn't figure out how it works. A little poking around shows several related solutions for this, all of which seem to rely on pymacs. If I were going to do a lot of python programming and didn't already have a fairly complicated emacs set up, I'd probably start by installing emacs-for-python. It looks to be a pretty complete setup, and even claims to have live warning/error detection.
In the spirit of helping others to help themselves, I'd like to point out how I came upon all of this information. My first step was to open a file in python-mode. I didn't actually have any python code available, so I just went to my scratch buffer and made it a python buffer (M-x python-mode). Then I asked for help about this strange new mode (C-h m) to see what it could do. It's author has kindly put a brief summary of what the mode can do which mentions eldoc-mode, Imenu, outline-mode, hippie-expand, rlcompleter, abbrev tables, and a bunch of other things. From there I started looking at the source code. For instance, to integrate with eldoc-mode, it defines a function called python-eldoc-function and gives that to the eldoc module for use in python buffers. Reading that code shows me how it interacts with the inferior buffer, etc.
I hope some of this helps.
Related
I'm creating a program in python (2.7) and I want to protect it from reverse engineering.
I compiled it using cx_freeze (supplies basic security- obfuscation and anti-debugging)
How can I add more protections such as obfuscation, packing, anti-debugging, encrypt the code recognize VM.
I thought maybe to encrypt to payload and decrypt it on run time, but I have no clue how to do it.
Generally speaking, it's almost impossible for you to make your program unbreakable as long as there's enough motive for the hackers.
But still you can make it harder to be reverse engineered, try to use cython to compile your core codes into pyd or so files.
There's no way to make anything digital safe nowadays.
What you CAN do is making it hard to a point where it's frustrating to do it, but I admit I don't know python specific ways to achieve that. The amount of security of your program is not actually a function of programsecurity, but of psychology.
Yes, psychology.
Given the fact that it's an arms race between crackers and anti-crackers, where both continuously attempt to top each other, the only thing one can do is trying to make it as frustrating as possible. How do we achieve that?
By being a pain in the rear!
Every additional step you take to make sure your code is hard to decipher is a good one.
For example could you turn your program into a single compiled block of bytecode, which you call from inside your program. Use an external library to encrypt it beforehand and decrypt it afterwards. Do the same with extra steps for codeblocks of functions. Or, have functions in precompiled blocks ready, but broken. At runtime, utilizing byteplay, repair the bytecode with bytes depending on other bytes of different functions, which would then stop your program from working when modified.
There are lots of ways of messing with people's heads and while I can't tell you any python specific ways, if you think in context of "How to be difficult", you'll find the weirdest ways of making it a mess to deal with your code.
Funnily enough this is much easier in assembly, than python, so maybe you should look into executing foreign code via ctypes or whatever.
Summon your inner Troll!
Story time: I was a Python programmer for a long time. Recently I joined in a company as a Python programmer. My manager was a Java programmer for a decade I guess. He gave me a project and at the initial review, he asked me that are we obfuscating the code? and I said, we don't do that kind of thing in Python. He said we do that kind of things in Java and we want the same thing to be implemented in python. Eventually I managed to obfuscate code just removing comments and spaces and renaming local variables) but entire python debugging process got messed up.
Then he asked me, Can we use ProGuard? I didn't know what the hell it was. After some googling I said it is for Java and cannot be used in Python. I also said whatever we are building we deploy in our own servers, so we don't need to actually protect the code. But he was reluctant and said, we have a set of procedures and they must be followed before deploying.
Eventually I quit my job after a year tired of fighting to convince them Python is not Java. I also had no interest in making them to think differently at that point of time.
TLDR; Because of the open source nature of the Python, there are no viable tools available to obfuscate or encrypt your code. I also don't think it is not a problem as long as you deploy the code in your own server (providing software as a service). But if you actually provide the product to the customer, there are some tools available to wrap up your code or byte code and give it like a executable file. But it is always possible to view your code if they want to. Or you choose some other language that provides better protection if it is absolutely necessary to protect your code. Again keep in mind that it is always possible to do reverse engineering on the code.
I'm creating a program in python (2.7) and I want to protect it from reverse engineering.
I compiled it using cx_freeze (supplies basic security- obfuscation and anti-debugging)
How can I add more protections such as obfuscation, packing, anti-debugging, encrypt the code recognize VM.
I thought maybe to encrypt to payload and decrypt it on run time, but I have no clue how to do it.
Generally speaking, it's almost impossible for you to make your program unbreakable as long as there's enough motive for the hackers.
But still you can make it harder to be reverse engineered, try to use cython to compile your core codes into pyd or so files.
There's no way to make anything digital safe nowadays.
What you CAN do is making it hard to a point where it's frustrating to do it, but I admit I don't know python specific ways to achieve that. The amount of security of your program is not actually a function of programsecurity, but of psychology.
Yes, psychology.
Given the fact that it's an arms race between crackers and anti-crackers, where both continuously attempt to top each other, the only thing one can do is trying to make it as frustrating as possible. How do we achieve that?
By being a pain in the rear!
Every additional step you take to make sure your code is hard to decipher is a good one.
For example could you turn your program into a single compiled block of bytecode, which you call from inside your program. Use an external library to encrypt it beforehand and decrypt it afterwards. Do the same with extra steps for codeblocks of functions. Or, have functions in precompiled blocks ready, but broken. At runtime, utilizing byteplay, repair the bytecode with bytes depending on other bytes of different functions, which would then stop your program from working when modified.
There are lots of ways of messing with people's heads and while I can't tell you any python specific ways, if you think in context of "How to be difficult", you'll find the weirdest ways of making it a mess to deal with your code.
Funnily enough this is much easier in assembly, than python, so maybe you should look into executing foreign code via ctypes or whatever.
Summon your inner Troll!
Story time: I was a Python programmer for a long time. Recently I joined in a company as a Python programmer. My manager was a Java programmer for a decade I guess. He gave me a project and at the initial review, he asked me that are we obfuscating the code? and I said, we don't do that kind of thing in Python. He said we do that kind of things in Java and we want the same thing to be implemented in python. Eventually I managed to obfuscate code just removing comments and spaces and renaming local variables) but entire python debugging process got messed up.
Then he asked me, Can we use ProGuard? I didn't know what the hell it was. After some googling I said it is for Java and cannot be used in Python. I also said whatever we are building we deploy in our own servers, so we don't need to actually protect the code. But he was reluctant and said, we have a set of procedures and they must be followed before deploying.
Eventually I quit my job after a year tired of fighting to convince them Python is not Java. I also had no interest in making them to think differently at that point of time.
TLDR; Because of the open source nature of the Python, there are no viable tools available to obfuscate or encrypt your code. I also don't think it is not a problem as long as you deploy the code in your own server (providing software as a service). But if you actually provide the product to the customer, there are some tools available to wrap up your code or byte code and give it like a executable file. But it is always possible to view your code if they want to. Or you choose some other language that provides better protection if it is absolutely necessary to protect your code. Again keep in mind that it is always possible to do reverse engineering on the code.
Python is a relatively new language for me and I already see some of the trouble areas of maintaining a scripting language based project. I am just wondering how the larger community , with a scenario when one has to maintain a fairly large code base written by people who are not around anymore, deals with the following situations:
Return type of a function/method. Assuming past developers didn't document the code very well, this is turning out to be really annoying as I am basically reading code line by line to figure out what a method/function is suppose to return.
Code refactoring: I figured a lot of code need to be moved around, edited/deleted and etc. But lot of times simple errors, which would otherwise be compile time error in other compiled languages e.g. - wrong number of arguments, wrong type of arguments, method not present and etc, only show up when you run the code and the code reaches the problematic area. Therefore, whether a re-factored code will work at all or not can only be known once you run the code thoroughly. I am using PyLint with PyDev but still I find it very lacking in this respect.
You are right, that's an issue with dynamically typed interpreted languages.
There are to important things that can help:
Good documentation
Extensive unit-testing.
They apply to other languages as well of course, but here they are especially important.
As far as I know If code is not documented at all and the author isn't around anymore it's up to you to find out what the ode actually does.
That's why people should always stick to certain guidelindes that can be enforced by stylecheckers like pep8. https://pypi.python.org/pypi/pep8
Comments and docstrings should be included in every method to avoid such situation you're describing. http://www.python.org/dev/peps/pep-0257/#what-is-a-docstring
Also unittests are very helpfull for refactoring since you can check if you broke something with the click of a button. http://docs.python.org/2/library/unittest.html
hope this helps
Others have already mentioned documentation and unit-testing as being the main tools here. I want to add a third: the Python shell. One of the huge advantages of a non-compiled language like Python is that you can easily fire up the shell, import your module, and run the code there to see what it does and what it returns.
Linked to this is the Python debugger: just put import pdb;pdb.set_trace() at any point in your code, and when you run it you will be dropped into the interactive debugger where you can inspect the current values of the variables. In fact, the pdb shell is an actual Python shell as well, so you can even change things there.
I'm trying to write some python, and I'm used to the lispy way of doing things, a REPL in EMACS and the ability to send arbitrary code snippets to the REPL. I like this way of developing code, and python's built-in IDLE seems to do it pretty well. However I do like EMACS as an editor.
What's the best thing analogous to SLIME for Python?
So far:
It seems that the trick is to open a python file, and then to use 'Start Interpreter' from the Python menu, after which you get an Inferior Python buffer. You can then use C-c C-c to load the whole buffer you're editing into the 'REPL', and use normal copy and paste to put snippets into the REPL.
This works as far as it goes. Is there any way to say 'reevaluate the big thing that the cursor is in now and display the answer', or 'reevaluate the thing the cursor is just at the end of and display the answer', like M-C-x and C-x-e in SLIME?
And it all seems to work better if you use the python-mode.el from Bozhidar's answer
There is ipython.el which you can use for some extended functionality over the vanilla python-mode. Ropemacs provides a few extra completion and refactoring related options that might help you. This is discussed here. However, I don't expect you're going to get anything close to SLIME.
I think the new python.el is a much better idea. It's under active development, it can spawn a python shell and send function definitions, buffers and files to it. It also has better than average re-indent support. It's rumoured that in Emacs 24 it might become the default python mode in Emacs.
python-mode is the default mode for editing Python in emacs. It includes commands for evaluating the buffer and running an inferior interpreter instance.
I often use ipython (or the regular python shell) to test python code snippets while coding, and it's been very useful. One shortcoming of this, though, is that if I want to test a multi-line segment of code, or want to write multiple lines of code before running, it isn't very convenient to have to do it "line by line". And even going back to change some of the lines is cumbersome because you have to re-type all the code that comes after it.
I'm playing with Groovy right now and I find that it has an excellent solution to this problem: the Groovy Console. You just write all the code you want, it's just like a regular editor; and then you hit run Ctrl+R (Cmd+R actually since I'm on a Mac) and it runs everything at once. If you want to change something (e.g. if there are errors), then that's easy too -- just change it and Ctrl+R again.
Is there an equivalent of this available for python? Or do you have any recommendations on a way to achieve similar behavior? I could just create a new file, save it, and then python <filename>.py from the shell. But that's just too many steps and would be cumbersome. Eclipse may be an option, but it's too heavyweight. I'm really looking for something lightweight that I can just spin up when I want to test something and then get rid of it just as quickly.
I'd be interested to hear any ideas/suggestions!
Thanks
You might give DreamPie a try. As far as I can tell from a quick read of the groovyConsole page you linked to, DreamPie features a similar input area/output area division (they call it "code box" and "history box"). The code you execute is by default cleared from the code box - which groovyConsole apparently doesn't do - but you can easily retrieve it (Ctrl+Up), or change a preference setting to "Leave code in the code box after execution".
Have you tried using IDLE, the standard Python IDE? You'd have to save the code as <filename>.py within IDLE, but after that you can run it using F5.
The Python docs link to this intro to IDLE, which may be helpful even if it's a little outdated.
I am using emacs and its python-mode.
C-c C-c: evals the current buffer
but you can also eval region (ie selection), functions etc ...
You can even make python-mode use ipython (like I do).
See http://ipython.scipy.org/dist/ipython.el . It works nicely
Did you try PyCrust? It has excellent multi-line editing, copy/paste support.
PyCrust can be found in wxPython Docs and Demos.