I'm currently developing a dynamic software system in python. It includes the use of dependency injection at many points, which is implemented with dynamic class attributes. My aim is to add a function dynamically to a class (with setattr) an this works quite fine. The only problem I have is that the IDE (in my case it's PyCharm) has no idea about those functions and marks it as "unresolved reference". Although the script runs without errors, it looks not very nice and I want the IDE to support other programmers, that don't know about those functions.
Here is an example of what I mean:
class A:
def __init__(self, func_obj):
setattr(self, 'custom_function', func_obj)
a = A(print) # Unresolved attribute reference 'custom_function' for class 'A' ...
a.custom_function('Hello World!')
As expected, this example prints "Hello World!". But I find it's ugly, that the IDE shows a warning for the last line, where I called "custom_function". And also there is no auto completion if you try to explore the script with duck typing.
So let me come to my question: Is there any way to tell the IDE that there are some dynamically added functions in this class? Maybe there is some way with meta classes or something like that... But I didn't find anything on Google and I have no idea where else to try.
I hope you can help me with that :)
I had a similar request and asked directly to PyCharm bug tracker: PY-28326. I think that their answer applies to your question too:
PyCharm at the moment doesn't support methods dynamically added to classes.
Related
I am trying to code up a module which has two classes. First class is called as TextProcessing
class TextProcessing(object):
""" To carry out text processing
"""
def __init__(self,):
pass
It has various methods in there for pre-processing text.
Similary other class is for other data wrangling on pre-processed data.
I am saving these two classes in a python file to make it a module.
Now lets say a user downloads this python module and would now want to run the various methods of each class.
I wanted to provide some sort of documentation about the module, methods of each class to a user when she imports the module so that she is aware of which function to call and what parameters to pass.
Think of how a scikit learn documentation is on their documentation page.
http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfTransformer.html
Even the documentation we get to see when we do
help(some_python_module)
is fine too.
Issue is I don't have a documentation page like sklearn to show documentation. And I wanted a user to know documentation of various methods she can use once she imports the module in python console.
Is there a way I can print that documentation info to the console when a user imports the module?
It can show the doc string of each Class and Method.
This is a very weird thing to do, but it's definitely possible.
The easiest thing to do is just to call help. While it's intended to be called from the interactive prompt, there's nothing stopping you from calling it from your own code.
Of course you could instead extract the docstrings (they're stored as __doc__ on every module, class, and function), textwrap them yourself, and print them out, but if you're trying to reproduce the same thing help does, that's a lot of work for no real benefit.
The only tricky bit is that the thing you want to invoke the help system on is "this current module". How do you refer to that? It's a bit clunky, but you have this current module's name as __name__, so you can look it up in sys.modules.
So:
"""Helpful module"""
import sys
class Spam:
"""Classy class"""
def eggs(self):
"Functional function"
return 2
help(sys.modules[__name__])
Now, when you import helpful for the first time in a session, it will print out the help.
Of course that will be pretty odd if someone's trying to run a script that does an import helpful, rather than doing it from an interactive session. So you may want to only do this in interactive sessions, by checking sys.flags:
if sys.flags.interactive:
help(sys.modules[__name__])
What if someone does an import otherthing, and that otherthing does an import helpful? You'll get the same help, which may be confusing.
If that's a problem, the only real option I can think of is to check whether the calling frame comes from the top-level script (and that the flags are interactive). That's pretty hacky, and something you shouldn't even consider unless you really need to, so I'll just direct you to the inspect module and hope you don't need it.
I would like to get the code completion in Eclipse with PyDev for the attributes of a class which is dynamically generated.
Basically, I have a class which is defined by reading out an XML document. Depending on what is written in this XML document, the class has different attributes dynamically defined (the XML-tags).
I would like to activate the code completion after calling the constructor of the class in my code.
The problem I see is that I have no control on the attributes of the class, which means : before running the code, I have no idea which attributes might be available. Does anyone have an idea ?
I tried to add the library to the Forces-Built In without success.
Regards
Well, PyDev can't really guess what you have if by looking at the code you don't know yourself ;)
Still, PyDev allows you to give it skeletons which can be used for code-completion purposes, so, an option here would be parsing the xml yourself and creating a class structure with attributes/methods as needed and ask PyDev to use that to give you completions.
You can see the Interpreter Configuration on the Getting Started manual: http://www.pydev.org/manual_101_interpreter.html#PyDevInterpreterConfiguration-PredefinedCompletions for info on how to actually do that.
Ok, i found yersterday some example with pypredef, and I started with this solution, and for now on it works well.
The only thing you need to care about is the identation. It looks like when you are implementing your .pypredef in Eclipse, you need to use 4 spacings als ident instead of a tab.
I'm learning python and came into a situation where I need to change the behvaviour of a function. I'm initially a java programmer so in the Java world a change in a function would let Eclipse shows that a lot of source files in Java has errors. That way I can know which files need to get modified. But how would one do such a thing in python considering there are no types?! I'm using TextMate2 for python coding.
Currently I'm doing the brute-force way. Opening every python script file and check where I'm using that function and then modify. But I'm sure this is not the way to deal with large projects!!!
Edit: as an example I define a class called Graph in a python script file. Graph has two objects variables. I created many objects (each with different name!!!) of this class in many script files and then decided that I want to change the name of the object variables! Now I'm going through each file and reading my code again in order to change the names again :(. PLEASE help!
Example: File A has objects x,y,z of class C. File B has objects xx,yy,zz of class C. Class C has two instance variables names that should be changed Foo to Poo and Foo1 to Poo1. Also consider many files like A and B. What would you do to solve this? Are you serisouly going to open each file and search for x,y,z,xx,yy,zz and then change the names individually?!!!
Sounds like you can only code inside an IDE!
Two steps to free yourself from your IDE and become a better programmer.
Write unit tests for your code.
Learn how to use grep
Unit tests will exercise your code and provide reassurance that it is always doing what you wanted it to do. They make refactoring MUCH easier.
grep, what a wonderful tool grep -R 'my_function_name' src will find every reference to your function in files under the directory src.
Also, see this rather wonderful blog post: Unix as an IDE.
Whoa, slow down. The coding process you described is not scalable.
How exactly did you change the behavior of the function? Give specifics, please.
UPDATE: This all sounds like you're trying to implement a class and its methods by cobbling together a motley patchwork of functions and local variables - like I wrongly did when I first learned OO coding in Python. The code smell is that when the type/class of some class internal changes, it should generally not affect the class methods. If you're refactoring all your code every 10 mins, you're doing something seriously wrong. Step back and think about clean decomposition into objects, methods and data members.
(Please give more specifics if you want a more useful answer.)
If you were only changing input types, there might be no need to change the calling code.
(Unless the new fn does something very different to the old one, in which case what was the argument against calling it a different name?)
If you changed the return type, and you can't find a common ancestor type or container (tuple, sequence etc.) to put the return values in, then yes you need to change its caller code. However...
...however if the function should really be a method of a class, declare that class and the method already. The previous paragraph was a code smell that your function really should have been a method, specifically a polymorphic method.
Read about code smells, anti-patterns and When do you know you're dealing with an anti-pattern?. There e.g. you will find a recommendation for the video "Recovery from Addiction - A taste of the Python programming language's concision and elegance from someone who once suffered an addiction to the Java programming language." - Sean Kelly
Also, sounds like you want to use Test-Driven Design and add some unittests.
If you give us the specifics we can critique it better.
You won't get this functionality in a text editor. I use sublime text 3, and I love it, but it doesn't have this functionality. It does however jump to files and functions via its 'Goto Anything' (Ctrl+P) functionality, and its Multiple Selections / Multi Edit is great for small refactoring tasks.
However, when it comes to IDEs, JetBrains pycharm has some of the amazing re-factoring tools that you might be looking for.
The also free Python Tools for Visual Studio (see free install options here which can use the free VS shell) has some excellent Refactoring capabilities and a superb REPL to boot.
I use all three. I spend most of my time in sublime text, I like pycharm for refactoring, and I find PT4VS excellent for very involved prototyping.
Despite python being a dynamically typed language, IDEs can still introspect to a reasonable degree. But, of course, it won't approach the level of Java or C# IDEs. Incidentally, if you are coming over from Java, you may have come across JetBrains IntelliJ, which PyCharm will feel almost identical to.
One's programming style is certainly different between a statically typed language like C# and a dynamic language like python. I find myself doing things in smaller, testable modules. The iteration speed is faster. And in a dynamic language one relies less on IDE tools and more on unit tests that cover the key functionality. If you don't have these you will break things when you refactor.
One answer only specific to your edit:
if your old code was working and does not need to be modified, you could just keep old names as alias of the new ones, resulting in your old code not to be broken. Example:
class MyClass(object):
def __init__(self):
self.t = time.time()
# creating new names
def new_foo(self, arg):
return 'new_foo', arg
def new_bar(self, arg):
return 'new_bar', arg
# now creating functions aliases
foo = new_foo
bar = new_bar
if your code need rework, rewrite your common code, execute everything, and correct any failure. You could also look for any import/instantiation of your class.
One of the tradeoffs between statically and dynamically typed languages is that the latter require less scaffolding in the form of type declarations, but also provide less help with refactoring tools and compile-time error detection. Some Python IDEs do offer a certain level of type inference and help with refactoring, but even the best of them will not be able to match the tools developed for statically typed languages.
Dynamic language programmers typically ensure correctness while refactoring in one or more of the following ways:
Use grep to look for function invocation sites, and fix them. (You would have to do that in languages like Java as well if you wanted to handle reflection.)
Start the application and see what goes wrong.
Write unit tests, if you don't already have them, use a coverage tool to make sure that they cover your whole program, and run the test suite after each change to check that everything still works.
For one of my projects I have a python program built around the python cmd class. This allowed me to craft a mini language around sql statements that I was sending to a database. Besides making it far easier to connect with python, I could do things that sql can't do. This was very important for several projects. However, I now need to add in if blocks for greater control flow.
My current thinking is that I will just add two new commands to the language, IF and END. These set a variable which determines whether or not to skip a line. I would like to know if anyone else has done this with the cmd module, and if so, is there a standard method I'm missing? Google doesn't seem to reveal anything, and the cmd docs don't reveal anything either.
For an idea that's similar to what I'm doing, go here. Questions and comments welcome. :)
Hmm, a little more complicated than what I was thinking, though having python syntax would be nice. I debated building a mini language for quite some time before I finally did it. The problem primarily comes in from the external limitations. I have a bunch of "data", which is being generous, to turn into sql. This is based on other "data" that won't pass through. It's also unique to each specific "version" of the problem. Doing straight data to sql would have been my first inclination, but was not practical.
For the curious, I spent a great deal of time going over the mini languages chapter in the art of unix programming, found here.
If I had built the thing in pure python, I wouldn't have had the flexibility I absolutely needed for the problem set.
The limitations of making a "mini language" have become apparent.
Proper languages have a tree-like structure and more complex syntax than cmd can handle easily.
Sometimes it's actually easier to use Python directly than it is to invent your own DSL.
Currently, your DSL probably reads a script-like file of commands.
Because of the way cmd works, your little comments get a string argument, which must be parsed. Then the command gets executed. And, further, each command is a method of the cmd.Cmd subclass.
Here's what you can do.
Each do_foo( self, args ) method becomes a stand-alone callable object. It will follow the Command design pattern. It will do exactly what the method function does now. No less. Exactly the same.
class Foo( object ):
def __init__( self, global_context ):
self.context= global_context
def __call__( self, args ):
... The rest of do_foo ...
Additionally, your existing cmd.Cmd subclass probably maintains some internal state.
All of the self.this and self.that instance variables must be changed to reference
and explicit context object.
class Context( object ): pass
Change self.this or self.that to self.context.this or self.context.that
Now, you can create your context and your various commands.
ctx = Context()
foo= Foo(ctx)
Your script changes syntax slightly. From:
foo argstring
bar argstring
to:
from mylanguage import foo, bar
foo( "argstring" )
bar( "argstring" )
This does Exactly what the CLI does now. No more. No less. Exactly the same. Slightly different syntax.
Now your script is no longer in a DSL that's hard to expand. It's in Python.
Having done that, you can now use Python syntax if statements.
You have the Exact functionality currently implemented in cmd with better syntax.
After examining the problem set some more, I've come to the conclusion that I can leave the minilanguage alone. It has all the features I need, and I don't have the time to rebuild the project from the ground up. This has been an interesting problem and I'm no longer sure I would build another minilanguage if I encountered the same situation. OTOH, it works very well here, and I am loathe to give up the advantages it has conferred.
I am using eclipse for python and I am facing a problem. I have many classes with many properties and want a list of objects from one of my declared classes. The problem is: When I am accessing any item from the list, the IDE does not know its type because in python we do not declare the variable with type, so there is no auto complete and I have to go to the class to copy the attribute name.
To make idea more clear:
class AutomataBranch(object):
def __init__(selfparams):
self.Name="";
self.nodes=[];
class LanguageAutomata(object):
def __init__(selfparams):
self.cfgAutomata=[];#This has AutomaBranch Type
Now in any method in LanguageAutomata class if I wrote:
cfgAutomata. Then it wont give me the Name attribute
Is there any solution for that?
Python is strongly typed and Python lists are too. Your problem come from the fact that Python is dynamically typed. Therefor a var can contain any type, and therefor no IDE can guess what is the type of your parameter, nor give you code completion for the methods.
This is how it is, there is no clean workaround. If it's a problem, then maybe dynamics language is not you predilection tool and you should use something that fit your development style. There are tools for everybody.
8 years later and we actually have a solution in Python 3.6.
PEP484 allows you to annotate your variables primarily for IDEs and linting:
Modifying #Hani's answer:
x : AutomataBranch = self.cfgAutomata[i]
This is now picked up by any good IDE to highlight errors and allow autocomplete.
I think you mean to say "statically typed" instead of "strongly typed." Python is strongly typed. You just don't know what that type is at compile time.
With that said, you really need to abandon the idea that you're going to find any IDEs that work as well for Python as they do for Java or C#. Python's dynamic typing makes this difficult. In fact, I tend to find that powerful IDEs are more of a burden than a help.
I think I found a good managable solution. Actually it is trivial but may help (I used it now).
When I want to access the list then I assign the object which I want to access to a variable ex:
x = AutomataBranch()
x = self.cfgAutomata[i]
The first line is used only to make the IDE knows that x is from AutomatBranch type. After that when I press x then all methods and properties are visualized.
I think it is some how good.