I have written a class in python 3.8 and for reasons, I need some of the class methods to be defined in a different file. The method I used for putting this function as the class method was by adding it later with setattr. This allows me to selectively add methods I want, I wont get into too many details on that because its irrelevant
myClass.py:
# This bit will scan the external file and add relevant methods to the class via a decorator
# just to give an idea, don't get hungh up on the implementation
def add_methods():
def decorator(Class):
import methods from external.py
for method in external.py:
if method is what I want:
setattr(Class, method_name, method)
#add_method()
def myClass():
def __init__(self):
dostuff
def method_1(self, arg1 , arg2):
dostuff
def method_2(self, arg1 , arg2, arg3):
dostuff
Inside another folder, we got the method file I mentioned
method_folder/external.py
def ext_added_method_1(self):
do stuff here
def ext_added_method_2(self):
do stuff here
Then finally, the class is used by a code
main.py
from myClass import myClass
handler = myClass()
dostuff
This all works just fine, the externally added methods are there and work just as expected. but there is one annoying bit that is when I'm working on the 'external.py' file which is where most of the coding is done anyway. My VSCode understandably does not know what is inside of the 'self' parameter of ext_added_method_1 and ext_added_method_2, so there is no linting.
The question is, is there a way of telling vscode, that on external.py, the 'self' parameter should be linted from the myClass object?
I don't need linting of the methods inside external.py (ext_added_method_1 and ext_added_method_2) I know that would be impossible since they are dynamically added. But for the methods that are defined in myClass.py, such as method_1 and method_2 I would like to have autocomplete and all that magical stuff.
Any way to achieve that without changing the architecture?
thank you!
Thx rioV8m, I wasn't familiar with that term. But then I looked it up and came up with the following solution
external.py
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from myClass import myClass
def ext_added_method_1(self: 'myClass'):
do stuff here
def ext_added_method_2(self: 'myClass'):
do stuff here
With that import check, I have no issues with circular imports so it basically ignores that, but is enough for VSCode to pick up on the class.
Also, I add the type in single quotes which are called 'forward references'. This is then only used by the editor to inform it of the class contents.
Related
Good evening, i need an advice, googling i couldn't find a proper direction.
I need to make a method available only within the class (i.e other methods or functions), if called from the program as a method of the object referring to the class i want:
the method to be invisible/not available to the intellisense
if i'm stubborn, and code it anyway, must raise an error.
Attaching a screenshot to make it more clear.
Any advice is appreciated, Thank you.
Screenshot of the problem
There's no private methods in python. Common usage dictates to precede a method that's only supposed to be used internally with one or two underscores, depending on the case. See here: What is the meaning of single and double underscore before an object name?
As others have mentioned there are no private methods in Python. I also don't know how to make it invisible for intelisense (probably there is some setting), but what you could theoretically do is this:
import re
def make_private(func):
def inner(*args, **kwargs):
name = func.__name__
pattern = re.compile(fr'(.*)\.{name}')
with open(__file__) as file:
for line in file:
lst = pattern.findall(line)
if (lst and not line.strip().startswith('#')
and not all(g.strip() == 'self' for g in lst)):
raise Exception()
return func(*args, **kwargs)
return inner
class MyClass:
#make_private
def some_method(self):
pass
def some_other_method(self):
self.some_method()
m = MyClass()
# m.some_method()
m.some_other_method()
It (make_private) is a decorator which basically when you call the function it is decorating, it first reads the entire file line by line and tries to find if in all of the file this method is called without being prefixed with self.. So if it is not then it is considered to be called from outside the class and an Exception is raised (probably add some message to it tho).
Issues could start once you have multiple files and this wouldn't entirely prevent someone from calling it if they really wanted for example if they did it like this:
self = MyClass()
self.some_method()
But mostly this would raise an exception.
OK Solved, to hide the method to the ide's Intellisense i added the double underscore (works fine with pycharm, not with vscode) then i used the accessify module to prevent forced execution calling myobj._myclass__somemethod()
from accessify import private
class myclass:
#private
def __somemethod(self)
I thought that the following code would result in an error because as far as I have read, a method in a Python class must either have "self" (or any other label, but "self" by convention) as its first argument, or "cls" or similar if the #classmethod decorator is used, or none if the #staticmethod decorator is used.
How come I get no error running this with Python 3.5 in the Terminal, even though test_method does not meet these requirements? It seems to work fine as a static method, but without the decorator.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import sys
class MyClass:
def test_method(args):
print(args[1])
#staticmethod
def static_method():
print("static_method")
#classmethod
def class_method(cls):
print("class_method")
def main(args):
MyClass.test_method(args)
if __name__ == '__main__':
sys.exit(main(sys.argv))
Output:
$ python3 testscript.py "testing"
$ testing
EDIT:
My question could also be phrased differently, drawing attention away from self and to #staticmethod: "How come I'm getting a seemingly working static method without the #staticmethod decorator?"
In Python 2, functions defined in a class body are automatically converted to "unbound methods", and cannot be called directly without a staticmethod decorator. In Python 3, this concept was removed; MyClass.text_method is a simple function that lives inside the MyClass namespace, and can be called directly.
The main reason to still use staticmethod in Python 3 is if you also want to call the method on an instance. If you don't use the decorator, the method will always be passed the instance as the first parameter, causing a TypeError.
There is nothing special about this. In python 3 there is no difference between a function defined inside a class or a function defined outside a class. Both of them are normal functions.
The self that you are talking about here or maybe cls comes into picture only when you access the function through an instance. Hence here you didn't get any error.
However if you modify your code just a little bit to look like the following, then you'd get an error that you expected.
def main(args):
MyClass().test_method(args)
# Should throw an error
EDIT:
#staticmethod will work on both class instances like MyClass().test_method(args)and just a regular direct call like MyClass.test_method(args)
However a regular method(without self in it) can't be called on a class instance. So you will always have to call it as MyClass.test_method(args)
self isn't necessarily required. However, if you want to reference any variable or value that is associated with the object(instantiation of the class) (E.g. for a class about cars, it's speed, self.speed) you'll need to have self as a parameter in the function. For this reason, it's common practice to always have self as an argument, otherwise you aren't really using the class for the right reason.
EDIT:
This will actually throw an error if you do the following:
class a():
def __init__(self, x):
self.asd = x
def hello(x):
print(x)
>>> g = a(4)
>>> g.hello(5)
as when calling "hello", both "self" and "4" will be passed as parameters. It would work in the following instance, which is what I was saying above:
>>> g = a
>>> g.hello(4)
or
>>> a.hello(4)
To add on to the existing answers here and provide a code example:
class MyClass:
def __init__(self):
pass
def myStaticMethod():
print("a static method")
#staticmethod
def myStaticMethodWithArg(my_arg):
print(my_arg)
print("a static method")
MyClass.myStaticMethod()
MyClass.myStaticMethodWithArg("skhsdkj")
abc = MyClass()
abc.myStaticMethodWithArg("avc")
Try removing the #staticmethod decorator and rerunning the code and see what happens! (The very last call will fail since the method is passed in both self and the string input. By adding the decorator, we can guide the interpreter to perform our desired action)
As you know, when the project's code is very large and there are so many attributes and functions defined in a Class, but some of them never be called by the instance of the Class, and maybe some of them has been discarded. Here is a example:
class Foo(object):
""""""
def __init__(self):
self.a = 1
self.b = 2
self.c = 3
...
self.y = 25
self.z = 26
def func1(self):
pass
def func2(self):
pass
def func3(self):
pass
...
...
def func100(self):
pass
if __name__ == '__main__':
f = Foo()
f.func1()
f.func2()
print f.a, f.b, f.z
In the above code, the instance f of class Foo just called func1() and func2(). And how to find all the attributes and functions of class that never called by the instance of class.
I have tried compiler module but that could not solve my question. And dir(my_instance) is just print all the functions and attributes defined the the class.
Thanks in advance.
You can try coverage.py. It's not static analysis, but actually runs your code and records which statements are executed, outputting annotated html or txt as you wish (quite nicely formatted as well). You can then look for functions and methods whose bodies are not executed at all.
This still doesn't take care of unused attributes. And I don't know the answer to that. Maybe comment them out one at a time and see if tests still pass...
It's pretty hard to prove something is or is not used in the general case. Python is a dynamic language; if even one bit of code calls into code the static analyzer doesn't fully analyze, it could be accessing the variables mentioned.
The pylint and flake8 tools will tell you about local and global names that aren't defined prior to use (unless you break them by using from x import * style imports), and about imports that are never used (an import that is never used is usually wrong, but even then, it could be an intentional part of the interface, where linters would have to be silenced), but I don't believe they can tell you that a given attribute is never accessed; after all, someone else could import your module and access said attributes.
Use the profile module in the standard library.
python -m cProfile -o output_file myscript.py
Then load the stats file and use print_callees() to get all the functions that were called--during that run of the program.
I don't know of any easy way to find out which attributes are used.
I am trying to use a nose_parameterized test and want to use it for a unittest method.
from nose.tools import assert_equal
from nose_parameterized import parameterized
import unittest
Class TestFoo(unittest.TestCase):
def setUp(self):
self.user1 = "Bar"
self.user2 = "Foo"
#parameterized.expand([
("testuser1",self.user1,"Bar"),
("testuser2",self.user2,"Foo")
]
def test_param(self,name,input,expected):
assert_equal(input,expected)
But self is not defined in the decorator function. Is there a workaround for this? I know that I can use global class variables but I need to use variables in setUp.
One workaround would be to use a string containing the attribute name in the decorator, and getattr in the test function:
#parameterized.expand([
("testuser1", "user1", "Bar"),
("testuser2", "user2", "Foo")
])
def test_param(self, name, input, expected):
assert_equal(getattr(self, input), expected)
With this approach, test_param assumes that the value of its input argument is the attribute name whose value should be checked against expected.
The decorator is not run when you seem to assume it will be run. In the following example:
class spam:
#eggs
def beans( self ):
pass
remember that the use of the decorator is the same as saying:
beans = eggs( beans )
inside the spam scope, immediately after the def statement itself is executed. When is a def statement executed? At the time the class and its methods are defined. The decorator modifies the class-level definition of the method spam.beans, not the instance-level value of self.beans. And of course, this occurs long before any instances of that class are ever created, i.e. before a reference to any one particular instance, self, is ever meaningful.
If you want to attach a particular callable (e.g. a modified test_param callable that has certain arguments pre-baked into it using functools.partial) to an instance self, you can of course do so inside one of the instance methods (e.g. __init__ or setUp).
Some people will describe the class-definition code as happening at "parse time" and instance-level code as happening at "run time". You may or may not find that a helpful way of thinking about it, although really almost everything is "run-time" in Python.
I have a Python library design that I'm trying to clean up, but I noticed that one piece isn't auto-completing in Eclipse/PyDev. I'm familiar enough for this not to be a problem for me, but its a library that others will end up using and auto-complete that doesn't work for a feature chunk won't do.
I'll just explain quickly what its trying to do, I re-created the design below. It all works, but auto-complete isn't useful. Maybe someone can just set me straight.
In module test_module2.py
import Main from test_module.py
from test_module import Main
class Second(object):
_main = Main
def set_main_variable(self, var):
self._main.variable = var
In module test_module.py
import sys
class Main(object):
variable = 0
second = None
def __init__(self):
for x in sys.modules.keys():
if x.endswith("test_module2"):
y = sys.modules[x]
self.second = y.Second()
self.second._main = self
def print_variable(self):
print self.variable
In application test.py
import test_module
import test_module2
if __name__ == "__main__":
m = test_module.Main()
m.second.set_main_variable(10)
m.print_variable() # prints 10
With this, if the test_module2.py module is imported, you can access it via a separate namespace member variable Main.second. Keeps things separate, and if the module isn't available, then you don't have access to Second's features.
In test_module2.py, Main is imported (you can't use Second without Main anyways) and _main is default to Main. This allows auto-complete in Second when you're working on the parent _main reference that's setup when Main is constructed and Second was imported. (this is good)
In Main, Second is optional and is never directly calls from Main methods. So auto-complete isn't necessary. (this is fine)
In __main__, auto-complete naturally works for Main member variables and methods, but doesn't work with m.second. I can't default m.second to anything because I'd end up having to import test_module2.py from test_module.py. It's optional and is defined by __main__'s import of test_module2.py. e.g. if __main__ imports test_module2.py, then Second is automatically available to the application.
Anyone have a better way to have an optionally imported module/class construct to a member variable, that will work with IDE auto-completion?
Thanks in advance for any ideas.
I'm going to come to the conclusion that auto-completion with PyDev will only see a variable set to a class's members if the parent class inherits from it, or defaults the variable to the class itself. For example:
class Main(object):
second = Second
Then in __main__ you can auto-complete: main.second...
That or Main has to inherit from Second.
Ok, I need to go back and complain that this library design isn't going to work with IDE auto-completion. See if I can use an wrapper class to inherit if the import test_module2 is present and clean things up.
My solution:
Here's what I came up with:
In module test_module2.py
import test_module
class Second(object):
_variable = 0 # overrided by Main's _variable
def set_main_variable(self, var):
self._variable = var
class Main(test_module.Main, Second):
def __init__(self):
super(Main, self).__init__()
In module test_module.py
class Main(object):
_variable = 0
second = None
def __init__(self):
super(Main, self).__init__()
def print_variable(self):
print self._variable
Now! In test.py if you import test_module or test_module2 (not both), you can construct Main with or without Second's added functionality. Second will have access to everything in Main, and because Main is inheriting Second, auto-complete works.
In application test.py
#import test_module
import test_module2
if __name__ == "__main__":
m = test_module.Main()
m.set_main_variable(10)
m.print_variable() # prints 10
I don't know if I can easily move Second's methods into a sub-namespace, like Main.second.set_variable(). I would have to explicitly set a Main reference within Second after Main constructs it as a variable, (in Main init, self.second = Second()) and not have Second inherited by Main. Then you could call m.second.set_main_variable(10), and keep all Second methods accesible from Main under the .second namespace.
First, Simple is better than complex -- your design appears to be complicated for no good reason. Maybe you should provide some details about your actual library.
Second, with Python as a dynamically typed language, it's just natural that there are cases when your IDE fails to auto-complete because your IDE has a static perspective. And you definitely shouldn't programm to suite the capabilites of your IDE.