We Keep Coding

Linting classes created at runtime in Python

For context, I am using the Python ctypes library to interface with a C library. It isn't necessary to be familiar with C or ctypes to answer this question however. All of this is taking place in the context of a python module I am creating.
In short, my question is: how can I allow Python linters (e.g. PyCharm or plugin for neovim) to lint objects that are created at runtime? "You can't" is not an answer ;). Of course there is always a way, with scripting and the like. I want to know what I would be looking at for the easiest way.
First I introduce my problem and the current approach I am taking. Second, I will describe what I want to do, and ask how.
Within this C library, a whole bunch of error codes are defined. I translated this information from the .h header file into a Python enum:
# CustomErrors.py
from enum import Enum
class CustomErrors(Enum):
ERROR_BROKEN = 1
ERROR_KAPUTT = 2
ERROR_BORKED = 3
Initially, my approach is to have a single exception class containing a type field which described the specific error:
# CustomException.py
from CustomErrors import CustomErrors
class CustomException(Exception):
def __init__(self, customErr):
assert type(customErr) is CustomError
self.type = customErr
super().__init__()
Then, as needed I can raise CustomException(CustomErrors.ERROR_KAPUTT).
Now, what I want to do is create a separate exception class corresponding to each of the enum items in CustomErrors. I believe it is possible to create types at runtime with MyException = type('MyException', (Exception,), {'__doc__' : 'Docstring for ABC class.'}).
I can create the exception classes at runtime like so:
#CustomException.py
from CustomErrors import CustomErrors
...
for ce in CustomErrors:
n = ce.name
vars()[n] = type(n, (Exception,), {'__doc__' : 'Docstring for {0:s} class.'.format(n)})
Note: the reason I want to create these at runtime is to avoid hard-coding of an Exception list that change in the future. I already have the problem of extracting the C enum automatically on the backburner.
This is all well and good, but I have a problem: static analysis cannot resolve the names of these exceptions defined in CustomException. This means PyCharm and other editors for Python will not be able to automatically resolve the names of the exceptions as a suggested autocomplete list when the user types CustomException.. This is not acceptable, as this is code for the end user, who will need to access the exception names for use in try-except constructs.
Here is the only solution I have been able to think of: writing a script which generates the .py files containing the exception names. I can do this using bash. Maybe people will tell me this is really the only option. But I would like to know what other approaches are suggested for solving this problem. Thanks for reading.

You can add a comment to tell mypy to ignore dynamically defined attribute errors. Perhaps the linters that you use share a similar way to silence such errors.
mypy docs on silencing errors based on error codes
This example shows how to ignore an error about an imported name mypy thinks is undefined:
# 'foo' is defined in 'foolib', even though mypy can't see the
# definition.
from foolib import foo # type: ignore[attr-defined]

Keeping alias types simple in Python documentation?

I'm trying to use the typing module to document my Python package, and I have a number of situations where several different types are allowable for a function parameter. For instance, you can either pass a number, an Envelope object (one of the classes in my package), or a list of numbers from which an Envelope is constructed, or a list of lists of numbers from which an envelope is constructed. So I make an alias type as follows:
NumberOrEnvelope = Union[Sequence[Real], Sequence[Sequence[Real]], Real, Envelope]
Then I write the function:
def example_function(parameter: NumberOrEnvelope):
...
And that looks great to me. However, when I create the documentation using Sphinx, I end up with this horrifically unreadable function signature:
example_function(parameter: Union[Sequence[numbers.Real], Sequence[Sequence[numbers.Real]], numbers.Real, expenvelope.envelope.Envelope])
Same thing also with the hints that pop up when I start to try to use the function in PyCharm.
Is there some way I can have it just leave it as "NumberOrEnvelope". Ideally that would also link in the documentation to a clarification of what "NumberOrEnvelope" is, though even if it didn't it would be way better than what's appearing now.

I had the same issue and used https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_type_aliases, introduced in version 3.3.
In your sphinx conf.py, insert this section. It does not seem to make much sense at the first sight, but does the trick:
autodoc_type_aliases = dict(NumberOrEnvelope='NumberOrEnvelope')
Warning: It only works in modules that start with from __future__ import annotation
Note: If there is a target in the documentation, type references even have a hyperlink to the definition. I have classes, documented elsewhere with autoclass, which are used as types of function parameters, and the docs show the nice names of the types with links.

Support for this appears to be in the works.
See Issue #6518.
That issue can be closed by the recent updates to Pull Request #8007 (under review).
If you want the fix ASAP, you can perhaps try using that build.

EDIT: This doesn't quite work, sadly.
Turns out after a little more searching, I found what I was looking for. Instead of:
NumberOrEnvelope = Union[Sequence[Real], Sequence[Sequence[Real]], Real, Envelope]
I found that you can create your own compound type that does the same thing:
NumberOrEnvelope = TypeVar("NumberOrEnvelope", Sequence[Real], Sequence[Sequence[Real]], Real, Envelope)
This displays in documentation as "NumberOrEnvelope", just as I wanted.

PTVS IntelliSense not working for Built-in Function

class Class:
def __init__(self, path):
self._path = path
string = open(self._path, 'r'). #HERE
When I try to type read() intelliSense says no completions.
However, I know open() function returns file object, which has read() function. I want to see all supported function after typing a dot.
PyCharm shows me recommanded function list, but PTVS does not support.
I want to know this is casual things in PTVS or only happening to me.
My current Python Enviroment is Anaconda 4.3.0 (Python 3.5.3)
How can I fix it?

We've already fixed the specific case of open for our upcoming update (not the one that released today - the next one), but in short the problem is that you don't really know what open is going to return. In our fix, we guess one of two likely types, which should cover most use cases.
To work around it right now, your best option is to assign the result of open to a variable and force it to a certain type using an assert statement. For example:
f = open(self._path, 'r')
import io
assert isinstance(f, io.TextIOWrapper)
f = open(self._path, 'rb')
import io
assert isinstance(f, io.BufferedIOBase)
Note that your code will now fail if the variable is not the expected type, and that the code for Python 2 would be different from this, but until you can get the update where we embed this knowledge into our code it is the best you can do.

Module Instantiation in myhdl

I'm currently looking into myHdl to see if it's worth using or not. However, I've come across a hiccup regarding the instantiation of modules. I've got two files, one that's a module and one that's the testbench. Inside the testbench, I've instantiated the module following the example they have on the website:
http://www.myhdl.org/examples/flipflops.html
The instantiation specifically is this line: dff_inst = dff(q, d, clk)
However, I get an error when I try to run the testbench:
Exception TypeError: 'isinstance() arg 2 must be a class, type, or tuple of classes and types' in <generator object _LabelGenerator at 0x7f6070b2ea50> ignored
I assume this has something to do with the fact that I have two separate files, so my guess is that python isn't finding the dff module(since it's in a separate file). I tried adding in an import dff line, but that simply gave me a 'module' object is not callable type error, which makes sense.
Looking in the documentation, they don't have a full .py file, so I'm not sure how they're linking these testbenches with the module. They specifically mention a hierarchy system and being able to instantiate other modules, but I can't seem to get it to work.
From what I understand from documentation, it looks like they're just writing the testbench and the module in the same file. However, to my understanding, it looks like they imply you can import modules, but I can't figure out how that's done. Is there just some simple thing I'm overlooking?

After experimenting a bit, it seems like I just need to use the following command: from dff import dff,
which makes a lot of sense.

How to properly extend other classes in Python? (python v3.3)

I've started to learn python in the past few days, and while exploring object-oriented programming I'm running into problems.
I'm using Eclipse while running the pydev plugin, am running on the python 3.3 beta, and am using a windows 64 bit system.
I can initialize a class fine and use any methods within it, as long as I'm not trying to extend the superclass (each class I've coded in a different source file)
For example, the following code compiles and runs fine.
class pythonSuper:
string1 = "hello"
def printS():
print pythonSuper.string1
and the code to access and run it...
from stackoverflow.questions import pythonSuper
class pythonSub:
pysuper = pythonSuper.pythonSuper()
pysuper.printS()
Like I said, that works. The following code doesn't
class pythonSuper: """Same superclass as above. unmodified, except for the spacing"""
string1 = "hello"
def printS(self):
print(pythonSuper.string1)
Well, that's not quite true. The superclass is absolutely fine, at least to my knowledge. It's the subclass that weirds out
from stackoverflow.questions import pythonSuper
class pythonSub(pythonSuper):
pass
pythonObject = pythonSub()
pythonSub.pythonSuper.printS()
when the subclass is run Eclipse prints out this error
Traceback (most recent call last):
File "C:\Users\Anish\workspace\Python 3.3\stackoverflow\questions\pythonSub.py",
line 7, in <module>
class pythonSub(pythonSuper):
TypeError: module.__init__() takes at most 2 arguments (3 given)
I have no idea what's going on. I've been learning python from thenewboston's tutorials, but those are outdated (I think his tutorial code uses python version 2.7). He also codes in IDLE, which means that his classes are all contained in one file. Mine, however, are all coded in files of their own. That means I have no idea whether the code errors I'm getting are the result of outdated syntax or my lack of knowledge on this language. But I digress. If anyone could post back with a solution and/or explanation of why the code is going wrong and what I could do to fix it. An explanation would be preferred. I'd rather know what I'm doing wrong so I can avoid and fix the problem in similar situations than just copy and paste some code and see that it works.
Thanks, and I look forward to your answers

I ran your code, albeit with a few modifications and it runs perfectly. Here is my code:
pythonSuper:
class pythonSuper:
string1 = 'hello'
def printS(self):
print(self.string1)
main:
from pythonSuper import pythonSuper as pySuper
class pythonSub(pySuper):
pass
pythonObject = pythonSub()
pythonObject.printS()
NOTE: The change I have made to your code is the following:
In your code, you have written pythonSub.pythonSuper.printS() which is not correct, because via pythonSub you already support a printS() method, directly inherited from the superclass. So there is no need to refer to the superclass explicitly in that statement. The statement that I used to substitute the aforementioned one, pythonObject.printS(), seems to have addressed this issue.

pythonSuper refers to the module, not the class.
class pythonSub(pythonSuper.pythonSuper):
pass