We Keep Coding

Erratic behavior with Mypy in VSCode: How to troubleshooting and solve?

I have been having issues with MyPy in VSCode. I usually have my venv activated, Pylance as the language server and the MyPy extension (to perform check on editor change event).
MyPy.
MyPy is installed on the current venv, is enabled, linting in enabled, and mypy is using the active interpreter, in my settings.json I have:
{
"python.languageServer": "Pylance",
"python.linting.enabled": true,
"python.linting.mypyEnabled": true,
"mypy.runUsingActiveInterpreter": true
}
MyPy does complain about "library stubs" not found, for packages that are not typed, it also points to some errors, but some very explicit ones are not being shown.
An example:
# file_a.py
def myfunc() -> dict:
return {}
# file_b.py
from file_a import myfunc
from file_z import func_dont_exist
# No error shown here
abc: bool = myfunc()
xyz: str = func_dont_exist()
# This points to an error, highlight the "1"
abc = "abc" + 1
If I run mypy . on the root, also no errors. I have mypy.ini file on the root, ignoring the migrations folder, .venv and usiing SQLMyPy Plugin.
If I disable the ignore_errors mypy point to a bunch of errors on the migration folders (expected), which tells me that the daemon is working.
I have updated my VSCode, tried to disable every extension I have, reloaded, restarted, checked the console, and nothing.
Running out of ideas on how to go by debugging this issue, every single question I find goes in the lines of: enable mypy on the settings, check the path, enable linting etc, but I've already done all of that.
Any ideas?

I had trouble with mypy in the past, like loading up mypy plugin. It was resolved when I started vscode from terminal with the right python environment activated
conda activate my-env
cd my/vs-code/project/dir
code .
Worth a try

Getting invalid syntax error when using Pylint but code runs fine

When I do pylint main.py, I get the following error:
E: 7, 0: invalid syntax (<string>, line 7) (syntax-error)
# main.py
import os
repo = os.environ.get('GITHUB_REPOSITORY')
branch = os.environ.get('GITHUB_REF')
commit = os.environ.get('GITHUB_SHA')
commit_url = f'https://github.com/{repo}/commit/{commit}'
repo_url = f'https://github.com/{repo}/tree/{branch}'
print(commit_url, repo_url)
The code is running as expected but pylint is giving this strange error. I am using Python 3.6.9 on Ubuntu 18.04.

It looks like PyLint isn't happy with your f-strings (introduced in 3.6) and is validating against the syntax of an older Python version. I'd check whether the PyLint you are using is running from the same Python environment your Python you are running the program with. I would guess it's running from your system Python, while your program is running from a virtual environment.
With pylint 2.5.3 and Python 3.8.2 the only complaint PyLint makes is about the lack of a module docstring.
************* Module main
main.py:1:0: C0114: Missing module docstring (missing-module-docstring)
-----------------------------------
Your code has been rated at 8.57/10

Use .format method like below
import os
repo = os.environ.get('GITHUB_REPOSITORY')
branch = os.environ.get('GITHUB_REF')
commit = os.environ.get('GITHUB_SHA')
commit_url = 'https://github.com/{}/commit/{}'.format(repo, commit)
repo_url = 'https://github.com/{}/tree/{}'.format(repo, branch)
print(commit_url, repo_url)
Check here, Python 3 returns "invalid syntax" when trying to perform string interpolation

flake8 to ignore warning F405 on multiple lines

I am having a fabric script and i am using below statement
from fabric.api import *
Now i know this isn't as per PEP8 standards but this is really needed specially for a library like fabric and we can really import everything from it. Because of this flake8 is complaining on multiple lines with F405 code.
I have disabled this for one line using #noqa but since there multiple lines with same PEP8 violation how can i ask flake8 to ignore this particular error code.
I have also tried # noqa: F405 at the beginning of the file but that didn't work.

Starting from version 3.7.0, flake8 supports per file ignores out of the box. You can check the documentation on the command line flag / config file option here

Putting
[flake8]
ignore = E405
in your .flake8 config file will work.
Flake8 itself does not support per-file configuration, see the post:
https://gitlab.com/pycqa/flake8/issues/156
But for advanced configuration, e.g. per-file, recommended way is to use flake8-putty
The homepage gives you an example:
Disable only D102 on foo.py
putty-ignore = foo.py : D102

Get the `m` and others of `pythonX.Ym`

(See this question for what the m means)
I need to construct the include path of .virtualenvs/foo/include/pythonX.Ym to compile something (i.e. -I...) against the virtualenv. I can get the X.Y using sys.version or sys.final_version.
How do I get the m to construct the include path?
EDIT: I tried sys.executable but that is pointing to .../foo/bin/python, which is unhelpful for this.

The easiest way to get the include path is to use the sysconfig.get_path() function:
import sysconfig
include_path = sysconfig.get_path('include')
This path is adjusted for virtualenvs already. For scripting purposes outside of Python, you can either print the path directly:
$ python -c 'import sysconfig; print(sysconfig.get_path("include"))'
or get all sysconfig data by running the module as a script:
$ python -m sysconfig
then parse the output that dumps to stdout.
Other than that, if you only want the executable name (with the m included), you can get that from the sys.executable variable; this includes the m suffix:
>>> import sys
>>> sys.executable
'/usr/bin/python3.5m'
As of Python 3.2, you can also use the sys.abiflags variable; it is set to m in this case:
>>> sys.abiflags
'm'
Also see PEP 3149.
For earlier Python versions, the various flags that influence the suffixes are available still via the aforementioned sysconfig module as configuration variables:
pymalloc = bool(sysconfig.get_config_var('WITH_PYMALLOC'))
pydebug = bool(sysconfig.get_config_var('WITH_PYDEBUG'))
wideunicode = bool(sysconfig.get_config_var('WITH_WIDE_UNICODE'))
Note that ubuntu merely compiles multiple binaries and adjusts the executable name to reflect the configuration option chosen; on other systems the ABI flags are not necessarily reflected in the executable name.

Python3 -m run configuration in Eclipse

Update 2021: Solution is built into PyDev/Eclipse
See accepted answer for details
Original Question (and old answers) for below
As many comments/questions/rants on SO and other places will tell you, Python3 packages using relative imports want to be run from a central __main__.py file. In the case where a module, say "modA" within a package, say "packA", that uses relative imports needs to be run (for instance because a test package is run if __name__ == '__main__'), we are told to run instead run python3 -m modA.packA from the directory above modA if sys.path() does not contain the directory above modA. I may dislike this paradigm, but I can work around it.
When trying to run modA from Eclipse/PyDev, however, I can't figure out how to specify a run configuration that will execute the module properly with the -m flag. Has anyone figured out how to set up a run configuration that will do this properly?
References: Relative imports for the billionth time ; Relative import in Python 3 is not working ; Multilevel relative import

Nowadays (since PyDev 5.4.0 (2016-11-28)) you can go to the Settings > PyDev > Run and select Launch modules with python -m mod.name instead of python filename.py ;)
See: https://www.pydev.org/history_pydev.html
For older versions of PyDev (old answer)
Unfortunately, right now, it's not automatic running with -m in PyDev, so, I'll present 3 choices to work in PyDev with relative imports which are preceded by a dot (in PyDev version 4.3.0):
Don't use relative imports, only absolute imports in your __main__ modules.
Create a separate module for the __main__ which will do an absolute import for the module you want to run and run that module instead (if you're distributing your application, this is probably needed anyways as the usual way for people to launch your code in Python is by passing the script as an argument to Python and not using the -m switch).
Add the -m module to the vm arguments in your run configuration by doing:
Make the run (which will fail because of the relative import)
Right-click the editor > Copy Context Qualified Name
Open the run configuration: Alt, R, N (i.e.: Toolbar > Run > Run Configuration)
Open arguments tab and add the '-m Ctrl+V' (to add the -m and the module name you copied previously).
Although this is definitely not ideal: you'll now receive an argument with the filename (as PyDev will always pass that to run the file) and the whole process is a nuisance.
As a note, I do hope to provide a way to make runs within PyDev using the -m soon (hopefully for PyDev 4.4.0)... although this may not be possible if the file being run is not under the PYTHONPATH (i.e.: to run an external file it still has to support the option without the -m).

There's a bit nasty trick possible here to work around this issue. I'm using PyDev 9.2.0
Put your venv right in the workspace, say under the dir "venv".
Refresh your eclipse workspace and ensure that it uses this venv (through your interpreter setup).
After the refresh, go to the run configuration and edit the "Main Module" by clicking the Browse button.
The venv will now appear.
Browse into the venv/lib/python3.8/site-packages
There you will find the pip-installed module source codes and you can select the module you want to run.

Update 2021: This answer is no longer needed. See accepted answer for details.
Here's what I was able to do after Fabio's great suggestion.
Create a program called /usr/local/bin/runPy3M with world read/execute permissions, with the following code:
#!/usr/local/bin/python3 -u
'''
Run submodules inside packages (with relative imports) given
a base path and a path (relative or absolute) to the submodule
inside the package.
Either specify the package root with -b, or setenv ECLIPSE_PROJECT_LOC.
'''
import argparse
import os
import re
import subprocess
import sys
def baseAndFileToModule(basePath, pyFile):
'''
Takes a base path referring to the root of a package
and a (relative or absolute) path to a python submodule
and returns a string of a module name to be called with
python -m MODULE, if the current working directory is
changed to basePath.
Here the CWD is '/Users/cuthbert/git/t/server/tornadoHandlers/'.
>>> baseAndFileToModule('/Users/cuthbert/git/t/', 'bankHandler.py')
'server.tornadoHandlers.bankHandler'
'''
absPyFilePath = os.path.abspath(pyFile)
absBasePath = None
if basePath is not None:
absBasePath = os.path.abspath(basePath)
commonPrefix = os.path.commonprefix([absBasePath, absPyFilePath])
uncommonPyFile = absPyFilePath[len(commonPrefix):]
else:
commonPrefix = ""
uncommonPyFile = absPyFilePath
if commonPrefix not in sys.path:
sys.path.append(commonPrefix)
moduleize = uncommonPyFile.replace(os.path.sep, ".")
moduleize = re.sub("\.py.?$", "", moduleize)
moduleize = re.sub("^\.", "", moduleize)
return moduleize
def exitIfPyDevSetup():
'''
If PyDev is trying to see if this program is a valid
Python Interpreter, it expects to function just like Python.
This is a little module that does this if the last argument
is 'interpreterInfo.py' and then exits.
'''
if 'interpreterInfo.py' in sys.argv[-1]:
interarg = " ".join([sys.executable] + sys.argv[1:])
subprocess.call(interarg.split())
exit()
return
exitIfPyDevSetup()
parser = argparse.ArgumentParser(description='Run a python file or files as a module.')
parser.add_argument('file', metavar='pyfile', type=str, nargs=1,
help='filename to run, with .py')
parser.add_argument('-b', '--basepath', type=str, default=None, metavar='path',
help='path to directory to consider the root above the package')
parser.add_argument('-u', action='store_true', help='unbuffered binary stdout and stderr (assumed)')
args = parser.parse_args()
pyFile = args.file[0]
basePath = args.basepath
if basePath is None and 'ECLIPSE_PROJECT_LOC' in os.environ:
basePath = os.environ['ECLIPSE_PROJECT_LOC']
modName = baseAndFileToModule(basePath, pyFile)
allPath = ""
if basePath:
allPath += "cd '" + basePath + "'; "
allPath += sys.executable
allPath += " -m " + modName
subprocess.call(allPath, shell=True) # maybe possible with runpy instead...
Then add a new interpreter to PyDev -- call it what you'd like (e.g., runPy3M) and point the interpreter to /usr/local/bin/runPy3M. Click okay. Then move it up the list if need be. Then add to environment for the interpreter, Variable: ECLIPSE_PROJECT_LOC (name here DOES matter) with value ${project_loc}.
Now submodules inside packages that choose this interpreter will run as modules relative to the sub package.
I'd like to see baseAndFileToModule(basePath, pyFile) added to runpy eventually as another run option, but this will work for now.
EDIT: Unfortunately, after setting all this up, it appears that this configuration seems to prevent Eclipse/PyDev from recognizing builtins such as "None", "True", "False", "isinstnance," etc. So not a perfect solution.