We Keep Coding

Python Path difference

what is the difference between them?
/Library/Developer/CommandLineTools/usr/bin/python3
/usr/bin/python3
MacBook-Pro Desktop % whereis python3
/usr/bin/python3
MacBook-Pro Desktop % /usr/bin/python3 --version
Python 3.8.9
MacBook-Pro Desktop % /Library/Developer/CommandLineTools/usr/bin/python3 --version
Python 3.8.9
>python3
Python 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from my_custom_module.__main__ import main
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'my_custom_module'
But it works with other
>/Library/Developer/CommandLineTools/usr/bin/python3
>>>from my_custom_module.__main__ import main
it works
There is weird thing happens I mean, when I type full path I get 3.8.8 but when I type short name "python3" I get 3.7.9.
though
whereis returns as follow
MacBook-Pro Desktop % whereis python3
/usr/bin/python3
/usr/bin/python3
Python 3.8.9 (default, Aug 3 2021, 19:21:54)
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
But
MacBook-Pro Desktop % python3
Python 3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08)
[Clang 6.0 (clang-600.0.57)] on darwin

I think your confusion lies in the whereis command. whereis python3 doesn't actually tell you what will be run when you use python3, it just searches the standard binary directories for the specified program (python3). I would recommend using which instead, as it searches the actual PATH that you use.
See man page for whereis, man page for which, comparison.
For example on my personal computer:
❯ whereis python3
/usr/bin/python3
❯ which python3
/usr/local/bin/python3
❯ /usr/bin/python3 --version
Python 3.8.9
❯ /usr/local/bin/python3 --version
Python 3.9.7
❯ python3 --version
Python 3.9.7

python3.5 sees but python3.9 does not see sqlite3

My machine:
Linux eyes 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u3 (2019-06-16) x86_64 GNU/Linux
i.e. python3.5 is its native python3
Have two python3 in the system:
root#machine: # update-alternatives --list python3
/usr/bin/python3.5
/usr/local/bin/python3.9
root#machine: # update-alternatives --config python3
There are 2 choices for the alternative python3 (providing
/usr/bin/python3).
Selection Path Priority Status
---------------------------------------------------------------
0 /usr/local/bin/python3.9 2 auto mode
1 /usr/bin/python3.5 1 manual mode
* 2 /usr/local/bin/python3.9 2 manual mode
+++++++++++ LET'S CHOOSE 1 FIRST +++++++++++
root#machine: # python3
Python 3.5.3 (default, Nov 18 2020, 21:09:16)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3 <------------ OK
>>> exit()
**+++++++++++ LET'S CHOOSE 2 NOW +++++++++++++
root#machine: # python3
Python 3.9.0 (default, Nov 23 2020, 09:50:01)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.9/sqlite3/__init__.py", line 23, in <module>
from sqlite3.dbapi2 import *
File "/usr/local/lib/python3.9/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3' <------- WHAT?
>>> exit()
++++++++++++++++++++++++++++++++++++++++++++
root#machine: # head ~/python3-9_distr/Python-3.9.0/config.log
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by python configure 3.9, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ ./configure --enable-optimizations --enable-loadable-sqlite-extensions
++++++++++++++++++++++++++++++++++++++++++++
/usr/lib/python3.5/lib-dynload# ls -la _sqlite3.cpython-35m-x86_64-linux-gnu.so
-rw-r--r-- 1 root root 96552 Nov 18 23:09 _sqlite3.cpython-35m-x86_64-linux-gnu.so
BUT /usr/local/lib/python3.9/lib-dynload DID NOT HAVE _sqlite3.*
I copied _sqlite3.cpython-35m-x86_64-linux-gnu.so and renamed to
-rwxr-xr-x 1 root root 96552 Nov 18 23:09 _sqlite3.cpython-3**9**m-x86_64-linux-gnu.so
Nothing had changed.
++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++
How come ? How can I make sqlite3 work in my python3.9 ?
UPDATE:
I have found a "solution". As I mentioned before, I copied
/var/lib/python3.5/lib-dynload/
_sqlite3.cpython-35m-x86_64-linux-gnu.so
and renamed to
/var/local/lib/python3.9/lib-dynload/
_sqlite3.cpython-39m-x86_64-linux-gnu.so
but it did not work.
Then I copied
_sqlite3.cpython-39m-x86_64-linux-gnu.so
to
_sqlite3.so in the same folder (/var/local/lib/python3.9/lib-dynload/)
It seemed to be working but later I got new problem.

Hi after you have installed python3.9 have you checked the PATH variable which python uses?
please go to a terminal and type
~$ python3.6
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
You should get something that looks like this
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/jishnu/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']
Note i'm using python3.8 hence output will be different
Do the same thing for your python3.9 and check if the package you are trying to access is installed in any of these paths.
If you want further help on how to add to path, refer this question
PYTHONPATH on Linux

Python: ImportError: cannot import module for user excepr for root

I installed a modue like this:
# pip3 install mega
And everything works fine. If I try to import from this module as root user, works OK:
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from mega import Mega
>>> mega = Mega()
>>> quit()
But if I try to do the same thing with a regular user, it doesn't work:
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from mega import Mega
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'Mega' from 'mega' (unknown location)
>>>
This happens with all non root users. Checking the PYTHONPATH, seems to be the same for root and for unprivileged users:
>>> import sys
>>> for p in sys.path:
... print(p)
...
/usr/lib/python37.zip
/usr/lib/python3.7
/usr/lib/python3.7/lib-dynload
/usr/local/lib/python3.7/dist-packages
/usr/lib/python3/dist-packages
>>>
I'm running Raspbian 10 on a Raspberry Pi 3b+
Any idea? Thank you in advance.

It was a matter of permissions. I don't know why, pip3 installs the module with no permission but for root user. A simple bash script corrects the issue:
#!/bin/sh
find /usr/local/lib/python3.7/dist-packages/ -maxdepth 1 -type d | while read line; do
sudo chmod -R a+rx $line
done
Now it works like a charm.

dotenv import in python3.7 (MacOS)

dotenv installed, to be sure I check this by calling pip3 and pip3.7, despite that it's same
(env) slonocomp:-wrapper oleg$ pip3.7 freeze | grep dotenv
django-dotenv==1.4.2
python-dotenv==0.11.0
(env) slonocomp:wrapper oleg$ pip3 freeze | grep dotenv
django-dotenv==1.4.2
python-dotenv==0.11.0
(env) slonocomp:wrapper oleg$
version of p3 and p3.7 actualy the same
(env) slonocomp:wrapper oleg$ python3 -V
Python 3.7.4
(env) slonocomp:wrapper oleg$ python3.7 -V
Python 3.7.4
But when I try to import dotenv in interactive mode (for demonstration) I got differences
(env) slonocomp:wrapper oleg$ python3
Python 3.7.4 (default, Oct 12 2019, 18:55:45)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dotenv
>>>
(env) slonocomp:wrapper oleg$ python3.7
Python 3.7.4 (default, Oct 12 2019, 18:55:45)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dotenv
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'dotenv'
>>>
(env) slonocomp:wrapper oleg$
The question is Why? And how to avoid this error.

How to edit python path

I use both python3.5.3 (default) and python 3.6.5 that I installed on raspberry pi.
The path for each version is as below.
I'd like to add the path, /usr/lib/python3/dist-packages, under python3.6 and remove it from python3.5. How could I do that?
user#raspberrypi:~ $ python3.6
Python 3.6.5 (default, Apr 5 2018, 18:01:08)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import pprint
>>> pprint.pprint(sys.path)
['',
'/usr/local/lib/python36.zip',
'/usr/local/lib/python3.6',
'/usr/local/lib/python3.6/lib-dynload',
'/usr/local/lib/python3.6/site-packages']
user#raspberrypi:~ $ python3.5
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170124] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import pprint
>>> pprint.pprint(sys.path)
['',
'/usr/lib/python35.zip',
'/usr/lib/python3.5',
'/usr/lib/python3.5/plat-arm-linux-gnueabihf',
'/usr/lib/python3.5/lib-dynload',
'/usr/local/lib/python3.5/dist-packages',
'/usr/lib/python3/dist-packages']

As each version of Python seems to maintain its own environment (and path) how's about adding this to the beginning of your program:
import sys
myPath='/usr/lib/python3/dist-packages' # or whatever else you want it to be
if '3.6' in sys.version and not myPath in sys.path:
sys.path.append(myPath)
elif '3.5' in sys.version and myPath in sys.path:
sys.path.remove(myPath)