We Keep Coding

Import on shared object fails with ModuleNotFound only when script is executed from command line

ran into a weird problem where there is a shared-object import error only when I run the script from command line. It succeed if i run the script in the python console using exec(...)
I have a class that needs a shared object: foo.py:
import os
cur_dir = os.curdir()
os.chdir('/tmp/dir_with_shared_object/')
import shared_object_class
os.chdir(cur_dir)
class Useful:
... # use the shared object import
Then there is a script like this:
from foo import Useful
If I enter python console and run:
exec(open('script.py').read())
Everything works fine.
If I run this on command line:
python script.py
I will get
ModuleNotFoundError: No module named 'shared_object_class'
The python is the same. It is 3.7.3, GCC 7.3.0 Anaconda. Anyone knows what is causing this discrepancy in behavior for shared object import?

A standard way of importing from a custom directory would be to include it in the PYTHONPATH environmental variable, with export PYTHONPATH=/tmp/dir_with_shared_object/.
update
It could also be done dynamically with
import sys
p = '/tmp/dir_with_shared_object/'
sys.path.insert(0, p)
PS
I think I have an explanation for why OP's original code didn't work. According to this python reference page, the import system searches, inter alia, in "[t]he directory containing the input script (or the current directory when no file is specified)." So the behavior in the REPL loop is different from how it is when running a script. Apparently the current directory is evaluated each time an import statement is encountered, while the directory containing the input script doesn't change.

Python Import cannot find shared object file (C++ extension)

I'm creating a C++ extension module for Python3. Compilation of the setup.py file compiles just fine, but when I go to import my new module, I get
ImportError: libMyLib.so: cannot open shared object file: No such file or directory
this is the path to my .so:
/path/to/lib-cc7/libMyLib.so
I've tried to import my libraries in the setup.py in different ways, I have tried setting and re-setting my LD_LIBRARY_PATH variable in the terminal as well as in my .bash_profile. I have tried setting the paths in sys.path.
When I run this code before the import statement:
print(os.environ.get("LD_LIBRARY_PATH"))
print(os.environ.get("PYTHONPATH"))
I get the path to the correct library directory.
When I run strace the path to other .so's that I need show up, and I see it searching for libMyLib.so, but it just searches in what seems like all of the other directories and /path/to/lib-cc7/. In other library searches it checks /path/to/lib-cc7/.
I have sanity checked that the library is there about 5 times.
It seems like no matter what I do,
import MyModule.MySubModule as SubModule
always returns the same import error. Is there anything else that I haven't tried? Why does it seem like Python is looking in the wrong place for my .so?
EDIT 1:
This is what my setup.py (in essence) looks like:
Submodule1 = Extension('Module.Submodule1', sources=['Module/Submodule1/submod1.cpp'], language = C++, libraries=[..long list of libraries..],)
Submodule2 = Extension('Module.Submodule2', sources=['Module/Submodule2/submod2.cpp'], language = C++, libraries=[..long list of libraries..],)
setup(name = "Module", version = '1.0',
packages = ['Module', 'Module.Submodule1', 'Module.Submodule2'],
ext_modules = [Submodule1, Submodule2], )

Duplicate a file with Python

I'm trying to figure out how to duplicate a single file for example: I have a file in "C:\ ..." and would like to duplicate this exact same file once. If also possible is there a way to use python to open specific documents?
thanks

I think you might be looking for copy2. This will copy the file contents and as much of the file metadata (permissions, ownership, etc) as it can on the platform. copystat has more notes on what can and can't be copied and how to find out on your platform.

just using system() from os module
os.system("cp resource_file target_file")

Using a module named shutil, the function copy2 can be called with the path of the source file and the corresponding destination directory you want to write to. For example,
import shutil
shutil.copy2('/src/test.txt','/dst/test_copy.txt')

you can copy a file via the command line on windows: open cmd.exe then type cd "C:\ ..." then type copy yourfile destination. more info here
you can make python do this for you: you will need the subprocess module which comes integrated into python so you dont have to donwload anything.
like this:
import subprocess
subprocess.run('copy yourfile destionation')
remember that for this to work your python script should be in the same folder as "yourfile", why? because i type "yourfile" as a relative path relative vs absolute path
subprocess works for python 3.3 and newer versions so another way to do it would be:
import os
os.system('copy youfile destionation')
to open specific documents with python take a quick look at here: https://www.pythonforbeginners.com/files/reading-and-writing-files-in-python

Why do I get a "no such file or directory" error when the file is known to exist?

I have uploaded a package to pypi and github. I have then installed the package and tried to use it. It contains a python script which need to read from a file. I have placed both in the same directory.
pip install pycricket
from pycricket import cricket
c = cricket.Cricket()
c.query()
Query() function involves reading from a file. When I see the 'pycricket' package in library, both script as well as file are in same folder.
query():
with open('matches.csv', 'r') as f:
#code
I don't know why I get the error.

You can inspect the current working directory with:
>>> import os
>>> os.getcwd()
If your data is in a different directory (unclear from the question, but likely given the error message), then change to the directory where the data is stores:
>>> os.chdir(path_to_data_directory)

How can I add autocomplete to Vim for non-standard Python libraries?

For example, I have a Python script using the Google App Engine SDK:
from google.appengine.ext import db
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from google.appengine.ext.webapp.util import run_wsgi_app
The module db has a submodule Key, so I try to use autocomplete on it:
db.KTab
But at the bottom of the Vim window, I get the following:
-- Omni completion (^O^N^P) Pattern not found
How do I include the path to non-standard Python libraries so that Vim autocompletion can find them? And also display their docstrings?

You need to add your library files to your tags file. For instance, if you have installed the Google App Engine via pip in a virtual environment located in env/:
virtualenv --no-site-package env/
source env/bin/activate
pip install google_appengine
... then you should execute:
ctags -R --python-kinds=-i -o tags env/
If you did not install google_appengine through pip, then you should locate the path to your python libraries (hint: it should be indicated by $PYTHONPATH. And according to this reference page: "on Unix, this is usually .:/usr/local/lib/python.") and replace env/ by the path you found.
Finally, your .vimrc file should parse your tags file. For instance, in my .vimrc, I have:
set tags+=/path/to/my/tags

I grabbed this from natw's vimrc (I think...maybe sontek), but it should do the trick, so long as your packages are findable by your current install of Python. This lets you use gf, but also sets up searching these files for autocompletion. Note the py <<EOF part, which starts a section interpreted in Python. This means you'd have to have the python interpreter installed in vim to use it.
function! LoadPythonPath()
py <<EOF
# load PYTHONPATH into vim, this lets you hover over a module name
# and type 'gf' (for goto file) and open that file in vim. Useful
# and easier than rope for simple tasks
import os.path
import sys
import vim
for p in sys.path:
if os.path.isdir(p):
vim.command(r"set path+=%s" % (p.replace(" ", r"\ ")))
EOF
endfunction
Btw, I don't like to have this load automatically, so I set it to a function that intelligently loads/unloads when I call it/first enter a Python doc. And I add a let g:PythonPathLoaded=1 to the previous function.
function! GetPythonPath()
if !exists("g:PythonPathLoaded")
call LoadPythonPath()
return
elseif g:PythonPathLoaded
return
else
call LoadPythonPath()
endif
endfunction
And I have an unload function too...though I'm not sure whether this makes a huge difference.
function! UnloadPythonPath()
py <<EOF
# load PYTHONPATH into vim, this lets you hover over a module name
# and type 'gf' (for goto file) and open that file in vim. Useful
# and easier than rope for simple tasks
for p in sys.path:
if os.path.isdir(p):
vim.command(r"set path-=%s" % (p.replace(" ", r"\ ")))
EOF
let g:PythonPathLoaded = 0
endfunction
Hope this helps! Plus, an added bonus is that this will load your packages regardless of whether you are using virtualenv (since it, I believe, runs whatever is set as 'python' at the moment).