We Keep Coding

YouCompleteMe/Python can complete for built-in libs, but not site-packages

I just installed ycm, everything looks good, but I found small problem. The problem is as following:
import os # os is built-in library
os. # ycm helps to complete members of the class.
import numpy # numpy is not built-in library, where its location is site-packages.
numpy. # nothing happened. ycm shows 'pattern not found' message.
I think, this would be a simple problem. But I could not find the solution yet. I think, there is some configuration file in which I can define 'search path' for my project.
It would be grateful if I can find a way to solve it.
Best,
Je-Hoon Song

I had the same issue with module 'mpmath' and fixed it in the following manner:
First I retrieved the path where the module was located:
%python3
>>>import mpmath
>>>print(mpmath.__file__)
/usr/lib/python3.4/site-packages/mpmath/__init__.py
Here I found the path of all my "installed" python3 packages to be:
/usr/lib/python3.4/site-packages/
I then simply added to my PYTHONPATH environment variable this path:
%export PYTHONPATH=/usr/lib/python3.4/site-packages/
Then when I used vim sample.py typing import mpmath and following it up with mpmath. YCM showed me all the autocompletions for the mpmath module.
Hope this helps.

I use anaconda python to be my python interpreter in ycm to solve this.
First I modified my vimrc according to full pythong setting in vim.
Then I change g:ycm_python_interpreter_path by
let g:ycm_python_interpreter_path = '/usr/local/anaconda3/bin/python3.8'
In this way I didn't change the system environment variables.
Addtional Info 1:
I think the main problem is that,
my Python interpreter for YCM is my system python (/usr/local/opt/python#3.9/bin/python3.9 ),
which only has limited locally built libraries.
So using the libraries comes with anaconda (/usr/local/anaconda3/bin/python3.8 ) can solve.
Additional Info 2:
By reading :YcmDebugInfo, the main different after edited g:ycm_python_interpreter_path is that:
-- Python completer debug information:
-- Python interpreter: /usr/local/opt/python#3.9/bin/python3.9
-- Python path: ['/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python39.zip', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', '/usr/local/lib/python3.9/site-packages']
-- Python version: 3.9.6
change to
-- Python completer debug information:
-- Python interpreter: /usr/local/anaconda3/bin/python3.8
-- Python path: ['/usr/local/anaconda3/lib/python38.zip', '/usr/local/anaconda3/lib/python3.8', '/usr/local/anaconda3/lib/python3.8/lib-dynload', '/usr/local/anaconda3/lib/python3.8/site-packages', '/usr/local/anaconda3/lib/python3.8/site-packages/aeosa', '/usr/local/anaconda3/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg']
-- Python version: 3.8.8
Additional Info 3: about how to read list of locally installed pyton modules
use https://stackoverflow.com/a/740018/11226687
e.g. in my case
$ /usr/local/opt/python#3.9/bin/python3.9
>>> help('modules')
# only return limitted modules
$ /usr/local/anaconda3/bin/python3
>>> help('modules')
# list out all the modules included in Anaconda, including numpy/matplotlib/scipy ect

numpy is kind of a difficult library because it dynamically builds its namespace on import, making it hard for static code analysis tools to know when you're write the code what names should be available. Because the names available in the namespace numpy are only really known at runtime, YCM probably doesn't have any useful suggestions for you.

One simple way to fix is activate your python environment, then open vim. For example
(django_mdn) ➜ locallibrary git:(master) ✗ vim
and in the vim run :echo $PATH.
Then you should be able to see that your venv path is at the first like this:
/Users/gwanghyeongim/.virtualenvs/django_mdn/bin:/usr/local/opt/tcl-tk/bin:...
Then see if your python packages are auto-complete.
It worked.
If you want to set a certain site-packages to be auto complete permanently, you need to make a file called .ycm_ extra_conf.py in your project root directory or global_extra_conf.py and set vim configuration if you want to set it globally.
P.S.
By running export PYTHONPATH=/usr/lib/python3.4/site-packages/ in the shell before opening vim didn't work for me. Besides, unless setting PYTHONPATH permanently, which will cause issue, you will have to set export PYTHONPATH everytime you want dependencies to be auto complete.

How to import module from current non-default directory

I'm using Python 2.7. I'm rather new to the python langauge. I have two python modules - "Trailcrest.py" and "Glyph.py", both in the same folder, but not in the Python27 folder.
I need to import "Trailcrest.py" into "Glyph.py", but I am getting the message that "no such module exists".
Additionally, whatever means I use to import the module needs to not be dependent on a solid-state path. This program is cross-platform, and the path can be changed depending on the user's preferences. However, these two modules will always be in the same folder together.
How do I do this?

If you have Trailcrest.py and Glyph.py in the same folder, importing one into the other is as simple as:
import Trailcrest
import Glyph
If this does not work, there seems to be something wrong with your Python setup. You might want to check what's in sys.path.
import sys
print sys.path

To elaborate a bit on Ferdinand Beyer's answer, sys.path is a list of file locations that the default module importer checks. Some, though not all installations of python will add the current directory or the directory of the __main__ module to the path. To make sure that the paths relative to a given module are importable in that module, do something like this:
import os.path, sys
sys.path.append(os.path.dirname(__file__))
But something like that shouldn't ever make it into a "production" product. Instead, use something like distutils to install the module's package into the python site-packages directory.

This can also be achieved using the environment variable PYTHONPATH which also influences Python's search path. This can be done in a shell script so that the Python files do not need to be altered. If you want it to import from the current working directory use the . notation in bash:
export PYTHONPATH=.
python python_prog.py

adding a subpackage from a different path

I have a python package called zypp. It is generated via swig and the rpm package (called python-zypp) puts it in:
rpm -ql python-zypp
/usr/lib64/python2.6/site-packages/_zypp.so
/usr/lib64/python2.6/site-packages/zypp.py
Now, I have a different project which provides an additional sets of APIs. Pure python. Plus some scripts.
The layout is:
bin/script1
python
python/zypp
python/zypp/plugins.py
python/zypp/__init__.py
plugins.py contains a Plugin class. I intended to put this in an rpm, and put it into
/usr/lib64/python2.6/site-packages/zypp/plugins.py
script1 uses this Plugin class. But as I test it running from git, I would like it to find the module from git too if it is not installed. So it has something like:
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), '../python'))
from zypp.plugins import Plugin
However, it seems that if python-zypp is installed on /usr/lib64/python2.6/site-packages/zypp.py, then script1 won't find the plugins submodule anymore. If I uninstall python-zypp, it does.
So my question is if it is possible to extend a module by adding submodules, being the submodules being located in a different load path. Or will they always clash?
An analogy would be, I have a module foo. And I provide foo.extras in a different load path (which may use foo indeed). The script won't find foo.extras if foo is found first in the system load path. If I only use the custom load path, the script may not find foo module if foo.extras uses it.
I have more experience with ruby, but in ruby I could have installed:
/usr/lib64/ruby/gems/1.8/gems/foo-1.0/lib/foo/*
And I could have in my script:
bin/script
lib/foo/extras/*
I could do in script:
$: << File.join(File.dirname(__FILE__), "../lib"
and then my script could
require foo
require foo/extras
No mater if foo/extras is installed on the system or in the custom load path. They don't clash.
The other way around, I found out that with PYTHONPATH the local zypp.plugins is found first. But then the installed zypp module is not found:
import zypp # works, but seems to import the local one
from zypp.plugins import Plugin # works, PYTHONPATH finds it first
repoinfo = zypp.RepoInfo() # does not work

If I understand your question correctly, you want to use the development version of that module instead of the installed module. Therefore, you can use
PYTHONPATH
From the Module Search Path documentation:
When a module named spam is imported, the interpreter searches for a file named spam.py in the current directory, and then in the list of directories specified by the environment variable PYTHONPATH. This has the same syntax as the shell variable PATH, that is, a list of directory names. When PYTHONPATH is not set, or when the file is not found there, the search continues in an installation-dependent default path; on Unix, this is usually .:/usr/local/lib/python.
So, if the GIT tree of the module directory was "/home/username/some/path", you would change the PYTHONPATH to "/home/username/some/path". Or if the PYTHONPATH variable is already in use, you would append ":/home/username/some/path" to it (note the colon separator). In order to make this permanent, add the line "PYTHONPATH=value" to the file "/etc/environment".
sys.path.insert
If you have a start script for your program, you could override the module search path using sys.path.insert(0, "somepath"). This is similar to the sys.path.append call you described but inserts the path into the beginning of the list.

Utilizing multiple python projects

I have a python app, that I'm developing. There is a need to use another library, that resides in different directory.
The file layout looks like this:
dir X has two project dirs:
current-project
xLibrary
I'd like to use xLibrary in currentProject. I've been trying writting code as if all the sources resided in the same directory and calling my projects main script with:
PYTHONPATH=.:../xLibrary ./current-project.py
but this does not work. I'd like to use its code base without installing the library globaly or copying it to my project's directory. Is it possible? Or if not, how should I deal with this problem.

It's generally a good programming practice to isolate packages into actual packages and treat them as such. If you're sure you'd like to continue with that approach though you can modify the search path from within python via:
import sys
sys.path.append( "<path_containing_the_other_python_files>" )
To avoid embedding absolute paths, you can use os.path.abspath(__file__) to obtain the absolute path to the currently executing .py file and follow up with a few os.path.dirname() calls to construct the proper relative path for inclusion to sys.path
A slightly altered approach that would allow you to get the best of both worlds would be to add an __init__.py file to xLibrary's directory then add the path containing 'xLibrary' to sys.path instead. Subsequent Python code could then import everything "properly" via from xLibrary import my_module rather than just import my_module which could be confusing to people accustomed to the standard package directory layout.

This depends how you use xLibrary from current-project.
If you do something like from xLibrary import module1 inside current-project, the xLibrary needs to be laid out as a Python package:
xLibrary/
xLibrary/__init__.py
xLibrary/module1.py # or whatever other modules the package consists of
In this case, you should include xLibrary's parent directory in PYTHONPATH:
PYTHONPATH=.:.. ./current-project.py
However, if xLibrary is just a collection of Python modules that you import individually (that is, if you do import module1 and import module2 ìn current-project, with xLibrary containing the files module1.py and module2.py), you should include xLibrary in PYTHONPATH just as you did:
PYTHONPATH=.:../xLibrary ./current-project.py

bash$ ln -s ../xLibrary xLibrary

First, it is probably better to use absolute paths in your PYTHONPATH variable.
Second, I don't think you need to add current directory to the path.
Other than that, it would be good to know what it is that doesn't work and what the error message is.
The command line you have there seems to be missing a semicolon
Try these two:
a=12 echo $a
b=12 ;echo $b
...and you'll see the difference.

Apart from the other suggestions, you may consider the virtualenv package. After writing a little setup.py file you can "virtually install" the library and avoid all the PYTHONPATH munging in general.
This is a really good practice and is encouraged by the python community.
Otherwise I prefer the use of sys.path method (by Rakis)

How do I tell a Python script (cygwin) to work in current (or relative) directories?

I have lots of directories with text files written using (g)vim, and I have written a handful of utilities that I find useful in Python. I start off the utilities with a pound-bang-/usr/bin/env python line in order to use the Python that is installed under cygwin. I would like to type commands like this:
%cd ~/SomeBook
%which pythonUtil
/usr/local/bin/pythonUtil
%pythonUtil ./infile.txt ./outfile.txt
(or % pythonUtil someRelPath/infile.txt somePossiblyDifferentRelPath/outfile.txt)
pythonUtil: Found infile.txt; Writing outfile.txt; Done (or some such, if anything)
However, my pythonUtil programs keep telling me that they can't find infile.txt. If I copy the utility into the current working directory, all is well, but then I have copies of my utilities littering the landscape. What should I be doing?
Yet Another Edit: To summarize --- what I wanted was os.path.abspath('filename'). That returns the absolute pathname as a string, and then all ambiguity has been removed.
BUT: IF the Python being used is the one installed under cygwin, THEN the absolute pathname will be a CYGWIN-relative pathname, like /home/someUser/someDir/someFile.txt. HOWEVER, IF the Python has been installed under Windows (and is here being called from a cygwin terminal commandline), THEN the absolute pathname will be the complete Windows path, from 'drive' on down, like D:\cygwin\home\someUser\someDir\someFile.txt.
Moral: Don't expect the cygwin Python to generate a Windows-complete absolute pathname for a file not rooted at /; it's beyond its event horizon. However, you can reach out to any file on a WinXP system with the cygwin-python if you specify the file's path using the "/cygdrive/driveLetter" leadin convention.
Remark: Don't use '\'s for separators in the WinXP path on the cygwin commandline; use '/'s and trust the snake. No idea why, but some separators may be dropped and the path may be modified to include extra levels, such as "Documents and Settings\someUser" and other Windows nonsense.
Thanks to the responders for shoving me in the right direction.

Look at os.getcwd:
http://docs.python.org/library/os.html#os-file-dir
Edit: For relative paths, please take a look at the os.path module:
http://docs.python.org/library/os.path.html
in particular, os.path.join and os.path.normpath. For instance:
import os
print os.path.normpath(os.path.join(os.getcwd(), '../AnotherBook/Chap2.txt'))

What happens when you type "ls"? Do you see "infile.txt" listed there?

os.chdir(my_dir)
or
os.chdir(os.getcwd())