I installed emacs-jedi to get some code completion for python in emacs. In general, I must say that I am very impressed! It works well out of the box and finds completions quickly for built -in libraries. However, I use python for scientific purposes and rely on numpy and scipy for my work. For some reason, I get no completions for these modules.
Example:
import numpy
testVector = numpy.array([1,2,3])
now typing testVector. and waiting, nothing shows up
I wonder why it does not work. It looks like sys.path problem but it should work without any configuration. But here is some idea for a brute force fix.
(1) Run the following script to get load path for numpy.
import os
import numpy
print(os.path.dirname(os.path.dirname(numpy.__file__)))
(2) Set jedi:server-args like this to forcefully add the path.
(setq jedi:server-args
'("--sys-path" "THE/PRINTED/PATH/FOR/NUMPY"
"--sys-path" "THE/PRINTED/PATH/FOR/SCIPY"))
See also: http://tkf.github.com/emacs-jedi/#jedi:server-args
Edit 1
Reading your comment on #syohex's answer, it looks like you mixed up some installation methods. jeid.el uses the virtualenv "env/" in the directory in which you have jedi.el, if it exists. el-get automatically creates "env/" if you have virtualenv. So, if you like system installation, you need to tell Jedi.el to ignore "evn/" by doing this:
(require 'jedi)
(setq jedi:server-command (list "python" jedi:server-script))
See also: http://tkf.github.com/emacs-jedi/#jedi:server-command
Edit 2
I have no clue why is that happening from your description. Here are several ways to narrow down the problem.
Run make tryout in the directory in which jedi.el is installed (like ~/.emacs.d/el-get/jedi/).
This opens a clean (i.e., it does not read your setup) Emacs process with minimal setup for jedi.el. Let's see if you can complete numpy and scipy.
Can you import numpy and scipy in Emacs? You could have different environment variable in Emacs and shell. Run M-! python -c 'import numpy' RET. If this does not give you an error, then it is fine.
Can you import numpy and scipy using env/bin/python? The best way to do it is to check it from Emacs.
So first go to the directory in which jedi.el is installed (e.g., C-x C-f ~/.emacs.d/el-get/jedi/ RET).
Then run M-! env/bin/python -c 'import numpy' RET. If this does not give you an error, then it should be possible to import numpy and scipy.
I hope at least one of them gives you an error, otherwise I need to think about another possibility.
I can get completion such case. Like following
You may use old requirement modules(jedi, epc, argparse).
You should update them and try again.
Related
I am new to Julia. I developed a few lines of code to get the results I needed from packages I was not able to find in Python or R. Now, I am trying to get this file to be easily accessible, and wrap the code in Python or R. Has anyone done this before? I have tried a few methods and have not found anything that has helped.
The most simple way to do this would be just a few lines of code that calls the .jl file, runs it (which the code is then added to a .txt file from julia), and then alerts you when the code is done.
Any help would be greatly appreciated. R is the preferable method and at this point Python would be appreciated as well.
Please find below instructions for Python, R and just an external process (which of course is an executable command that can be run from any other process). I recommend putting your code in a package and loading it in one of those languages rather than executing this as an external process.
Python
Use Python Anaconda (not in-built system Python) and install Julia
Run Julia and install PyCall
using Pkg
ENV["PYTHON"]="/path/to/your/python/executable"
pkg"add PyCall"
pkg"build PyCall"
Put your code into a Julia package
using Pkg
Pkg.generate("MyPackage")
In the folder src you will find MyPackage.jl, edit it to look like this:
module MyPackage
function main(x,y)
#do very complex staff or place it in your_other_file.jl
2x.+y
end
include("your_other_file.jl")
export main, and_whatever_other_functio_you_defined
end
Install pyjulia
python -m pip install julia
(On Linux systems you might want to use python3 instead of python command)
For this step note that while an external Python can be used with Julia. However, for a convenience it might be worth
to consider using a Python that got installed together with Julia as PyCall.
In that case for installation use a command such this one:
%HOMEPATH%\.julia\conda\3\python -m pip install julia
or on Linux
~/.julia/conda/3/python -m pip install julia
Note that if you have JULIA_DEPOT_PATH variable defined you can replace %HOMEPATH%\.julia or ~/.julia/ with its value.
Run the appropiate Python and tell it to configure the Python-Julia integration:
import julia
julia.install()
Now you are ready to call your Julia code:
>>> from julia import Pkg
>>> Pkg.activate(".\\MyPackage") #use the correct path
Activating environment at `MyPackage\Project.toml`
>>> from julia import MyPackage
>>> MyPackage.main([1,2],5)
[7,9]
Gnu R
Configure your system PATH variable to point to your Julia location. Hence when you type julia in the console it should start Julia
Run the script below to install R-Julia integration
install.packages("JuliaCall")
library(JuliaCall)
julia <- julia_setup()
Follow the above instructions for Python (step 3 only) and create the package named MyPackage
Run the code
library(JuliaCall)
julia_eval("using Pkg;Pkg.activate(\"C:/temp/rrr/MyPackage\")")
julia_library("MyPackage")
julia_eval("MyPackage.main(3,5)")
Bash (or just any language)
Build the package following instructions for Python (step 3 only)
Configure the system PATH variable
Being in the package directory run the command (note string(:.) is a Julian trick that I use to avoid apostrophe escaping in bash commands):
julia -e "using Pkg;Pkg.activate(string(:.));Pkg.instantiate();using MyPackage;MyPackage.main(3,4)"
This will install all dependencies for your package. In order to skip the installation remove Pkg.instantiate() from the above command.
The answer from #przemyslaw-szufel is correct but maybe a bit overcomplicated. You don't necessarily need to wrap your code in a module or define a custom environment (yes it is good practice, but a step at the time...)
First create a file juliaScripts.jl with content:
function getAnElement(array,n)
return array[n]
end
To run functions defined in a .jl file in R
Then in R you just do:
> install.packages("JuliaCall")
> library(JuliaCall)
> julia_setup() # on every new R session !
> julia_source("juliaScript.jl")
> out <- julia_call("getAnElement",c(10,20,30),2)
> out
[1] 20
Note that the R vector has been automatically converted to a Julia Array.
To run functions defined in a .jl file in Python
In Python, it is even easier:
$ python3 -m pip install --user julia
>>> import julia
>>> julia.install() # only once, not every session
>>> jl=julia.Julia(compiled_modules=False)
>>> from julia import Main
>>> Main.include("juliaScript.jl")
>>> Main.getAnElement([1,2,3],2)
20
Also in Python arrays (native python lists as well Numpy arrays and other commonly used data structures) are automatically converted between Python and Julia.
Not to make advertising, but more details on interfacing R <-> Julia or Python <-> Julia are on my Apress(2019) book "Julia Quick Syntax reference" in Chapter 7 "Interfacing Julia with other languages" (I shouldn't say it, but you can easily find the pdf online in well-known sites...)
Using the JuliaConnectoR package:
library(JuliaConnectoR)
fun <- juliaCall("include", "/path/to/file.jl") # you may need to provide the full path
For more info on JuliaConnectoR, see the link above as well as this paper, which additionally compares it to alternative packages suck as JuliaCall and XRJulia.
all.
I recently started working with Jenkins, in an attempt to replace cronjob with Jenkins pipeline. I have really a bit knowledge of programming jargon. I learned what I learned from questions on stackoverflow. So, if you guys need any more info, I would really appreciate if you use plain English.
So, I installed the lastest version of Jenkins and suggested plugins plus all the plugins that I could find useful to python running.
Afterwards, I searched stackoverflow and other websites to make this work, but all I could do was
#!/usr/bin/env python
from __future__ import print_function
print('Hello World')
And it succeeded.
Currently, Jenkins is running on Ubuntu 16.04, and I am using anaconda3's python (~/anaconda3/bin/python).
When I tried to run a bit more complicated python code (by that I mean import pandas), it gives me import error.
What I have tried so far is
execute python script build: import pandas - import error
execute shell build: import pandas (import pandas added to the code that worked above)
python builder build: import pandas - invalid interpreter error
pipeline job: sh python /path_to_python_file/*.py - import error
All gave errors. Since 'hello world' works, I believe that using anaconda3's python is not an issue. Also, it imported print_function just fine, so I want to know what I should do from here. Change workspace setting? workdirectory setting? code changes?
Thanks.
Since 'hello world' works, I believe that using anaconda3's python is not an issue.
Your assumption is wrong.
There are multiple ways of solving the issue but they all come down to using the correct python interpreter with installed pandas. Usually in ubuntu you'll have at least two interpreters. One for python 2 and one for python 3 and you'll use them in shell by calling either python pth/to/myScript.py or python3 pth/to/myScript.py. python and python3 are in this case just a sort of labels which point to the correct executables, using environmental variable PATH.
By installing anaconda3 you are adding one more interpreter with pandas and plenty of other preinstalled packages. If you want to use it, you need to tell somehow your shell or Jenkins about it. If import pandas gives you an error then you're probably using a different interpreter or a different python environment (but this is out of scope here).
Coming back to your script
Following this stack overflow answer, you'll see that all the line #!/usr/bin/env python does, is to make sure that you're using the first python interpreter on your Ubuntu's environment path. Which almost for sure isn't the one you installed with anaconda3. Most likely it will be the default python 2 distributed with ubuntu. If you want to make sure which interpreter exactly is running your script, instead of 'Hello World' put inside:
#!/usr/bin/env python
import sys
print(sys.executable) # this line will give you the exact path to the interpreter
print(sys.version) # this one will give you the version
Ok, so what to do?
Well, run your script using the correct interpreter. Remove #!/usr/bin/env python from your file and if you have a pipeline, add there:
sh "/home/yourname/anaconda3/bin/python /path_to_python_file/myFile.py"
It will most likely solve the issue. It's also quite flexible in the sense that if you ever want to use this python file on a different machine, you won't have your username hardcoded inside.
I ran cProfile on a python 3 script, worked nicely, then tried to visualize it using runsnake. Howvever, I got an empty screen and the error 'bad marshal data'.
I removed .pyc file but that did not work either.
The code I used to install runsnake was:
sudo apt-get install python-profiler python-wxgtk2.8 python-setuptoolD
sudo easy-install installSquareMap RunSnakeRun
I am using UBUNTU.
Many thanks.
note: I should add I installed everything while py3k was activated
TL;DR: This error occurs when profiling in Python 2.x and viewing the profile in Python 3.x or vice versa.
I had the same problem. As far as I can tell, the RunSnakeRun package has not been ported to Python3. At least, I could pip it to python2 but not to python3 (SyntaxError). Further, I think the output format of cProfile is not compatible between python 2/3. I did not take the time to find an definitive confirmation of this, but in the doc of cProfile class pstats.Stats(*filenames, stream=sys.stdout), they do say "The file selected by the above constructor must have been created by the corresponding version of profile or cProfile. To be specific, there is no file compatibility guaranteed with future versions of this profiler, and there is no compatibility with files produced by other profilers.". This seems to be the origin of your problem. For e.g., I made a profile output from python3
import cProfile
cProfile.run('some code to profile', 'restats')
and tried to open it in RunSnakeRun and got the same marhsal error you got. Further, if I do
import pstats
p = pstats.Stats('restats')
p.strip_dirs().sort_stats(-1).print_stats()
in python3, it works like a charm. If I do it in python2, it gives the marshal error. Now, RunSnakeRun is executed in python2 (unless you found some way to make it run in python3). So, my guess is that you have performed your profiling in python3 and are using tools relying on python2 to analyze them, which tools are expecting the output to be compatible with python2.
The RunSnakeRun project seems to be inactive for a while now (copyright on the home page is 2005-2011) and there is no indication that it will be ported to python3.... Maybe considering alternative visualization tool might be the best way to go for you if you want to develop in Python3. pyprof2calltree in combination with KCachegrind worked fine for me in Linux. It can provide a similar visual view of the profiling output as you would get from RunSnakeRun.
Also ran into the same problem and I think there's no (good) way to use runsnake for Python3 (as it already was mentioned in the previous answer). However, SnakeViz might help. It's a relatively intuitive graphical overview of profiling data that, like runsnake, builds on top of profile outputs. Nice bonus: works also for Jupyter notebooks!
I just installed ycm, everything looks good, but I found small problem. The problem is as following:
import os # os is built-in library
os. # ycm helps to complete members of the class.
import numpy # numpy is not built-in library, where its location is site-packages.
numpy. # nothing happened. ycm shows 'pattern not found' message.
I think, this would be a simple problem. But I could not find the solution yet. I think, there is some configuration file in which I can define 'search path' for my project.
It would be grateful if I can find a way to solve it.
Best,
Je-Hoon Song
I had the same issue with module 'mpmath' and fixed it in the following manner:
First I retrieved the path where the module was located:
%python3
>>>import mpmath
>>>print(mpmath.__file__)
/usr/lib/python3.4/site-packages/mpmath/__init__.py
Here I found the path of all my "installed" python3 packages to be:
/usr/lib/python3.4/site-packages/
I then simply added to my PYTHONPATH environment variable this path:
%export PYTHONPATH=/usr/lib/python3.4/site-packages/
Then when I used vim sample.py typing import mpmath and following it up with mpmath. YCM showed me all the autocompletions for the mpmath module.
Hope this helps.
I use anaconda python to be my python interpreter in ycm to solve this.
First I modified my vimrc according to full pythong setting in vim.
Then I change g:ycm_python_interpreter_path by
let g:ycm_python_interpreter_path = '/usr/local/anaconda3/bin/python3.8'
In this way I didn't change the system environment variables.
Addtional Info 1:
I think the main problem is that,
my Python interpreter for YCM is my system python (/usr/local/opt/python#3.9/bin/python3.9 ),
which only has limited locally built libraries.
So using the libraries comes with anaconda (/usr/local/anaconda3/bin/python3.8 ) can solve.
Additional Info 2:
By reading :YcmDebugInfo, the main different after edited g:ycm_python_interpreter_path is that:
-- Python completer debug information:
-- Python interpreter: /usr/local/opt/python#3.9/bin/python3.9
-- Python path: ['/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python39.zip', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9', '/usr/local/Cellar/python#3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', '/usr/local/lib/python3.9/site-packages']
-- Python version: 3.9.6
change to
-- Python completer debug information:
-- Python interpreter: /usr/local/anaconda3/bin/python3.8
-- Python path: ['/usr/local/anaconda3/lib/python38.zip', '/usr/local/anaconda3/lib/python3.8', '/usr/local/anaconda3/lib/python3.8/lib-dynload', '/usr/local/anaconda3/lib/python3.8/site-packages', '/usr/local/anaconda3/lib/python3.8/site-packages/aeosa', '/usr/local/anaconda3/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg']
-- Python version: 3.8.8
Additional Info 3: about how to read list of locally installed pyton modules
use https://stackoverflow.com/a/740018/11226687
e.g. in my case
$ /usr/local/opt/python#3.9/bin/python3.9
>>> help('modules')
# only return limitted modules
$ /usr/local/anaconda3/bin/python3
>>> help('modules')
# list out all the modules included in Anaconda, including numpy/matplotlib/scipy ect
numpy is kind of a difficult library because it dynamically builds its namespace on import, making it hard for static code analysis tools to know when you're write the code what names should be available. Because the names available in the namespace numpy are only really known at runtime, YCM probably doesn't have any useful suggestions for you.
One simple way to fix is activate your python environment, then open vim. For example
(django_mdn) ➜ locallibrary git:(master) ✗ vim
and in the vim run :echo $PATH.
Then you should be able to see that your venv path is at the first like this:
/Users/gwanghyeongim/.virtualenvs/django_mdn/bin:/usr/local/opt/tcl-tk/bin:...
Then see if your python packages are auto-complete.
It worked.
If you want to set a certain site-packages to be auto complete permanently, you need to make a file called .ycm_ extra_conf.py in your project root directory or global_extra_conf.py and set vim configuration if you want to set it globally.
P.S.
By running export PYTHONPATH=/usr/lib/python3.4/site-packages/ in the shell before opening vim didn't work for me. Besides, unless setting PYTHONPATH permanently, which will cause issue, you will have to set export PYTHONPATH everytime you want dependencies to be auto complete.
I'm having trouble importing numpy into my script because of some path-name issues.
I'm running my script with python 2.7, rather than the default 2.6 on my server because I need some of the updates in the Collections module. I cant seem to import numpy like:
from numpy.random import poisson
So I am trying to use the python2.7 specific links to numpy on my server, which are installed in:
/opt/lib/python2.7/numpy
But this period in the path is really making this difficult. I cannot change the path in anyway.
I've found a similar problem here, but frankly the code just doesn't make enough sense to me for me to feel safe using it (plus several commenters seemed to suggest it was a bad idea.). If someone has another suggestion, or if you can give me a good explanation of the code there I would appreciate it.
Try setting PYTHONPATH to point to /opt/lib/python2.7, after which import numpy et cetera should pull libraries from there.
$ PYTHONPATH=/opt/lib/python2.7 python27 my_script.py