When I was loading certain modules [namely pygments.lexers Bash Lexer and pygments.formatters LatexFormatter] I was was getting an error that python couldn't find the modules. I then realised that this problem was being caused by my PYTHONPATH, which is set up for using paraview with python. It brings its own version of pygments, which doesn't work with nbconvert from the jupyter notebook for some reason [Note it is not totally disfunctional, as PythonLexer, and a few others were called without a problem, it was only the ones that I've mentioned above that couldn't be found].
I have a similar problem with mayavi, which wouldn't work with paraview's version of vtk.
Both of these problems can be resolved simply enough by commenting out the python path in the bashrc, but obviously then paraview won't work.
Is there any way to, for example, reduce the priority of the PYTHONPATH so that the system codes in /etc... are called preferentially, but paraview can still find the ones that it needs in the PYTHONPATH?
I am using python 2.76 on linux mint 17.3, paraview is version 4.4.0, installed from source code as per here
Ordering the entries in PYTHONPATH is partly right, but the system paths don't seem to get included until you run python, and then they get put at the end. So to put a system path in front, explicitly add it:
export PYTHONPATH="[path/to/system/files]:$PYTHONPATH"
It is kind of a hack, because the system path you add will be duplicated in sys.path. But it works.
Yes, you can import sys and manipulate sys.path as an ordinary list at runtime. You can rearrange what's there, or just insert at the beginning (sys.path.insert(0, 'path')). Do this before your import statement. If this will cause problems elsewhere, put it back after your import statement.
Note, this is fairly hacky. But it sounds like you might have a case for it, although I have not looked at these specific tools together.
Edit: this is more relevant if you want to control the python path at the level of individual imports within the course of one execution of python. If you want to control the path at the level of one full execution of Python, you can also set the python path on the command line for just that execution like this:
PYTHONPATH=/replacement/path/here python your_script.py
This is more verbose (unless you wrap it in a shell script or alias) than just calling python, but it lets you control the path one script at a time, wheres putting it into .bashrc/.bash_profile or similar changes it for your whole shell session.
export PYTHONPATH=$PYTHONPATH:<your_path> will give priority to system paths and only if something is not there will look on your path.
export PYTHONPATH=<your_path>:$PYTHONPATH will search <your_path> firstly and then $PYTHONPATH for what it does not find.
If something exists in both but you want to use one version of it for one program and the other for another then you might want to look on different bashrc profiles.
Related
Recently, I've installed Anaconda3 in a MacOS laptop, but I need to set back the default python version to 2.7 because of some old codes involving the following syntax:
#! /usr/bin/env python
I read some comments and it seems like the easiest way to do it is by using an alias,
alias python="python2"
but even after setting this on my bash_profile, I'm still getting this:
$ which python
/anaconda3/bin/python
What am I doing wrong? Any suggestion is welcome. Thank you in advance!
IMO, you don't want to use an alias to do this. Rather, you want to manipulate your filesystem and/or PATH variable so that the first instance of a binary named 'python' found by the system on your PATH is the one you want to have running by default.
If it made sense to do so, and it was easy to do, an obvious fix would be to uninstall Anaconda. But I assume you still need it.
You could also dig into what the Mac might be doing in your case to manage multiple Python versions in some formal way. This might be the best solution, but I can't tell you how that works. You could Google for that information probably. If you want to take a more basic approach, read on.
What I'd suggest you do first is run the command ls -l `which python`. This will tell you which Python binary is currently the default one, and show you details about that binary. I'm guessing that that "binary" is actually a symbolic link, pointing to some other file. If so, you can just delete that link, or rename it to something else. The same thing will work if what you see is not a symbolic link, but rather a real executable, but it's a bigger deal to decide to delete or rename a real executable. If you just rename this file, you can obviously reverse what you've done any time you want.
Once you do this, do ls -l `which python` again. This should give you a new answer and be pointing at the original python installation. You can confirm which version of python is now the default by running python.
I have a question about adding project path to python, for facilitating import effort.
Situation
When I write code in Python, I usually add necessary path to sys.path by using
import sys
sys.path.append("/path/to/dir/") # almost every `.py` need this
Sometimes, when my project gets bigger with many levels of directories, this approach seems bulky and error-prone (especially when I re-organize my files)
Recently, I start using a bash script (located at project root directory) that adding the sys.path.append with project root argument to .py file in the project. With this approach, I hardly have to manually care about importing a module.
Question
My question is: Is that a good practice? I find it convenient for myself, compared to my old method, but since the bash script is a separated file, I need 2 command to run any script in my project (one for the bash and one for the .py). I can include the command calling .py to the bash, but it far less flexible than directly call it from terminal.
Really want to hear some advices! Thanks in advance. Any suggestion will be gratefully appreciated!
It is generally not good practice to use manipulate sys.path within a python library or program. You should add the relevant paths to the PYTHONPATH in the calling environment for your python program:
PYTHONPATH="/path/to/other/projects/directory:$PYTHONPATH" python ...
or
export PYTHONPATH="/path/to/other/projects/directory:$PYTHONPATH"
python ...
This allows you to easily manipulate the paths that your program or library will search for dependencies easily without modifying your code.
It is also very easy to manage this in your personal development environment by modifying your bashrc or in your production environments in your init script (or other wrapper script) and provides you with one place to update each time you add or modify your project paths.
Given that you mention that you have almost one directory per .py file, you should also consider how your code might be organized into packages to further simplify your setup.
It's not a particularly good practice, though you could get away with it. Better to look into virtualenv though (or pipenv) for a smoother workflow.
I write Python programs with eclipse IDE in Linux Backtrack5.
Scapy,Python 2.6 are available in Backtrack.Using auto-completion feature I've seen in all modules,But when i run the program, I see the following error:
from scapy.contrib.ospf import OSPF_Hdr
ImportError : No module named contrib.ospf
I've just started programming with Python, and many of the sources that I've encountered this problem.
The problem for me is very interesting, when coding everything is OK, but it is not running!
In addition to resolving this problem, I want to know why this problem occurs??And in general how to fix?
Maybe my question is a duplicate, but a complete solution and we need it, in which case it does not duplicate any other person who asks the question.
Your search path is wrong.
2 common causes are:
Pythonversions in development and execution are different. Are you sure you use python2.6 both times? Because not all packages are available for all versions of python.
-the search path in your IDE is different from the search path in your shell. I don't have experience with eclipse for python, but it happened to me with other IDEs.
So in bash you can write:
export PYTHONPATH="$PYTHONPATH:/where/module/lives/"
to include the path.
How to include this in your bashprofile:
http://blog.streamitive.com/2011/12/01/export-pythonpathpythonpath/
You can include the path via python itself:
http://www.johnny-lin.com/cdat_tips/tips_pylang/path.html
When to include via bash and when to include via python itself:
include via python for controlled programms, when you know 100% your module will always be in the same location.
Else, you should set the path explicitly. Maybe in a startup shell script, in the crontab command, or wherever you are calling your python program.
The PYTHONPATH the IDE is using isn't the same as the PYTHONPATH you are running the program with
Edit: based on a Ulf Rompe's comment, it is important you use "1" instead of "0", otherwise you will break sys.path.
I have been doing python for quite a while now (over a year), and I am always confused as to why people recommend you use sys.path.append() instead of sys.path.insert(). Let me demonstrate.
Let's say I am working on a module named PyWorkbooks (that is installed on my computer), but I am simultaneously working on a different module (let's say PyJob) that incorporates PyWorkbooks. As I'm working on PyJob I find errors in PyWorkbooks that I am correcting, so I would like to import a development version.
There are multiple ways to work on both (I could put my PyWorkbooks project inside of PyJob, for instance), but sometimes I will still need to play with the path. However, I cannot simply do a sys.path.append() to the folder where PyWorkbooks is at. Why? Because python will find my installed PyWorkbooks first!
This is why you have to do a sys.path.insert(1, path_to_dev_pyworkbooks)
In summary:
sys.path.append(path_to_dev_pyworkbooks)
import PyWorkbooks # does NOT import dev pyworkbooks, imports installed one
or:
sys.path.insert(1, path_to_dev_pyworkbooks) # based on comments you should use **1 not 0**
import PyWorkbooks # imports correct file
This has caused a few hangups for me in the past, and I would really like it if we (as a community) started recommending sys.path.insert(1, path), as if you are manually inserting a path I think it is safe to say that that is the path you want to use!
Or do I have something wrong? It's a question that sometimes bothers me and I wanted it in the open!
If you really need to use sys.path.insert, consider leaving sys.path[0] as it is:
sys.path.insert(1, path_to_dev_pyworkbooks)
This could be important since 3rd party code may rely on sys.path documentation conformance:
As initialized upon program startup, the first item of this list,
path[0], is the directory containing the script that was used to
invoke the Python interpreter.
If you have multiple versions of a package / module, you need to be using virtualenv (emphasis mine):
virtualenv is a tool to create isolated Python environments.
The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.
Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.
Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.
In all these cases, virtualenv can help you. It creates an environment that has its own installation directories, that doesn’t share libraries with other virtualenv environments (and optionally doesn’t access the globally installed libraries either).
That's why people consider insert(0, to be wrong -- it's an incomplete, stopgap solution to the problem of managing multiple environments.
you are confusing the concept of appending and prepending. the following code is prepending:
sys.path.insert(1,'/thePathToYourFolder/')
it places the new information at the beginning (well, second, to be precise) of the search sequence that your interpreter will go through. sys.path.append() puts things at the very end of the search sequence.
it is advisable that you use something like virtualenv instead of manually coding your package directories into the PYTHONPATH everytime. for setting up various ecosystems that separate your site-packages and possible versions of python, read these two blogs:
python ecosystems introduction
bootstrapping python virtual environments
if you do decide to move down the path to environment isolation you would certainly benefit by looking into virtualenvwrapper: http://www.doughellmann.com/docs/virtualenvwrapper/
Anyone know this? I've never been able to find an answer.
If you're prone to installing python in various and interesting places on your PATH (as in $PATH in typical Unix shells, %PATH on typical Windows ones), using /usr/bin/env will accomodate your whim (well, in Unix-like environments at least) while going directly to /usr/bin/python won't. But losing control of what version of Python your scripts run under is no unalloyed bargain... if you look at my code you're more likely to see it start with, e.g., #!/usr/local/bin/python2.5 rather than with an open and accepting #!/usr/bin/env python -- assuming the script is important I like to ensure it's run with the specific version I have tested and developed it with, NOT a semi-random one;-).
From wikipedia
Shebangs specify absolute paths to system executables; this can cause
problems on systems which have non-standard file system layouts
Often, the program /usr/bin/env can be used to circumvent this
limitation
it finds the python executable in your environment and uses that. it's more portable because python may not always be in /usr/bin/python. env is always located in /usr/bin.
It finds 'python' also in /usr/local/bin, ~/bin, /opt/bin, ... or wherever it may hide.
You may find this post to be of interest:
http://mail.python.org/pipermail/python-list/2008-May/661514.html
This may be a better explanation:
http://mail.python.org/pipermail/tutor/2007-June/054816.html