We Keep Coding

Python package cannot be loaded into PBS queue file

The skopt package (https://scikit-optimize.github.io/stable/install.html) was installed on a cluster I use.
When I run the code in python directly in the terminal (i.e., cluster terminal), no problem occurs and the code works as expected.
However when I simply place the command to execute the code in a PBS queue system file (e.g., python3 ./code.py), I cannot load the installed package and I get the following message:
Traceback (most recent call last):
File "./test.py", line 22, in <module>
from skopt import gp_minimize
ModuleNotFoundError: No module named 'skopt'
Could someone give me an insight on how to solve the problem?
If I need to provide any more information, let me know.
ps1: the packages is installed on /usr/local/lib/python3.6/site-packages/skopt directory.
ps2: I tried to load export PYTHONPATH=$PYTHONPATH:[$HOME]/.usr/local/lib/python3.6/site-packages/skopt into the PBS file and it didn't work.

This has happened to me before but its a pretty simple fix
pip3 converts to python3
pip converts to python
Use those commands instead of random. Check my comment for further details.

The solution was easier than I thought, as the skopt package was in the path "/usr/local/lib/python3.6/site-packages/" just use the command "export PYTHONPATH=$PYTHONPATH:/usr/local/ lib/python3.6/site-packages", before running the code in python
I was using the "export" command wrongly.

Lots of trouble running a python script from the command line

Context for how you answer: I am new to the command line environment, except when it comes to basic git commands. I've always just used an IDE like PyCharm or NetBeans to run stuff for school projects. Please frame your answer accordingly.
I have a very small python script that pulls down a URL:
import sys
sys.path.append(r'C:\Users\WNeill\PycharmProjects\bloomskyGrantGrove\venv\Lib\site-packages\bloomsky_api')
import bloomsky_api as bs
client = bs.BloomSkyAPIClient(api_key='pr-XXXXXXXXXX')
data = client.get_data()[0] # Dictionary formatted like JSON, if you want data besides the latest image
with open("image_URL.txt", 'w') as file:
print(data.get('outdoor').get('image_url'), file=file)
I did the sys.path.append() because I read in a different question that it would solve my problems of "module not found" when running my scripts from the command line.
Well, it did, sort of... now, it finds my imports, but apparently my imports have imports...
$ py -m bloomtest.py
Traceback (most recent call last):
File "C:\Program Files\Python38\lib\runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "C:\Program Files\Python38\lib\runpy.py", line 109, in _get_module_details
__import__(pkg_name)
File "C:\Users\WNeill\PycharmProjects\bloomskyGrantGrove\bloomtest.py", line 4, in <module>
import bloomsky_api as bs
File "C:\Users\WNeill\PycharmProjects\bloomskyGrantGrove\venv\Lib\site-packages\bloomsky_api\bloomsky_api.py", line 2, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
So what do I do to make this work when my dependencies have dependencies?

Don't use sys.path.append. Imagine sending your code to someone else, their packages won't be in the same path and they won't be able to run your program. You might not plan on distributing your code but it's just bad practice.
Instead you should use pip to install your packages, as i assume you've been using the PyCharm package manager. I think it automatically installs with current versions of python (not sure though, I'm on Linux) and it's used like so:
pip install BloomSky-API
it'll automatically get all the dependencies and put them in the right places.
I've never used the py command before (am I missing out?), try using python bloomtest.py to run it instead just to be sure. You might get an error telling you that python is an unrecognized command or file, if that's the case it means your PATH is not set up correctly. I've found the easiest way to resolve this is to simply reinstall python, making sure to check the checkbox that says to add python to your PATH.
I'd usually post suggestions like this in a comment if I'm not sure if it solves the problem you're having, but the answer is too long to fit in a comment. Hope this helps!

I want to post the final solution to my problem, which I stumbled on thanks to DutChen18's answer.
He said I should use pip install to install all of my packages, which is one thing that I already do. I don't know much about the command line, except basic git and that. Trying to do it again gave me requirement already satisfied errors.
However, I was using an embedded terminal in PyCharm: C:\Program Files\Git\bin\bash.exe, which comes when you download Git. This works great in PyCharm because it automatically starts in the working directory of your project. Very convenient for me, a new command line user.
I decided to open up Git Bash separately from PyCharm and run pip install again. The first thing I found is that it didn't work without python -m pip install, unlike in the PyCharm embedded terminal.
Once I figured that out, I tried to python -m pip install BloomSky-API, but this time it didn't tell me that the libraries were already installed. All of a sudden, I could run my python script from the command line.
I have ZERO clue as to why this happened or why it now works, and I would love to hear a more technical explanation now that I have things working.

Problem with Python project after clone the project from Git repo

I create a Python project using Flask and it works perfectly.
Then I upload it to my private Azure repository.
After that, another coworker clone the project and try to run in his machine, but it didn't work. He got the following error:
File "run.py", line 1, in <module>
from api import app
File "E:\PUC_Conexao_Colaborativa_API\api\__init__.py", line 7, in <module>
from api.instance.config import app_config, ambiente
ModuleNotFoundError: No module named 'api.instance'
I look on Stackoverflow and found this post:
Python error "ImportError: No module named"
One of the answer say that:
You edited init.py on windows.
The windows editor added something non-printing, perhaps a carriage-return (end-of-line in Windows is CR/LF; in unix it is LF only), or perhaps a CTRL-Z (windows end-of-file).
I remember when I install "Git for Windows" exe, it asks for how I want to config the line ending conversions and I select "Checkout Windows-style, commit Unix-style line endings".
Is this the cause that my Python project fails to run in another machines?
The project runs fine on my machine and I copy the entire project and give to the another coworker using a pendrive. When the coworker run the project using the files in the pendrive, it runs perfectly, just like in my machine.
But if I or anyone else try to run the project downloading it from the repository, if fails to run with the error that I mention above.
I suppose the error is caused by the conversion of Git when I push the project to the repo. If am I correct, how can I fix that?

App Engine dev server: bad runtime process port [''] No module named google.appengine.dist27.threading

When I try to run any of my app engine projects by python GoogleAppEngineLauncher
I got the error log as follows:
Does anyone have any ideas of what's going on?
I tried remove the SDK and reinstall it. Nothing happens. Still got the same error.
Everything is working fine and I don't think I made any changes before this happens.
The only thing that I can think of is that I install bigquery command line tool before this happens. But I don't think this should be the reason of this.
bad runtime process port ['']
Traceback (most recent call last):
File
"/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/_python_runtime.py",
line 197, in
_run_file(file, globals()) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/_python_runtime.py",
line 193, in _run_file
execfile(script_path, globals_) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/runtime.py",
line 175, in
main() File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/runtime.py",
line 153, in main
sandbox.enable_sandbox(config) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py",
line 159, in enable_sandbox
import('%s.threading' % dist27.name) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py",
line 903, in load_module
raise ImportError('No module named %s' % fullname) ImportError: No module named google.appengine.dist27.threading

The most probable reason is having another python package that is coming from google. Run
python in verbose mode with python -vvvvv and try following command import google.
If the above import is successful make sure its coming from /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google which is the path of python google appengine libraries on my system (OSX, fresh install of google appengine sdk).
If that is not the case (but the import is successful) then its most likely other existing google libraries which are creating issue and the path of the same would be visible on python prompt after the successful import google command like following:
>>> import google
import google # loaded from Zip
/Library/Python/2.7/site-packages/protobuf-2.4.1-py2.7.egg/google/init.pyc
In this case my google protobuf package was the culprit.
Solution:
Use virtualenv: If you haven't used python virtualenv before, may be this is the best time to use it. (a) Make sure that your PYTHONPATH variable is not set to include anything that has a google package! Unset it running unset PYTHONPATH (in your bash prompt) unless you are very sure what you have there. (b) Create a new python virtualenv, activate it and try to run google appengine commands in it:
virtualenv --no-site-packages /tmp/googleapps
source /tmp/googleapps/bin/activate
dev_appserver.py path_to_google_app
Remove conflicting packages from path: Move conflicting packages a new virtualenv. This is likely to break other stuff so not recommended.
I had hit the same issue today. This being the top result on google but lacked any answers, I add this after fixing same issue on my system. There may be other possible reasons which thankfully I haven't come across.
Good luck!

A recent upgrade of the development SDK started causing this problem for me. After much turmoil, I found that the problem was that the SDK was in a sub-directory of my project code. When I ran the SDK from a different (parent) directory the error went away.

Issue with my Python environment

Brand new to Python, coming from Ruby.
I have a script that works perfectly if I run it form ipython or ipython qtconsole. I then tried to turn it into an executable script -- threw #!/usr/bin/env python at the top.
Running the script throws an error:
$ ./script/myscript.py
Traceback (most recent call last):
File "./script/myscript.py", line 6, in <module>
import yaml
ImportError: No module named yaml
Obviously there's something wrong with how python is loading modules (as it works perfectly fine from the REPL) but I have no idea how to fix it.
Thanks!

What is likely happening is you have more than one version of Python installed on your system, and the yaml module is only installed in one of them. When you run ipython it's using one version, but your script's shebang line is finding another version. Run
head `which ipython`
and see if it matches up to the result of which python (I'm betting it won't). Once you know the path to the python binary being used by ipython, you can specifically define it in your script's shebang line.
As a long-term fix, edit your $PATH variable and put the directory containing your desired version of Python ahead of the directory shown by which python, so that you can continue to use #!/usr/bin/env python as a shebang.

ipython must be pointing at a different version of python than what is in PYTHONPATH.
You can find out by looking at cat /usr/local/bin/ipython.
Look at
ipython reads wrong python version