Context for how you answer: I am new to the command line environment, except when it comes to basic git commands. I've always just used an IDE like PyCharm or NetBeans to run stuff for school projects. Please frame your answer accordingly.
I have a very small python script that pulls down a URL:
import sys
sys.path.append(r'C:\Users\WNeill\PycharmProjects\bloomskyGrantGrove\venv\Lib\site-packages\bloomsky_api')
import bloomsky_api as bs
client = bs.BloomSkyAPIClient(api_key='pr-XXXXXXXXXX')
data = client.get_data()[0] # Dictionary formatted like JSON, if you want data besides the latest image
with open("image_URL.txt", 'w') as file:
print(data.get('outdoor').get('image_url'), file=file)
I did the sys.path.append() because I read in a different question that it would solve my problems of "module not found" when running my scripts from the command line.
Well, it did, sort of... now, it finds my imports, but apparently my imports have imports...
$ py -m bloomtest.py
Traceback (most recent call last):
File "C:\Program Files\Python38\lib\runpy.py", line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "C:\Program Files\Python38\lib\runpy.py", line 109, in _get_module_details
__import__(pkg_name)
File "C:\Users\WNeill\PycharmProjects\bloomskyGrantGrove\bloomtest.py", line 4, in <module>
import bloomsky_api as bs
File "C:\Users\WNeill\PycharmProjects\bloomskyGrantGrove\venv\Lib\site-packages\bloomsky_api\bloomsky_api.py", line 2, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
So what do I do to make this work when my dependencies have dependencies?
Don't use sys.path.append. Imagine sending your code to someone else, their packages won't be in the same path and they won't be able to run your program. You might not plan on distributing your code but it's just bad practice.
Instead you should use pip to install your packages, as i assume you've been using the PyCharm package manager. I think it automatically installs with current versions of python (not sure though, I'm on Linux) and it's used like so:
pip install BloomSky-API
it'll automatically get all the dependencies and put them in the right places.
I've never used the py command before (am I missing out?), try using python bloomtest.py to run it instead just to be sure. You might get an error telling you that python is an unrecognized command or file, if that's the case it means your PATH is not set up correctly. I've found the easiest way to resolve this is to simply reinstall python, making sure to check the checkbox that says to add python to your PATH.
I'd usually post suggestions like this in a comment if I'm not sure if it solves the problem you're having, but the answer is too long to fit in a comment. Hope this helps!
I want to post the final solution to my problem, which I stumbled on thanks to DutChen18's answer.
He said I should use pip install to install all of my packages, which is one thing that I already do. I don't know much about the command line, except basic git and that. Trying to do it again gave me requirement already satisfied errors.
However, I was using an embedded terminal in PyCharm: C:\Program Files\Git\bin\bash.exe, which comes when you download Git. This works great in PyCharm because it automatically starts in the working directory of your project. Very convenient for me, a new command line user.
I decided to open up Git Bash separately from PyCharm and run pip install again. The first thing I found is that it didn't work without python -m pip install, unlike in the PyCharm embedded terminal.
Once I figured that out, I tried to python -m pip install BloomSky-API, but this time it didn't tell me that the libraries were already installed. All of a sudden, I could run my python script from the command line.
I have ZERO clue as to why this happened or why it now works, and I would love to hear a more technical explanation now that I have things working.
Related
When I try to run my python projects, in some cases I get this error:
File "/usr/local/bin/AAA/camera_service/camera_service_main.py", line 6, in <module>
from views.hires_camera_handler_view import hires_camera_handler_blueprint
File "/usr/local/bin/AAA/camera_service/views/hires_camera_handler_view.py", line 7, in <module>
from hires_camera_handler.hires_camera_handler import HiResCameraHandler
File "/usr/local/bin/AAA/camera_service/hires_camera_handler/hires_camera_handler.py", line 3, in <module>
from ids_peak import ids_peak
File "/home/izx/anaconda3/envs/py38/lib/python3.8/site-packages/ids_peak/ids_peak.py", line 18, in <module>
from . import _ids_peak_python_interface
ImportError: libpython3.8.so.1.0: cannot open shared object file: No such file or directory
The object file exists in ~/anaconda3/envs/py38/lib
I can make the code run in my terminal by adding
export LD_LIBRARY_PATH=~/anaconda3/envs/py38/lib
However, there seems to be a deeper problem here, because more of my projects behave differently then on other ubuntu installs. I could of course reinstall ubuntu. But I would like to understand what I did wrong here.
Also, the solution above doesn't solve the issue. I can still not run my tests in PyCharm and that would be really beneficial. Probably I can somehow set this value in my pycharm run, however, setting the path variable didn't work and I wouldn't know how to do it.
I have tried various solutions. installing libpython3.8 (its already installed), installing libpython3.8-dev (its not available on ubuntu 22.04 jammy). I've also added the value to bashrc but as expected that doesn't solve it for PyCharm.
Can anyone explain to me exactly what the problem is here? Why is my conda environment not able to find its own lib folder? Why does adding the path to LD_LIBRARY_PATH work? It doesnt make sense to me that my conda environment needs the shared files when it got its own. What is the default location for conda to look for the shared files? How can this happen all of sudden when I have installed anaconda many times on many systems? I know these are a lot of questions, but I hope they can give me the answers I need to understand this problem instead of follow other SO-posts blindly.
Thank you
If your system is Ubuntu, you can find this file as I do and copy it to /lib/ or /lib64/
find / -name libpython3.8.so.1.0
sudo cp /home/zhou/anaconda3/envs/paddlep/lib/libpython3.8.so.1.0 /usr/lib/
or
sudo cp /home/zhou/anaconda3/envs/paddlep/lib/libpython3.8.so.1.0 /usr/lib64/
It is always worth reading the docs from anaconda:
https://docs.conda.io/projects/conda-build/en/latest/resources/use-shared-libraries.html#shared-libraries-in-macos-and-linux
You have to tell python where to find the additional modules by setting the PYTHONPATH in ~/.bashrc; Make sure to reload / restart the terminal or your IDE after updating ~/.bashrc
e.g.
export PYTHONPATH="${PYTHONPATH}:/Users/<myuser>/project_root_folder_path"
see Permanently add a directory to PYTHONPATH?
I apologize if this has been asked before, but I'm having trouble finding a similar question on here.
I am attempting to build some code for work, but have never done a project so large. I decided to set up a virtual environment in python, which I hoped would help with cross OS compatibility and packaging changes. I had python 3.10.5 installed globally on my user account on Ubuntu 22.04. I followed a tutorial which told me to install pyenv and pipx, which I did. I initially ran into a problem with spyder not launching, but following another stackoverflow page, I was able to delete the ini and it launched just fine.
I could not get spyder to load any libraries like numpy, no matter what directory I set the interpreter to. I eventually decided that maybe pyenv's shims and symlinks were messing something up. I uninstalled pyenv after much trouble, during which I continued to get redirection errors to the now removed directory, despite a) the directory being removed, b) the .bashrc file being changed according to the github instructions on removal. I also uninstalled pipx. I then installed virtualenv like I should have in the first place, but now spyder won't launch. The original ini file fix did not work this time. Also some of the files seem to point towards pipx; I'm thinking maybe I should reinstall pipx, uninstall spyder, and then uninstall pipx again. But I am afraid to stack more complications on top of the issue.
In summary, I believe all my file pathing and installs are scrambled and screwed up.
My current solution has been to make a new user account to isolate everything and try again. I am wondering if someone can help point me to:
a) a correct and reliable guide on virtual environments in python that doesn't rely on shims and symlinks. I'm thinking virtualenv but am unsure.
b) a way to clean up my main home user account file system by removing packages, resetting the PATH and possibly clean installing python. I do not think the pyenv github is giving correct instructions on removing the shims, because I kept getting errors about the pyenv directory even after removal and changing the .bashrc.
EDIT: Sorry, forgot to add: when I try to run spyder via terminal I get a long traceback
Traceback (most recent call last):
File "/usr/bin/spyder", line 33, in <module>
sys.exit(load_entry_point('spyder==4.2.1', 'gui_scripts', 'spyder')())
File "/usr/lib/python3/dist-packages/spyder/app/start.py", line 213, in main
mainwindow.main(options, args)
File "/usr/lib/python3/dist-packages/spyder/app/mainwindow.py", line 3624, in main
mainwindow = create_window(app, splash, options, args)
File "/usr/lib/python3/dist-packages/spyder/app/mainwindow.py", line 3482, in create_window
main.setup()
File "/usr/lib/python3/dist-packages/spyder/app/mainwindow.py", line 803, in setup
self.completions = CompletionManager(self)
File "/usr/lib/python3/dist-packages/spyder/plugins/completion/plugin.py", line 97, in __init__
plugin_client = Plugin(self.main)
File "/usr/lib/python3/dist-packages/spyder/plugins/completion/kite/plugin.py", line 50, in __init__
self.installer = KiteInstallerDialog(
File "/usr/lib/python3/dist-packages/spyder/plugins/completion/kite/widgets/install.py", line 287, in __init__
self._integration_widget = KiteIntegrationInfo(self)
File "/usr/lib/python3/dist-packages/spyder/plugins/completion/kite/widgets/install.py", line 58, in __init__
image = image.scaled(image_width, image_height, Qt.KeepAspectRatio,
TypeError: arguments did not match any overloaded call:
scaled(self, int, int, aspectRatioMode: Qt.AspectRatioMode = Qt.IgnoreAspectRatio, transformMode: Qt.TransformationMode = Qt.FastTransformation): argument 1 has unexpected type 'float'
scaled(self, QSize, aspectRatioMode: Qt.AspectRatioMode = Qt.IgnoreAspectRatio, transformMode: Qt.TransformationMode = Qt.FastTransformation): argument 1 has unexpected type 'float'
The skopt package (https://scikit-optimize.github.io/stable/install.html) was installed on a cluster I use.
When I run the code in python directly in the terminal (i.e., cluster terminal), no problem occurs and the code works as expected.
However when I simply place the command to execute the code in a PBS queue system file (e.g., python3 ./code.py), I cannot load the installed package and I get the following message:
Traceback (most recent call last):
File "./test.py", line 22, in <module>
from skopt import gp_minimize
ModuleNotFoundError: No module named 'skopt'
Could someone give me an insight on how to solve the problem?
If I need to provide any more information, let me know.
ps1: the packages is installed on /usr/local/lib/python3.6/site-packages/skopt directory.
ps2: I tried to load export PYTHONPATH=$PYTHONPATH:[$HOME]/.usr/local/lib/python3.6/site-packages/skopt into the PBS file and it didn't work.
This has happened to me before but its a pretty simple fix
pip3 converts to python3
pip converts to python
Use those commands instead of random. Check my comment for further details.
The solution was easier than I thought, as the skopt package was in the path "/usr/local/lib/python3.6/site-packages/" just use the command "export PYTHONPATH=$PYTHONPATH:/usr/local/ lib/python3.6/site-packages", before running the code in python
I was using the "export" command wrongly.
I'm trying to convert my python file to an executable using PyInstaller. The program uses the Google Cloud Translate API to translate given text between languages. When running python quicktrans.py in the terminal, the program works fine. Then I ran pyinstaller quicktrans.py, SHIFT + right-clicked the directory the executable was in, and ran the .exe file in the terminal. This is the traceback that it spit out (Note this is not the whole traceback because it is a little lengthy):
File "c:\users\kalab\realpython\quicktrans\google\cloud\connection.py", line 31, in <module>
get_distribution('google-cloud-core').version)
File "site-packages\pkg_resources\__init__.py", line 559, in get_distribution
File "site-packages\pkg_resources\__init__.py", line 433, in get_provider
File "site-packages\pkg_resources\__init__.py", line 970, in require
File "site-packages\pkg_resources\__init__.py", line 856, in resolve
pkg_resources.DistributionNotFound: The 'google-cloud-core' distribution was not found and is required by the application
Failed to execute script quicktrans
I've tried looking into this and some reason it's giving me a pip-like error. I've been trying to fix this for hours and no luck.
Note: To install its client library, as per the documentation, you must run pip install --upgrade google-cloud-translate
I'm thinking this might have something to do with this because the last application I used dealt with the Facebook client module and you only had to do pip install facebook-sdk and the executable made by PyInstaller ran with no issues.
If you want to examine my code used in my program, it's hosted on my GitHub.
Thanks to anyone helping me out here!
It is basically package building name issue. Pyinstaller tries to import
google.cloud
where Google cloud package is now called
gcloud
. So you need to create a hook file for that names
C:\Users\\AppData\Local\Programs\Python\Python36-32\Lib\site-packages\PyInstaller\hooks\hook-gcloud.py
File contents:
from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('gcloud')
In my experience base on the helps in https://github.com/GoogleCloudPlatform/google-cloud-python/issues/1187 :
Go \Anaconda3\Lib\site-packages\PyInstaller\hooks folder (if you use anaconda otherwise you need to find it under python folder)
Find the hook-google-cloud.py (If exist, otherwise you need to creat the hook.
Write to existing code as shown below
'''
Copyright (c) 2017, PyInstaller Development Team.
Distributed under the terms of the GNU General Public License with exception
for distributing bootloader.
The full license is in the file COPYING.txt, distributed with this software.
'''
from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('google-cloud-core')
datas += copy_metadata('google-cloud-translate')
datas += copy_metadata('google-api-core')
Hope you find this explaination helpful. Thank you.
Alternate hook tweak
I'm running into this same essential problem with the Google speech engine.
It's odd how everyone here seems to have success with slightly alternate solutions to this. I really don't understand how the "patches" to the hook which leave copy_metadata('google-cloud-core') in place can work? The error thrown back reads The 'google-cloud-core' distribution was not found..., so how can one execute that line as is?
This is my replacement for the file content of hook-google.cloud.py, in order to build an exe using google speech:
# PATCH: PROVIDED ALTERNATE PACKAGE NAME
from PyInstaller.utils.hooks import copy_metadata
try:
datas = copy_metadata('google-cloud-core')
except:
datas = copy_metadata('google-cloud-speech')
My personal solution:
Change all calls to get_distribution with it returned values (0.21.0 in my case)
Remove from pkg_resources import get_distribution from import
for all files in the package.
I had the exact same issue. I solved it by doing this:
Goto the Pyinstaller hooks folder (~\Lib\site-packages\PyInstaller\hooks)
Find the file hook-google.cloud.py, open it, and add the following code to it
datas += copy_metadata('google-cloud-translate')
datas += copy_metadata('google-api-core')
The issue seems to be that get_distribution is not working with the default google.cloud.translate hook, so I just added this to a hook that was working.
Hope this helps someone.
I was using PyCharm venv for my project, and the only solution that worked for me is changing the project over to a system interpreter (and install the required packages to that).
Today, when I tried to build an EXE out of my Python script, I got the same error:
pkg_resources.DistributionNotFound: The 'google-cloud-core' distribution was not found and is required by the application
I thought the reason was one of the listed ones in this thread, since I was sure I had all dependencies installed with pipenv because my code compiled and I could debug and run the code without issues. Note I used pipenv shell in an empty folder and created my app in it, installing all necessary libraries using pipenv install ..., and one of the libraries was google-cloud-dialogflow (the app is a chatbot manager).
The solution was simply to run pipenv install google-cloud-core.
Now, pyinstaller chatbot_manager.py --onefile --windowed created the c:\Users\...\dist\chatbot_manager.exe file without issues.
When I try to run any of my app engine projects by python GoogleAppEngineLauncher
I got the error log as follows:
Does anyone have any ideas of what's going on?
I tried remove the SDK and reinstall it. Nothing happens. Still got the same error.
Everything is working fine and I don't think I made any changes before this happens.
The only thing that I can think of is that I install bigquery command line tool before this happens. But I don't think this should be the reason of this.
bad runtime process port ['']
Traceback (most recent call last):
File
"/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/_python_runtime.py",
line 197, in
_run_file(file, globals()) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/_python_runtime.py",
line 193, in _run_file
execfile(script_path, globals_) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/runtime.py",
line 175, in
main() File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/runtime.py",
line 153, in main
sandbox.enable_sandbox(config) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py",
line 159, in enable_sandbox
import('%s.threading' % dist27.name) File "/Users/txzhang/Documents/App/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py",
line 903, in load_module
raise ImportError('No module named %s' % fullname) ImportError: No module named google.appengine.dist27.threading
The most probable reason is having another python package that is coming from google. Run
python in verbose mode with python -vvvvv and try following command import google.
If the above import is successful make sure its coming from /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google which is the path of python google appengine libraries on my system (OSX, fresh install of google appengine sdk).
If that is not the case (but the import is successful) then its most likely other existing google libraries which are creating issue and the path of the same would be visible on python prompt after the successful import google command like following:
>>> import google
import google # loaded from Zip
/Library/Python/2.7/site-packages/protobuf-2.4.1-py2.7.egg/google/init.pyc
In this case my google protobuf package was the culprit.
Solution:
Use virtualenv: If you haven't used python virtualenv before, may be this is the best time to use it. (a) Make sure that your PYTHONPATH variable is not set to include anything that has a google package! Unset it running unset PYTHONPATH (in your bash prompt) unless you are very sure what you have there. (b) Create a new python virtualenv, activate it and try to run google appengine commands in it:
virtualenv --no-site-packages /tmp/googleapps
source /tmp/googleapps/bin/activate
dev_appserver.py path_to_google_app
Remove conflicting packages from path: Move conflicting packages a new virtualenv. This is likely to break other stuff so not recommended.
I had hit the same issue today. This being the top result on google but lacked any answers, I add this after fixing same issue on my system. There may be other possible reasons which thankfully I haven't come across.
Good luck!
A recent upgrade of the development SDK started causing this problem for me. After much turmoil, I found that the problem was that the SDK was in a sub-directory of my project code. When I ran the SDK from a different (parent) directory the error went away.