I have a Python 2.7 script that uses BeautifulSoup4 and requests modules.
The issue is, that I need to deploy this script on a machine to which we can not directly install any new modules/libaries via pip install or anything else.
We can copy this script and any files it needs to run to that machine, but we can not directly install any modules.
I have tried PyInstaller, PEX and Nuitka to create an executable file or a bundle (in any format, for example .zip) so that we can copy the entire file or bundle into the machine and run the python script from there, without the need to do pip install or installing the modules manually via Wheel file. All without success.
Environment details:
Target machine on which the script needs to run: RHEL-based Linux OS with Python 2.7.
My development machine: Windows 10 but I also have access to Fedora Linux machine both with Python 3 and Python 2.7.
The import section of my script looks like this:
from __future__ import with_statement
from __future__ import absolute_import
import requests
import re
from bs4 import BeautifulSoup
from io import open
Can someone, please, help me out here?
We have the script ready to be deployed, but we are not able to run it in our target machine because of the missing modules/libraries.
Thank you very much
EDIT:
Mentioning this since it may not be clear at first - we do not have the issue with a network connection or anything of this gender. We were prohibited to use pip install or a manual method of installing a module. Therefore, we can only bundle the modules directly with the script or something so that would not need to directly install the modules on the target machine itself.
I am creating a Python Web App in Google App engine.
When I
sudo pip install
a third party library and then try to import it, I get the error 'ImportError: No module named x'. Where x is the name of that library. In my case as an example: Boto, Boto3, Fask etc.
If I go into shell in GAE and type python >> import X the library can be used inside the python environment. When deploying the app though or running the app in the virtaul server in Google App Engine I get the module import error.
I even tried methods like: python >> import sys >> sys.path.insert(0, "path_here")
export PYTHONPATH and selected where those libraries are located
I even followed several Q&A here in Stackoverflow without any success, can somebody please give me a proper way to fix the import error in Google App Engine?
FYI
I am not using any local environment in my pc, I am working directly through the GAE bash console, the launch code editor in GAE and I am running the command dev_appserver.py $PWD
When I do
pip freeze
I can see that the modules are currently installed and deployed on the GAE virtual environment. Is there a problem with my path? What's the best approach to make GAE load my already installed third party libraries.
UPDATE:
Importing the library directly on the python shell from Google App Engine works just fine. Importing the library on my python app index.py file results in the error.
Python import directly from Shell
Python import to the index.py file
Though this is an old thread, adding this answer now:
Run command : gcloud components list
This will show the different components installed and not in your environment.
Install app-engine-python components if not installed:
gcloud components install app-engine-python
gcloud components install app-engine-python-extras
If it doesn't work:
In Windows, uninstall and download and install Google-sdk (check the python version you need). Delete all the files the installer ask you to delete in the last step and run the gcloud component commands again.
I got this error in Google App Engine's Python have used Google Translate API,
But I don't know how to fix,
<module>
from apiclient.discovery import build
ImportError: No module named apiclient.discovery
I'll try to set environment which indicates to Google App Engine SDK,
And upload to Google Apps Engine again, always get the error,
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it.
Please tell me how to fix,
Thanks
UPDATE : Fixed
Follow Nijjin's help,
I fixed problems by adding the following folders,
apiclient, gflags, httplib2, oauth2client, uritemplate
If you still got problem, please consider below Answer of this page to get more info. ex. : Varum answer, etc ...
You should be able to get these dependencies with this simple install:
sudo pip install --upgrade google-api-python-client
This is described on the quick start page for python.
apiclient was the original name of the library.
At some point, it was switched over to be googleapiclient.
If your code is running on Google App Engine, both should work.
If you are running the application yourself, with the google-api-python-client installed, both should work as well.
Although, if we take a look at the source code of the apiclient package's __init__.py module, we can see that the apiclient module was simply kept around for backwards-compatibility.
Retain apiclient as an alias for googleapiclient.
So, you really should be using googleapiclient in your code, since the apiclient alias was just maintained as to not break legacy code.
# bad
from apiclient.discovery import build
# good
from googleapiclient.discovery import build
If none of the above solutions work for you, consider if you might have installed python through Anaconda. If this is the case then installing the google API library with conda might fix it.
Run:
python --version
If you get something like
Python 3.6.4 :: Anaconda, Inc.
Then try:
conda install google-api-python-client
As bgoodr has pointed out in a comment you might need to specify the channel (think repository) to get the google API library. At the time of writing this means running the command:
conda install -c conda-forge google-api-python-client
See more at https://anaconda.org/conda-forge/google-api-python-client
apiclient is not in the list of third party library supplied by the appengine runtime: http://developers.google.com/appengine/docs/python/tools/libraries27 .
You need to copy apiclient into your project directory & you need to copy these uritemplate & httplib2 too.
Note: Any third party library that are not supplied in the documentation list must copy to your appengine project directory
Make sure you only have google-api-python-client installed. If you have apiclient installed, it will cause a collision. So, run the following:
sudo pip uninstall apiclient
I fixed the problem by reinstalling the package with:
pip install --force-reinstall google-api-python-client
For app engine project you gotta install the lib locally by typing
pip install -t lib google-api-python-client
read more here
There is a download for the Google API Python Client library that contains the library and all of its dependencies, named something like google-api-python-client-gae-<version>.zip in the downloads section of the project. Just unzip this into your App Engine project.
for python3 this worked for me:
sudo pip3 install --upgrade google-api-python-client
I had the same problem because of a bug in the installation of the URITemplate module.
This solved the problem:
pip install --force-reinstall uritemplate.py
I got this same error when working on a project to parse recent calendar events from Google Calendar.
Using the standard install with pip did not work for me, here is what I did to get the packages I needed.
Go directly to the source, here is a link for the google-api-python-client, but if you need a different language it should not be too different.
https://github.com/google/google-api-python-client
Click on the green "Clone or Download" button near the top left and save it as a zip file. Move the zip to your project folder and extract it there. Then cut all the files from the folder it creates back into the root of your project folder.
Yes, this does clutter your work space, but many compilers have ways to hide files.
After doing this the standard
from googleapiclient import discovery
works great.
Hope this helps.
"google-api-python-client" requires:
pip install uritemplate.py
to fix problem on GAE Development Server:
from googleapiclient.discovery import build
ImportError: No module named googleapiclient.discovery
I installed google-api-python-client using pip but it is still showing me error so I try upgrading it and it help me to get out of error
if you are using "windows" then
pip install --upgrade google-api-python-client
will help you because it help me so it will help you too :)
I encountered the same issue.
This worked:
>>> import pkg_resources
>>> pkg_resources.require("google-api-python-client")
[google-api-python-client 1.5.3 (c:\python27), uritemplate 0.6 (c:\python27\lib\site-packages\uritemplate-0.6-py2.7.egg), six 1.10.0 (c:\python27\lib\site-packages\six-1.10.0-py2.7.egg), oauth2client 3.0.0 (c:\python27\lib\site-packages\oauth2client-3.0.0-py2.7.egg), httplib2 0.9.2 (c:\python27\lib\site-packages\httplib2-0.9.2-py2.7.egg), simplejson 3.8.2 (c:\python27\lib\site-packages\simplejson-3.8.2-py2.7-win32.egg), six 1.10.0 (c:\python27\lib\site-packages\six-1.10.0-py2.7.egg), rsa 3.4.2 (c:\python27\lib\site-packages\rsa-3.4.2-py2.7.egg), pyasn1-modules 0.0.8 (c:\python27\lib\site-packages\pyasn1_modules-0.0.8-py2.7.egg), pyasn1 0.1.9 (c:\python27\lib\site-packages\pyasn1-0.1.9-py2.7.egg)]
>>> from apiclient.discovery import build
>>>
It only worked with me when I used sudo:
sudo pip install --upgrade google-api-python-client
I was getting the same error, even after following Google's guide at https://developers.google.com/drive/api/v3/quickstart/python, then I realized I had to invoke like this:
python3 quickstart.py
Instead of:
python quickstart.py <-- WRONG
(Note the "3")
Worked flawlessly.
I'm using Ubuntu 18.04.4 LTS.
use this
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
The same error can be seen if you are creating a Python module and your executing the script after installing it via pip or pipx command.
In this case ensure you have declared what the project minimally needs to run correctly into install_requires section of your setup.py file, so in this case:
install_requires=[
"google-api-python-client>=1.12.3",
"google-auth-httplib2>=0.0.4",
"google-auth-oauthlib>=0.4.1"
]
This can also happen if the interpreter on your IDE is pointing to the wrong virtual environment. In VSCODE I've set it manually to the correct interpreter and my problem was solved.
(May 2021) It's been about 8 years since the original question, and since then, several product changes have occurred, so new developers arriving here looking to use the Google Translate API on Python App Engine have a few changes to make:
Product info/costs: The Google Translate API is now available as the Google Cloud Translation API. It's not free (meaning you need to create a billing account backed by a financial instrument like a credit card), but you get a quota of translated characters per month. See its pricing page for more info. Similarly, while you used to be able to create an App Engine app without a credit card, you can no longer do so via the new policy as of Nov 2019. It still has a generous "Always Free" tier quota which you must exceed in order to incur charges. Also see the App Engine pricing page for more info.
Client libraries: Rather than using apiclient or googleapiclient which are part of the Google APIs client library which is a low-level, multi-product, platform-level client library, we recommend the Google Cloud client libraries which are higher-level and product-focused. That means there's a specific Cloud Translation client library (actually two: basic/v2/Python 2 or advanced/v3/Python 3) — these are higher-level and much easier to use:
Add client lib: pip install -U pip google-cloud-translate (or pip3)
With it, your code sample can be as simple as:
'translate_demo.py - demo the Cloud Translation API'
from __future__ import print_function
import google.auth
from google.cloud import translate
TRANSLATE = translate.TranslationServiceClient()
_, PROJECT_ID = google.auth.default()
PARENT = 'projects/{}'.format(PROJECT_ID)
TARGET_LANG = 'es'
TEXT = 'Hello world'
DATA = {
'parent': PARENT,
'contents': [TEXT],
'target_language_code': TARGET_LANG,
}
try: # Python 3/advanced/v3
rsp = TRANSLATE.translate_text(request=DATA)
except TypeError: # Python 2/basic/v2
rsp = TRANSLATE.translate_text(**DATA)
print(TEXT, '=', rsp.translations[0].translated_text)
It also works on Python 2 and 3 without any modification:
$ python2 translate_demo.py
Hello world = Hola Mundo
$ python3 translate_demo.py
Hello world = Hola Mundo
This code snippet can be adapted for App Engine fairly easily (more below), especially if you're prototyping since you can take advantage of the default service account so you don't have to muck around with service accounts, like making a new one, creating a public/private key-pair, and having to download the JSON credentials file and pointing the GOOGLE_APPLICATION_CREDENTIALS environment variable to it, etc. When you're ready to go into production and need to create your own service account, then check out this page in the docs.
Furthermore, there has been significant changes in App Engine itself: the original Python 2 App Engine service had a bunch of built-in proprietary APIs (Datastore, Memcache, Task Queues, etc.). Due to user feedback regarding "vendor lock-in," the next generation Python 3 App Engine service was made to free developers from those services. Instead, you'd leverage any equivalent productized services, i.e., Cloud Datastore, Cloud Memorystore, and Cloud Tasks instead. The Google Cloud team has created a migration guide and I've augmented that guide with hands-on tutorials, code samples, and videos to help folks migrate to these unbundled services as you port your app to Python 3.
If you're considering Google Cloud serverless compute platforms beyond App Engine, such as Cloud Functions (FaaS) or Cloud Run (containerized/managed CaaS), then check out this Translation API sample app I created (where I basically stole the above code snippet from) that can be deployed 8 different ways, Python 2 and 3, locally with Flask's development server, to App Engine, Cloud Functions, or Cloud Run, all with just minor config changes. It's meant to show flexibility in our platforms as well as to help users understand the differences between them better.