I am trying to develop a simple forum site for my udacity assignment. It is not a strict requirement to use the bcrypt for password hashing, but I'd like to do it because I also like to know how to use third party libraries which are not provided by Google.
Following instructions provided here (installing a third-party library), I have created a folder named lib, and installed bcrypt library with following command:
python -m pip install -t lib/ bcrypt
I have the lib folder automatically structred like this:
I also created an appengine_config.py file with following content, as per instructions in above manual:
# appengine_config.py
from google.appengine.ext import vendor
# add lib folder as vendor directory
vendor.add('lib')
At this point, I am unable to import the bcrypt to my scripts. The import commands I tried so far are as follows:
from lib import bcrypt
ImportError: No module named lib
import bcrypt
ImportError: No module named bcrypt._bcrypt
from lib.bcrypt import bcrypt
ImportError: No module named lib.bcrypt
What am I missing?
As Avinash Raj pointed out, and as already pointed out in referenced manual, one cannot use python libraries with c extensions. So I downloaded the py-bcrypt, it worked like a charm.
For any newbie like me who needs it, here is the steps you have to take:
Inside your project folder, create a folder called "lib"
Extract the zip downloaded from github above, to the folder 'lib'. Do not use
- in your folder name. Name it something like pybcrypt
Create the appengine_config.pyfile, as outlined in here
Import the library to your script, like so: from pybcrypt import bcrypt
Pat yourself on the back.
Here is another option, you need to setup Wheel package before you can import bcrypt
pip install wheel
pip install bcrypt
from flask_bcrypt import Bcrypt
Related
tl;dr - My python package requires import package.package instead of working with just import package. How do I get it to work with the latter?
I am trying to set up my first python package and am running into some issues with the import part of the process.
My python package file setup looks like this on my computer:
my-package
- build
- dist
- package
- package.egg-info
- LICENSE
- README.md
- Setup.py
Inside package is the following:
__init__.py
package.py
__init__.py reads name = 'package', and package.py contains all of the content of the package.
EDIT: I have attempted using various different versions of __init__.py, including adding import package, import package.package, or import package.package as package below the line name = 'package', but all of resulted in the same issue.
Using the Packaging Python Projects tutorial, I've been able to get my package upload to TestPyPi, but when I install the package to my computer, none of the functions/methods are available, and when I run "import package" and do help(package), I get the following:
Help on package package:
NAME
package
PACKAGE CONTENTS
package
DATA
name = 'package'
FILE
url/to/package
When I run import package.package and help(package), I can access the methods/functions, and get the expected help text for the package's content.
My question is, how to I configure the package file on my computer in such a way that once it is upload to TestPyPi and then downloaded, import package works, instead of needing to run import package.package?
When you write import package, you can access names in package/__init__.py as package.foo.
So, inside __init__.py, if you import all the necessary functions/variables/etc from package.py, those names will be visible to clients that just import package.
So, if you have this in package/__init__.py:
from .package import (foo, bar, baz)
Then in your other code you can do this:
from package import foo
And you dont have to worry about from package.package import foo.
Try to run this file:
#!flask/bin/python
from flask import Flask, jsonify
#!/usr/bin/python
import psycopg2
And this return:
ImportError: No module named psycopg2
I have file with only psycopg2 and it is work. And I have another file with only flask and it is work too. But when I try to run file with both imports it crush.
How to fix it (include flask/bin/python and /usr/bin/python, that flask and psycopg2 work toogether)?
#! PATH is shebang and Unix/Linux use it for path for executor. (/path/to/script.py insted of python /path/to/script.py)
However, it might help to extend the path in which to look for packages.
Run for your extra enviroment (python bin) this:
# Example:
python -c "import PACKAGE; print(PACKAGE.__path__)"
# Psycopg2
python -c "import psycopg2; print(psycopg2.__path__)"
Output:
['/path/to/some/python/psycopg2']
(My output: /home/usr/miniconda3/envs/free/lib/python3.6/site-packages/psycopg2)
#!flask/bin/python
from flask import Flask, jsonify
import sys
sys.path.insert(0, '/path/to/some/python') # Without package name
import psycopg2
If you add a path with a package name, you must then be one level down, but beware, __init__.py files can edit paths through the "namespace" so it's not ideal. But you only work with the package itself. Without it, you work with packages that are available in that directory.
You have probably two different python installations running.
One in [some path]/flask/bin/python and one in /usr/bin/python. The modules are installed in different environments so you can not use them together.
To fix this install flask module in your 'main' python installation or install psycopg2 in your flask environment.
You need to install a package module.
In terminal execute:
pip install psycopg2
more information in PyPi - PSYCOPG2
I am using google-auth to allow firebase authentication in my GAE project.
Everything works fine when I run the code locally using dev_appserver.py or when I deploy it to google app engine.
But I get this ImportError exceptions when I try to use Django's manage.py script to create/run migrations.
ImportError: Could not import 'firebase.authentication.FirebaseAuthentication' for API setting 'DEFAULT_AUTHENTICATION_CLASSES'. ImportError: No module named auth.transport.requests.
The google-auth module is installed under lib directory and has this structure:
- lib
- google
- auth
- oauth2
These import cause the ImportErrors:
import google.auth.transport.requests
from google.oauth2 import id_token
My guess is that there might be naming conflicts as other imports work fine.
Please help!
If you want to use 3rd party libraries that are not included in this list, then you'll have to add them manually since you have done that by adding lib folder and including all the package folder follow these steps.
Create your_app_directory/appengine_config.py file.
Add these following lines inside that file
from google.appengine.ext import vendor
vendor.add('lib')
This should solve the import error problem.
As the title says.
So I've added the following to appengine_config.py with no luck:
# appengine_config.py
from google.appengine.ext import vendor
# Add any libraries install in the "lib" folder.
vendor.add('lib')
I did a print sys.path and verified the lib dir contains google/cloud/bigquery
I can import it if I run python myself:
from google.cloud import bigquery
print bigquery.__path__
['/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery']
From a Google App Engine end points:
import google.cloud;print google.cloud.__path__
['/usr/local/lib/python2.7/dist-packages/google/cloud']
Big Query is at that system location. /usr/local/lib/python2.7/dist-packages/google/cloud/biqguery/ exists. However if I try the following from app engine end point:
from google.cloud import bigquery
ImportError: No module named google.cloud.bigquery
The file in question includes and that does not appear to help:
from __future__ import absolute_import
Update
I setup a venv and install everything like this: pip install -t lib/ -r requirements.txt --upgrade. From there if I try import google; print google.__path__ I get:
['lib/google', '/usr/lib/google-cloud-sdk/platform/google_appengine/google']
I think the second path might be the cause.
I've reviewed the following with no success:
Error importing Google Cloud Bigquery api module in python app
https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2366
How to import BigQuery in AppEngine for Python
You need to install google-cloud-bigquery into you app's lib dir, that's where the development server is looking at, not on your system's libs. From Installing a third-party library:
Create a directory to store your third-party libraries, such as lib/.
mkdir lib
Use pip (version 6 or later) with the -t <directory> flag to copy the libraries into the folder you created in the previous
step. For example:
pip install -t lib/ <library_name>
How to add third party python libraries in Google App Engine, which are not provided by Google? I am trying to use BeautifulSoup in Google App Engine and unable to do so. But my question is for any library I want to use in Google App Engine.
Google has provided a documented way for included third-party libraries in your GAE project.
See the "Adding Third-party Packages to the Application" section of the Libraries in Python 2.7 docs.
If you want to include additional pure-python third-party packages, you can do so by setting up vendoring. Vendoring allows you to install packages to a subdirectory of your project and include them in your code. To use vendoring, create (or modify) appengine_config.py in the root of your project.
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
And then just put all your libs' source code in your lib dir
> pip install beautifulsoup4 -t lib
So your project directory structure looks like this:
project
- lib
- bs4
- your_code.py
This will allow your project's source files to import libs' packages/modules as if they were added to your PYTHON_PATH. For example:
# file: your_code.py
import bs4 # no need for 'from lib import bs4'
# do stuff with bs4...
You can also easily install everything from a requirements.txt file by doing the following command
> pip install -t lib -r requirements.txt
Actually I think this answer fits better here.
If you want to use 3rd party libraries that are not included in this list, then you'll have to add them manually.
In order to include manually any other library you have to have them inside the directory where the app.yaml lives. So for example if you have the following structure:
hello
├── libs
│ └── bs4
├── hello.py
└── app.yaml
then in your hello.py you have to put these two lines in the beginning of the file:
import sys
sys.path.insert(0, 'libs')
After doing that you'll be able to use any 3rd party library that you're going to put in that libs directory.
For example:
from bs4 import BeautifulSoup
You simply copy the folder containing the library you want to use into your app engine project.
Then when you deploy it's uploaded with your application and is available for use.
EDIT: Jesse's answer is how I now do this. So do that!
The way it worked here is:
import sys
# sys.path.insert(0, 'libs') #"Old" way, not working for me.
sys.path.append(os.path.join(os.path.dirname(__file__), "libs")) # This works!
Then import normally:
from bs4 import BeautifulSoup
Just put Beautifulsoup in the root of your project and upload it all
pip install -t lib package_name
lib: the location for third party libraries
Then you are good to use this package like a normal library you use from ipython or terminal.