I try to launch Google Cloud Function.
io lib is used:
from io import BytesIO
But when I try to add 'io' to requirements.txt function deploy with error.
The mistake happen only when I add this string to the file.
How I can fix it?
Related
I am doing some machine learning project and want to run the project on google colab since my own machine is too weak for it and hangs when i try to run the project on it. My project has the structure as shown in picture .
project structure.
I have multiple .py files each importing modules from one another. I converted the project to .zip file in my pc and then used the upload tab on google colab to upload the project. i unzipped the file and tried to run one code from the "examples" folder which is importing some function from the modAL function like this
from modAL.models import ActiveLearner.
this import is failing on google colab with error " no module named modAL " . Can someone please tell me how to get around this issue? The code works just fine on my own laptop.
I found this explanation: https://zerowithdot.com/colab-workspace/ - very useful.
After creating a space in your google drive
from os.path import join
from google.colab import drive
ROOT = "/content/drive"
drive.mount(ROOT)
fetch the git repo
GIT_USERNAME = "xxx"
GIT_TOKEN = "xxx"
GIT_REPOSITORY = "Repo"
!mkdir "{PROJECT_PATH}"
!git clone https://{GIT_TOKEN}#github.com/{GIT_USERNAME}/{GIT_REPOSITORY}.git "
{PROJECT_PATH}"
Finally use importlib to get access to definitions
from importlib.machinery import SourceFileLoader
somemodule = SourceFileLoader('somelib', join(PROJECT_PATH,
'utils/somelib.py')).load_module()
If the project is public (probably possible to make it work otherwise too) you can create a package [1] and install it with pip:
!pip install git+https://github.com/myuser/myproject
[1] https://packaging.python.org/tutorials/packaging-projects/
I am trying to process an audio file from my S3 through a Lambda function which should call the google speech api to to speech-to-text with the file. However, it doesn't seem like Lambda has this module installed.
Is there a way to import the module externally/manually?
import json
import boto3
import array as arr
import botocore
from botocore.vendored import requests
import os
import urllib
from google.cloud import speech
from google.cloud.speech import enums
from google.cloud.speech import types
def lambda_handler(event, context):
output = 'test'
return output
Expected is that the import works, however the error output is the following:
"errorMessage": "Unable to import module 'lambda_function': No module
named 'google'",
You have to add the google library to your deployment package. Depending on your deployment process, there's various ways how to go about it. The documentation is a good place to start learning about that.
BTW, instead of Google's speech-to-text you can use the one from AWS called Transcribe. You don't have to install any additional library to use that, boto3 will do.
Check the python version used for installing the dependencies and give the same runtime in aws Lambda. I used python 3.8 for installing and gave 3.7 in runtime and ran into this error.
A better way would be to create a file called requirements.txt and add all the dependencies in there
google-cloud-speech==0.36.0
On my understanding, this problem could be solved by adding the required modules as a Layer in Lambda. Then it will be available to your script.
Also, that Layer could be re-used for other Lambdas in the future.
I'm trying to use the Freesound API in a Google Colaboratory notebook (running Python 3) to generate a database of sounds for which to do machine learning with. However I have been unable to use the definitions in a module I imported.
I've looked at other similar questions but they did not seem to address my issue (most were cases of trying to import a module in the standard python library and instead importing a .py file of the same name) and I apologize if this particular issue has been covered somewhere else.
The boilerplate code is as follows:
#clone relevant Git repo
!git clone https://github.com/MoltenMuffins/freesound-python
!ls
#Import packages
import os
import sys
import requests
#Open module file and import module
open('freesound.py','wb')
import freesound
There is some code after that but it is not relevant to the issue. Running this last code block is what gives me the Attribute Error despite FreesoundClient being defined in the freesound.py file cloned from the repo:
freesound_client = freesound.FreesoundClient()
I would greatly appreciate an explanation regarding this issue!
Here's a link to the colabs notebook if you'd like to take a look
I would follow the repo's instructions of using their setup.py to do the installation:
After cloning the git repo, you want to change your working directory to the freesound-python directory and run setup.py
import os
os.chdir('/content/freesound-python')
!python setup.py install
# now import the module
import freesound
I am currently working on a project on Machine Learning on Google Colab which uses Tensorflow API. I created a folder and uploaded it on google drive to run on google Colab.
I successfully mounted the google drive and can run the script,
But when I try importing another module from the script present in the same folder, it throws an error
from . import inference
ImportError: cannot import name 'inference'
I tried finding out a solution for this but found results stating how to import a module directly to the colab notebook.
Please tell me what I am missing here.
Edit:
The folder structure is
-nmt
-nmt.py
-train.py
-inference.py
-utils
-evaluation.py
and so on.
And I am running the python file from the nmt folder. I am getting relative import errors.
Try this. First, upload the .py file and then save it locally. Then you may use it as you like it.
from google.colab import files
src = list(files.upload().values())[0]
open('library_you_want_to_use.py','wb').write(src)
import library_you_want_to_use
I'm using virtualenv with my application, and I've installed gdata, jira, and gspread using env/bin/pip install <lib name> in terminal under my project folder. I'm following the documentation from the Google API but it is not working?
In the documentation, in order to do error handling you need to do:
from gdata import errors
And in order to create an instance of the Drive API service (in order to later on create a file) you need to do:
from gdata.discovery import build
However the files are different, there is no "discovery" or "errors" and when I run env/bin/python run.py I get this error:
Traceback (most recent call last):
File "run.py", line 3, in <module>
from gdata import errors
ImportError: cannot import name errors
(same with discovery)
I thought that maybe they mean from apiclient import errors literally in the documentation, so I tried pip installing apiclient and replacing gdata with apiclient but it still does not work.
I downloaded the gdata.zip file and unzipped it and looked through the sample code (especially for spreadsheet since that's what I'm trying to create) and they take a very different approach than the documentation and I'm very confused. My goal is to use their API to just create a spreadsheet from the code, but I do not plan on using their API to edit the spreadsheet itself, I plan on using gspread (Github).
I've done a lot of research and I've been directed to a lot of different places and I might have perhaps mixed up the code? Does anyone know what I did wrong/have a fix? A huge thanks in advance.
This kind of import error is usually caused by the user installing another module of the same name. Do you by any chance have a gdata.py somehwere on your Python path?
You can verify whether this is causing the issue by:
import gdata
print gdata.__file__
This tells you where the interpreter is loading the code from.