We Keep Coding

How to install utils.lib in anaconda to run a jupyter notebook for python?

In a jupyter notebook running on anaconda there is a line "import utils.lib as lib". When I run it, I get the error message "ModuleNotFoundError: No module named 'utils.lib'".
I tried to search for the utils.lib on Internet so that I can install it. But I could not find it. Please let me know how to install it. Thank you -- Manoranjan Dash

You don't provide the code context for the import line. You don't have to always provide the complete context; however, you didn't provide anything and that is why the Bot tried to encourage you to improve things. 'Additional context' would also be important to supply in your post such as the source of the notebook or any related blog post, etc..
The only place I see that import come up on the internet besides your post is here. The code there shows how to get that and install it under 'Downloading the utils and installing':
LIB_DIRECTORY_PATH = DIR+'/utils'
# Check if utils directory already exist, otherwise download, and install
import os
import shutil
if not os.path.isdir(LIB_DIRECTORY_PATH):
if not os.path.isdir(DIR+'/utils'):
os.mkdir(DIR+'/utils')
print('Downloading utils')
user = "ruslanmv"
repo = "Speech-Recognition-with-RNN-Neural-Networks"
src_dir = "utils"
pyfile = "lib.py"
url = f"https://raw.githubusercontent.com/{user}/{repo}/master/{src_dir}/{pyfile}"
!wget --no-cache --backups=1 {url}
print("Installing library...")
shutil.move(DIR+'/lib.py', DIR +'/utils/lib.py')
print("Done.")
Source of that above code: https://ruslanmv.com/blog/Speech-Recognition-with-RNN-Neural-Networks
Code that the code block retrieves and places in the correct location is found at this repo. It isn't something you install. You need to place it alongside the notebok. (Ideally it is set up so you just download or clone the repository, it looks like to me.) The utils directory and it's content is what you need to get or make/copy and place along with your notebook.
Direct link to raw code it gets:
https://raw.githubusercontent.com/ruslanmv/Speech-Recognition-with-RNN-Neural-Networks/master/utils/lib.py

How can I access other files in the same Docker container?

I'm new to Docker. When I am running my docker container, the last line in the Dockerfile is the following:
CMD ["python3", "./poses/server/server_multithreaded.py"]
In the server_multithreaded.py file described above, I am importing another file, as seen below:
from poses.poseapp.poseapp_sockets import PoseAppWSockets
When I run container using the command docker run -p 8089:8089 zoheezus/capstone I get the following error:
Traceback (most recent call last):
File "./poses/server/server_multithreaded.py", line 18, in <module>
from poses.poseapp.poseapp_sockets import PoseAppWSockets
ImportError: No module named 'poses'
From what I understand, the 'poses' directory is not accessible or I am not accessing it the right way. What do I need to do for server_multithreaded.py to be able to access the other files when I run it?
The file structure of the project is the following:

Python won't add current working path into module search path, it will just add top level script's path into search path, of course PYTHONPATH, sys path etc. also be searched.
For your case, the current working path /usr/pose_recognizer won't be searched, only /usr/pose_recognizer/poses/server will be searched. So, definitely, it won't find module named 'poses'.
To make it work for you, give you some options:
Option 1: Execute the server_multithreaded.py as module:
python -m poses.server.server_multithreaded
Option 2: Change sys.path in server_multithreaded.py as next before from poses.poseapp.poseapp_sockets import PoseAppWSockets:
import sys
import os
file_path = os.path.abspath(os.path.dirname(__file__)).replace('\\', '/')
lib_path = os.path.abspath(os.path.join(file_path, '../..')).replace('\\', '/')
sys.path.append(lib_path)
Option 3: Change PYTHONPATH in dockerfile:
WORKDIR /usr/pose_recognizer
ENV PYTHONPATH=.
CMD ["python3", "./poses/server/server_multithreaded.py"]

this problem is most likely not related to docker but to your PYTHONPATH environment variable (inside the docker container). You must make sure that capstone-pose-estimation is in it. Furthermore, you should make poses and poseapp a package (containing __init___.py) in order to import from it

ModuleNotFoundError - Airflow error while import Python file

I created a very simple DAG to execute a Python file using PythonOperator. I'm using docker image to run Airflow but it doesn't recognize a module where I have my .py file
The structure is like this:
main_dag.py
plugins/__init__.py
plugins/njtransit_scrapper.py
plugins/sql_queries.py
plugins/config/config.cfg
cmd to run docker airflow image:
docker run -p 8080:8080 -v /My/Path/To/Dags:/usr/local/airflow/dags puckel/docker-airflow webserver
I already tried airflow initdb and restarting the web server but it keeps showing the error ModuleNotFoundError: No module named 'plugins'
For the import statement I'm using:
from plugins import njtransit_scrapper
This is my PythonOperator:
tweets_load = PythonOperator(
task_id='Tweets_load',
python_callable=njtransit_scrapper.main,
dag=dag
)
My njtransit_scrapper.py file is just a file that collects all tweets for a tweeter account and saves the result in a Postgres database.
If I remove the PythonOperator code and imports the code works fine. I already test almost everything but I'm not quite sure if this is a bug or something else.
It's possible that when I created a volume for the docker image, it's just importing the main dag and stopping there causing to not import the entire package?

To help others who might land on this page and get this error because of the same mistake I did, I will record it here.
I had an unnecessary __init__.py file in dags/ folder.
Removing it solved the problem, and allowed all the dags to find their dependency modules.

How to run a Python script in a '.py' file from a Google Colab notebook?

%%javascript
IPython.OutputArea.prototype._should_scroll = function(lines) {
return false;
}
%run rl_base.py
I run this giving error saying rl_base.py file not found. I have uploaded the same to gdrive in colab and from the same folder I am running my .ipynb file, containing the above code

If you have the test.py file in the corresponding folder in drive as in the below attached image, then the command which you use to run the test.py file is as mentioned below,
!python gdrive/My\ Drive/Colab\ Notebooks/object_detection_demo-master/test.py
Additional Info:
If you jusst want to run !python test.py then you should change directory, by the following command before it,
%cd gdrive/My\ Drive/Colab\ Notebooks/object_detection_demo-master/

When you run your notebook from Google drive, an instance is created only for the notebook. To make the other files in your Google drive folder available you can mount your Google drive with:
from google.colab import drive
drive.mount('/content/gdrive')
Then copy the file you need into the instance with:
!cp gdrive/My\ Drive/path/to/my/file.py .
And run your script:
!python file.py

You should not upload to gdrive. You should upload it to Colab instead, by calling
from google.colab import files
files.upload()

## 1. Check in which directory you are using the command
!ls
## 2.Navigate to the directory where your python script(file.py) is located using the command
%cd path/to/the/python/file
## 3.Run the python script by using the command
!python file.py

A way is also using colabcode.. You will have full ssh access with Visual Studio Code editor.
# install colabcode
!pip install colabcode
# import colabcode
from colabcode import ColabCode
# run colabcode with by deafult options.
ColabCode()
# ColabCode has the following arguments:
# - port: the port you want to run code-server on, default 10000
# - password: password to protect your code server from being accessed by someone else. Note that there is no password by default!
# - mount_drive: True or False to mount your Google Drive
!ColabCode(port=10000, password="abhishek", mount_drive=True)
It will prompt you with a link to visual studio code editor with full access to your colab directories.

Here is a simple answer along with a screenshot
Mount the google drive
from google.colab import drive
drive.mount('/content/drive')
Call the py file path
import sys
import os
py_file_location = "/content/drive/MyDrive/Colab Notebooks"
sys.path.append(os.path.abspath(py_file_location))

It seems necessary to put the .py file's name in ""
!python "file.py"

python import error "No module named appengine.ext"

after runing this code,I found import error:-
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
class MainPage(webapp.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Hello, webapp World!')
application = webapp.WSGIApplication([('/', MainPage)],debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
how to use google.apengine.ext

import sys
sys.path.insert(1, '/Users/<username>/google-cloud-sdk/platform/google_appengine')
sys.path.insert(1, '/Users/<username>/google-cloud-sdk/platform/google_appengine/lib/yaml/lib')
sys.path.insert(1, 'lib')
if 'google' in sys.modules:
del sys.modules['google']
this solves the problems for me

It looks like the App Engine SDK is not installed, or at least the Python runtime cannot find it.
read and follow the instructions here: https://cloud.google.com/appengine/downloads#Google_App_Engine_SDK_for_Python
They tell you, how to install App Engine SDK for Python.

Try:
import google
print google.__path__
to see what exactly you're importing.

I had this same issue because I pip installed gcloud before downloading and installing the SDK. The pip install created a python package google which didn't contain the appengine submodule (which is found in the SDK folder). I uninstalled the gcloud and related packages. Then just pip installed the google-cloud-bigquery which is the only package I needed from gcloud. Everything works fine now.

I faced similar error while calling Google Analytics API using AWS Lambda.
Workaround from (Schweigi1) helped me.
import googleapiclient
from googleapiclient.discovery_cache.base import Cache
class MemoryCache(Cache):
_CACHE = {}
def get(self, url):
return MemoryCache._CACHE.get(url)
def set(self, url, content):
MemoryCache._CACHE[url] = content
Usage:
service = googleapiclient.discovery.build("analyticsreporting", "v4", http=http, credentials=credentials,cache=MemoryCache())
Hope this helps someone who is facing this issue in AWS Lambda.

First possible reason:
you don't install the python library in google cloud sdk, so you can run in cmd (as administrator):
gcloud components install app-engine-python.
Second possible reason:
your IDE is not success get into google libraries, they exist in:
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine
or in:
C:\Users\[your user]\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine
You can see in attached link explain how to add these libraries to IDE external libraries: https://stackoverflow.com/a/24206781/8244338

I got this error in python:
from google.appengine.api import search
ImportError: No module named appengine.api
I thought this would be something along the similar lines of what is happening in this thread.
So, my solution was to run "dev_appserver.py 'your yaml file' ". I got this solution following the below links:
1) https://cloud.google.com/appengine/docs/standard/python/tools/using-local-server
2) https://www.youtube.com/watch?v=BdqUY8lCuBI
Hope this helps!

check if you named some file google.py :) in the same package, because this can shadow the import of google.appengine.ext. I had the same error:
python import error “No module named appengine.ext”
and deleteting the file solved the problem.