I have installed all the Node.js libraries as required for my nodejs application. But the nodejs application at the backend uses python libraries which has some lines of code such as import nltk and all other libraries. and these import statements when uploaded on a heroku platform is raising an error.
Traceback (most recent call last): File "mainfile.py", line 2, in import functionfile as f File "/app/functionfile.py", line 1, in import nltk ModuleNotFoundError: No module named 'nltk'
I have uploaded all the files but still it is raising an error.
complete code deployed on heroku is available in github link - https://github.com/rinku16/patranker
nodejs application code files is present in app folder
Issue: How can i use this nltk and other python libraries in my application.
Below are the python libraries required in the python file
# Import function file as reference to use defined python functions
import functionfile as f
import sys
import nltk
from nltk.corpus import stopwords
#for generating sentence token
from nltk.tokenize import sent_tokenize
#for generating word token
from nltk.tokenize import word_tokenize
# BeautifulSoup for scraping and Requests to make HTTP requests.
from bs4 import BeautifulSoup
# BeautifulSoup is in bs4 package
import requests
#Counter class For Getting Most Occurred Biwords
from collections import Counter
#FOR MAKING DATA FRAME AND WRITING THE PATENT EXTRACTED DATA TO CSV FILE
from pandas import DataFrame
import pandas as pd
import os
import os.path
from os import path
import re
from datetime import datetime, timedelta
from datetime import date
import math
For example, for nltk library below is path of the file i got
>>> import nltk
>>> nltk.__file__ 'C:\\Users\\Carthaginian\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\nltk\\__init__.py'
how i use this path to clone it in my nodejs application. please see this.
You can create a custom_lib folder and clone all the libraries in it. Provide the correct path using import or require in Node.js project. Use gulp runner or any task runner to copy these custom_lib to your build folder and remember follow the same hierarchy in the build folder structure.
You can also write a custom script to copy these python_lib to the build folder. You can add this custom script in package.json in scripts.
We have the same issue in our project and it work like a charm.
Related
I am attempting to run a script which calls another python file (copied along with its entire github repo), but I am getting a ModuleNotFoundError:
This is despite putting the files into the correct directories.
How I run it
Activate my python3.9 virtual environment
(newpy39) aevas#aevas-Precision-7560:~/Desktop/Dashboard_2021_12$ python3 dashboard.py
where aevas is my username, and ~/Desktop/Dashboard_2021_12 is the folder
No additional arguments required to run it.
Imports for dashboard.py
import sys
sys.path.insert(1, 'targetTrack')
# Qt imports
from PyQt5.QtCore import QThread, pyqtSignal
import argparse
import configparser
import platform
import shutil
import time
import cv2
import torch
import torch.backends.cudnn as cudnn
from yolov5.utils.downloads import attempt_download
from yolov5.models.experimental import attempt_load
from yolov5.models.common import DetectMultiBackend
from yolov5.utils.datasets import LoadImages, LoadStreams
from yolov5.utils.general import LOGGER, check_img_size, non_max_suppression, scale_coords, check_imshow, xyxy2xywh
from yolov5.utils.torch_utils import select_device, time_sync
from yolov5.utils.plots import Annotator, colors
from deep_sort_pytorch.utils.parser import get_config
from deep_sort_pytorch.deep_sort import DeepSort
Part 1: Models not found, despite models being the parent folder.
python3 dashboard.py
Traceback (most recent call last):
File "/home/aevas/Desktop/Dashboard_2021_12/dashboard.py", line 25, in <module>
from trackerThread import trackerDeepSORT
File "/home/aevas/Desktop/Dashboard_2021_12/trackerThread.py", line 15, in <module>
from yolov5.models.experimental import attempt_load
File "/home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5/models/experimental.py", line 14, in <module>
from models.common import Conv
ModuleNotFoundError: No module named 'models'
Part 2: After changing import models.common to import common, it turns out even common cannot be found despite being in the same folder !
python3 dashboard.py
Traceback (most recent call last):
File "/home/aevas/Desktop/Dashboard_2021_12/dashboard.py", line 25, in <module>
from trackerThread import trackerDeepSORT
File "/home/aevas/Desktop/Dashboard_2021_12/trackerThread.py", line 15, in <module>
from yolov5.models.experimental import attempt_load
File "/home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5/models/experimental.py", line 14, in <module>
from common import Conv
ModuleNotFoundError: No module named 'common'
This is how the files are like in the folder models
and this is how the import portion of experimental.py looks like
# YOLOv5 🚀 by Ultralytics, GPL-3.0 license
"""
Experimental modules
"""
import math
import numpy as np
import torch
import torch.nn as nn
from common import Conv
from utils.downloads import attempt_download
I have consulted the following links, but to no avail:
https://towardsdatascience.com/how-to-fix-modulenotfounderror-and-importerror-248ce5b69b1c
Python can't find module in the same folder
ModuleNotFoundError: No module named 'models'
https://github.com/ultralytics/yolov5/issues/353
I understand that I can change it to import .common and then the module can be successfully imported. However, the next line import utils causes a similar error. utils is on the same level as models. Which means there is going to be a cascade of moduleNotFoundError errors at this rate. I also understand using the folder ()method is inadvisable, hence I did not continue with it.
As I had mentioned that I had copied the entire cloned github repo over, when executed standalone, the github repo works perfectly fine. There are no differences between the experimental.py.
What could be wrong?
The problem is that the models module lives in the /home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5 folder, but when you run dashboard.py from /home/aevas/Desktop/Dashboard_2021_12/, the Python interpreter does not look inside the /home/aevas/Desktop/Dashboard_2021_12/targetTrack/yolov5 folder to find modules; it only looks as far as files/folders in /home/aevas/Desktop/Dashboard_2021_12/.
The usual way to fix this would be by installing yolov5 as a package, but it does not appear that yolov5 contains the code necessary to be installed in such a way. To work around this, you need to let Python know that it needs to look inside the yolov5 folder to find the module.
The quick-and-dirty way of doing this is to add the folder to the PYTHONPATH environment variable.
A slightly less quick-and-dirty, but still pretty ugly way of doing it is by dynamically adding the yolov5 folder to sys.path before you try and import the models module. You would do this by adding something like the following to the top of dashboard.py:
from pathlib import Path
import sys
sys.path.append(str(Path(__file__, "..", "targetTrack", "yolov5").resolve()))
I am trying to implement the code in https://github.com/kexinyi/ns-vqa.
However, when I try the command, python tools/preprocess_questions.py \ ... in the section of "Getting started". I see a message No module named 'utils.programs'.
Then I install utils and which makes import utils work, but import utils.programs does not work.
Does anyone have any idea to solve it?
import os
import argparse
import json
import h5py
import numpy as np
import utils.programs as program_utils # this one cannot be imported
import utils.preprocess as preprocess_utils
import utils.utils as utils
Solution:
Add the below lines at the beginning of preprocess_questions.py file.
import sys
sys.path.insert(0, "..")
This should solve your problem.
Explanation:
It is failing because preprocess_questions.py don't know about the path of utils.programs to import. With the above lines added to the path using .., the required file will be imported.
For more on this refer how importing works in python.
I'm very new to deep learning and python and I'm trying to recreate the project at https://github.com/Nagakiran1/Extending-Google-BERT-as-Question-and-Answering-model-and-Chatbot
As I saw that in the project there is a file named Bert_QuestionAnswer.ipynb and with data.txt are the only difference I see from the original Bert repository, I just simply loaded it in my google drive and opened it as a notebook to see it in use.
When I run the first portion dough I get the ModuleNotFoundError: No module named 'modeling'errror.
What library is it part of?
For somoebody this was the problem :
It looks like it's trying to import from the github repo source rather
than the pip package.
If you are running this in a directory that contains the BERT github
repo, try running it elsewhere.
As always many thanks for the help.
This is the code of the file that throws me the error :
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from IPython.core.debugger import set_trace
import collections
import json
import math
import os
import random
import modeling
import optimization
import tokenization
import six
import os
import tensorflow as tf
import logging
logging.getLogger('tensorflow').disabled = True
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import warnings
warnings.filterwarnings("ignore")
import time
from pandas import Series
from nltk.tokenize import sent_tokenize
import gensim.downloader as api
from gensim.parsing.preprocessing import remove_stopwords
word_vectors = api.load("glove-wiki-gigaword-100") # load pre-trained word-vectors from gensim-data
You need to tell python where this module is:
import sys
sys.path.append("/path/to/your/bert/repo")
Because python will search in his system folders and in the current working directory. If you don't run it in the repo, python doesn't find this module.
PEP8 suggests that:
Imports should be grouped in the following order:
standard library imports
related third party imports
local application/library specific imports
You should put a blank line between each group of imports.
I am using Flake8Lint which Sublime Text plugin for lint Python files.
My code as below:
import logging
import re
import time
import urllib
import urlparse
from flask import Blueprint
from flask import redirect
from flask import request
from flask.ext.login import current_user
from flask.ext.login import login_required
from my_application import one_module
it will show the warning as below:
import statements are in the wrong order, from my_application should be before from from flask.ext.login
but flask is the third party library, it should before my my_application import. This is why? How to fix it?
The flake8-import-order plugin needs to be configured to know which names should be considered local to your application.
For your example, if using a .flake8 ini file in your package root directory, it should contain:
[flake8]
application_import_names = my_application
Alternatively you can use only relative imports for application local imports:
from __future__ import absolute_import
import os
import sys
import requests
from . import (
client
)
...
I gave my friends a project. and I use lots of 3rd part packages among the scripts.
How could I generate all the necessary packages automatically.
Otherwise,
my friend should run my script and catch the exception and install the missing package, again and again.
By the way, is it possible to install all the necessary packages at once ?
# -*- coding: utf8 -*-
from flask import request, url_for
from flask import json
from flask import Response
from flask import Flask, request, jsonify
from flask_request_params import bind_request_params
import datetime
import pandas as pd
import pymongo
import pdb
from webargs import Arg
from webargs.flaskparser import use_args, use_kwargs
import urlparse
from mongo import Mongo
import yaml
import time, functools
from functools import wraps
from pdb import set_trace
from flask import g
from pandas_helper import PandasHelper
import errors
That’s what requirement files in pip are for:
"Requirements files" are files containing a list of items to be installed using pip install
Basically they document your dependencies, so when you do a pip install later, pip will get all the required dependencies to run your script.