I'm very new to deep learning and python and I'm trying to recreate the project at https://github.com/Nagakiran1/Extending-Google-BERT-as-Question-and-Answering-model-and-Chatbot
As I saw that in the project there is a file named Bert_QuestionAnswer.ipynb and with data.txt are the only difference I see from the original Bert repository, I just simply loaded it in my google drive and opened it as a notebook to see it in use.
When I run the first portion dough I get the ModuleNotFoundError: No module named 'modeling'errror.
What library is it part of?
For somoebody this was the problem :
It looks like it's trying to import from the github repo source rather
than the pip package.
If you are running this in a directory that contains the BERT github
repo, try running it elsewhere.
As always many thanks for the help.
This is the code of the file that throws me the error :
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from IPython.core.debugger import set_trace
import collections
import json
import math
import os
import random
import modeling
import optimization
import tokenization
import six
import os
import tensorflow as tf
import logging
logging.getLogger('tensorflow').disabled = True
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import warnings
warnings.filterwarnings("ignore")
import time
from pandas import Series
from nltk.tokenize import sent_tokenize
import gensim.downloader as api
from gensim.parsing.preprocessing import remove_stopwords
word_vectors = api.load("glove-wiki-gigaword-100") # load pre-trained word-vectors from gensim-data
You need to tell python where this module is:
import sys
sys.path.append("/path/to/your/bert/repo")
Because python will search in his system folders and in the current working directory. If you don't run it in the repo, python doesn't find this module.
Related
I have installed all the Node.js libraries as required for my nodejs application. But the nodejs application at the backend uses python libraries which has some lines of code such as import nltk and all other libraries. and these import statements when uploaded on a heroku platform is raising an error.
Traceback (most recent call last): File "mainfile.py", line 2, in import functionfile as f File "/app/functionfile.py", line 1, in import nltk ModuleNotFoundError: No module named 'nltk'
I have uploaded all the files but still it is raising an error.
complete code deployed on heroku is available in github link - https://github.com/rinku16/patranker
nodejs application code files is present in app folder
Issue: How can i use this nltk and other python libraries in my application.
Below are the python libraries required in the python file
# Import function file as reference to use defined python functions
import functionfile as f
import sys
import nltk
from nltk.corpus import stopwords
#for generating sentence token
from nltk.tokenize import sent_tokenize
#for generating word token
from nltk.tokenize import word_tokenize
# BeautifulSoup for scraping and Requests to make HTTP requests.
from bs4 import BeautifulSoup
# BeautifulSoup is in bs4 package
import requests
#Counter class For Getting Most Occurred Biwords
from collections import Counter
#FOR MAKING DATA FRAME AND WRITING THE PATENT EXTRACTED DATA TO CSV FILE
from pandas import DataFrame
import pandas as pd
import os
import os.path
from os import path
import re
from datetime import datetime, timedelta
from datetime import date
import math
For example, for nltk library below is path of the file i got
>>> import nltk
>>> nltk.__file__ 'C:\\Users\\Carthaginian\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages\\nltk\\__init__.py'
how i use this path to clone it in my nodejs application. please see this.
You can create a custom_lib folder and clone all the libraries in it. Provide the correct path using import or require in Node.js project. Use gulp runner or any task runner to copy these custom_lib to your build folder and remember follow the same hierarchy in the build folder structure.
You can also write a custom script to copy these python_lib to the build folder. You can add this custom script in package.json in scripts.
We have the same issue in our project and it work like a charm.
I'm trying to run this python project on my linux machine. I did setup everything according to the requirement but when I try to run the project with the ./generate.sh executable file I got the following error.
Import Error: No module name tqdm found.
Here are the imports exists in file.
import os.path as path
import ast
from glob import glob
import signal
import imp
import logging
import time
import numpy as np
import socket
import tqdm
import sys
import click
import tensorflow as tf
from tensorflow.python.client import timeline
I check with pip3 show tqdm command it shows me the package detail. I also try to uninstall and install again the project but got no luck.
If i remove the tqdm import from the file then it shows me this error.
File "./run.py", line 16, in <module>
import click
ImportError: No module named click
Can someone guide me what I'm doing wrong here?
it seems you are importing it wrong, from tqdm docs:
from tqdm import tqdm
I've checked it for both python2 and 3 and this is the way to use it.
The project you are trying to use at least 3 years old, so maybe things have changed since then, so if it wont work for you even with proper import statement you can simply remove it.
Every loop working with tqdm will work without it as well.
For example:
from tqdm import tqdm
for i in tqdm(range(10000)):
pass
is the same as:
for i in range(10000)):
pass
I am trying to implement the code in https://github.com/kexinyi/ns-vqa.
However, when I try the command, python tools/preprocess_questions.py \ ... in the section of "Getting started". I see a message No module named 'utils.programs'.
Then I install utils and which makes import utils work, but import utils.programs does not work.
Does anyone have any idea to solve it?
import os
import argparse
import json
import h5py
import numpy as np
import utils.programs as program_utils # this one cannot be imported
import utils.preprocess as preprocess_utils
import utils.utils as utils
Solution:
Add the below lines at the beginning of preprocess_questions.py file.
import sys
sys.path.insert(0, "..")
This should solve your problem.
Explanation:
It is failing because preprocess_questions.py don't know about the path of utils.programs to import. With the above lines added to the path using .., the required file will be imported.
For more on this refer how importing works in python.
I've been following a tensorflow tutorial https://www.tensorflow.org/official_models/fine_tuning_bert
In the first code snippet, I saw a lot of imports from official module
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import tensorflow_hub as hub
import tensorflow_datasets as tfds
tfds.disable_progress_bar()
from official.modeling import tf_utils
from official import nlp
from official.nlp import bert
# Load the required submodules
import official.nlp.optimization
import official.nlp.bert.bert_models
import official.nlp.bert.configs
import official.nlp.bert.run_classifier
import official.nlp.bert.tokenization
import official.nlp.data.classifier_data_lib
import official.nlp.modeling.losses
import official.nlp.modeling.models
import official.nlp.modeling.networks
And problem is that i found no module name official.
I guess this official module somehow related to problem specific or for BERT model(from tf-hub).
As bert model uses specific text preprocessing and official module is providing this.
So, where i can find, download, use and make imports from this official module to work? I've been using python 3.7, tf-2.2, tf-hub-0.8.0
Please help me out
The official modules of TensorFlow can be found in the TensorFlow Model Garden Repository
I'm currently learning Python and using spyder 3 as editor.
There are several python packages that I use regularly and, to avoid including them in each new script, I put a list of imports in a script file called autoload.py and hoped that by calling autoload.py the packages in question are automatically loaded. Unfortunately this does not work.
To illustrate, the file autoload.py contains:
import pandas as pd
import os
import matplotlib.pyplot as plt
from functools import reduce
import collections as clct
import numpy as np
import platform
Your help will be appreciated.
Simple, autoload.py contains:
import pandas as pd
import os
import matplotlib.pyplot as plt
from functools import reduce
import collections as clct
import numpy as np
import platform
Your file.py contains:
from autoload import *
file.py load autoload.py content, but surely your IDE throw syntax error before running code. If you try to run, works perfectly. I have tested it in PyCharm and it works.
Anyway, I have to say you that is very bad practice.
Regards.
This worked for me in Spyder 3.1.4(Python 3.6):
autoload.py
import sys
import easygui
test.py
import autoload
print('test')
print(easygui.msgbox('Hello'))
sys.exit()
Like others have mentioned, this is probably not a good approach.