We Keep Coding

When using brownie I can't use external modules, pip install doesn't work brownie

PROBLEM: Can't import PIL module when using python 3 and brownie
When I run 'brownie run scripts/create_metadata.py --network rinkeby' I get the following error: ModuleNotFoundError: No module named 'PIL'.
The simple solution would be to pip install pillow. The problem is that I do have it installed. Pillow shows up when I run 'pip list' in my cmd.
The error only occures when using brownie, I CAN use the PIL module when running the script regularly in python. But when I run the script using 'brownie run...' I do get the error. Below is the full output and the code.
Any tips would be greatly appreciated.
This is the code for scripts/create_metadata.py:
from brownie import StreamCollectible, network
from metadata.sample_metadata import metadata_template
from pathlib import Path
import requests
import json
import os
import math
import PIL
# from PIL import Image, ImageFont, ImageDraw
PINATA_BASE_URL = "https://api.pinata.cloud/"
endpoint = "pinning/pinFileToIPFS"
headers = {
"pinata_api_key": str(os.getenv("PINATA_API_KEY")),
"pinata_secret_api_key": str(os.getenv("PINATA_API_SECRET")),
}
def main():
stream_collectible = StreamCollectible[-1]
number_of_collectibles = stream_collectible.tokenCounter()
print(f"There are {number_of_collectibles} stream collectibles")
for token_id in range(number_of_collectibles):
metadata_file_name = f"./metadata/{network.show_active()}/{token_id}"
metadata = metadata_template
if Path(metadata_file_name).exists():
print(f"{metadata_file_name} already exists! Delete it to overwrite it")
else:
print(f"Creating Metadata file: {metadata_file_name}")
metadata["name"] = f"Stream Token #{token_id}"
metadata["description"] = ""
img_path = f"./images/{token_id}.jpg"
create_img(token_id)
img_uri = None
if os.getenv("UPLOAD_IPFS") == "true":
img_uri = upload_pinata_IPFS(token_id)
metadata["image"] = img_uri
with open(metadata_file_name, "w") as file:
json.dump(metadata, file)
def upload_pinata_IPFS(file_name):
file_path = f"./images/{file_name}.jpg"
with Path(file_path).open("rb") as fp:
image_binary = fp.read()
response = requests.post(
str(PINATA_BASE_URL + endpoint),
files={"file": (str(file_name), str(image_binary))},
headers=headers,
)
JSON_response = response.json()
URI = f"https://ipfs.io/ipfs/{JSON_response['IpfsHash']}?filename={file_name}.json"
print("JSON RESPONSE")
print(JSON_response)
print("")
print("URI")
print(URI)
return URI
def create_img(n):
width = 17
nList = list(str(n))
bImage = Image.new("L", (width, width), 255)
for i in range(len(nList)):
xAxis = (7 - 2 * (len(nList) - 1)) + 4 * i
nImage = Image.open(f"template/{nList[i]}.png")
bImage.paste(nImage, (xAxis, 6))
if not os.path.exists("./images"):
os.makedirs("./images")
bImage.save(f"./images/{n}.png")
n -= 1
This is the full output:
INFO: Could not find files for the given pattern(s).
Brownie v1.17.2 - Python development framework for Ethereum
Erc721StreamtokenProject is the active project.
File "C:\Users\user\.local\pipx\venvs\eth-brownie\lib\site-packages\brownie\_cli\run.py", line 50, in main
return_value, frame = run(
File "C:\Users\user\.local\pipx\venvs\eth-brownie\lib\site-packages\brownie\project\scripts.py", line 53, in run
module = _import_from_path(script)
File "C:\Users\user\.local\pipx\venvs\eth-brownie\lib\site-packages\brownie\project\scripts.py", line 149, in _import_from_path
_import_cache[import_str] = importlib.import_module(import_str)
File "C:\Program, line line, in in
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File "<frozen, line line, in in
File ".\scripts\create_metadata.py", line 8, in <module>
import PIL
ModuleNotFoundError: No module named 'PIL'

When using brownie installed via pipx, you actually install brownie in it's own virtual environment. When you run brownie, it doesn't look into your locally installed packages. You have to inject your modules into the brownie virtual environment, like so:
pipx inject eth-brownie Pillow
Or whatever your module is

Python ModuleNotFoundError when trying to run main.py

I am pretty new to python and have been working on a data validation program. I am trying to run my main.py file but I am getting the following error.
Traceback (most recent call last):
File "/Users/user/data_validation/main.py", line 5, in <module>
from src.compile_csv import combine_csv_files
File "/Users/user/data_validation/src/compile_csv.py", line 5, in <module>
from helpers.helper_methods import set_home_path
ModuleNotFoundError: No module named 'helpers'
I am using python version 3.94
This is my folder structure
data_validation
- src
- files
validation.csv
- folder_one
combined.csv
- folder_two
combined_two.csv
- helpers
helper_methods.py
compile_csv.py
mysql.py
split.py
main.py
Both compile_csv.py and split.py use methods from helpers.helper_methods.py
My main.py which is throwing the error when being run, looks like the following:
import os
import sys
import earthpy as et
from src.mysql import insert_data, output_non_matches
from src.compile_csv import combine_csv_files
from src.split import final_merged_file
from src.helpers.helper_methods import set_home_path, does_file_exist
home_path = et.io.HOME
file_path = home_path, "data_validation", "src", "files"
folder_name = sys.argv[1]
def configure_file_path():
master_aims_file = os.path.join(
file_path, "validation.csv")
output_csv = os.path.join(
file_path, "output.csv.csv")
gdpr_file_csv = set_home_path(folder_name + "_output.csv")
output_csv = does_file_exist(os.path.join(
file_path, folder_name + "_mismatch_output.csv"))
return output_csv, master_aims_file, gdpr_file_csv
output_csv, master_aims_file, gdpr_file_csv = configure_file_path()
if __name__ == "__main__":
print("🏁 Finding names which do not match name in master file")
if (folder_name == "pd") and (folder_name == "wu"):
final_merged_file()
else:
combine_csv_files()
insert_failures = insert_data(output_csv, master_aims_file)
output_failures = output_non_matches(output_csv, gdpr_file_csv)
if insert_failures or output_failures:
exit(
"⚠️ There were errors in finding non-matching data, read above for more info"
)
os.remove(os.path.join(home_path, "data_validation", "members_data.db"))
exit(
f"✅ mismatches found and have been outputted to {output_csv} in the {folder_name} folder")
From what I understand in python 3 we do not need to use __init__.py and you can use . for defining the path during import, So I am not entirely sure as to what I am doing wrong.
I am executing the file from /Users/user/data_validation and using the following command python main.py pd

There it is. The error is happening in your compile_csv.py file. I'm guessing in that file you have from helpers.helper_methods import blah. But you need to change it to
from .helpers.helper_methods import blah
OR
from src.helpers.helper_methods import blah
Reason being is that imports are relative to cwd not to the file where the code is running. So you need to add the import relative to /Users/user/data_validation.

you can try by setting PYTHONPATH variable. check more about it here

From *folder_name* import *variable* Python 3.4.2

File setup:
...\Project_Folder
...\Project_Folder\Project.py
...\Project_folder\Script\TestScript.py
I'm attempting to have Project.py import modules from the folder Script based on user input.
Python Version: 3.4.2
Ideally, the script would look something like
q = str(input("Input: "))
from Script import q
However, python does not recognize q as a variable when using import.
I've tried using importlib, however I cannot figure out how to import from the Script folder mentioned above.
import importlib
q = str(input("Input: "))
module = importlib.import_module(q, package=None)
I'm not certain where I would implement the file path.

Repeat of my answer originally posted at How to import a module given the full path?
as this is a Python 3.4 specific question:
This area of Python 3.4 seems to be extremely tortuous to understand, mainly because the documentation doesn't give good examples! This was my attempt using non-deprecated modules. It will import a module given the path to the .py file. I'm using it to load "plugins" at runtime.
def import_module_from_file(full_path_to_module):
"""
Import a module given the full path/filename of the .py file
Python 3.4
"""
module = None
try:
# Get module name and path from full path
module_dir, module_file = os.path.split(full_path_to_module)
module_name, module_ext = os.path.splitext(module_file)
# Get module "spec" from filename
spec = importlib.util.spec_from_file_location(module_name,full_path_to_module)
module = spec.loader.load_module()
except Exception as ec:
# Simple error printing
# Insert "sophisticated" stuff here
print(ec)
finally:
return module
# load module dynamically
path = "<enter your path here>"
module = import_module_from_file(path)
# Now use the module
# e.g. module.myFunction()

I did this by defining the entire import line as a string, formatting the string with q and then using the exec command:
imp = 'from Script import %s' %q
exec imp

Python: Import networkx as nx: Global name not defined

I wrote a module (processing_0) in which I import all packages and modules required for my project.
import networkx as nx
import matplotlib.pyplot as plt
import numpy as np
import collections
import la
import csv
import fishery
import re
from collections import OrderedDict
import processing_1
import processing_2
import processing_3
from processing_1 import readingraph, readinpathgraph, preparefisher, inEEG
from processing_2 import pathwayprofile
from processing_3 import checkkin
from fishery import fisher
The modules that I wrote (processing_1/2/3) all require access to networkx (nx).
As part of the master module, I have a the startup function:
def startup():
EEG = readingraph("/.../file1")
EET = readingraph("/.../file2")
EEL = readingraph("/.../file3")
return EEG, EET, EEL
However, after importing processing_0 and trying to run startup() that uses readingraph from processing_1, I keep getting the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "processing_0.py", line 31, in startup
EEG = readingraph("/.../file1")
File "processing_1.py", line 4, in process
graph = nx.read_adjlist(filename)
NameError: global name 'nx' is not defined
Is there any way to globally import networkx as nx and make it accessible to all imported modules?

if you are using linux ubuntu, do these followings in order.
sudo apt-get update
sudo apt-get install python-networkx
go to pycharm env and > file>setting> interpreter and structure to
configure your python env and add packages, there is all packages
that are in env, then click on + to add newpackage
type networkx in search text box, and then select it from package
list and click on install
after finish it , click ok and close windows

In every file that you use networkx you need to import it. So just repeat the line
import networkx as nx
inside the file processing_1.py

keyring module is not included while packaging with py2exe

I am making an app using python 2.7 on windows and keyring-3.2.1 . In my python code on eclipse, I used
import keyring
keyring.set_password("service","jsonkey",json_res)
json_res= keyring.get_password("service","jsonkey")
is working fine as I am storing json response in keyring. But, when I converted python code into exe by using py2exe, it shows import error keyring while making dist. Please suggest how to include keyring in py2exe.
Traceback (most recent call last):
File "APP.py", line 8, in <module>
File "keyring\__init__.pyc", line 12, in <module>
File "keyring\core.pyc", line 15, in <module>
File "keyring\util\platform_.pyc", line 4, in <module>
File "keyring\util\platform.pyc", line 29, in <module>
AttributeError: 'module' object has no attribute 'system'
platform_.py code is :
from __future__ import absolute_import
import os
import platform
def _data_root_Windows():
try:
root = os.environ['LOCALAPPDATA']
except KeyError:
# Windows XP
root = os.path.join(os.environ['USERPROFILE'], 'Local Settings')
return os.path.join(root, 'Python Keyring')
def _data_root_Linux():
"""
Use freedesktop.org Base Dir Specfication to determine storage
location.
"""
fallback = os.path.expanduser('~/.local/share')
root = os.environ.get('XDG_DATA_HOME', None) or fallback
return os.path.join(root, 'python_keyring')
# by default, use Unix convention
data_root = globals().get('_data_root_' + platform.system(), _data_root_Linux)
platform.py code is:
import os
import sys
# While we support Python 2.4, use a convoluted technique to import
# platform from the stdlib.
# With Python 2.5 or later, just do "from __future__ import absolute_import"
# and "import platform"
exec('__import__("platform", globals=dict())')
platform = sys.modules['platform']
def _data_root_Windows():
try:
root = os.environ['LOCALAPPDATA']
except KeyError:
# Windows XP
root = os.path.join(os.environ['USERPROFILE'], 'Local Settings')
return os.path.join(root, 'Python Keyring')
def _data_root_Linux():
"""
Use freedesktop.org Base Dir Specfication to determine storage
location.
"""
fallback = os.path.expanduser('~/.local/share')
root = os.environ.get('XDG_DATA_HOME', None) or fallback
return os.path.join(root, 'python_keyring')
# by default, use Unix convention
data_root = globals().get('_data_root_' + platform.system(), _data_root_Linux)

The issue you're reporting is due to an environment that contains invalid modules, perhaps from an improper installation of one version of keyring over another. You will want to ensure that you've removed remnants of the older version of keyring. In particular, make sure there's no file called keyring\util\platform.* in your site-packages.
After doing that, however, you'll encounter another problem. Keyring loads its backend modules programmatically, so py2exe won't detect them.
To work around that, you'll want to add a 'packages' declaration to your py2exe options to specifically include the keyring.backends package. I invoked the following setup.py script with Python 2.7 to convert 'app.py' (which imports keyring) to an exe:
from distutils.core import setup
import py2exe
setup(
console=['app.py'],
options=dict(py2exe=dict(
packages='keyring.backends',
)),
)
The resulting app.exe will import and invoke keyring.