problem
I am trying to publish and use a package I created, here is the directory structure
my_package_folder
|
|-build
|-dist
|-setup.py
|-my_package_name
||
||-my_package_file.py
||-__init__.py
||
||-request_folder
|||
|||-request_file.py
|||-__init__.py
setup.py
VERSION = '1.0.0'
DESCRIPTION = 'api package'
LONG_DESCRIPTION = 'longer package description'
# Setting up
setup(
name="my_package_settings_name",
version=VERSION,
author="John Smith",
author_email="<email#email.com>",
url='https://github.com/the-project/project-name',
description="api client package",
long_description="api client",
packages=find_packages(),
install_requires=[],
keywords=['python'],
classifiers=[
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Operating System :: MacOS :: MacOS X",
"Operating System :: Microsoft :: Windows",
]
)
my_package_file.py
from request_folder.request_file import Request
class Api:
def __init__(self): #not important implementation details
request.py
import json
import urllib.parse
import urllib.request
from urllib.request import urlopen
class Request:
def __init__(self): # unimportant implementation details
Then I run the following commands:
python setup.py sdist bdist_wheel
twine upload dist/*
this uploads the package to pypi and succeeds. However, when I try and download and use my package, i am unable to import and use the code the import fails
pip install my_package_settings_name==1.0.0
script.py
from my_package_settings_name import Api
the import fails to find the package. I am not the best at python so I think there must be something small I am missing. Any help or suggestions would be appreciated.
The problem that is confusing you is that your package name is actually different from the python modules that it contains. You've uploaded "my_package_settings_name", which contains the module my_package_name. Thus, assuming that your Api class is defined inside of my_package_name/init.py, you should use:
from my_package_name import Api
Related
I am trying to install a locally built Python library and install it on SQL server. I have managed to make the wheel file using setuptools and install it on the sql server. The wheel set up looks like this.
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="Q",
version="0.0.1",
author="Ben Watson",
author_email="ben.watson#XXXXX.com",
description="Package for SQL server that .....",
long_description=long_description,
long_description_content_type="text/markdown",
license_files = ('LICENSE.txt',),
#packages=setuptools.find_packages(),
packages=setuptools.find_namespace_packages(),
classifiers=[
"Programming Language :: Python :: 3.7.1",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.7.1',
)
I can see that the install worked. Tested this and get the following output.
STDOUT message(s) from external script: Package Q is version 0.0.1
I now try and run the code as such;
EXECUTE sp_execute_external_script
#language = N'Python',
#script = N'
import Q as Qtest
x=Qtest.test()
print(x)
'
The issue is that locally running the file throws the error;
ModuleNotFoundError: No module named 'Q'
Not sure what I am missing here. Most of the help online point to a path issue. SQL Server has specific requirements for the path, so I am reluctant to alter this in any way.
Regards
Ben
I have a package which I have written and installed onto a virtual environment in editable mode. I can only import from this package only when the modules and items within those modules are imported using the 'from' syntax.
In a python file outside the package, I can import specific modules from the package using from package import module and import specific functions/objects from these modules via from package.module import x in external scripts/python interpreter.
However, when I try to import the whole module, I find that the package has no accessible modules; i.e. if I were to write:
import package
x = package.module.x
Then I would receive the error:AttributeError: module 'package' has no attribute 'module'.
Intriguingly , if I use a from import and then attempt the same command again, the error does not occur and the attribute, object or function 'x' imports properly.
I believe that the problem should have something to do with how the package is found in the setup.py, but I don't know enough about python packaging to understand what is going on
I have this module installed in editable mode through pip on my anaconda virtual environment, and it uses the standard cookiecutter python format. There are no sub-packages, and the init.py contains only basic bibliographic information ('_name_ ' and '_email_' and so on).
here is my setup:
"""The setup script."""
from setuptools import setup, find_packages
with open('README.rst') as readme_file:
readme = readme_file.read()
with open('HISTORY.rst') as history_file:
history = history_file.read()
requirements = ['Click>=7.0', ]
test_requirements = ['pytest>=3', ]
setup(
author="Alexander Pasha",
author_email={email},
python_requires='>=3.9',
classifiers=[
'Development Status :: 2 - Pre-Alpha',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Natural Language :: English',
'Programming Language :: Python :: 3.9',
],
description={description},
install_requires=requirements,
license="MIT license",
long_description=readme + '\n\n' + history,
include_package_data=True,
keywords={name},
name={name},
packages=find_packages(include=['package', 'package.*']),
test_suite='tests',
tests_require=test_requirements,
url={GitHub},
version='0.1.0',
zip_safe=False,
)
for reference, I'm using python 3.9.12
For many python libraries, the argument used with import is the same as the one used to install the library with pip.
E.g.
pip install numpy
pip install scipy
pip install pandas
correspond to
import numpy
import scipy
import pandas
but this pattern doesn't seem to work for all libraries. E.g. (found here):
pip install Pillow
is required to get this to succeed
import PIL
Based on the pattern in the first examples, I would have expected pip install PIL to install PIL, but instead we use pip install Pillow. Why is this and how does this work?
Basically, what you import is usually the module name. For example, your package might be developed in the following hierarchy:
MyLib
- __init__.py
- my_script1.py
- my_script2.py
However, when you make your library as a "package" available in pip, usually you will need to prepare your setup.py file, which will be automatically run when people use pip install to install your package.
The setup.py can be something like this:
from distutils.core import setup
setup(
name = 'YOURPACKAGENAME', # How you named your package folder (MyLib)
packages = ['YOURPACKAGENAME'], # Chose the same as "name"
version = '0.1', # Start with a small number and increase it with every change you make
license='MIT', # Chose a license from here: https://help.github.com/articles/licensing-a-repository
description = 'TYPE YOUR DESCRIPTION HERE', # Give a short description about your library
author = 'YOUR NAME', # Type in your name
author_email = 'your.email#domain.com', # Type in your E-Mail
url = 'https://github.com/user/reponame', # Provide either the link to your github or to your website
download_url = 'https://github.com/user/reponame/archive/v_01.tar.gz', # I explain this later on
keywords = ['SOME', 'MEANINGFULL', 'KEYWORDS'], # Keywords that define your package best
install_requires=[ # I get to this in a second
'validators',
'beautifulsoup4',
],
classifiers=[
'Development Status :: 3 - Alpha', # Chose either "3 - Alpha", "4 - Beta" or "5 - Production/Stable" as the current state of your package
'Intended Audience :: Developers', # Define that your audience are developers
'Topic :: Software Development :: Build Tools',
'License :: OSI Approved :: MIT License', # Again, pick a license
'Programming Language :: Python :: 3', #Specify which pyhton versions that you want to support
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
],
)
Therefore, in the above example, people who install your package via pip should run pip install YOURPACKAGENAME. After that, they need to run import MyLib in the code.
TD; DL:
What you import is a module name, but what you installed via pip is the name of the package, they can be different. But usually, I would say that I like people to use the same name for both to avoid any confusing.
Ref:
https://medium.com/#joel.barmettler/how-to-upload-your-python-package-to-pypi-65edc5fe9c56
I'm deploying my first python project, but having issues with installation. I've followed the practices outlined in https://packaging.python.org/guides/distributing-packages-using-setuptools/#uploading-your-project-to-pypi. My project is organized with a top-level executable script bin/gsat that calls imports other modules like so:
import gsat.input_validation as input_validation
The modules are in src/gsat/ , following the arrangement in the example project at https://github.com/pypa/sampleproject
If I install locally from the project source , using develop mode:
pip install -e .
... then I have no issues installing and the software works.
But if I install it from PyPI:
pip install "gsat"
... then it won't run because the import statements fail to find the modules. Error:
File "/Library/Frameworks/Python.framework/Versions/3.7/bin/gsat", line 10, in <module>
import gsat.input_validation as input_validation
ModuleNotFoundError: No module named 'gsat'
The full project is at https://github.com/MikeAxtell/gsat , commit c680172. The project is also on PyPI as "gsat". The distribution files are being made like:
python3 setup.py sdist bdist_wheel
... and the fun setup.py file is below. I'm sure this is some noob issue; I am new to python packaging and python programming in general, so thanks in advance for help!
setup.py:
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="gsat",
version="0.1a",
author="Michael J. Axtell",
author_email="mja18#psu.edu",
description="General Small RNA-seq Analysis Tool",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/MikeAxtell/gsat",
scripts=['bin/gsat'],
package_dir={'': 'src'},
packages=setuptools.find_packages(where='gsat'),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>3.5, <4',
install_requires=['biopython','python-Levenshtein'],
)
Looks like there is an issue in this line:
packages=setuptools.find_packages(where='gsat'),
I believe it should read the following instead:
packages=setuptools.find_packages(where='src'),
I have published a module to pypi called 'surrealism' that generates surreal sentences and error messages. It contains a SQLite3 database containing all of the words and sentences needed for my module.
All of the following install methods work fine:
python setup.py install
pip install surrealism
easy_install surrealism
and the module works fine.
However, when installing into a virtualenv, things go wrong. surrealism.py get installed into C:\Users\me\virtualenvs\surrealism\Lib\site-packages, but surrealism.sqlite doesn't get installed?
If I run python and try and import the module, my module creates a new sqlite3 database called surrealism.sqlite at C:\Users\me\virtualenvs\surrealism
The contents of my setup.py follows:
#!/usr/bin/env python
from setuptools import setup
long_desc = open('readme.rst').read()
setup(name = 'surrealism',
version = '0.5.2',
py_modules = ['surrealism'],
author = 'Morrolan',
author_email = 'morrolan#icloud.com',
url = 'https://github.com/Morrolan/surrealism',
license = 'GNU General Public License (GPL)',
description = 'Surreal sentence and error message generator.',
long_description = long_desc,
platforms = ['Windows','Unix','OS X'],
download_url = "https://pypi.python.org/pypi/surrealism/",
keywords = ["surreal", "surrealism", "error message"],
classifiers = [
"Programming Language :: Python",
"Programming Language :: Python :: 2.6",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3.3",
"Development Status :: 4 - Beta",
"Environment :: Other Environment",
"Intended Audience :: Developers",
"Intended Audience :: Education",
"License :: OSI Approved :: GNU General Public License (GPL)",
"Operating System :: OS Independent",
"Topic :: Education",
"Topic :: Software Development :: Libraries :: Python Modules",
],
install_requires=['setuptools'],
)
In surrealism.py, I reference/connect to the SQLite3 database in a fairly rudimentary way:
CONN = sqlite3.connect('surrealism.sqlite')
But so far it hasn't caused me any problems.
is there a more explicit way to reference surrealism.sqlite, or do I have to specify something in setup.py to force the installation?
Kind Regards,
Morrolan
the crucial problem is exactly the way you are connecting to your sqlite database; That will refer to a file in the current directory;, anywhere the program that invokes it is trying to run. What you want to say is
... sqlite3.connect(where_this_python_lib_is_installed + '...sqlite')
So that it doesn't matter where it's installed. There is a fairly standard way to do that, using the pkg_resources library. Since we're trying to discover a sqlite database, that means we need a real file on disk, not a string or file-like object; so the right method to use here pkg_resources.resource_filename, we just need to change the connect call to:
from pkg_resources import resource_filename
CONN = sqlite3.connect(resource_filename(__name__, 'surrealism.sqlite'))
But wait... That only works if the package data is in a package, but you currently have a module. Not a big problem, though; we'll rename surrealism.py to surrealism/__init__.py, surrealism.sqlite to surrealism/surrealism.sqlite, and make the appropriate changes in MANIFEST.in. We'll also need to tell setuptools about this. Change py_modules=["surrealism"], in your setup.py to packages=["surrealism"].
Almost there, The last thing we need to do is get setuptools to actually install that file from source. The first is pretty obvious, we need to tell it which files to copy; Add
package_data={'surrealism': ['surrealism.sqlite']},
To your setup.py, the second change is more subtle. In most cases, setuptools tries to install packages as zip files. This is usually a good thing; but in our case, we need to pass the filename of a real file to sqlite.connect, so we have to tell it not to try to zip the package. For that, just add
zip_safe=False,
To your setup.py.