We Keep Coding

SQL Server 2017. ModuleNotFoundError: No module named 'Q'

I am trying to install a locally built Python library and install it on SQL server. I have managed to make the wheel file using setuptools and install it on the sql server. The wheel set up looks like this.
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="Q",
version="0.0.1",
author="Ben Watson",
author_email="ben.watson#XXXXX.com",
description="Package for SQL server that .....",
long_description=long_description,
long_description_content_type="text/markdown",
license_files = ('LICENSE.txt',),
#packages=setuptools.find_packages(),
packages=setuptools.find_namespace_packages(),
classifiers=[
"Programming Language :: Python :: 3.7.1",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.7.1',
)
I can see that the install worked. Tested this and get the following output.
STDOUT message(s) from external script: Package Q is version 0.0.1
I now try and run the code as such;
EXECUTE sp_execute_external_script
#language = N'Python',
#script = N'
import Q as Qtest
x=Qtest.test()
print(x)
'
The issue is that locally running the file throws the error;
ModuleNotFoundError: No module named 'Q'
Not sure what I am missing here. Most of the help online point to a path issue. SQL Server has specific requirements for the path, so I am reluctant to alter this in any way.
Regards
Ben

How to get rid of library root mark in PyCharm

I've started working on a Pyramid application, which requires setup.py usage, but as soon as I built the app, my app folder was marked as library root.
It is not convenient, because when I open a file, it is also opened under External Libraries unfolding it. This can be "fixed" by removing check on Always Select Opened File, but I like this feature, so I don't want to disable it.
I've also tried to tweak Project Structure in settings, but it didn't help.
How to get rid of this _library_root_ mark?
UPD. Contents of setup.py:
setup(
name='app',
version=0.1,
description='Blog with CMS',
classifiers=[
"Programming Language :: Python",
"Framework :: Pylons",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Internet :: WWW/HTTP :: WSGI :: Application"
],
keywords="web services",
author='',
author_email='',
url='',
packages=find_packages(),
include_package_data=True,
zip_safe=False,
install_requires=['cornice', 'waitress'],
entry_points="""\
[paste.app_factory]
main=app:main
""",
paster_plugins=['pyramid']
)

The reason why it happens is PyCharm adds sd-blog/app to the interpreter paths once you pip install -e in order to be able to provide completion for app objects. Possible workaround:
remove app from the interpreter paths
mark sd-blog/app as a source root to restore the code insight broken in the step 1

Modules missing when installing python project using pip

I'm deploying my first python project, but having issues with installation. I've followed the practices outlined in https://packaging.python.org/guides/distributing-packages-using-setuptools/#uploading-your-project-to-pypi. My project is organized with a top-level executable script bin/gsat that calls imports other modules like so:
import gsat.input_validation as input_validation
The modules are in src/gsat/ , following the arrangement in the example project at https://github.com/pypa/sampleproject
If I install locally from the project source , using develop mode:
pip install -e .
... then I have no issues installing and the software works.
But if I install it from PyPI:
pip install "gsat"
... then it won't run because the import statements fail to find the modules. Error:
File "/Library/Frameworks/Python.framework/Versions/3.7/bin/gsat", line 10, in <module>
import gsat.input_validation as input_validation
ModuleNotFoundError: No module named 'gsat'
The full project is at https://github.com/MikeAxtell/gsat , commit c680172. The project is also on PyPI as "gsat". The distribution files are being made like:
python3 setup.py sdist bdist_wheel
... and the fun setup.py file is below. I'm sure this is some noob issue; I am new to python packaging and python programming in general, so thanks in advance for help!
setup.py:
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="gsat",
version="0.1a",
author="Michael J. Axtell",
author_email="mja18#psu.edu",
description="General Small RNA-seq Analysis Tool",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/MikeAxtell/gsat",
scripts=['bin/gsat'],
package_dir={'': 'src'},
packages=setuptools.find_packages(where='gsat'),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>3.5, <4',
install_requires=['biopython','python-Levenshtein'],
)

Looks like there is an issue in this line:
packages=setuptools.find_packages(where='gsat'),
I believe it should read the following instead:
packages=setuptools.find_packages(where='src'),

pip missing dependencies in setup.py

I'm trying to use pip to get an environment set up, and running pip install -e ./ from the root directory of my project isn't picking everything up. I've got a setup.py file with a requires section that looks like so:
requires = [
'phonenumbers',
'inflect',
'repoze.sendmail==4.1',
'pyramid',
'pyramid_chameleon',
'pyramid_debugtoolbar',
'pyramid_mailer',
'pyramid_tm',
'transaction',
'zope.sqlalchemy',
'waitress',
'pyramid_beaker',
'cryptacular',
'pycrypto',
'webtest',
'alembic',
'psycopg2',
'python-dateutil',
'sqlalchemy-utils',
'cryptacular',
'arrow',
'jsonpickle',
'sqlalchemy',
'pyramid_storage',
'boto',
'requests'
]
When the command is run, some libraries, such as boto, won't be installed. Does anyone know why these packages would be missed?
Edit: Here's the call to setup in setup.py, with some irrelevent bits ommited:
dependency_links = [
'git+https://github.com/benthor/inflect.py#egg=inflect',
]
setup(
classifiers=[
"Programming Language :: Python",
"Framework :: Pyramid",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Internet :: WWW/HTTP :: WSGI :: Application",
],
author='',
author_email='',
url='',
keywords='web wsgi bfg pylons pyramid',
packages=find_packages(),
include_package_data=True,
zip_safe=False,
test_suite='test',
install_requires=requires,
dependency_links=dependency_links
)

The requires argument to setup() doesn't actually do anything, and for all intents and purposes should be considered deprecated and useless.
Use install_requires instead.

How to specify install location of additional files during the creation of module

I have published a module to pypi called 'surrealism' that generates surreal sentences and error messages. It contains a SQLite3 database containing all of the words and sentences needed for my module.
All of the following install methods work fine:
python setup.py install
pip install surrealism
easy_install surrealism
and the module works fine.
However, when installing into a virtualenv, things go wrong. surrealism.py get installed into C:\Users\me\virtualenvs\surrealism\Lib\site-packages, but surrealism.sqlite doesn't get installed?
If I run python and try and import the module, my module creates a new sqlite3 database called surrealism.sqlite at C:\Users\me\virtualenvs\surrealism
The contents of my setup.py follows:
#!/usr/bin/env python
from setuptools import setup
long_desc = open('readme.rst').read()
setup(name = 'surrealism',
version = '0.5.2',
py_modules = ['surrealism'],
author = 'Morrolan',
author_email = 'morrolan#icloud.com',
url = 'https://github.com/Morrolan/surrealism',
license = 'GNU General Public License (GPL)',
description = 'Surreal sentence and error message generator.',
long_description = long_desc,
platforms = ['Windows','Unix','OS X'],
download_url = "https://pypi.python.org/pypi/surrealism/",
keywords = ["surreal", "surrealism", "error message"],
classifiers = [
"Programming Language :: Python",
"Programming Language :: Python :: 2.6",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3.3",
"Development Status :: 4 - Beta",
"Environment :: Other Environment",
"Intended Audience :: Developers",
"Intended Audience :: Education",
"License :: OSI Approved :: GNU General Public License (GPL)",
"Operating System :: OS Independent",
"Topic :: Education",
"Topic :: Software Development :: Libraries :: Python Modules",
],
install_requires=['setuptools'],
)
In surrealism.py, I reference/connect to the SQLite3 database in a fairly rudimentary way:
CONN = sqlite3.connect('surrealism.sqlite')
But so far it hasn't caused me any problems.
is there a more explicit way to reference surrealism.sqlite, or do I have to specify something in setup.py to force the installation?
Kind Regards,
Morrolan

the crucial problem is exactly the way you are connecting to your sqlite database; That will refer to a file in the current directory;, anywhere the program that invokes it is trying to run. What you want to say is
... sqlite3.connect(where_this_python_lib_is_installed + '...sqlite')
So that it doesn't matter where it's installed. There is a fairly standard way to do that, using the pkg_resources library. Since we're trying to discover a sqlite database, that means we need a real file on disk, not a string or file-like object; so the right method to use here pkg_resources.resource_filename, we just need to change the connect call to:
from pkg_resources import resource_filename
CONN = sqlite3.connect(resource_filename(__name__, 'surrealism.sqlite'))
But wait... That only works if the package data is in a package, but you currently have a module. Not a big problem, though; we'll rename surrealism.py to surrealism/__init__.py, surrealism.sqlite to surrealism/surrealism.sqlite, and make the appropriate changes in MANIFEST.in. We'll also need to tell setuptools about this. Change py_modules=["surrealism"], in your setup.py to packages=["surrealism"].
Almost there, The last thing we need to do is get setuptools to actually install that file from source. The first is pretty obvious, we need to tell it which files to copy; Add
package_data={'surrealism': ['surrealism.sqlite']},
To your setup.py, the second change is more subtle. In most cases, setuptools tries to install packages as zip files. This is usually a good thing; but in our case, we need to pass the filename of a real file to sqlite.connect, so we have to tell it not to try to zip the package. For that, just add
zip_safe=False,
To your setup.py.