ImportError python module - python

I have a python module that I have added a new file to. When I try to import the new file as
from my_module.new_file import new_class
it throws up an import error, where as the same thing works fine for the original files that were in there.
I'm sure this is down to the module being cached somewhere, and not updating the cache (if I remove the module folder completely importing the original code still works).
Can someone tell me where python 3.6 caches modules, how to remove the cached module, or force python to load the new file into the module.

Found the answer:
import importlib
importlib.reload(my_module)

Related

Python: Sqlalchemy messing up pyinstaller?

I am trying to package my program using pyinstaller. The code runs fine on windows, and uses SqlAlchemy, OpenCV and pyodbc packages.
I ran pyinstaller to create the executable and tried to run it. I'm getting an error:
ImportError: No module named ConfigParser
now, I reran the same thing and looked at logs from pyinstaller and got a warning:
WARNING: Hidden import "sqlalchemy.sql.functions.func" not found!
along with a few others.
then there was a warning about trying to import ConfigParser in lower and uppercase.
Attempted to add Python module twice with different upper/lowercases: ConfigParser
What might be the issue here?
So, I figured it out. To an extent.
Seems like pyInstaller doesn't deal with SWIG files that well.
In sqlalchemy.utils there's a file called compat.py. It is there to make the module compatible with all versions of python.
for example, in python2.x, there's ConfigParser whereas in py3, it is named configparser
So there is a part in compat.py to deal with it:
if py3:
import configparser
# Some other such import statements
elif py2:
import ConfigParser as configparser
Now, pyinstaller gets stumped here as it just focuses on the import, and hence it tries to import both and fails miserably.
My crude workaround to this involved modifying the compat.py file and retaining only the parts relevant to the python version I have (2.x).
Running pyinstaller again proved to be a success! :)
Although this is all very crude and there's probably something better out there, but I couldn't find anything, so I'm sharing what worked for me.

python. package works in folder A but not in folder B

A noob python question.
I have a package called TorCtl, I saved it in /root/A and create a file (useIt.py) in /root/A which uses the package and it works absolutely fine.
now the problem starts when I try to use this package within another project.
I copied TorCtl package into /root/B/C and created a new file (useIt2.py) (this file is used from another file that is the main for this package.)
when I run main I get import error from TorCtl package. for example:
from elixir import *
ImportError: No module named elixir
I have no idea why it should work in one folder and not another.
Can anyone point out what I am doing wrong please?

Creating (not distributing) and importing Python 3.3/4 packages vs Python 2.7

Okay, so in the past, I've made my own Python packages with Python 2.x (most recently, 2.7.5). It has worked fine. Let me explain how I did that, for reference:
Make a directory within the working directory. We'll call it myPackage.
Make a file called __init__.py in the directory myPackage.
Make sure all the modules that you want to be part of the package are imported within __init__.py. These modules are typically in the myPackage folder.
From a Python program in the working directory, type import myPackage (and it imports fine, and is usable).
However, in Python 3, I get errors with that. (ImportError: No module named 'Whatever the first imported module is')
I researched the problem and found the following:
Starred imports don't work in Python 3.3.
The __init__.py file is not required in Python 3.3.
So, I removed the stars from my imports, and leaving the __init__.py file in, I still got errors (ImportError: No module named 'Whatever the first imported module is'). So, I removed the __init__.py file, and I don't get any errors, but my package doesn't include any of my modules.
Okay, so I discovered by doing a web search for python3 __init__.py or some such that I can do the following, although I don't have any clue if this is the standard way of doing things:
In the modules in the package, make sure there are no plain imports (not just no starred ones). Only do from myModule import stuff. However, you need to put a . in front of myModule: e.g. from .myModule import stuff. Then, I can import myPackage.oneOfMyModules
I found that by following this rule in the __init__.py file, it also works.
Once again, I don't know if this is how it's supposed to work, but it seems to work.
I found this page that is supposed to have something to do with the changes in packages in Python 3.something, but I'm not sure how it relates to what I'm doing:
http://legacy.python.org/dev/peps/pep-0420/
So, what is the standard way to do this? Where is it documented (actually saying the syntax)? Is the way I'm doing it right? Can I do regular imports instead of from package import module?
After analyzing some Python 3 packages installed on my system (I should have tried that to start with!) I discovered that they often seem to do things a little differently. Instead of just doing from .myModule import stuff they would do from myPackage.myModule import stuff (inside the modules in the package). So, that works, too, I suppose, and seems to be more frequently used.

How to call a python file, that needs to import packages?

I'm following a tutorial to call python code from a C++ program from the python docs.
Everything works just fine when trying to call the multiply example. Now if I add a line to the python source code importing a library, lets say openpyxl,
from openpyxl import load_workbook
I receive an error from python
ImportError: No module named openpyxl
I thought if I import a system library, I wouldn't have any problems, but I also get an error if I try to import datetime.
I don't have any error if I import the file from the python console. The openpyxl library is installed in my system.
So my question is: how to import python source code that needs to import packages?
EDIT: Ok, I forgot to mention something, I have not been completely honest with you guys, I'm sorry.
Trying to run the example I run into a problem: I couldn't make python found my multiply.py file, and the line PyImport_Import always return null.
My solution was to add the path in which I knew my python source was by using PySys_SetPath. The problem is that I just realized that this function doesn't append a new directory, it just overwrites the PYTHONPATH. So now python can find multiply.py, but absolutly anything else.
Of course I've deleted that line but now I have another question, why does python can't find my source if the file is just in the same directory of the C++ compiled program?
The I realized that my sys.path from my python console was a little different from the path showed in my embedded python: the first one had at the beginning of the list an empty string ''. I'm not a python expert, but when I add that line to my path I could import the multiply.py so it seems that was the reason I couldn't import modules that were located to relative to my executable was the missing of this empty path -but still don't know what it means-.
I have to thank to #paul-evans who give me the idea of adding the path to find my files.
This is what PYTHONPATH is for. You can set it as an environment variable containing a list module directories, or in the code itself something like:
import sys
sys.path.append("path/to/openpyxl/module")

Make "shortcut" from Python module

Let's say I have vtk module in my Python site packages, and from application with own Python distribution I want to access this module.
I tried couple of things like:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
import vtk
lut = vtk.vtkLookupTable()
but it fails to load module properly:
AttributeError: 'module' object has no attribute 'vtkLookupTable'
If I do same from default Python interpreter all is fine.
Now I thought to make a wrapper of vtk in this application site packages, with simple __init__.py resolving paths, so that when I do import vtk it will hopefully load right thing, but I have no experience with Python packages to try to make this work
To put it simple, how can I wrap module from arbitrary folder, in Python site packages by making folder with same name as referenced package and simple __init__.py file?
Remove these lines:
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
The site-packages will already be on your python path. Adding a package/folder within that python path (especially at the first level), will just mess with your imports. How is this vtk package structured?
/path/to/site-packages/
vtk/
__init__.py
vtk.py
In this case, to access a function within vtk:
from vtk import vtk
lut = vtk.vtkLookupTable()
It all comes down to how the folder is arranged. You could also do this:
import vtk
lut = vtk.vtk.vtkLookupTable()
Do not try to hack python importing by creating proxy modules simply because you're not understanding how python importing is working. The error was quite clear. The attribute vtkLookupTable did not exist on whatever it was you imported. You imported the wrong thing. Fix it.
You should very very very very rarely have to manipulate the sys.path manually. When you do have to, you should know that it's the right reason - not to work around something you're not fully understanding.
I had trouble with python paths when I first started with python. It can be frustrating, but coming to understand how it works is necessary. What can help you is something like the following:
import vtk
print dir(vtk)
That will print the attributes of vtk, so you can explore exactly what is in the package or module in cases like this where you think you're importing the right thing.
After re-reading your question, it seems like this is a different python install you're talking about. The answer is to install this package into the other python install, or include this package as a top level import by copying the folder into the root level of your application.
"C:\Python27\Lib\site-packages" is already on your python path. So appending path is unnecessary. Remove:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
Create a new folder called 'vtk\' in "C:\Python27\Lib\site-packages", then create a new python file named __init__.py in "C:\Python27\Lib\site-packages\vtk" and put your own module vtk.py in this directory.
Using:
import vtk
or
from vtk import vtk
to use your own module.

Categories