Python os.walk() failing - python

I have created a script to give me the list of files in a folder directory. Yet, I am occasionally getting this error. What does this mean?
portion of the error:
Script failed due to an error:
Traceback (most recent call last):
File "<script>", line 12, in <module>
File "C:\Program Files\Nuix\Nuix 6\lib\jython.jar\Lib\os.py", line 309, in walk
File "C:\Program Files\Nuix\Nuix 6\lib\jython.jar\Lib\os.py", line 299, in walk
File "C:\Program Files\Nuix\Nuix 6\lib\jython.jar\Lib\genericpath.py", line 41, in isdir
File "C:\Program Files\Nuix\Nuix 6\lib\jython.jar\Lib\genericpath.py", line 41, in isdir
java.lang.AbstractMethodError: org.python.modules.posix.PythonPOSIXHandler.error(Ljnr/constants/platform/Errno;Ljava/lang/String;Ljava/lang/String;)V
at jnr.posix.BaseNativePOSIX.stat(BaseNativePOSIX.java:309)
at jnr.posix.CheckedPOSIX.stat(CheckedPOSIX.java:265)
at jnr.posix.LazyPOSIX.stat(LazyPOSIX.java:267)
The script:
import os
import codecs
import shutil
import datetime
import sys
exportpath = 'P:/Output/Export7/{6136BAF2-85BA-4E64-8C11-A2C59398FC02}/'
tempnativefolder = 'NATIVESOrig'
for dir, sub, file in os.walk(exportpath + tempnativefolder):
for fname in file:
#source path
source = os.path.join(dir, fname).replace('\\', '/')
print source
print("Natives moved to subfolders")

I found out that the presence of these characters(see "diamond with question mark" character in screenshot) in the file name causes the issue. Once I replaced those, my script works. thanks so much.

What the error means: AbstractMethodError means that some code tried to call a method which was not implemented.
PythonPOSIXHandler implements jnr.posix.POSIXHandler. JRuby also uses JNR and the interface is subtly different between the two. JRuby's newer copy of JNR has that one additional #error(Errno, String, String) method and Jython's implementation lacks that method, because it's compiled against the interface when the method didn't exist.
I usually see this problem in the other direction - where stuff in Jython's jar breaks JRuby. I assume it entirely depends on the order of the jars in the classpath.

Related

sys.path.insert cannnot import other python files

What the title says. I have the following file structure inside the pathC:\Users\User\Desktop\all python file\5.0.8:
5.0.8\
tree one\
little main.py
sample_tree1.py
tree two\
sample_tree2.py
the following is what's inside little main.py:
import sys
import sample_tree1
sys.path.insert(0, r'C:\Users\User\Desktop\all python file\5.0.8\tree two\sample_tree2.py')
import sample_tree2
I want to import sample_tree2.py, but an error is raised as soon as I run the little main.py:
Traceback (most recent call last):
File "<ipython-input-1-486a3fafa7f2>", line 1, in <module>
runfile('C:/Users/User/Desktop/all python file/5.0.8/tree one/little main.py', wdir='C:/Users/User/Desktop/all python file/5.0.8/tree one')
File "C:\Users\User\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)
File "C:\Users\User\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/User/Desktop/all python file/5.0.8/tree one/little main.py", line 12, in <module>
import sample_tree2
ModuleNotFoundError: No module named 'sample_tree2'
So what happened? I followed this post's top answer in order to import a file from another branch of path, but it doesn't work.
Thank you in advance
EDIT:
I am looking for a solution that:
1) does not require changing the current working directory
2) does not require changing the names of the files
3) does not require changing the configuration of the python terminal stuff
EDIT2:
Added some capscreens of files and folder structures, and error messages
The sys.path.insert() command inserts a path into the system path and should not contain the filename.
Please try below using your structure:
little main.py:
import sys
import sample_tree1
sys.path.insert(0, r'/my/absolute/path/5.0.8/treetwo')
print(sys.path) # view the path and verify your insert
import sample_tree2
print(sample_tree2.tree2_func())
sample_tree2.py in treetwo
def tree2_func():
return 'called tree2'
Outputs:
['/my/absolute/path/5.0.8/treetwo', '... other paths']
called tree2

gensim file not found error

I am executing the following line:
id2word = gensim.corpora.Dictionary.load_from_text('wiki_en_wordids.txt')
This code is available at "https://radimrehurek.com/gensim/wiki.html". I downloaded the wikipedia corpus and generated the required files and wiki_en_wordids.txt is one of those files. This file is available in the following location:
~/gensim/results/wiki_en
So when i execute the code mentioned above I get the following error:
Traceback (most recent call last):
File "~\Python\Python36-32\temp.py", line 5, in <module>
id2word = gensim.corpora.Dictionary.load_from_text('wiki_en_wordids.txt')
File "~\Python\Python36-32\lib\site-packages\gensim\corpora\dictionary.py", line 344, in load_from_text
with utils.smart_open(fname) as f:
File "~\Python\Python36-32\lib\site-packages\smart_open\smart_open_lib.py", line 129, in smart_open
return file_smart_open(parsed_uri.uri_path, mode)
File "~\Python\Python36-32\lib\site-packages\smart_open\smart_open_lib.py", line 613, in file_smart_open
return open(fname, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'wiki_en_wordids.txt'
Even though the file is available in the required location I get that error. Should I place the file in any other location? How do I determine what the right location is?
The code requires an absolute path here. Relative path should be used when entire operation is carried out in the same directory location, but in this case, the file name is passed as argument to some other function which is located at different location.
One way to handle this situation is using abspath -
import os
id2word = gensim.corpora.Dictionary.load_from_text(os.path.abspath('wiki_en_wordids.txt'))

rar file module not working in python 2.7

i have a code like this
import rarfile
pwd = None
rar = rarfile.RarFile(source_filename)
rar.extractall(dest_dir,None,pwd) # error from here
this code in working in ubuntu.
when i run this on windows i get error like this
Traceback (most recent call last):
File "1_bete_rar.pyw", line 132, in extract
File "1_bete_rar.pyw", line 176, in unrar_file
File "rarfile.pyc", line 586, in extractall
File "rarfile.pyc", line 1112, in _extract
File "rarfile.pyc", line 1704, in custom_popen
File "subprocess.pyc", line 711, in __init__
File "subprocess.pyc", line 948, in _execute_child
WindowsError: [Error 2] The system cannot find the file specified
what is the problem with my code? how can i extract rar file with python in windows?
As the rarfile FAQ states (and as is made evident by traces of subprocess in the stack trace),
[rarfile] depends on unrar command-line utility to do the actual decompression.
Note that by default it expect it to be in PATH. If unrar launching fails, you need to fix this.
So get UnRAR from http://www.rarlab.com/rar_add.htm and put it somewhere in your PATH (such as the directory you're running your script from).
Looks like source_filename isn't pointing to a valid RAR file, do this little check before, just to be sure:
import os.path
os.path.isfile(source_filename) # what's the value returned?
If the file exists, then check if the path is in the correct format. For example, this won't work:
source_filename = 'c:\documents\file.rar'
Try this instead:
source_filename = 'c:\\documents\\file.rar'
Or even better, use raw strings:
source_filename = r'c:\documents\file.rar'
One common problem using python with Windows is that the path separator is \, but this is a special character that needs to be escaped in Python. If you print the source_filename you should be able to see if this is set correctly.
e.g.
source_filename = 'c:\users\prosserc\documents\myfile.txt'
will not work correctly. Here are a couple of alternatives:
Use a raw string:
source_filename = r'c:\users\prosserc\documents\myfile.txt'
or use os.path.join to join to an eviornment variable such as user_profile
source_filename = os.path.join(os.getenv('userprofile'), 'documents', 'myfile.txt')

comtypes internal method fails on LP_BSTR

I'm trying to access the .olb files that ship with ArcGIS10.1 using the comtypes module. Some of the .olb files work (esriGeometry.olb) and some of them don't (esriSystem.olb), and some of them work some of the time (esriSearch.olb).
The following code
from comtypes.client import GetModule
olb_path = 'C:\\Program Files (x86)\\ArcGIS\\Desktop10.1\\com\\esriSystem.olb'
m = GetModule(path)
raises this traceback and exception
Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
test3()
File "D:\Data\MatthewPlourde\JAMES\gis_tools\tool.py", line 139, in test3
m = GetModule(path)
File "C:\Python27\ArcGIS10.1\lib\site-packages\comtypes\client\_generate.py", line 112, in GetModule
mod = _CreateWrapper(tlib, pathname)
File "C:\Python27\ArcGIS10.1\lib\site-packages\comtypes\client\_generate.py", line 188, in _CreateWrapper
mod = _my_import(fullname)
File "C:\Python27\ArcGIS10.1\lib\site-packages\comtypes\client\_generate.py", line 26, in _my_import
return __import__(fullname, globals(), locals(), ['DUMMY'])
File "C:\Python27\ArcGIS10.1\lib\site-packages\comtypes\gen\_5E1F7BC3_67C5_4AEE_8EC6_C4B73AAC42ED_0_10_1.py", line 5705, in <module>
( ['in'], POINTER(_midlSAFEARRAY(POINTER(BSTR))), 'pParameters' ),
File "C:\Python27\ArcGIS10.1\lib\site-packages\comtypes\safearray.py", line 18, in _midlSAFEARRAY
sa_type = _make_safearray_type(itemtype)
File "C:\Python27\ArcGIS10.1\lib\site-packages\comtypes\safearray.py", line 53, in _make_safearray_type
raise TypeError(itemtype)
TypeError: <class 'comtypes.errorinfo.LP_BSTR'>
Apparently comtypes.safearray._make_safearray_type doesn't know what to do with <class 'comtypes.errorinfo.LP_BSTR'>. If there's anyone out there using ArcGIS10.1, I'd be grateful to know whether you can reproduce this error, and especially grateful if you know the cause.
I found a solution posted on the ArcGIS forums. It simply involves modifying automation.py in the comtypes source. Add the entry POINTER(BSTR): VT_BYREF|VT_BSTR to the _ctype_to_vartype dictionary.
After this, all the .olb's load.
I too, am having this exact error and I can't get past it. If you find out what happens please update this. The only thing I can find is something saying that there may be mixing 32 bit with 64 bit libs.
(also, I don't see where I can send a reply to your question... only an answer. I don't use stackexchange much)
Let me add something that may help you, this was taken from: http://forums.arcgis.com/threads/15848-ArcMap-10-ArcObjects-and-Python-very-cool-but-help-with-a-couple-of-problems
import logging
# grab rootlogger
_loggy = logging.getLogger()
_loggy.setLevel(logging.DEBUG)
_loggy.addHandler(logging.FileHandler("derpdebug.log"))
import os
import comtypes.client
# change com_dir to whatever it is for you
com_dir = r'C:\Program Files (x86)\ArcGIS\Desktop10.0\com'
coms = [os.path.join(com_dir, x) for x in os.listdir(com_dir) if os.path.splitext(x)[1].upper() == '.OLB']
map(comtypes.client.GetModule, coms)
# check add whatever you want here.
import comtypes.gen.esriArcMapUI
import comtypes.gen.esriGeodatabase
print dir(comtypes.gen.esriArcMapUI)
Then I just did:
grep -v ^Release derpdebug.log >readable.log

cx_freeze / pptx "PackageNotFoundError"

I'm working on a Python-programm that use the modul "pptx" (edit and manipulate powerpoint). The programm works without problem in the interpretor, but once I start it as a .exe file (built with cx_freeze or py2exe) I have this error-message when I click on the main-button :
Exception in Tkinter callback
Traceback (most recent call last):
File "Tkinter.pyc", line 1470, in __call__
File "pptx_02.py", line 187, in ButtonErstellen
File "pptx\api.pyc", line 29, in __init__
File "pptx\presentation.pyc", line 87, in __init__
File "pptx\presentation.pyc", line 170, in __open
File "pptx\packaging.pyc", line 88, in open
File "pptx\packaging.pyc", line 671, in __new__
PackageNotFoundError: Package not found at 'C:\Users\Moi\Programmation\Python\build\exe.win32-2.7\library.zip\pptx\templates\default.pptx'
The problem is that the file "default.pptx" actually exists at this adress.
While looking in the functions of "pptx", I may had an idea about the reason of this problem (but no idea of the solution).
For the last error : File "pptx\packaging.pyc", line 671, in new
is the following code:
class FileSystem(object):
"""
Factory for filesystem interface instances.
A FileSystem object provides access to on-disk package items via their URI
(e.g. ``/_rels/.rels`` or ``/ppt/presentation.xml``). This allows parts to
be accessed directly by part name, which for a part is identical to its
item URI. The complexities of translating URIs into file paths or zip item
names, and file and zip file access specifics are all hidden by the
filesystem class. |FileSystem| acts as the Factory, returning the
appropriate concrete filesystem class depending on what it finds at *path*.
"""
def __new__(cls, file):
# if *file* is a string, treat it as a path
if isinstance(file, basestring):
path = file
if is_zipfile(path):
fs = ZipFileSystem(path)
elif os.path.isdir(path):
fs = DirectoryFileSystem(path)
else:
raise PackageNotFoundError("Package not found at '%s'" % path)
else:
fs = ZipFileSystem(file)
return fs
The problem may be that the final file (default.pptx) isnot a zip but is in a zip. But I have no idea how I could fix it or if it realy is the problem.
If somebody as an idea ...
Thank you !

Categories