I'm having trouble using shelve on my dev server. My local machine will create a new file just fine when I call shelve.open(). But when I push the file to the remote server, an error happens. It looks like it is trying to find the file first and it's failing because it isn't there. I'm not changing the file extension. My local machine is Ubuntu 14 and my remote is CentOS 6. Here is the python stack trace. Python 2.6.
File "/var/lib/jenkins/................/translation.py", line 51, in _market
lang_db = shelve.open(market + "_pydb")
File "/usr/lib64/python2.6/shelve.py", line 239, in open
return DbfilenameShelf(filename, flag, protocol, writeback)
File "/usr/lib64/python2.6/shelve.py", line 223, in __init__
Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
File "/usr/lib64/python2.6/anydbm.py", line 83, in open
return mod.open(file, flag, mode)
File "/usr/lib64/python2.6/dbhash.py", line 19, in open
return bsddb.hashopen(file, flag, mode)
File "/usr/lib64/python2.6/bsddb/__init__.py", line 361, in hashopen
d.open(file, db.DB_HASH, flags, mode)
DBNoSuchFileError: (2, 'No such file or directory')
For the record, I changed the default to dumbdbm and it's still giving me an IO error (file not found).
import anydbm
anydbm._defaultmod = __import__('dumbdbm')
I found out that when I was pushing to the repository, git wasn't including the directory I was looking in because it was empty. If you try to use shelve.open with an argument that is in a directory that doesn't exist, it will give you an IO error. I simply forced the creation of the directory and now it works.
Related
Im trying to run a python Server using CherryPy for a WebSite but when I run it this error pops up.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/cherrypy/_cprequest.py", line 638, in respond
self._do_respond(path_info)
File "/usr/local/lib/python3.10/dist-packages/cherrypy/_cprequest.py", line 694, in _do_respond
self.hooks.run('before_handler')
File "/usr/local/lib/python3.10/dist-packages/cherrypy/_cprequest.py", line 95, in run
self.run_hooks(iter(sorted(self[point])))
File "/usr/local/lib/python3.10/dist-packages/cherrypy/_cprequest.py", line 117, in run_hooks
hook()
File "/usr/local/lib/python3.10/dist-packages/cherrypy/_cprequest.py", line 65, in __call__
return self.callback(**self.kwargs)
File "/usr/local/lib/python3.10/dist-packages/cherrypy/_cptools.py", line 280, in _lock_session
cherrypy.serving.session.acquire_lock()
File "/usr/local/lib/python3.10/dist-packages/cherrypy/lib/sessions.py", line 550, in acquire_lock
self.lock = zc.lockfile.LockFile(path)
File "/usr/local/lib/python3.10/dist-packages/zc/lockfile/__init__.py", line 117, in __init__
super(LockFile, self).__init__(path)
File "/usr/local/lib/python3.10/dist-packages/zc/lockfile/__init__.py", line 87, in __init__
fp = open(path, 'a+')
FileNotFoundError: [Errno 2] No such file or directory: '/var/www/html/cncsessions\\/session-73ab2ecbe9bd50153b4f20828fcc08bff6e9cd6e.lock'
It's my first time using this module and I don't know what's wrong.
I'm using Ubuntu 22, Python 3.10.6
Hard to say without seeing your exact code that this is calling.
Judging by the Error you are trying to use Sessions.
The sessions are looking for
/var/www/html/cncsessions
To place the session files in
But it gives an error. It looks like the path might be wrong. There's a double backslash at the end there and a forward slash.
\\/
If you haven't given up on this/figured it out already I would try changing this path to just this
/var/www/html/cncsessions
Also be sure you do not store your session data in your web root. Looks like from that path you might be doing that! Anything in a webroot will be served via public webserver. There's little to no chance anyone would guess the file names though.
I'm trying to use dask from Pycharm using a remote (SSH) interpreter.
Here's some code:
from dask.distributed import Client
client = Client(processes=True)
On Python 2 this seems to work fine, but on Python 3 this fails with:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/forkserver.py", line 278, in main
code = _serve_one(child_r, fds,
File "/usr/lib/python3.8/multiprocessing/forkserver.py", line 317, in _serve_one
code = spawn._main(child_r, parent_sentinel)
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 125, in _main
prepare(preparation_data)
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/usr/lib/python3.8/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "/usr/lib/python3.8/runpy.py", line 261, in run_path
code, fname = _get_code_from_file(run_name, path_name)
File "/usr/lib/python3.8/runpy.py", line 231, in _get_code_from_file
with open(fname, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/<input>'
distributed.nanny - WARNING - Restarting worker
/home/ubuntu exists and is writable- I'm not sure what /home/ubuntu/input is though!
Does anyone have an idea as to what is going wrong here?
EDIT
so after a bit of tracing back I can see that in multiprocess/spawn.py on line 226, there is a prepare function which takes a map of properties describing the processed to be spawned. For some reason the 'init_main_from_path' in that map is being set to /home/ubuntu/<input> which is a directory that doesn't exist anywhere (I'm not even usre this is a valid directory name!). It looks like the PyCharm console is setting this to help itself with the remote execution, but it's messing up the ability of multiprocess to correctly spawn a process. If I hack multiprocess/spawn.py to remove the rogue 'init_main_from_path' key then I can start the client.
I'm attempting to download all the files in an SFTP directory to a local folder using the pysftp library. My code looks like this:
import pysftp
sftp = pysftp.Connection('server', username = 'name', password = 'password')
sftp.get_d('Daily_Reports', '/home/jchrysostom/Documents/SupplyChain/Daily_Reports/')
Daily_Reports is a folder that exists on the SFTP server - I have verified this. I have also verified that /home/jchrysostom/Documents/SupplyChain/Daily_Reports/ exists. I can cd to it in terminal with no problems.
However, when I run this python script, I get the following error: IOError: [Errno 2] File not found.
Any ideas what may be causing this?
UPDATE: A little investigation shows that the files actually have downloaded. In fact, all of them have downloaded just fine. However, I'm unable to run the rest of the script, because it's erroring out for some reason. Is this just a bug in the library?
UPDATE 2 - Full Traceback, as requested:
Traceback (most recent call last):
File "supplychain.py", line 20, in <module>
sftp.get_d('Daily_Reports','/home/jchrysostom/Documents/SupplyChain/Daily_Reports/')
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 255, in get_d
preserve_mtime=preserve_mtime)
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 497, in cd
self.cwd(original_path)
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 510, in chdir
self._sftp.chdir(remotepath)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 580, in chdir
if not stat.S_ISDIR(self.stat(path).st_mode):
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 413, in stat
t, msg = self._request(CMD_STAT, path)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 729, in _request
return self._read_response(num)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 776, in _read_response
self._convert_status(msg)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 802, in _convert_status
raise IOError(errno.ENOENT, text)
IOError: [Errno 2] File not found
As best as I can tell, this is a bug in pysftp. The files are being copied successfully, but (at least according to the traceback here) the library is blowing up when it tries to change back to the original remote working directory on the FTP server.
Workaround is to iterate over the files in the directory and get() each individually...
for filename in sftp.listdir('Daily_Reports'):
sftp.get('Daily_Reports/' + filename, localpath = '/home/jchrysostom/Documents/SupplyChain/Daily_Reports/' + filename)
I'm new to Python and I writing a script that uses the pysftp module. There is one function in the pysftp module that I'm having trouble with -- it's the rename function.
Here is the call
srv = pysftp.Connection(host = 'xxx.com', username = 'xxx', password = 'xxx')
y = srv.rename('ftptest.txt', 'renamedfile.txt')
The error I'm getting is
Traceback (most recent call last):
File "./ftp.py", line 49, in <module>
y = srv.rename('ftptest.txt', 'renamedfile.txt')
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 740, in rename
self._sftp.rename(remote_src, remote_dest)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 365, in rename
self._request(CMD_RENAME, oldpath, newpath)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 729, in _request
return self._read_response(num)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 776, in _read_response
self._convert_status(msg)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 804, in _convert_status
raise IOError(errno.EACCES, text)
IOError: [Errno 13] Permission denied
I have verified that the file exists and it has the 777 for permissions.
With SFTP protocol (which lacks concept of a current working directory), using an absolute paths is recommended. With a relative path, the result is server implementation specific. So you cannot be sure what files you are actually trying to rename.
On *nix systems, to rename a file, you need write permissions to a containing folder. The permissions to the file you are renaming do not matter.
I recently upgraded my GAE Python app to Python 2.7. Since then, I periodically get the following error with the dev server and the dev server serves up a blank page:
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 168, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 206, in _LoadHandler
handler = __import__(path[0])
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/main.py", line 2, in <module>
import views
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/views.py", line 3, in <module>
from pytz.gae import pytz
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/pytz/__init__.py", line 34, in <module>
from pkg_resources import resource_stream
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1818, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1690, in FindAndLoadModule
description)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1615, in LoadModuleRestricted
return source_file.load_module(submodule_fullname)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/dist/py_zipimport.py", line 246, in load_module
submodname, is_package, fullpath, source = self._get_source(fullmodname)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/dist/py_zipimport.py", line 207, in _get_source
source = self.zipfile.read(relpath.replace(os.sep, '/'))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 867, in read
return self.open(name, "r", pwd).read()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 882, in open
zef_file = open(self.filename, 'rb')
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 578, in __init__
raise IOError(errno.EACCES, 'file not accessible', filename)
IOError: [Errno 13] file not accessible: '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg'
INFO 2012-01-21 20:50:44,222 dev_appserver.py:2832] "POST /manage HTTP/1.1" 500 -
Some notes:
This doesn't happen on the production server.
On the dev server, my app will work for a few minutes and then this error happens.
If I stop and restart my app on the dev server, it will work again for a few minutes.
I am using the latest version of gae-pytz and you can see that it fails in an import there.
The [...] that I removed are similar to the stuff you see near the end.
I don't know why setuptools is being invoked at the end.
I'm using a Mac with Lion.
I can use the dev server, but it is really annoying to stop and restart every few minutes. Any ideas how to fix this?
The actual problem from the stack trace, is your code is trying to import setup tools from site-packages, which the dev server won't do.
'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg'
You will need to include setuptools in you application code base. The fact that it works sometimes, suggests that you code paths through various modules vary, and maybe (depending on what your tesing in dev) different import orders mean setup tools has been imported somewhere else, or is only required at certain points in your code.
Have a look at line 4th line of the stack trace where pytz is imported, the next line is from pkg_resources import resource_stream thats whats triggering the rest of the import issue. I use a fake truncated pkg_resources at the root of my project, that doesn't end up trying to import stuff from setup tools. You can run the dev server in debug import mode which will tell you a lot more
Here is a fake pkg_resources.
"""Package resource API
--------------------
A resource is a logical file contained within a package, or a logical
subdirectory thereof. The package resource API expects resource names
to have their path parts separated with ``/``, *not* whatever the local
path separator is. Do not use os.path operations to manipulate resource
names being passed into the API.
The package resource API is designed to work with normal filesystem packages,
.egg files, and unpacked .egg files. It can also work in a limited way with
.zip files and with custom PEP 302 loaders that support the ``get_data()``
method.
"""
import sys, os, zipimport, time, re, imp, new
try:
frozenset
except NameError:
from sets import ImmutableSet as frozenset
from os import utime #, rename, unlink # capture these to bypass sandboxing
from os import open as os_open
There are probably other/better ways of doing this, but it works for me.
Oh, I would also suggest you use http://code.google.com/p/gae-pytz/ instead of pytz.
Cheers
An alternative answer to the above that I prefer.
The __init__.py file for pytz contains the following lines:
#try:
# from pkg_resources import resource_stream
#except ImportError:
resource_stream = None
I commented out the first three lines and that fixed the problem.
The problem is a bug in the App Engine dev server with Python 2.7. The solution is here:
File not accesible error (setuptools) in logs