I was trying to run docker-compose using Fabric, but when I run the docker-compose up command using Fabric I got the following error:
[52577] Failed to execute script docker-compose
Traceback (most recent call last):
File "docker-compose", line 6, in <module>
File "compose/cli/main.py", line 71, in main
File "compose/cli/main.py", line 127, in perform_command
File "compose/cli/main.py", line 1080, in up
File "compose/cli/main.py", line 1076, in up
File "compose/project.py", line 475, in up
File "compose/service.py", line 352, in ensure_image_exists
File "compose/service.py", line 1217, in pull
File "compose/progress_stream.py", line 101, in get_digest_from_pull
File "compose/service.py", line 1182, in _do_pull
File "site-packages/docker/api/image.py", line 381, in pull
File "site-packages/docker/auth.py", line 48, in get_config_header
File "site-packages/docker/auth.py", line 96, in resolve_authconfig
File "site-packages/docker/auth.py", line 127, in _resolve_authconfig_credstore
File "site-packages/dockerpycreds/store.py", line 20, in __init__
File "site-packages/dockerpycreds/utils.py", line 12, in find_executable
File "distutils/spawn.py", line 176, in find_executable
File "os.py", line 669, in __getitem__
KeyError: 'PATH'
Failed to up the env
When I run docker-compose up from the terminal it works fine.
#task
def up(c):
if c.run('docker-compose up', warn=True).failed:
print ("Failed to up the env")
As my comment suggested, it seems docker-composer up when it needs to do a build at least, needs the PATH environment variable to be set. To fix, I just forwarded the PATH from the current session to the child process (in PHP sorry!):
$compose = new Process( 'docker-compose up -d', '', [
'VOLUME' => getcwd(),
'PATH' => getenv( 'PATH' ),
] );
Related
I am developing a code where I need to create a Manager to synchronize a Queue. In the project I am using hydra and everything works fine until I create a multiprocessing Manager Object. This is the raised error with HYDRA_FULL_ERROR=1:
Primary config module 'config' not found.
Check that it's correct and contains an __init__.py file
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Traceback (most recent call last):
File "mre.py", line 8, in <module>
main()
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\main.py", line 32, in decorated_main
_run_hydra(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\utils.py", line 355, in _run_hydra
run_and_report(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\utils.py", line 207, in run_and_report
return func()
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\utils.py", line 356, in <lambda>
lambda: hydra.run(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\hydra.py", line 107, in run
return run_job(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\core\utils.py", line 125, in run_job
ret.return_value = task_function(task_cfg)
File "mre.py", line 7, in main
m = Manager()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\context.py", line 57, in Manager
m.start()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\managers.py", line 583, in start
self._address = reader.recv()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\connection.py", line 250, in recv
buf = self._recv_bytes()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\connection.py", line 305, in _recv_bytes
waitres = _winapi.WaitForMultipleObjects(
KeyboardInterrupt
^C
C:\Users\UX331U\Desktop>SET HYDRA_FULL_ERROR=1
C:\Users\UX331U\Desktop>python mre.py
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\spawn.py", line 125, in _main
prepare(preparation_data)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\UX331U\Desktop\mre.py", line 8, in <module>
main()
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\main.py", line 32, in decorated_main
_run_hydra(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\utils.py", line 355, in _run_hydra
run_and_report(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\utils.py", line 210, in run_and_report
raise ex
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\utils.py", line 207, in run_and_report
return func()
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\utils.py", line 356, in <lambda>
lambda: hydra.run(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\hydra.py", line 100, in run
cfg = self.compose_config(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\hydra.py", line 505, in compose_config
self.config_loader.ensure_main_config_source_available()
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\config_loader_impl.py", line 135, in ensure_main_config_source_available
self.missing_config_error(
File "C:\Users\UX331U\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\hydra\_internal\config_loader_impl.py", line 109, in missing_config_error
raise MissingConfigException(
hydra.errors.MissingConfigException: Primary config module 'config' not found.
Check that it's correct and contains an __init__.py file
I also attach a minimal code snippet to reproduce the error.
import hydra
from multiprocessing import Manager
#hydra.main(config_path='config', config_name='config')
def main(cfg):
m = Manager() # The error is in this line
m.Queue()
main()
The config.yaml is correctly created and located since when I don't use the Manager, the entire code works with no errors:
- PROJECT ----- config ---- config.yaml
|
|
----- main.py
As far as I have understood, the problem comes from the fact that hydra changes the directory when initialized and afterwards, when the Manager is created, hydra is called again to be initialized but as the directory has been changed, it can not find the config folder. If this is true, is it a bug or can it be solved?
Thank you so much for your help.
I'm not able to reproduce the error on my machine.
You can stop hydra from changing directories at runtime by passing hydra.job.chdir=False at the command line -- see the docs on how to disable changing current working dir to job's output dir.
python main.py hydra.job.chdir=False
Colleague provided me set of py files for testing, which are running fine on his machine. However when I run the py files, I am getting following error:
Traceback (most recent call last):
File "D:\workspaces\pyworkspace\Analytics Python\python_server\server1.py", line 84, in <module>
start_server()
File "D:\workspaces\pyworkspace\Analytics Python\python_server\server1.py", line 57, in start_server
socketObj.bind(server_address)
LookupError: unknown encoding: idna
Here, it says to add import encodings.idna. So I added that import (even though eclipse says "unused import"), but still getting the same error. So I put some breakpoints and started up debugging, but debugger didnt hit the breakpoints. The program termniates without hitting any breakpoints giving following error:
pydev debugger: starting (pid: 8316)
Traceback (most recent call last):
File "_pydevd_bundle\pydevd_cython_win32_34_64.pyx", line 741, in _pydevd_bundle.pydevd_cython_win32_34_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_34_64.c:15515)
File "_pydevd_bundle\pydevd_cython_win32_34_64.pyx", line 254, in _pydevd_bundle.pydevd_cython_win32_34_64.PyDBFrame.do_wait_suspend (_pydevd_bundle/pydevd_cython_win32_34_64.c:5631)
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 764, in do_wait_suspend
self._activate_mpl_if_needed()
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 416, in _activate_mpl_if_needed
activate_function()
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 407, in <lambda>
self.mpl_modules_for_patching = {"matplotlib": lambda: activate_matplotlib(do_enable_gui),
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydev_ipython\matplotlibtools.py", line 93, in activate_matplotlib
is_interactive = is_interactive_backend(backend)
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydev_ipython\matplotlibtools.py", line 56, in is_interactive_backend
from matplotlib.rcsetup import interactive_bk, non_interactive_bk # #UnresolvedImport
ImportError: cannot import name 'interactive_bk'
Traceback (most recent call last):
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 1621, in <module>
main()
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 1615, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 1022, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\_pydev_imps\_pydev_execfile.py", line 25, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "D:\Mahesh\workspaces\pyworkspace\Analytics Python\python_server\server.py", line 84, in <module>
start_server()
File "D:\Mahesh\workspaces\pyworkspace\Analytics Python\python_server\server.py", line 84, in <module>
start_server()
File "_pydevd_bundle\pydevd_cython_win32_34_64.pyx", line 982, in _pydevd_bundle.pydevd_cython_win32_34_64.SafeCallWrapper.__call__ (_pydevd_bundle/pydevd_cython_win32_34_64.c:19346)
File "_pydevd_bundle\pydevd_cython_win32_34_64.pyx", line 498, in _pydevd_bundle.pydevd_cython_win32_34_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_34_64.c:18639)
File "_pydevd_bundle\pydevd_cython_win32_34_64.pyx", line 750, in _pydevd_bundle.pydevd_cython_win32_34_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_34_64.c:15669)
File "_pydevd_bundle\pydevd_cython_win32_34_64.pyx", line 741, in _pydevd_bundle.pydevd_cython_win32_34_64.PyDBFrame.trace_dispatch (_pydevd_bundle/pydevd_cython_win32_34_64.c:15515)
File "_pydevd_bundle\pydevd_cython_win32_34_64.pyx", line 254, in _pydevd_bundle.pydevd_cython_win32_34_64.PyDBFrame.do_wait_suspend (_pydevd_bundle/pydevd_cython_win32_34_64.c:5631)
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 764, in do_wait_suspend
self._activate_mpl_if_needed()
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 416, in _activate_mpl_if_needed
activate_function()
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydevd.py", line 407, in <lambda>
self.mpl_modules_for_patching = {"matplotlib": lambda: activate_matplotlib(do_enable_gui),
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydev_ipython\matplotlibtools.py", line 93, in activate_matplotlib
is_interactive = is_interactive_backend(backend)
File "D:\Programs\Eclipse Oxygen\plugins\org.python.pydev_6.2.0.201711281614\pysrc\pydev_ipython\matplotlibtools.py", line 56, in is_interactive_backend
from matplotlib.rcsetup import interactive_bk, non_interactive_bk # #UnresolvedImport
ImportError: cannot import name 'interactive_bk'
Why is this happening? (I am using python 3.4, eclipse oxygen 4.7 and PyDev 6.2.0.201711281614)
Apparently the problem is because pydev is expecting matplotlib to have some modules which aren't available in your installation... so, the import:
from matplotlib.rcsetup import interactive_bk, non_interactive_bk # #UnresolvedImport
is failing (and that's expected in PyDev if you have matplotlib installed).
Not sure why... I need to investigate whether you have a broken matplotlib or if matplotlib actually changed (in which case the pydev debugger would need to be fixed).
Can you add details on the matplotlib version you have?
I've been using supervisord to run celery for my django project for a while, but suddenly celerybeat won't start. It gives the following traceback:
Traceback (most recent call last):
File "[...]celery/apps/beat.py", line 112, in start_scheduler
beat.start()
File "[...]celery/beat.py", line 454, in start
humanize_seconds(self.scheduler.max_interval))
File "[...]kombu/utils/__init__.py", line 322, in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File "[...]celery/beat.py", line 494, in scheduler
return self.get_scheduler()
File "[...]celery/beat.py", line 489, in get_scheduler
lazy=lazy)
File "[...]celery/utils/imports.py", line 53, in instantiate
return symbol_by_name(name)(*args, **kwargs)
File "[...]celery/beat.py", line 358, in __init__
Scheduler.__init__(self, *args, **kwargs)
File "[...]celery/beat.py", line 185, in __init__
self.setup_schedule()
File "[...]celery/beat.py", line 377, in setup_schedule
self._store['entries']
File "/usr/local/lib/python2.7/shelve.py", line 122, in __getitem__
value = Unpickler(f).load()
UnpicklingError: pickle data was truncated
Haven't been able to find anything on this.
I pieced it together.
The issue was caused by a corrupt celerybeat-schedule file. To locate the file enter:
find ~/ -name celerybeat-schedule -print
Then delete or rename the file:
mv [filename] [newfilename]
Then restart your processes.
I'm developing a Django app, using Celery and RabbitMQ as worker. I'm starting Celery with the following command (on Fedora)
python manage.py celery worker --loglevel=info
However, I'm getting the following error:
ImportError: No module named processe
In my office, we are using Ubuntu and are not getting any errors like this.
Here's the full traceback:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
utility.execute()
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/djcelery/management/commands/celery.py", line 22, in run_from_argv
['%s %s' % (argv[0], argv[1])] + argv[2:],
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/celery.py", line 901, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/base.py", line 187, in execute_from_commandline
return self.handle_argv(prog_name, argv[1:])
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/celery.py", line 893, in handle_argv
return self.execute(command, argv)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/celery.py", line 868, in execute
return cls(app=self.app).run_from_argv(self.prog_name, argv)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/celery.py", line 148, in run_from_argv
return self(*args, **options)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/celery.py", line 118, in __call__
ret = self.run(*args, **kwargs)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/celery.py", line 220, in run
return self.target.run(*args, **kwargs)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/bin/celeryd.py", line 141, in run
kwargs.get('pool_cls') or self.app.conf.CELERYD_POOL)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/celery/concurrency/__init__.py", line 26, in get_implementation
return symbol_by_name(cls, ALIASES)
File "/home/gurpinars/projects/github/Blog-Env/lib/python2.7/site-packages/kombu/utils/__init__.py", line 80, in symbol_by_name
module = imp(module_name, package=package, **kwargs)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named processes
Additionally, here is my pip freeze output:
Django==1.5.2
PIL==1.1.7
amqp==1.0.13
anyjson==0.3.3
billiard==2.7.3.32
celery==3.0.23
django-celery==3.0.23
django-debug-toolbar==0.9.4
ipdb==0.7
ipython==1.0.0
kombu==2.5.14
python-dateutil==2.1
pytz==2013d
redis==2.8.0
six==1.4.1
wsgiref==0.1.2
Any suggestions as to how I can resolve this issue?
Problem solved.On ubuntu,rabbitmq start automatically but fedora we had to start and restart manually when changed conf file.
Setting up the testing framework and I can't seem to get past this error:
Traceback (most recent call last):
File "/usr/local/bin/nosetests", line 8, in <module>
load_entry_point('nose==1.1.2', 'console_scripts', 'nosetests')()
File "/Library/Python/2.7/site-packages/nose-1.1.2-py2.7.egg/nose/core.py", line 118, in __init__
**extra_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/Library/Python/2.7/site-packages/nose-1.1.2-py2.7.egg/nose/core.py", line 135, in parseArgs
self.config.configure(argv, doc=self.usage())
File "/Library/Python/2.7/site-packages/nose-1.1.2-py2.7.egg/nose/config.py", line 338, in configure
self.plugins.configure(options, self)
File "/Library/Python/2.7/site-packages/nose-1.1.2-py2.7.egg/nose/plugins/manager.py", line 271, in configure
cfg(options, config)
File "/Library/Python/2.7/site-packages/nose-1.1.2-py2.7.egg/nose/plugins/manager.py", line 94, in __call__
return self.call(*arg, **kw)
File "/Library/Python/2.7/site-packages/nose-1.1.2-py2.7.egg/nose/plugins/manager.py", line 162, in simple
result = meth(*arg, **kw)
File "build/bdist.macosx-10.7-intel/egg/nosegae.py", line 80, in configure
File "/usr/local/bin/dev_appserver.py", line 76, in <module>
DIR_PATH = get_dir_path(os.path.join('lib', 'ipaddr'))
File "/usr/local/bin/dev_appserver.py", line 66, in get_dir_path
'file and %s.' % sibling)
ValueError: Could not determine directory that contains both, this file and lib/ipaddr.
It's a correct error coming from dev_appserver.py... there is no lib/ipaddr in the /usr/local/bin directory.
Why is nosetests triggering the raising of this error? Why does dev_appengine try to look for lib/ipaddr in the first place?