Is there an import of rules_python that defines toolchains but is compatible with bazel release 0.5.4? If not, what's the minimum version of bazel that does implement toolchains?
$bazel info release
release 0.5.4
I've inherited an application that builds with bazel 0.5.4, and would prefer not to destabilize the application with a significant upgrade. But, it does require toolchains in order to find python 3.
Suppose the application consists of the WORKSPACE and BUILD files from the most recent rules release README, plus a small python objective in the BUILD:
py_test(
name = "sandbox_test",
srcs = ["sandbox_test.py"],
default_python_version = "PY3",
srcs_version = "PY3",
)
What additional WORKSPACE definitions could enable this to run? With the latest rules_python provided:
git_repository(
name = "rules_python",
commit = "740825b7f74930c62f44af95c9a4c1bd428d2c53",
remote = "https://github.com/bazelbuild/rules_python.git",
)
toolchain.bzl cannot be found:
ERROR: error loading package 'toolchain_demo': Extension file not found. Unable to load file '#bazel_tools//tools/python:toolchain.bzl': file doesn't exist or isn't a file
ERROR: error loading package 'toolchain_demo': Extension file not found. Unable to load file '#bazel_tools//tools/python:toolchain.bzl': file doesn't exist or isn't a file
Is there a way to patch toolchain definitions into the WORKSPACE, or are toolchains with 0.5.4 impossible?
Related
I'm trying to build a pex_binary in mac OS X for my Apache Heron application (written in Python), but it fails with error.
Details of bazel build error below.
$bazel build pmTop
ERROR: /Arun/Python/Heron/PatMon/WORKSPACE:1:1: name 'git_repository' is not defined
ERROR: Error evaluating WORKSPACE file
ERROR: error loading package '': Encountered error while reading extension file 'pex/pex_rules.bzl': no such package '#io_bazel_rules_pex//pex': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'pex/pex_rules.bzl': no such package '#io_bazel_rules_pex//pex': error loading package 'external': Could not load //external package
INFO: Elapsed time: 0.104s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
Contents of my WORKSPACE below.
git_repository(
name = "io_bazel_rules_pex",
remote = "https://github.com/benley/bazel_rules_pex.git",
tag = "0.3.0",
)
load("#io_bazel_rules_pex//pex:pex_rules.bzl", "pex_repositories")
pex_repositories()
Bazel version details below.
$bazel version
Build label: 0.25.0
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed May 1 21:47:49 2019 (1556747269)
Build timestamp: 1556747269
Build timestamp as int: 1556747269
Not sure, why it's unable to fetch the pex_rules.bzl package. I'm not behind firewall. Appreciate any pointers to fix this issue.
The git_repository rule is not known. Add the following statement on the top of your WORKSPACE file
load("#bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
I want to profile my tensorflow application using tfprof. I have a running tensorflow 1.3 installation where the the tfprof command line tool is missing. I also tried the provided pip packages locally, but there I also can't find tfprof.
Is there a way to compile and link the tfprof command line tool agains my running tensorflow application?
I already git-cloned the tensorflow repository and tried to build it with bazel 0.5.2
$ bazel build --config opt tensorflow/core/profiler/...
WARNING: Output base '/home/USERNAME/.cache/bazel/_bazel_USERNAME/e5cce820cc082410b4fcc604db349066' is on NFS. This may lead to surprising failures and undetermined behavior.
WARNING: Config values are not defined in any .rc file: opt
ERROR: /tmp/tensorflow/tensorflow/core/BUILD:1416:1: no such target '//tensorflow/tools/git:gen/spec.json': target 'gen/spec.json' not declared in package 'tensorflow/tools/git' defined by /tmp/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /tmp/tensorflow/tensorflow/core/BUILD:1416:1: no such target '//tensorflow/tools/git:gen/head': target 'gen/head' not declared in package 'tensorflow/tools/git' defined by /tmp/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /tmp/tensorflow/tensorflow/core/BUILD:1416:1: no such target '//tensorflow/tools/git:gen/branch_ref': target 'gen/branch_ref' not declared in package 'tensorflow/tools/git' defined by /tmp/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: Analysis of target '//tensorflow/core/profiler:profiler' failed; build aborted.
INFO: Elapsed time: 167.083s
or just copy the command mentioned here
bazel build --config opt third_party/tensorflow/core/profiler/...
WARNING: Output base '/home/USERNAME/.cache/bazel/_bazel_USERNAME/e5cce820cc082410b4fcc604db349066' is on NFS. This may lead to surprising failures and undetermined behavior.
WARNING: Config values are not defined in any .rc file: opt
ERROR: no targets found beneath 'third_party/tensorflow/core/profiler'.
I think the path is not right. You should use pathtensorflow/core/profiler/, if your current directory is the cloned tensorflow repository.
Run ./configure script from your tensorflow directory to set the environment variables.
My BUILD file is pretty simple, just
load("#protobuf_bzl//:protobuf.bzl", "py_proto_library")
py_proto_library(
name = "struct_py_pb2",
srcs = ["struct.proto"],
)
But bazel gives a bunch of baffling error messages like:
$ bazel build google/genomics/v1:all
ERROR: thomaswc//v1/BUILD:22:1: no such package '': BUILD file not found on package path and referenced by '//v1:struct_py_pb2'
ERROR: Analysis of target '//v1:struct_py_pb2' failed; build aborted: no such package '': BUILD file not found on package path
INFO: Elapsed time: 0.581s
FAILED: Build did NOT complete successfully (2 packages loaded)
currently loading: #protobuf_bzl//
I see other projects on github using bazel and py_proto_library, though, so I know it must be possible. Is there some WORKSPACE or .bzl magic that I need?
After a bunch of digging, I found a work-around: the default values of default_runtime and protoc are screwed up, so you need to override them:
py_proto_library(
name = "struct_py_pb2",
srcs = ["struct.proto"],
default_runtime = "#com_google_protobuf//:protobuf_python",
protoc = "#com_google_protobuf//:protoc",
)
I'm testing the learn to fly tutorial for pysdl2. (I'm also new to python)
http://pysdl2.readthedocs.io/en/rel_0_9_4/tutorial/index.html
I'm getting an error, thinking it's just some path problem.
Error:
guillaume#ubuntu:~/script$ python sdlTest.py
Traceback (most recent call last):
File "sdlTest.py", line 4, in <module>
RESOURCES = sdl2.ext.Resources(__file__, "resources")
File "/usr/lib/python2.7/dist-packages/sdl2/ext/resources.py", line 139, in __init__
self.scan(path, subdir, excludepattern)
File "/usr/lib/python2.7/dist-packages/sdl2/ext/resources.py", line 313, in scan
raise ValueError("invalid path '%s'" % path)
ValueError: invalid path 'sdlTest.py'
Current code:
import sys
import sdl2.ext
RESOURCES = sdl2.ext.Resources(__file__, "resources")
I'm on ubuntu 16.04
installed python version
guillaume#ubuntu:~/script$ python -V
Python 2.7.12
python-sdl2 version:
guillaume#ubuntu:~/script$ dpkg -s python-sdl2
Package: python-sdl2
Status: install ok installed
Priority: optional
Section: python
Installed-Size: 392
Maintainer: Ubuntu Developers <ubuntu-devel-discuss#lists.ubuntu.com>
Architecture: all
Source: pysdl2
Version: 0.9.3+dfsg2-1
Depends: python:any (<< 2.8), python:any (>= 2.7.5-5~), libsdl2-2.0-0, libsdl2-gfx-1.0-0, libsdl2-image-2.0-0, libsdl2-mixer-2.0-0, libsdl2-ttf-2.0-0
Recommends: python-numpy
Suggests: pysdl2-doc
Description: Python bindings to the SDL2 C-library (Python 2 build)
PySDL2 is a ctypes based wrapper around the Simple DirectMedia Layer 2 library
to allow portable low level access to a video framebuffer, audio output, mouse
and keyboard.
.
This module is built for Python version 2.x.
Original-Maintainer: Debian Python Modules Team <python-modules-team#lists.alioth.debian.org>
Homepage: https://bitbucket.org/marcusva/py-sdl2
PYTHONPATH:
guillaume#ubuntu:~/script$ echo $PYTHONPATH
/usr/lib/python2.7/dist-packages/sdl2:
guillaume#ubuntu:~/script$ ls /usr/lib/python2.7/dist-packages/sdl2/
audio.py filesystem.py keycode.pyc render.py stdinc.pyc
audio.pyc filesystem.pyc loadso.py render.pyc surface.py
blendmode.py gamecontroller.py loadso.pyc rwops.py surface.pyc
blendmode.pyc gamecontroller.pyc log.py rwops.pyc syswm.py
clipboard.py gesture.py log.pyc scancode.py syswm.pyc
clipboard.pyc gesture.pyc messagebox.py scancode.pyc timer.py
cpuinfo.py haptic.py messagebox.pyc sdlgfx.py timer.pyc
cpuinfo.pyc haptic.pyc mouse.py sdlgfx.pyc touch.py
dll.py hints.py mouse.pyc sdlimage.py touch.pyc
dll.pyc hints.pyc pixels.py sdlimage.pyc version.py
endian.py __init__.py pixels.pyc sdlmixer.py version.pyc
endian.pyc __init__.pyc platform.py sdlmixer.pyc video.py
error.py joystick.py platform.pyc sdlttf.py video.pyc
error.pyc joystick.pyc power.py sdlttf.pyc
events.py keyboard.py power.pyc shape.py
events.pyc keyboard.pyc rect.py shape.pyc
ext keycode.py rect.pyc stdinc.py
guillaume#ubuntu:~/script$
"resource" refers to the name of a sub-directory from where you are running your sdl2-python script.
In the sdl2 example on importing, they need a directory called "resource" to load images that will be displayed them in the sdl2 window in their subsequent examples..
We need some resources from the resources folder, so that we have a
test image around to display on the window later on. In your own
applications, it is unlikely that you will ever need to import them,
but we need them here, so we use the sdl2.ext.Resources class to have
them available.
If you want to continue to use the line:
RESOURCES = sdl2.ext.Resources(__file__, "resources")
then from the directory where you are running your script with this line, create a child-directory called "resources" and run your script. You will see the error message will not appear. You can use names other than "resources", just make sure a child directory with the same name exist.
More importantly, to use sdl2, you just need the following 2 lines to load the sdl2 modules and you are good to go..
import sdl2
import sdl2.ext
You have likely failed to include a resources folder in your project dir.
The error is somewhat ambiguous but this fixed it for me.
Edit2: added entire stderr
Edit1: deleted my supposed answer. Updated description of problem and added full buildout.cfg text
Edit0: fixed links
I am trying to port a Pyramid project I have to Google App Engine. I am attempting to leverage Tobias Rodaebel's (thanks!) buildout recipe that aids in this. I used the pyramid_appengine scaffold, essentially following the procedure described here.
Things went relatively smoothly until I tried to add additional eggs to the buildout. I attempt to add the pymongo package to the ./buildout.cfg file,
[buildout]
include-site-packages=false
find-links=http://dist.plone.org/thirdparty/
extends = versions.cfg
versions = versions
update-versions-file = versions.cfg
show-picked-versions = true
develop=src/bkk
parts=bkk
tests
service-bkk
ae-sdk-version=1.9.18
ae-runtime=2.7
ae-sdk-location = ${buildout:parts-directory}/google_appengine
ae-extra-paths =
${buildout:bin-directory}
${buildout:directory}/parts/bkk
${buildout:directory}/parts/google_appengine
${buildout:directory}/parts/google_appengine/lib/antlr3
${buildout:directory}/parts/google_appengine/lib/django
${buildout:directory}/parts/google_appengine/lib/fancy_urllib
${buildout:directory}/parts/google_appengine/lib/yaml/lib
unzip=true
supervisor-port = 9999
supervisor-conf-dir = ${buildout:directory}/conf
supervisor-log-dir = ${buildout:directory}/var/log
service-conf-templates = ${buildout:directory}/conf.tmpl
project-name=bkk
[bkk]
recipe=rod.recipe.appengine
packages=
pyramid
pymongo
pyramid_jinja2
repoze.lru
zope.interface
zope.deprecation
venusian
translationstring
jinja2
webob
src=src/bkk
server-script=devappserver
zip-packages=false
use_setuptools_pkg_resources=true
url=https://storage.googleapis.com/appengine-sdks/featured/google_appengine_${buildout:ae-sdk-version}.zip
[bootstrap]
recipe=zc.recipe.egg
eggs=pastescript
extra-paths=${buildout:ae-extra-paths}
[tests]
recipe = zc.recipe.egg
eggs =
WebTest
WebOb
pytest
pytest-cov
interpreter = python
extra-paths=${buildout:ae-extra-paths}
[supervisor]
recipe = zc.recipe.egg
[mk-supervisor-log-dir]
recipe = collective.recipe.cmd:py
on_install = true
cmds =
>>> if not os.path.isdir('${buildout:supervisor-log-dir}'):os.makedirs('${buildout:supervisor-log-dir}')
[service-supervisor]
recipe = collective.recipe.template
input = ${buildout:service-conf-templates}/supervisord.conf_tmpl
output = ${buildout:directory}/etc/supervisord.conf
depends = ${mk-supervisor-log-dir:recipe} ${supervisor:recipe}
[service-bkk]
recipe = collective.recipe.template
input = ${buildout:service-conf-templates}/service-${buildout:project-name}.conf_tmpl
output = ${buildout:supervisor-conf-dir}/service-${buildout:project-name}.conf
port=8000
admin_port=8010
api_port=8020
depends = ${service-supervisor:recipe}
and also unzip the corresponding egg into the ./eggs directory. I end up getting the following error:
Develop: '/Users/npk1/Dev/bkk/bkk_gae/bkk/src/bkk'
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'paster_plugins'
warnings.warn(msg)
warning: no files found matching '*.ini'
warning: no files found matching '*.rst'
warning: no files found matching '*.jpg' under directory 'bkk'
warning: no files found matching '*.pt' under directory 'bkk'
warning: no files found matching '*.txt' under directory 'bkk'
warning: no files found matching '*.mak' under directory 'bkk'
warning: no files found matching '*.mako' under directory 'bkk'
warning: no files found matching '*.js' under directory 'bkk'
warning: no files found matching '*.html' under directory 'bkk'
warning: no files found matching '*.xml' under directory 'bkk'
Uninstalling bkk.
Unused options for buildout: 'ae-runtime' 'include-site-packages' 'unzip'.
Installing bkk.
rod.recipe.appengine: Google App Engine distribution already downloaded.
While:
Installing bkk.
An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/Users/npk1/Dev/bkk/bkk_gae/bkk/eggs/zc.buildout-2.3.1-py2.7.egg/zc/buildout/buildout.py", line 1946, in main
getattr(buildout, command)(args)
File "/Users/npk1/Dev/bkk/bkk_gae/bkk/eggs/zc.buildout-2.3.1-py2.7.egg/zc/buildout/buildout.py", line 626, in install
installed_files = self[part]._call(recipe.install)
File "/Users/npk1/Dev/bkk/bkk_gae/bkk/eggs/zc.buildout-2.3.1-py2.7.egg/zc/buildout/buildout.py", line 1370, in _call
return f()
File "/Users/npk1/Dev/bkk/bkk_gae/bkk/eggs/rod.recipe.appengine-2.0.6-py2.7.egg/rod/recipe/appengine/__init__.py", line 380, in install
self.copy_packages(ws, temp_dir)
File "/Users/npk1/Dev/bkk/bkk_gae/bkk/eggs/rod.recipe.appengine-2.0.6-py2.7.egg/rod/recipe/appengine/__init__.py", line 290, in copy_packages
raise KeyError, '%s: package not found.' % p
KeyError: 'pyramid_jinja2: package not found.'
Walking through the code it seems that the zc.recipe is assembling its own list of dependencies and checking them against my projects and pymongo does not show up, though perhaps I am wrong.
Does anyone know the process when adding new packages to an existing recipe? Is that the wrong way to think about it? I will continue to try to digest buildout's documentation, as I am new to the utility.
Thanks
Pyramid has a cookbook of recipes.
These two recipes describe how to deploy Pyramid on GAE. Both provide guidance using a different method than the path you started down.
Pyramid on Google’s App Engine (using buildout)
Pyramid on Google’s App Engine (using appengine-monkey)