How to build Python PEX using Bazel? - python

I'm trying to build a pex_binary in mac OS X for my Apache Heron application (written in Python), but it fails with error.
Details of bazel build error below.
$bazel build pmTop
ERROR: /Arun/Python/Heron/PatMon/WORKSPACE:1:1: name 'git_repository' is not defined
ERROR: Error evaluating WORKSPACE file
ERROR: error loading package '': Encountered error while reading extension file 'pex/pex_rules.bzl': no such package '#io_bazel_rules_pex//pex': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'pex/pex_rules.bzl': no such package '#io_bazel_rules_pex//pex': error loading package 'external': Could not load //external package
INFO: Elapsed time: 0.104s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
Contents of my WORKSPACE below.
git_repository(
name = "io_bazel_rules_pex",
remote = "https://github.com/benley/bazel_rules_pex.git",
tag = "0.3.0",
)
load("#io_bazel_rules_pex//pex:pex_rules.bzl", "pex_repositories")
pex_repositories()
Bazel version details below.
$bazel version
Build label: 0.25.0
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed May 1 21:47:49 2019 (1556747269)
Build timestamp: 1556747269
Build timestamp as int: 1556747269
Not sure, why it's unable to fetch the pex_rules.bzl package. I'm not behind firewall. Appreciate any pointers to fix this issue.

The git_repository rule is not known. Add the following statement on the top of your WORKSPACE file
load("#bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

Related

NCCL install / Tensorflow complie error with RTX 2080ti

I am installing Cuda 10, CuDnn 7.3 and NCCL 2.3 on Ubuntu 18.04 with a 2080ti. I am having trouble with the NCCL part. When i try to compile Tensorflow it says error and cannot find NCCL-SLA.txt file. But when i search for that file i cant find it anywhere either? So when i look online people say you can move it from another directory but its not there for me so i do not know what to do.
Here is the error code i am getting:
ERROR: missing input file '#local_config_nccl//:nccl/NCCL-SLA.txt'
ERROR: /home/josh/tensorflow/tensorflow/tools/pip_package/BUILD:166:1: //tensorflow/tools/pip_package:build_pip_package: missing input file '#local_config_nccl//:nccl/NCCL-SLA.txt'
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/josh/tensorflow/tensorflow/tools/pip_package/BUILD:166:1 1 input file(s) do not exist
INFO: Elapsed time: 17.209s, Critical Path: 11.30s
INFO: 353 processes, local.
FAILED: Build did NOT complete successfully
If you can’t find the file NCCL-SLA.txt, try copy the file LICENSE.txt to a new file named NCCL-SLA.txt in the same directory

bazel can't build py_proto_library

My BUILD file is pretty simple, just
load("#protobuf_bzl//:protobuf.bzl", "py_proto_library")
py_proto_library(
name = "struct_py_pb2",
srcs = ["struct.proto"],
)
But bazel gives a bunch of baffling error messages like:
$ bazel build google/genomics/v1:all
ERROR: thomaswc//v1/BUILD:22:1: no such package '': BUILD file not found on package path and referenced by '//v1:struct_py_pb2'
ERROR: Analysis of target '//v1:struct_py_pb2' failed; build aborted: no such package '': BUILD file not found on package path
INFO: Elapsed time: 0.581s
FAILED: Build did NOT complete successfully (2 packages loaded)
currently loading: #protobuf_bzl//
I see other projects on github using bazel and py_proto_library, though, so I know it must be possible. Is there some WORKSPACE or .bzl magic that I need?
After a bunch of digging, I found a work-around: the default values of default_runtime and protoc are screwed up, so you need to override them:
py_proto_library(
name = "struct_py_pb2",
srcs = ["struct.proto"],
default_runtime = "#com_google_protobuf//:protobuf_python",
protoc = "#com_google_protobuf//:protoc",
)

Building boost examples in linux

I've installed all the boost and boost-related packages from synaptic manager. If I ran "bjam" from the directory getting error :
error: Unable to load Jamfile.
error: Could not find a Jamfile in directory '/usr/share/boost- build/kernel'.
error: Attempted to find it with pattern '[Bb]uild.jam [Jj]amfile.v2 [Jj]amfile [Jj]amfile.jam'.
error: Please consult the documentation at 'http://www.boost.org'.

Cannot build the bootloader of pyinstaller on solaris 10(i386)

Trying to build the bootloader of pyinstaller by this instruction, but fail
error messages :
[37/40] cc_link: build/debug/linux/getpath_1.o build/debug/common/pyi_python_1.o build/debug/common/pyi_global_1.o build/debug/common/pyi_launch_1.o build/debug/common/pyi_pythonlib_1.o build/debug/common/pyi_utils_1.o build/debug/common/pyi_archive_1.o build/debug/common/main_1.o build/debug/common/pyi_path_1.o -> build/debug/run_d
ld: fatal: library -lz: not found
ld: fatal: File processing errors. No output written to /home/devel3/nwtham/python27/pyinstaller/bootloader/build/debug/run_d
Waf: Leaving directory `/home/devel3/nwtham/python27/pyinstaller/bootloader/build'
Build failed: -> task failed (err #1):
{task: cc_link getpath_1.o,pyi_python_1.o,pyi_global_1.o,pyi_launch_1.o,pyi_pythonlib_1.o,pyi_utils_1.o,pyi_archive_1.o,main_1.o,pyi_path_1.o -> run_d}
I try to install libz1-1.2.8,REV=2013.09.23-SunOS5.10-i386-CSW.pkg.gz from here, but it still get the same error messages. I check on the folder /opt/csw/lib and found a hyperlink named "libz.so.1" and change it to "libz.so", but I still got the same error message.
My LD_LIBRARY_PATH
/opt/csw/lib/python2.7:/opt/csw/lib:/usr/lib:/usr/local/gcc/lib:/usr/local/lib:/usr/sfw/lib:/usr/dt/lib:/usr/openwin/lib/:/usr/ucblib:/opt/sybase/lib
Maybe I install a wrong package, which package should I install?
Python2.7 and gcc4.9.2 all download from here.
edit :
part of the PATH :
/usr/bin:/usr/local/bin:/opt/csw/gcc4/bin:/usr/sfw/bin:/opt/csw/bin

pip fails to install PIL or Pillow with mt.exe error

On one of my Windows 7 development machines, I am attempting to install the Python Image Library.
My machines are similar. Both run Windows 7 Professional, x64. Both use Python 2.7.3 (32bit). On one of the machine pip install PIL works fine. On the other it fails with the trace ending with this:
build\temp.win-amd64-2.7\Release\_imaging.pyd.manifest : general error c1010070:
Failed to load and parse the manifest. The system cannot find the file specified.
error: command 'mt.exe' failed with exit status 31
How can I resolve this error?
Thanks to http://bugs.python.org/issue4431, this error was fixed by modifying:
C:\<Python dir>\Lib\distutils\msvc9compiler.py
and adding:
ld_args.append('/MANIFEST')
after the MANIFESTFILE line so it looks like:
# Embedded manifests are recommended - see MSDN article titled
# "How to: Embed a Manifest Inside a C/C++ Application"
# (currently at http://msdn2.microsoft.com/en-us/library/ms235591(VS.80).aspx)
# Ask the linker to generate the manifest in the temp dir, so
# we can embed it later.
temp_manifest = os.path.join(
build_temp,
os.path.basename(output_filename) + ".manifest")
ld_args.append('/MANIFESTFILE:' + temp_manifest)
ld_args.append('/MANIFEST')
If you still get the error, then change the if arg.startswith("/MANIFESTFILE:") to if arg.startswith("/MANIFEST:") in the manifest_get_embed_info(self, target_desc, ld_args) method.
Download the compressed package from pypi, and try building and installing in your machine. This link could give you some hints. That exactly deals with your problem only but the installation varies.
If you've reached here looking for
general error c1010070:
Failed to load and parse the manifest. The system cannot find the file specified.
error: command 'mt.exe' failed with exit status 31
Here's a workaround that worked in Windows 8/x64/Python 3.3/VS 11:
# py 3.3 seems to be compiled against VS 2010 compiler, force using VS11 cl.exe for us
$env:VS100COMNTOOLS=$env:VS110COMNTOOLS
# Modify C:\Python33\lib\distutils\msvc9compiler.py
# Comment line 670: ld_args.append('/MANIFESTFILE:' + temp_manifest)
# Basically it will instruct build to not look for manifest file

Categories