cloudera-scm-agent fails when using python 2.7 - python

Not sure how I can get around this...on the cloudera-manager site it says their software requires either pyhton2.6 or python2.7
however when i try to start the cloudera-scm-agent it complains that:
/opt/versioned_apps/cm-4.8.2/lib64/cmf/agent/build/env/bin/python: error while loading shared libraries: libpython2.6.so.1.0: cannot open shared object file: No such file or directory
I'm running centos7 (which is not supported out of the box).
To make matters worst, I don't know anything about python either (sorry)...so if i need to install anything, please provide step-by-step instructions :-)

Related

Cannot import eurostag.dll into Python

I am new with programming, so it is maybe harder for me to understand but I have the following issue:
I have a script that imports "eurostag.dll", which according to its manual, should work until Python 3.6. (but the manual is not updated, so it may work also with later updates, I assume).\ The issue is that I have Python 3.8. and when running the script I receive the following message:
"Failed to load EUROSTAG library (Could not find module 'D:\Eurostag\eustag_esg.dll' (or one of its dependencies). Try using the full path with constructor syntax.)"
I have tried to move the .dll library where the script is, but nothing changed. I tried also changing the directory with os.chdir, but the same message appears (with the same 'D:\Eurostag\eustag_esg.dll', so the directory was not changed.
Does anybody know if there is any workaround for this?
Thank you!

pack individual .jpg image files into RecordIO (.rec) files

I'm trying to go through the MXNET tutorial (https://statist-bhfz.github.io/cats_dogs_finetune), but have trouble making the RecordIO ".rec" files used to process the pictures.
I have tried the suggested approach:
python C:/mxnet-20170203/tools/im2rec.py --list=1 --recursive=1 --train-ratio=0.8 cats_dogs train_pad_224x224
python C:/mxnet-20170203/tools/im2rec.py --num-thread=4 --pass-through=1 cats_dogs_train.lst train_pad_224x224
python C:/mxnet-20170203/tools/im2rec.py --num-thread=4 --pass-through=1 cats_dogs_val.lst train_pad_224x224
But i get the error: "AttributeError: 'module' object has no attribute 'MXIndexedRecordIO'"
when doing so.
Is there a way to generate the ".rec" files in R directly? And if not, how do i get pass the error?
Thanks.
Kr,
Daniel
The error you are getting seems related to the fact that you are trying to use a newer version of im2rec.py than the one that came with your installed version of mxnet, hence the missing modules.
try using the im2rec.py script located in your own mxnet installation folder, somewhere like this python2.7/site-packages/mxnet/tools/im2rec.py
Also note that if you upgrade your version of mxnet, the syntax of im2rec has changed to:
im2rec.py --list --recursive --train-ratio=0.8 cats_dogs train_pad_224x224

Issue with using protobufs with python ImportError: cannot import name descriptor_pb2

Context
Steps taken:
Environment Setup
I've installed protobufs via Home Brew
I've also followed the steps in the proto-bufs python folder's readme on installing python protobufs - namely running the python setup.py install command
I've using the protobuf-2.4.1 files
Coding
I have a python file (generated from a .proto file I compiled) that contains the statement, among other import statements, but I believe this one is the one causing issues:
from google.protobuf import descriptor_pb2
The above python file, I'm importing in another python file, it's
this python file that I want to write up logic for parsing the
protobufs data files I receive
Error received
I get this error when running that file:
Steps taken to fix
Searched google for that error - didn't find much
Looked at this question/answer Why do I see "cannot import name descriptor_pb2" error when using Google Protocol Buffers?
I don't really understand the above questions selected answer,I tried to run the command in the above answer protoc descriptor.proto --python_out=gen/ by coping and pasting it in the terminal in different places but couldn't get it to work
Question
How do I fix this error?
What is the underlying cause?
How do I check if the rest of my protobuf python compiler/classes are set up correctly?
I've discovered the issue. I had not run the python install instructions the first time I tried to compile this file. I recompiled the file and this issue was fixed.

Python Throws ImportError: No module named..." Error

I'm trying to install the Chilkat library for Python in order to use its encryption functionality, but being new to Python in every possible way, I'm getting stuck entirely too early. I've installed the library as instructed by the docs and verified that the files are in the "right place" (/usr/local/lib/python2.7/site-packages/) on my Ubuntu 12.04 server.
I've also downloaded the test script. When I try to run it, however:
ImportError: No module named chilkat
I know this is stupid basic, but here I am. In the docs they do mention a possible issue with sys.prefix. That (presumably default, since I've never touched it) value on my machine is /usr. I moved everything there, but still get the same error.
Help? Where is the most "pythonic" place to put these files and how can I get Python to recognize them universally?
Thanks.
For anyone searching, I just ended up adding site-packages to my sys.path by adding a .pth file to dist-packages which was already in my path.
echo "/usr/local/lib/python2.7/site-packages" > /usr/local/lib/python2.7/dist-packages/site-packages.pth

How to bundle Python dependancies in IronWorker?

I'm writing a simple IronWorker in Python to do some work with the AWS API.
To do so I want to use the boto library which is distributed via PyPi repository. The boto library is not installed by default in the IronWorker runtime environment.
How can I bundle the boto library dependancy with my IronWorker code?
Ideally I'm hoping I can use something like the gem dependancy bundling available for Ruby IronWorkers - i.e in myRuby.worker specify
gemfile '../Gemfile', 'common', 'worker' # merges gems from common and worker groups
In the Python Loggly sample, I see that the hoover library is used:
#here we have to include hoover library with worker.
hoover_dir = os.path.dirname(hoover.__file__)
shutil.copytree(hoover_dir, worker_dir + '/loggly') #copy it to worker directory
However, I can't see where/how you specify which hoover library version you want, or where to download it from.
What is the official/correct way to use 3rd party libraries in Python IronWorkers?
Newer iron_worker version has native support of pip command.
So, you need:
runtime "python"
exec "something.py"
pip "boto"
pip "someotherpip"
full_remote_build true
[edit]We've worked on our toolset a bit since this answer was written and accepted. The answer from my colleague below is the recommended course moving forward.[/edit]
I wrote the Python client library for IronWorker. I'm also employed by Iron.io.
If you're using the Python client library, the easiest (and recommended) way to do this is to just copy over the library's installed folder, and include it when uploading the package. That's what the Python Loggly sample is doing above. As you said, that doesn't specify a version or where to download the library from, because it doesn't care. It just takes the one installed on your system and uses it. Whatever you get when you enter "import boto" on your local machine is what would be uploaded.
The other option is using our CLI to upload your worker, with a .worker file.
To do this, here's what you'd need to do:
Create a botoworker.worker file:
runtime "binary"
build 'pip install --install-option="--prefix=`pwd`/pips" boto'
file 'botoworker.py'
exec "botoworker.sh"
That second line is the pip command that will be run to install the dependency. You can modify it like you would any pip command run from the command line. It's going to execute that command on the worker during the "build" phase, so it's only executed once instead of every time you run a task.
The third line should be changed to the Python file you want to run--it's your Python worker file. Here's the one we used to test this:
import boto
If you save that as botoworker.py, the above should work without any modification. :)
The fourth line is a shell script that's going to actually run your worker. I've included the one we used below. Just save it as botoworker.sh, and you won't have to worry about modifying the .worker file above.
PYTHONPATH="$HOME/pips/lib/python2.7/site-packages:$PYTHONPATH" python botoworker.py "$#"
You'll notice it refers to your Python file--if you don't name your Python file botoworker.py, remember to change it here, too. All this does is set your PYTHONPATH to include the installed library, and then runs your Python file.
To upload this, just make sure you have the CLI installed (gem install iron_worker_ng, making sure your Ruby version is 1.9.3 or higher) and then run "iron_worker upload botoworker" in your shell, from the same directory your botoworker.worker file is in.
Hope this helps!

Categories