I would like to install the Python fastText wrapper of the Facebook C++ library on Azure ML Studio. This library is installed and works properly on my laptop.
I tried to follow the instructions in this Stack Overflow thread for the upload on Azure without success.
The code in my "Execute Python Script" is minimal: I am just unzipping and loading the fastText package that I installed locally on my machine and then calling a help function on the "train_supervised" attribute of the fastText module to verify that the package is imported correctly
# The script MUST contain a function named azureml_main
# which is the entry point for this module.
import fastText
# The entry point function can contain up to two input arguments:
# Param<dataframe1>: a pandas.DataFrame
# Param<dataframe2>: a pandas.DataFrame
def azureml_main(dataframe1 = None, dataframe2 = None):
print(help(fastText.train_supervised))
# Return value must be of a sequence of pandas.DataFrame
return dataframe1,
Upon running this minimal Azure experiment I get the following error:
Traceback (most recent call last): File "C:\server\invokepy.py", line 199, in batch odfs = mod.azureml_main(*idfs) File "C:\temp\e6acccec62994066a25e0d758090e749.py", line 44, in azureml_main print(help(fastText.train_supervised))AttributeError: module 'fastText' has no attribute 'train_supervised'Process returned with non-zero exit code 1---------- End of error message from Python interpreter ---------- Process exited with error code -2
I have also tried to create a virtual environment on my local machine (using conda) and to install fastText and its dependencies in it, but I did not manage. The goal would have been then to zip and upload those libraries to Azure. This is because, for compatibility with the Azure Python environment, I need the 3.5.1 Python version (Anaconda 4.0).
Any help/guidance appreciated!
There is a more completed answer for the SO thread post by me: Updating pandas to version 0.19 in Azure ML Studio.
And you should install Cypython and fasttext via pip in a virtualenv, then you need to package these modules below (list by pip freeze) to a zip file and upload it to Azure ML Studio.
Cython==0.29.10
fasttext==0.8.3
future==0.17.1
numpy==1.16.4
However, there are two issue in your code as below.
To import fasttext in Python, not fastText which will cause ModuleNotFoundError: No module named 'fastText'.
Actually, there is not an attribute named train_supervised, I got the error AttributeError: module 'fasttext' has no attribute 'train_supervised'. And I tried to find it via the online fastText API Reference All Functions & All Variables, it doesn't exist really, just supervised.
Hope it helps.
Related
I am trying to modify an AI for a game on the steam store. The AI communicates through the game with the use of a mod called the communication mod. The AI is made using a python project. The package I am trying to modify is https://github.com/ForgottenArbiter/spirecomm and the mod is https://github.com/ForgottenArbiter/CommunicationMod.
I want to add the pandas package and the job lib package as imports so I can use a model I have made for the AI. When I try to run the game + mod after adding the pandas and joblib packages as imports I get this error in the error log.
Traceback (most recent call last):
File "/Users/ross/downloads/spirecomm-master/main.py", line 6, in <module>
from spirecomm.ai.agent import SimpleAgent
File "/Users/ross/Downloads/spirecomm-master/spirecomm/ai/agent.py", line 10, in <module>
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
This issue only happens when the game is running and the mod tries to run. if I just run the file in terminal it is able to compile/run and send the ready signal
I have checked that I have these modules installed and it is installed. I am on an M1 Mac and I have several different versions of python installed but I have checked them all and it is installed for each of them. I have also opened the python package using pycharm and added pandas and joblib to the python interpreter as a package.
Another thing I have tried is modifying the setup.py file to say that pandas and joblib are required. I then ran it again but I am not sure if this had any effect because I have already run it before.
There is limited help that can be provided without knowing the framework that you are is using but hopefully this will give you some starting points to help.
If you are getting a "No module named 'pandas'" error, it is because you have imported pandas in your code but your python interpreter cannot find it. There are two major reasons this will happen, either it is not installed (which you say has definitely happened) or it is not in the location the interpreter expects (most likely).
The first thing you can do is make sure the pandas install is in the PYTHONPATH. To do this look at Permanently add a directory to PYTHONPATH?.
Secondly, you say you have several versions of python and have installed the module for all versions but you most likely have several instances of at least one version. Many IDEs, such as PyCharm, create a virtual environment when you create a new project and place in it a new instance of python interpreter, or more accurately a link to one version of python. Within this virtual environment, the IDE then loads the modules it has been told to load and thus makes them available for import to the code using that particular environment.
In your case I suspect you may be using a virtual environment that has not had pandas loaded into it. You need to investigate your IDEs documentation if you do not know how to load it. Alternatively you can instruct the IDE to use a different virtual environment (one that does have pandas loaded) - again search documentation for how to do this.
Finally, if all else fails, you can specifically tell your code where to look for the module using the sys.path.append command in your code.
import sys
sys.path.append('/your/pandas/module/path')
import pandas
I'm trying to run some code python code in Google colab. This code uses some custom modules I wrote, some of which were written in cython. On my local machine, I'd normally compile the cython code first before running my main python script. What is the correct way of doing this on colab, if there is one? Right now, the script throws an error when it tries to import my cython modules. Specifically, I get the error
ImportError: model/cython_library/core.so: invalid ELF header
which makes sense, since the core.so file was compiled on my local machine instead of gcloud.
I am trying to use pyinstaller to "freeze" a python app I've written that uses pytorch. I see that pyinstaller has a hook for torch: https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/hooks/hook-torch.py
But it doesn't appear to work. I still get the following error:
File "torch/_utils_internal.py", line 53, in get_source_lines_and_file
OSError: Can't get source for . TorchScript requires source access in order to carry out
compilation, make sure original .py files are available. Original
error: could not get source code [28326] Failed to execute script
timeslide
I also tried this work-around with no success:
https://github.com/pyinstaller/pyinstaller/issues/2666#issuecomment-370789381
I just need a way to get pytorch to install in a pyinstaller app. It doesn't need to be clean or beautiful. Can anyone suggest a solution?
(macOS and python 3.7)
I am working one file python project.
I integrated google-cloud-API for realtime speech streaming and recognition.
It works with python aaa.py command well.
Now I need windows build file(.exe), so I used pyinstaller program and I got aaa.exe file successfully.
But I got this error while running speech streaming by using Google cloud API.
[Errno 2] No such file or directory:
'D:\AI\ai\dist\AAA\google\cloud\gapic\speech\v1\speech_client_config.json'
So I copied this speech_client_config.json file in needed path, after that I got below error again.
Exception in 'grpc._cython.cygrpc.ssl_roots_override_callback'
ignored E0511 01:13:14.320000000 3108
src/core/lib/security/security_connector/security _connector.cc:1170]
assertion failed: pem_root_certs != nullptr
Then, I can not find solution to get working version with google-cloud API.
I am using python version 2.7.14
I need your friendly help.
Thanks.
I had the same problem. If you are willing to distribute roots.pem with your executable (just search for the file - it should be buried deep within the installation directory of grpcio), I had luck fixing this by setting GRPC_DEFAULT_SSL_ROOTS_FILE_PATH environment variable to the full path of this roots.pem file.
Update 2021
To anyone who is experiencing this issue. I got it working thanks to these amazing people. See the full conversation on this github issue.
Here is the link
Step 1
Credits to #cbenhagen & #rising-stark on this github link.
A PyInstaller hook called hook-grpc.py looking like this would do the trick:
Create a python file named hook-grpc.py with this code.
from PyInstaller.utils.hooks import collect_data_files
datas = collect_data_files('grpc')
Step 2
Put the hook-grpc.py file in your \site-packages\PyInstaller\hooks directory of the python environment you are running on. So basically you can find it at
C:\Users\yourusername\AppData\Local\Programs\Python\Python37\Lib\site-packages\PyInstaller\hooks
Note:
Just change the yourusername and Python37 to your
respective username and python version you are using.
For Anaconda users it might be different. Check this site
to find the anaconda python environment path you are using.
Step 3
Once you've done that you can now convert your .py python program to .exe using pyinstaller and it should work.
This looks to me like a SSL credentials mistake. I think you are not being allowed to GC. Check this code snippet and this documentation.
Context
Steps taken:
Environment Setup
I've installed protobufs via Home Brew
I've also followed the steps in the proto-bufs python folder's readme on installing python protobufs - namely running the python setup.py install command
I've using the protobuf-2.4.1 files
Coding
I have a python file (generated from a .proto file I compiled) that contains the statement, among other import statements, but I believe this one is the one causing issues:
from google.protobuf import descriptor_pb2
The above python file, I'm importing in another python file, it's
this python file that I want to write up logic for parsing the
protobufs data files I receive
Error received
I get this error when running that file:
Steps taken to fix
Searched google for that error - didn't find much
Looked at this question/answer Why do I see "cannot import name descriptor_pb2" error when using Google Protocol Buffers?
I don't really understand the above questions selected answer,I tried to run the command in the above answer protoc descriptor.proto --python_out=gen/ by coping and pasting it in the terminal in different places but couldn't get it to work
Question
How do I fix this error?
What is the underlying cause?
How do I check if the rest of my protobuf python compiler/classes are set up correctly?
I've discovered the issue. I had not run the python install instructions the first time I tried to compile this file. I recompiled the file and this issue was fixed.