We Keep Coding

OpenAI GPT3 Search API not working locally

I am using the python client for GPT 3 search model on my own Jsonlines files. When I run the code on Google Colab Notebook for test purposes, it works fine and returns the search responses. But when I run the code on my local machine (Mac M1) as a web application (running on localhost) using flask for web service functionalities, it gives the following error:
openai.error.InvalidRequestError: File is still processing. Check back later.
This error occurs even if I implement the exact same example as given in OpenAI documentation. The link to the search example is given here.
It runs perfectly fine on local machine and on colab notebook if I use the completion API that is used by the GPT3 playground. (code link here)
The code that I have is as follows:
import openai
openai.api_key = API_KEY
file = openai.File.create(file=open(jsonFileName), purpose="search")
response = openai.Engine("davinci").search(
search_model = "davinci",
query = query,
max_rerank = 5,
file = file.id
)
for res in response.data:
print(res.text)
Any idea why this strange behaviour is occurring and how can I solve it? Thanks.

The problem was on this line:
file = openai.File.create(file=open(jsonFileName), purpose="search")
It returns the call with a file ID and status uploaded which makes it seem like the upload and file processing is complete. I then passed that fileID to the search API, but in reality it had not completed processing and so the search API threw the error openai.error.InvalidRequestError: File is still processing. Check back later.
The returned file object looks like this (misleading):
It worked in google colab because the openai.File.create call and the search call were in 2 different cells, which gave it the time to finish processing as I executed the cells one by one. If I write all of the same code in one cell, it gave me the same error there.
So, I had to introduce a wait time for 4-7 seconds depending on the size of your data, time.sleep(5) after openai.File.create call before calling the openai.Engine("davinci").search call and that solved the issue. :)

Consume an AUTO ML Machine learning forecasting passing to a values of csv file

I have successfully run an AUTO ML forecasting model on Azure Machine Learning. Model is deployed on container service.
I would like to pass on a bunch of test values to the model from csv and generate forecast for a set of two variables, i.e. cluster and week number. The same options are available in Azure ML test functionality but how do I pass it on to all the csv values I have?
I tried using powerBI but it errors out once I try to apply the model. After reading 215 kb of file, it fails with error "context deadline exceeded"
I also tried to invoke the rest endpoint via excel add in - but it errors out saying bad url.
What's the easy way to pass on all the values of csv to rest endpoint and consume the result and store it back to csv?

The easiest way of testing the deployed service, in my opinion, is by modifying the consumption code pre-generated by Azure Machine Learning.
This code is available C#, Python and R. You can get it from Azure Machine Learning workspace > Endpoints > Select your deployed inference service > Select the tab Consume. The sample HTTP call is completely agnostic from the SDK, meaning that it should work on any environment without having to install any client.
Once you get access to this code, you can modify the json containing the sample data ("example_value") to pass to the service by the data is present in the csv. In Python, for instance, this is easily accomplished with Pandas.read_csv() method.
Eventually, you can modify the json to pass more than 1 record at the time. The output response will be modified accordingly.
Please, pay attention that the json must contain strings and must be encoded to be used in the HTTP protocol.

An unknown error has occurred in Cloud Function: GCP Python

Log Viewer
Unknown Error Image
I am running into an Unknown Error while executing a Cloud Function in GCP (Python).
Steps:
Running Cloud Function to retrieve the data from BigQuery DataStore and save the CSV file in GCP Storage.
Running Cloud Function to retrieve the data from BigQuery DataStore and save the CSV file in GCP Storage.
Running Cloud Function to retrieve the data from BigQuery DataStore and save the CSV file in GCP Storage.
It is executing successfully and files are stored in Storage. If you view the Logs it is showing Finished with Status Code 200 (attached is the log view image), which is success code.
However, in the end we are getting Unknown Error with some tracking number as per the attached screen shot.
Have anyone seen this earlier and suggestions for resolution.

Based on my follow up with Google Support, it seems this is related to Cloud Console itself.
The error message which we are experiencing is related to Cloud Function's Tester UI timing out. Currently it is set to 1 minute maximum even when Cloud Function itself has a timeout window different (between 1 min to 9mins maximum). So if we are using the CF UI Testing (Test Function option in CF), it will time out in 1 min, even though CF will successfully execute (Success Code 200 in view log)
As per the Google Support, CF Product team is working on delivering a more descriptive message (for 1 min UT Testing timeout) instead of this error. Also they are not sure if CF’s Product Team is going to set the CF’s testing UI timeout same as the CF timeout. No ETA yet.
So we will running our CF differently and not using CF UI Console for testing.

GEE Python API: Export image to Google Drive fails

Using GEE Python API in an application running with App Engine (on localhost), I am trying to export an image to a file in Google Drive. The task seems to start and complete successfully but no file is created in Google Drive.
I have tried to execute the equivalent javascript code in GEE code editor and this works, the file is created in Google Drive.
In python, I have tried various ways to start the task, but it always gives me the same result: the task completes but no file is created.
My python code is as follows:
landsat = ee.Image('LANDSAT/LC08/C01/T1_TOA/LC08_123032_20140515').select(['B4', 'B3', 'B2'])
geometry = ee.Geometry.Rectangle([116.2621, 39.8412, 116.4849, 40.01236])
task_config = {
'description': 'TEST_todrive_desc',
'scale': 30,
'region': geometry,
'folder':'GEEtest'
}
task = ee.batch.Export.image.toDrive(landsat, 'TEST_todrive', task_config)
ee.batch.data.startProcessing(task.id, task.config)
# Note: I also tried task.start() instead of this last line but the problem is the same, task completed, no file created.
# Printing the task list successively
for i in range(10):
tasks = ee.batch.Task.list()
print(tasks)
time.sleep(5)
In the printed task list, the status of the task goes from READY to RUNNING and then COMPLETED. But after completion no file is created in Google Drive in my folder "GEEtest" (nor anywhere else).
What am I doing wrong?

I think that the file is been generated and stored on the google drive of the 'Service Account' used for Python API not on your private account that is normally used when using the web code editor.

You can't pass a dictionary of arguments directly in python. You need to pass it using the kwargs convention (do a web search for more info). Basically, you just need to preface the task_config argument with double asteriks like this:
task = ee.batch.Export.image.toDrive(landsat, 'TEST_todrive', **task_config)
Then proceed as you have (I assume your use of task.config rather than task_config in the following line is a typo). Also note that you can query the task directly (using e.g. task.status()) and it may give more information about when / why the task failed. This isn't well documented as far as I can tell but you can read about it in the API code.

Using Microsoft Azure to run "a bunch of Python scripts on demand"

I'm trying to define an architecture where multiple Python scripts need to be run in parallel and on demand. Imagine the following setup:
script requestors (web API) -> Service Bus queue -> script execution -> result posted back to script requestor
To this end, the script requestor places a script request message on the queue, together with an API endpoint where the result should be posted back to. The script request message also contains the input for the script to be run.
The Service Bus queue decouples producers and consumers. A generic set of "workers" simply look for new messages on the queue, take the input message and call a Python script with said input. Then they post back the result to the API endpoint. But what strategies could I use to "run the Python scripts"?

One possible strategy could be to use Webjobs. Webjobs can execute Python scripts and run on a schedule. Let's say that you run a Webjob every 5 minutes, the Python script can pool the queue, do some processing and post the results back to you API.

Per my exprience, I think there are two strategies below you could use.
Developing a Python script for Azure HDInsight. Azure HDInsight as a platform based on Hadoop that has the power of parallel compute, you can try to refer to the doc https://azure.microsoft.com/en-us/documentation/articles/hdinsight-hadoop-streaming-python/ to know it.
Develoging a Python script based on the parallel compute framework like dispy or jug running Azure VMs.
Hope it helps. Best Regards.