We Keep Coding

Why is my Android app crashing every time I run a python script with chaquopy?

I am building an Android app that will allow the user to get a picture either by taking it in real time or uploading it from their saved images. Then, it will go through a machine learning script in python to determine their location. Before I completely connect to the algorithm, I am trying a test program that just returns a double.
from os.path import dirname, join
import csv
import random
filename = join(dirname(__file__), "new.csv")
def testlat():
return 30.0
def testlong():
return 30.0
These returned values are used in a Kotlin file that will then send those values to the Google Maps activity on the app for the location to be plotted.
class MainActivity : AppCompatActivity() {
var lat = 0.0
var long = 0.0
var dynamic = false
private val cameraRequest = 1888
lateinit var imageView: ImageView
lateinit var button: Button
private val pickImage = 100
private var imageUri: Uri? = null
var active = false
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
// Accesses the info image button
val clickMe = findViewById<ImageButton>(R.id.imageButton)
// Runs this function when the info icon is pressed by the user
// It will display the text in the variable infoText
clickMe.setOnClickListener {
Toast.makeText(this, infoText, Toast.LENGTH_LONG).show()
}
if (ContextCompat.checkSelfPermission(applicationContext, Manifest.permission.CAMERA)
== PackageManager.PERMISSION_DENIED) {
ActivityCompat.requestPermissions(
this,
arrayOf(Manifest.permission.CAMERA),
cameraRequest
)
}
imageView = findViewById(R.id.imageView)
val photoButton: Button = findViewById(R.id.button2)
photoButton.setOnClickListener {
val cameraIntent = Intent(MediaStore.ACTION_IMAGE_CAPTURE)
startActivityForResult(cameraIntent, cameraRequest)
dynamic = true
}
/*
The below will move to external photo storage once button2 is clicked
*/
button = findViewById(R.id.button)
button.setOnClickListener {
val gallery = Intent(Intent.ACTION_PICK, MediaStore.Images.Media.INTERNAL_CONTENT_URI)
startActivityForResult(gallery, pickImage)
}
// PYTHON HERE
if (! Python.isStarted()) {
Python.start(AndroidPlatform(this))
}
}
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
super.onActivityResult(requestCode, resultCode, data)
if (resultCode == RESULT_OK && requestCode == pickImage) {
imageUri = data?.data
imageView.setImageURI(imageUri)
// PYTHON HERE
val py = Python.getInstance()
val pyobj = py.getModule("main")
this.lat = pyobj.callAttr("testlat").toDouble()
this.long = pyobj.callAttr("testlong").toDouble()
/* Open the map after image has been received from user
This will be changed later to instead call the external object recognition/pathfinding
scripts and then pull up the map after those finish running
*/
val mapsIntent = Intent(this, MapsActivity::class.java)
startActivity(mapsIntent)
}
}
}
I set up chaquopy and the gradle is building successfully, but everytime I get to the python part of emulating the app, it crashes. I'm not quite sure why that is; I thought maybe the program was too much for the phone to handle but it is a very basic python script so I doubt that's the issue.

If your app crashes, you can find the stack trace in the Logcat.
In this case, it's probably caused by the line return = 30.0. The correct syntax is return 30.0.

Convert from Tensorflow -> CoreML 3.0 for slot/intent detection

I am trying to use some of the models created by this codebase (Slot-Filling-Understanding-Using-RNNs) in my Swift application.
I was able to convert lstm_nopooling, lstm_nopooling300 and lstm to convert to CoreML.
In model.py I used this code:
def save_model(self):
joblib.dump(self.summary, 'models/' + self.name + '.txt')
self.model.save('models/' + self.name + '.h5')
try:
coreml_model = coremltools.converters.keras.convert(self.model, input_names="main_input", output_names=["intent_output","slot_output"])
coreml_model.save('models/' + self.name + '.mlmodel')
except:
pass
print("Saved model to disk")
I am trying to convert the vectors back to an intent and slots.
I have this, but
func tokenizeSentences(instr: String) -> [Int] {
let s = instr.lowercased().split(separator: " ")
var ret = [Int]()
if let filepath = Bundle.main.path(forResource: "atis.dict.vocab", ofType: "csv") {
do {
let contents = try String(contentsOfFile: filepath)
print(contents)
var lines = contents.split { $0.isNewline }
var pos = 0
for word in s {
if let index = lines.firstIndex(of: word) {
print(index.description + " " + word)
ret.append(index)
}
}
return ret
} catch {
// contents could not be loaded
}
} else {
// example.txt not found!
}
return ret
}
func predictText(instr:String) {
let model = lstm_nopooling300()
guard let mlMultiArray = try? MLMultiArray(shape:[20,1,1],
dataType:MLMultiArrayDataType.int32) else {
fatalError("Unexpected runtime error. MLMultiArray")
}
let tokens = tokenizeSentences(instr: instr)
for (index, element) in tokens.enumerated() {
mlMultiArray[index] = NSNumber(integerLiteral: element)
}
guard let m = try? model.prediction(input: lstm_nopooling300Input.init(main_input: mlMultiArray))
else {
fatalError("Unexpected runtime error. MLMultiArray")
}
let mm = m.intent_output
let length = mm.count
let doublePtr = mm.dataPointer.bindMemory(to: Double.self, capacity: length)
let doubleBuffer = UnsafeBufferPointer(start: doublePtr, count: length)
let output = Array(doubleBuffer)
print("******** intents \(mm.count) ********")
print(output)
let mn = m.slot_output
let length2 = mn.count
let doublePtr2 = mm.dataPointer.bindMemory(to: Double.self, capacity: length2)
let doubleBuffer2 = UnsafeBufferPointer(start: doublePtr2, count: length2)
let output2 = Array(doubleBuffer2)
print("******** slots \(mn.count) ********")
print(output2)
}
}
When I run my code I get this, truncated, for intents:
******** intents 540 ********
[0.0028914143331348896, 0.0057610333897173405, 4.1651015635579824e-05,
0.15935245156288147, 5.6665314332349226e-05, 5.7797817134996876e-05, 0.0044302307069301605, 0.00012486864579841495, 0.0004683282459154725, 0.003053907072171569, 3.806956738117151e-05, 0.012112349271774292, 5.861848694621585e-05, 0.0031344725284725428,
The problem, I believe, is that the ids are in a pickle file, so in atis/atis.train.pkl perhaps.
All I did was train the models and convert those I could to CoreML and now I am trying to use it, but not certain what to do next.
I have a textfield and I enter 'current weather in london' and I hope to get something similar to (this is from running example.py)
{'intent': 'weather_intent', 'slots': [{'name': 'city', 'value': 'London'}]}
Here is the coreml input/output

Thanks to #MatthijsHollemans I was able to figure out what to do.
In data_processing.py I added these:
with open('atis/wordlist.csv', 'w') as f:
for key in ids2words.keys():
f.write("%s\n"%(ids2words.keys[key]))
with open('atis/wordlist_slots.csv', 'w') as f:
for key in ids2slots.keys():
f.write("%s\n"%(ids2slots[key]))
with open('atis/wordlist_intents.csv', 'w') as f:
for key in ids2intents.keys():
f.write("%s\n"%(ids2intents[key]))
This allows me to tokenize correctly, using wordlist.csv.
Then when I get the response back, and using mm.count was wrong, it should have been output.count for example, I could see the intents.
Look for the element that has the largest value, and then you look up in wordlist_intents.csv (I turned this into an array, probably should be a dictionary) to find the likely intent.
I still need to do the slots, but the basic idea is the same.
The key was to output the dictionary used in python to a csv file and then import that into the project.
UPDATE
I realized that when mm.count was 540 that is because it can have 20 words in the sentence and so it can return that many. So in my case I needed to split the word by space and then loop that many times as I won't get more slots than I have words.
I am doing this in SwiftUI so I also had to create an observable so I could use EnvironmentObject to pass in the terms.
So, to properly loop over the double array in memory I am including the latest code that does what I expect.
func predictText(instr:String) {
let model = lstm_nopooling300()
guard let mlMultiArray = try? MLMultiArray(shape:[20,1,1],
dataType:MLMultiArrayDataType.int32) else {
fatalError("Unexpected runtime error. MLMultiArray")
}
let tokens = tokenizeSentences(instr: instr)
let sent = instr.split(separator: " ")
print(instr)
print(tokens)
for (index, element) in tokens.enumerated() {
mlMultiArray[index] = NSNumber(integerLiteral: element)
}
guard let m = try? model.prediction(input: lstm_nopooling300Input.init(main_input: mlMultiArray))
else {
fatalError("Unexpected runtime error. MLMultiArray")
}
let mm = m.intent_output
let length = mm.count
let doublePtr = mm.dataPointer.bindMemory(to: Double.self, capacity: length)
var intents = [String]()
for i in 0...sent.count - 1 {
let doubleBuffer = UnsafeBufferPointer(start: doublePtr + i * 27, count: 27)
let output = Array(doubleBuffer)
let intent = convertVectorToIntent(vector: output)
intents.append(intent)
}
print(intents)
let mn = m.slot_output
let length2 = mn.count
let doublePtr2 = mn.dataPointer.bindMemory(to: Double.self, capacity: length2)
var slots = [String]()
for i in 0...sent.count - 1 {
let doubleBuffer2 = UnsafeBufferPointer(start: doublePtr2 + i * 133, count: 133)
let output2 = Array(doubleBuffer2)
var slot = ""
slot = convertVectorToSlot(vector: output2)
slots.append(slot)
slots.append(sent[i].description)
}
print(slots)
}

terraform) how to stop instance after running script(inline)?

I create 30 instance to run crawl script, and want to stop aws instances after complete running this command, python3 aws_crawl.py, but I cannot find how to stop current aws instance by command line.
Please see my main.tf code below:
resource "aws_instance" "crawl_worker" {
ami = "${var.ami_id}"
instance_type = "t3a.nano"
security_groups = ["${var.aws_security_group}"]
key_name = "secret"
count = 10
tags = {
Name = "crawl_worker_${count.index}"
}
connection {
type = "ssh"
host = "${self.public_ip}"
user = "ubuntu"
private_key = "${file("~/secret.pem")}"
timeout = "50m"
}
provisioner "remote-exec" {
inline = [
"python3 aws_crawl.py"
]
}
}

Fixtures in Golang testing

Coming from the python world, fixtures are very useful (Fixtures defines a Python contract for reusable state / support logic, primarily for unit testing). I was wondering if there's similar support in Golang which can allow me to run my tests with some predefined fixtures like setting up server, tearing it down, doing some repeated tasks each time a test is run ? Can someone point me to some examples of doing the same in Golang ?

If you want to use the standard Go testing tools, you can define a function with the signature TestMain(m *testing.M) and put your fixture code in there.
From the testing package wiki:
It is sometimes necessary for a test program to do extra setup or teardown before or after testing. It is also sometimes necessary for a test to control which code runs on the main thread. To support these and other cases, if a test file contains a function:
func TestMain(m *testing.M)
then the generated test will call TestMain(m) instead of running the tests directly. TestMain runs in the main goroutine and can do whatever setup and teardown is necessary around a call to m.Run. It should then call os.Exit with the result of m.Run. When TestMain is called, flag.Parse has not been run. If TestMain depends on command-line flags, including those of the testing package, it should call flag.Parse explicitly.
A simple implementation of TestMain is:
func TestMain(m *testing.M) {
flag.Parse()
os.Exit(m.Run())
}

I know this is an old question, but this still came up in a search result so I thought I'd give a possible answer.
You can isolate code out to helper functions that return a "teardown" function to clean up after itself. Here's one possible way to go about starting a server and have it close at the end of the test case.
func setUpServer() (string, func()) {
h := func(w http.ResponseWriter, r *http.Request) {
code := http.StatusTeapot
http.Error(w, http.StatusText(code), code)
}
ts := httptest.NewServer(http.HandlerFunc(h))
return ts.URL, ts.Close
}
func TestWithServer(t *testing.T) {
u, close := setUpServer()
defer close()
rsp, err := http.Get(u)
assert.Nil(t, err)
assert.Equal(t, http.StatusTeapot, rsp.StatusCode)
}
This starts up a server with net/http/httptest and returns its URL along with a function that acts as the "teardown." This function is added to the defer stack so it's always called regardless of how the test case exits.
Optionally, you can pass in the *testing.T if you have a more complicated set up going on in there and you need to handle errors. This example shows the set up function returning a *url.URL instead of a URL formatted string, and the parse can possibly return an error.
func setUpServer(t *testing.T) (*url.URL, func()) {
h := func(w http.ResponseWriter, r *http.Request) {
code := http.StatusTeapot
http.Error(w, http.StatusText(code), code)
}
ts := httptest.NewServer(http.HandlerFunc(h))
u, err := url.Parse(ts.URL)
assert.Nil(t, err)
return u, ts.Close
}
func TestWithServer(t *testing.T) {
u, close := setUpServer(t)
defer close()
u.Path = "/a/b/c/d"
rsp, err := http.Get(u.String())
assert.Nil(t, err)
assert.Equal(t, http.StatusTeapot, rsp.StatusCode)
}

I wrote golang engine for use fixtures similar to pytest:
https://github.com/rekby/fixenv
Usage example:
package example
// db create database abd db struct, cached per package - call
// once and same db shared with all tests
func db(e Env)*DB{...}
// DbCustomer - create customer with random personal data
// but fixed name. Fixture result shared by test and subtests,
// then mean many calls Customer with same name will return same
// customer object.
// Call Customer with other name will create new customer
// and resurn other object.
func DbCustomer(e Env, name string) Customer {
// ... create customer
db(e).CustomerStore(cust)
// ...
return cust
}
// DbAccount create bank account for customer with given name.
func DbAccount(e Env, customerName, accountName string)Account{
cust := DbCustomer(e, customerName)
// ... create account
db(e).AccountStore(acc)
// ...
return acc
}
func TestFirstOwnAccounts(t *testing.T){
e := NewEnv(t)
// background:
// create database
// create customer bob
// create account from
accFrom := DbAccount(e, "bob", "from")
// get existed db, get existed bob, create account to
accTo := DbAccount(e, "bob", "to")
PutMoney(accFrom, 100)
SendMoney(accFrom, accTo, 20)
if accFrom != 80 {
t.Error()
}
if accTo != 20 {
t.Error()
}
// background:
// delete account to
// delete account from
// delete customer bob
}
func TestSecondTransferBetweenCustomers(t *testing.T){
e := NewEnv(t)
// background:
// get db, existed from prev test
// create customer bob
// create account main for bob
accFrom := DbAccount(e, "bob", "main")
// background:
// get existed db
// create customer alice
// create account main for alice
accTo := DbAccount(e, "alice", "main")
PutMoney(accFrom, 100)
SendMoney(accFrom, accTo, 20)
if accFrom != 80 {
t.Error()
}
if accTo != 20 {
t.Error()
}
// background:
// remove account of alice
// remove customer alice
// remove account of bob
// remove customer bob
}
// background:
// after all test finished drop database

How do I get event log information from pyethereum?

I've got some tests that create contracts with pyethereum and do various things with them, but I'm puzzled over how to get information about events they log.
A simplified example:
from ethereum import tester as t
s = t.state()
code = """contract LogTest {
event LogMyNumber(uint);
function LogTest() {
}
function logSomething() {
LogMyNumber(4);
}
}"""
logtest = t.state().abi_contract(code, language='solidity', sender=t.k0)
logtest.logSomething()
#number_i_logged = WHAT DO I DO HERE?
#print "You logged the number %d" % (number_i_logged)
I run this and get:
No handlers could be found for logger "eth.pow"
{'': 4, '_event_type': 'LogMyNumber'}
That json that's getting printed is the information I want, but can someone explain, or point me to an example, of how I might capture it and load it into a variable in python so that I can check it and do something with it? There seems to be something called log_listener that you can pass into abi_contract that looks like it's related but I couldn't figure out what to do with it.

I know you've been waiting for an answer for quite a long time, but in case someone else is wondering, here is goes:
log_listeners you mentioned is the way to go. You can find some sample code using it in pyethereum's tests, and here is your fixed code:
from ethereum import tester as t
s = t.state()
code = """contract LogTest {
event LogMyNumber(uint loggedNumber);
function LogTest() {
}
function logSomething() {
LogMyNumber(4);
}
}"""
c = s.abi_contract(code, language='solidity', sender=t.k0)
o = []
s.block.log_listeners.append(lambda x: o.append(c._translator.listen(x)))
c.logSomething()
assert len(o) == 1
assert o == [{"_event_type": 'LogMyNumber', "loggedNumber": 4}]
number_i_logged = o[0]["loggedNumber"]
print "You logged the number %d" % (number_i_logged)