Coming from the python world, fixtures are very useful (Fixtures defines a Python contract for reusable state / support logic, primarily for unit testing). I was wondering if there's similar support in Golang which can allow me to run my tests with some predefined fixtures like setting up server, tearing it down, doing some repeated tasks each time a test is run ? Can someone point me to some examples of doing the same in Golang ?
If you want to use the standard Go testing tools, you can define a function with the signature TestMain(m *testing.M) and put your fixture code in there.
From the testing package wiki:
It is sometimes necessary for a test program to do extra setup or teardown before or after testing. It is also sometimes necessary for a test to control which code runs on the main thread. To support these and other cases, if a test file contains a function:
func TestMain(m *testing.M)
then the generated test will call TestMain(m) instead of running the tests directly. TestMain runs in the main goroutine and can do whatever setup and teardown is necessary around a call to m.Run. It should then call os.Exit with the result of m.Run. When TestMain is called, flag.Parse has not been run. If TestMain depends on command-line flags, including those of the testing package, it should call flag.Parse explicitly.
A simple implementation of TestMain is:
func TestMain(m *testing.M) {
flag.Parse()
os.Exit(m.Run())
}
I know this is an old question, but this still came up in a search result so I thought I'd give a possible answer.
You can isolate code out to helper functions that return a "teardown" function to clean up after itself. Here's one possible way to go about starting a server and have it close at the end of the test case.
func setUpServer() (string, func()) {
h := func(w http.ResponseWriter, r *http.Request) {
code := http.StatusTeapot
http.Error(w, http.StatusText(code), code)
}
ts := httptest.NewServer(http.HandlerFunc(h))
return ts.URL, ts.Close
}
func TestWithServer(t *testing.T) {
u, close := setUpServer()
defer close()
rsp, err := http.Get(u)
assert.Nil(t, err)
assert.Equal(t, http.StatusTeapot, rsp.StatusCode)
}
This starts up a server with net/http/httptest and returns its URL along with a function that acts as the "teardown." This function is added to the defer stack so it's always called regardless of how the test case exits.
Optionally, you can pass in the *testing.T if you have a more complicated set up going on in there and you need to handle errors. This example shows the set up function returning a *url.URL instead of a URL formatted string, and the parse can possibly return an error.
func setUpServer(t *testing.T) (*url.URL, func()) {
h := func(w http.ResponseWriter, r *http.Request) {
code := http.StatusTeapot
http.Error(w, http.StatusText(code), code)
}
ts := httptest.NewServer(http.HandlerFunc(h))
u, err := url.Parse(ts.URL)
assert.Nil(t, err)
return u, ts.Close
}
func TestWithServer(t *testing.T) {
u, close := setUpServer(t)
defer close()
u.Path = "/a/b/c/d"
rsp, err := http.Get(u.String())
assert.Nil(t, err)
assert.Equal(t, http.StatusTeapot, rsp.StatusCode)
}
I wrote golang engine for use fixtures similar to pytest:
https://github.com/rekby/fixenv
Usage example:
package example
// db create database abd db struct, cached per package - call
// once and same db shared with all tests
func db(e Env)*DB{...}
// DbCustomer - create customer with random personal data
// but fixed name. Fixture result shared by test and subtests,
// then mean many calls Customer with same name will return same
// customer object.
// Call Customer with other name will create new customer
// and resurn other object.
func DbCustomer(e Env, name string) Customer {
// ... create customer
db(e).CustomerStore(cust)
// ...
return cust
}
// DbAccount create bank account for customer with given name.
func DbAccount(e Env, customerName, accountName string)Account{
cust := DbCustomer(e, customerName)
// ... create account
db(e).AccountStore(acc)
// ...
return acc
}
func TestFirstOwnAccounts(t *testing.T){
e := NewEnv(t)
// background:
// create database
// create customer bob
// create account from
accFrom := DbAccount(e, "bob", "from")
// get existed db, get existed bob, create account to
accTo := DbAccount(e, "bob", "to")
PutMoney(accFrom, 100)
SendMoney(accFrom, accTo, 20)
if accFrom != 80 {
t.Error()
}
if accTo != 20 {
t.Error()
}
// background:
// delete account to
// delete account from
// delete customer bob
}
func TestSecondTransferBetweenCustomers(t *testing.T){
e := NewEnv(t)
// background:
// get db, existed from prev test
// create customer bob
// create account main for bob
accFrom := DbAccount(e, "bob", "main")
// background:
// get existed db
// create customer alice
// create account main for alice
accTo := DbAccount(e, "alice", "main")
PutMoney(accFrom, 100)
SendMoney(accFrom, accTo, 20)
if accFrom != 80 {
t.Error()
}
if accTo != 20 {
t.Error()
}
// background:
// remove account of alice
// remove customer alice
// remove account of bob
// remove customer bob
}
// background:
// after all test finished drop database
Related
So I'm trying to send a transaction with python that use the function collectFromTiles in this smart contract:
https://polygonscan.com/address/0x068adc06faff086ee9ef826c0b83e7710e223c3d#code
This is the function's code from polygonscan:
function collectFromTiles(CollectOrder[] memory _orders,
uint8 _v,
bytes32 _r,
bytes32 _s) checkSignature(_v, _r, _s) external override {
uint arrayLen = _orders.length;
uint[] memory ids = new uint[](arrayLen);
uint[] memory amounts = new uint[](arrayLen);
uint cId;
uint cAmount;
// I hate this loop
for (uint i; i < _orders.length; i++) {
(cAmount, cId) = _handle_collect(_orders[i].x, _orders[i].y, _orders[i].z);
ids[i] = cId;
amounts[i] = cAmount;
}
// Reusing cId, cAmount to save on variables here
mint.trustedBatchMint(msg.sender, ids, amounts);
}
Next is the function to get the v,r,s:
modifier checkSignature(uint8 _v,
bytes32 _r,
bytes32 _s){
if (enableSignature) {
address signer = ECDSA.recover(_hashTypedDataV4(keccak256(
abi.encode(COLLECT_MESSAGE, msg.sender, nonces[msg.sender]++)
)), _v, _r, _s);
require(signers[signer], "MISSION_CONTROL: INVALID_SIGNATURE");
}
_;
}
and finally is the COLLECT_MESSAGE:
bytes32 private constant COLLECT_MESSAGE = keccak256("CollectMessage(address _user,uint256 _nonce)");
I'm looking for an outline of how I can extract the v,r,s using web3.py
I think he wants to do a transaction like this
https://polygonscan.com/tx/0x0a37f5af0aa5b15431e339a7dc4ce3bdc77f229e8bd69025a4740812e0c55e6f
where the collectFromTiles function is called
the ABI contract is not in this contract. it's in the contract
https://polygonscan.com/address/0x068adc06faff086ee9ef826c0b83e7710e223c3d#writeContract
I think he wants to know. how to get all parameters to call this method.
I'm using python web3
enter image description here
enter image description here
I am building an Android app that will allow the user to get a picture either by taking it in real time or uploading it from their saved images. Then, it will go through a machine learning script in python to determine their location. Before I completely connect to the algorithm, I am trying a test program that just returns a double.
from os.path import dirname, join
import csv
import random
filename = join(dirname(__file__), "new.csv")
def testlat():
return 30.0
def testlong():
return 30.0
These returned values are used in a Kotlin file that will then send those values to the Google Maps activity on the app for the location to be plotted.
class MainActivity : AppCompatActivity() {
var lat = 0.0
var long = 0.0
var dynamic = false
private val cameraRequest = 1888
lateinit var imageView: ImageView
lateinit var button: Button
private val pickImage = 100
private var imageUri: Uri? = null
var active = false
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
// Accesses the info image button
val clickMe = findViewById<ImageButton>(R.id.imageButton)
// Runs this function when the info icon is pressed by the user
// It will display the text in the variable infoText
clickMe.setOnClickListener {
Toast.makeText(this, infoText, Toast.LENGTH_LONG).show()
}
if (ContextCompat.checkSelfPermission(applicationContext, Manifest.permission.CAMERA)
== PackageManager.PERMISSION_DENIED) {
ActivityCompat.requestPermissions(
this,
arrayOf(Manifest.permission.CAMERA),
cameraRequest
)
}
imageView = findViewById(R.id.imageView)
val photoButton: Button = findViewById(R.id.button2)
photoButton.setOnClickListener {
val cameraIntent = Intent(MediaStore.ACTION_IMAGE_CAPTURE)
startActivityForResult(cameraIntent, cameraRequest)
dynamic = true
}
/*
The below will move to external photo storage once button2 is clicked
*/
button = findViewById(R.id.button)
button.setOnClickListener {
val gallery = Intent(Intent.ACTION_PICK, MediaStore.Images.Media.INTERNAL_CONTENT_URI)
startActivityForResult(gallery, pickImage)
}
// PYTHON HERE
if (! Python.isStarted()) {
Python.start(AndroidPlatform(this))
}
}
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
super.onActivityResult(requestCode, resultCode, data)
if (resultCode == RESULT_OK && requestCode == pickImage) {
imageUri = data?.data
imageView.setImageURI(imageUri)
// PYTHON HERE
val py = Python.getInstance()
val pyobj = py.getModule("main")
this.lat = pyobj.callAttr("testlat").toDouble()
this.long = pyobj.callAttr("testlong").toDouble()
/* Open the map after image has been received from user
This will be changed later to instead call the external object recognition/pathfinding
scripts and then pull up the map after those finish running
*/
val mapsIntent = Intent(this, MapsActivity::class.java)
startActivity(mapsIntent)
}
}
}
I set up chaquopy and the gradle is building successfully, but everytime I get to the python part of emulating the app, it crashes. I'm not quite sure why that is; I thought maybe the program was too much for the phone to handle but it is a very basic python script so I doubt that's the issue.
If your app crashes, you can find the stack trace in the Logcat.
In this case, it's probably caused by the line return = 30.0. The correct syntax is return 30.0.
Here I would like to offer a complete test case showing a simple TBB parallel_for construct causing deadlock in a Python application. Python frontend is combined with TBB backend using pybind11:
void backend_tbb(vector<int>& result, std::function<int (int)>& callback)
{
int nthreads = tbb::task_scheduler_init::default_num_threads();
const char* cnthreads = getenv("TBB_NUM_THREADS");
if (cnthreads) nthreads = std::max(1, atoi(cnthreads));
tbb::task_group group;
tbb::task_arena arena(nthreads, 1);
tbb::task_scheduler_init init(nthreads);
group.run( [&] {
tbb::parallel_for(tbb::blocked_range<int>(0, result.size()),
[&](const tbb::blocked_range<int>& range)
{
for (int i = range.begin(); i != range.end(); i++)
result[i] = callback(i);
});
});
arena.execute( [&] { group.wait(); });
}
void backend_serial(vector<int>& result, std::function<int (int)>& callback)
{
for (int i = 0; i < result.size(); i++)
result[i] = callback(i);
}
PYBIND11_MODULE(python_tbb, m)
{
pybind11::bind_vector<std::vector<int> >(m, "stdvectorint");
m.def("backend_tbb", &backend_tbb, "TBB backend");
m.def("backend_serial", &backend_serial, "Serial backend");
}
With backend_tbb uncommented, app deadlocks infinitely:
from python_tbb import *
import numpy as np
def callback(a) :
return int(a) * 10
def main() :
length = 10
result1 = stdvectorint(np.zeros(length, np.int32))
result2 = stdvectorint(np.zeros(length, np.int32))
backend_serial(result1, callback)
# XXX Uncomment this to get the program hang
#backend_tbb(result2, callback)
for i in range(length) :
print("%d vs %d" % (result1[i], result2[i]))
if __name__ == "__main__" :
main()
I've tried gil_scoped_acquire/gil_scoped_release, but no change. Similar solution reportedly works for OpenMP loop - but again no luck when I try to do the same for TBB. Please kindly advice on this case, thanks!
The issue is that TBB tasks get spawned inside task_arena instance associated with the task_group, but the waiting is done inside another task_arena instance, called arena. This can lead to the deadlock. To fix the issue, try wrapping the call to group.run() into task_arena.execute() similarly as it is done for group.wait().
However, in this case, the latter wrapping seems superfluous. So, you might want to combine two wrappings into one
arena.execute() {
group.run( /* ... */ );
group.wait();
}
which, in this particular example, makes the use of task_group unnecessary since the master thread spawns the tasks and immediately joins for participating in their execution, similarly as it is done in tbb::parallel_for. Thus, task_group can be merely removed.
I have a task wherein I have to make 700 REST API calls. I have used a loop, but the run time of the loop is more than the timeout of AWS Lambda service. Is there any way I can make the calls concurrently and merge the results. I am using node JS, but a solution in python is also welcome
Here is a sample of the code I am running now:
HitBTCData = {}
for item in exchanges:
itemlist = item.split('-')
response = requests.get('https://min-
api.cryptocompare.com/data/histominute?
fsym='+itemlist[0]+'&tsym='+itemlist[1]+'&limit=2000&aggregate=1
&e=HitBTC').json()
if itemlist[0]+itemlist[1] not in HitBTCData:
HitBTCData[itemlist[0]+itemlist[1]] = []
HitBTCData[itemlist[0]+itemlist[1]].append(response['Data'])
while response['Type'] != 1:
time = response['TimeFrom']
response = requests.get('https://min-
api.cryptocompare.com/data/histominute?
fsym='+itemlist[0]+'&tsym='+itemlist[1]+
'&limit=2000&aggregate=1&toTs='+str(time)+'&e=HitBTC').json()
if response['Type'] != 1:
HitBTCData[itemlist[0]+itemlist[1]]
.append(response['Data'])
HitBTCData[itemlist[0]+itemlist[1]] =
HitBTCData[itemlist[0]+itemlist[1]][::-1]
Use a loop in Node.js and the calls will run concurrently, but to make sure all of your calls are processed before exiting the lambda, you need to have some form of counter to make sure all of the queries have ended.
example:
var request = require('request');
var results = [];
var runningQueries = 0;
exchanges.forEach(function (item) {
++runningQueries;
callAPI (item, function (err, data) {
if (err) {
results.push(err)
--runningQueries;
} else {
results.push(data)
--runningQueries;
}
if (runningQueries == 0) {
// all call results have been collated, so do something
}
})
})
You'll need to build your own callAPI function. Also, I'm being a bit lazy with what I'm pushing to the array here, but it's just to give you a framework.
You could very easily do this with Promise.all():
const urls = [...];
const requestFunction = async (url) => {
// all your ifs go here
let data = await request(url);
// transform/sanitize your data
return data;
}
const allResults = await Promise.all(urls.map(requestFunction));
// you now have an array of results
The only downside to Promise.all() is that if any of the promises rejects, everything fails. So I would suggest catching errors and resolving with null or something similar.
You could do a for loop to generate your array of URLs, new Array(700).map(...), or since it looks like you already have some kind of array, just .map() that directly and do more transforms inside the request function.
I have a C header file which contains a series of classes, and I'm trying to write a function which will take those classes, and convert them to a python dict. A sample of the file is down the bottom.
Format would be something like
class CFGFunctions {
class ABC {
class AA {
file = "abc/aa/functions"
class myFuncName{ recompile = 1; };
};
class BB
{
file = "abc/bb/functions"
class funcName{
recompile=1;
}
}
};
};
I'm hoping to turn it into something like
{CFGFunctions:{ABC:{AA:"myFuncName"}, BB:...}}
# Or
{CFGFunctions:{ABC:{AA:{myFuncName:"string or list or something"}, BB:...}}}
In the end, I'm aiming to get the filepath string (which is actually a path to a folder... but anyway), and the class names in the same class as the file/folder path.
I've had a look on SO, and google and so on, but most things I've found have been about splitting lines into dicts, rather then n-deep 'blocks'
I know I'll have to loop through the file, however, I'm not sure the most efficient way to convert it to the dict.
I'm thinking I'd need to grab the outside class and its relevant brackets, then do the same for the text remaining inside.
If none of that makes sense, it's cause I haven't quite made sense of the process myself haha
If any more info is needed, I'm happy to provide.
The following code is a quick mockup of what I'm sorta thinking...
It is most likely BROKEN and probably does NOT WORK. but its sort of the process that I'm thinking of
def get_data():
fh = open('CFGFunctions.h', 'r')
data = {} # will contain final data model
# would probably refactor some of this into a function to allow better looping
start = "" # starting class name
brackets = 0 # number of brackets
text= "" # temp storage for lines inside block while looping
for line in fh:
# find the class (start
mt = re.match(r'Class ([\w_]+) {', line)
if mt:
if start == "":
start = mt.group(1)
else:
# once we have the first class, find all other open brackets
mt = re.match(r'{', line)
if mt:
# and inc our counter
brackets += 1
mt2 = re.match(r'}', line)
if mt2:
# find the close, and decrement
brackets -= 1
# if we are back to the initial block, break out of the loop
if brackets == 0:
break
text += line
data[start] = {'tempText': text}
====
Sample file
class CfgFunctions {
class ABC {
class Control {
file = "abc\abc_sys_1\Modules\functions";
class assignTracker {
description = "";
recompile = 1;
};
class modulePlaceMarker {
description = "";
recompile = 1;
};
};
class Devices
{
file = "abc\abc_sys_1\devices\functions";
class registerDevice { recompile = 1; };
class getDeviceSettings { recompile = 1; };
class openDevice { recompile = 1; };
};
};
};
EDIT:
If possible, if I have to use a package, I'd like to have it in the programs directory, not the general python libs directory.
As you detected, parsing is necessary to do the conversion. Have a look at the package PyParsing, which is a fairly easy-to-use library to implement parsing in your Python program.
Edit: This is a very symbolic version of what it would take to recognize a very minimalistic grammer - somewhat like the example at the top of the question. It won't work, but it might put you in the right direction:
from pyparsing import ZeroOrMore, OneOrMore, \
Keyword, Literal
test_code = """
class CFGFunctions {
class ABC {
class AA {
file = "abc/aa/functions"
class myFuncName{ recompile = 1; };
};
class BB
{
file = "abc/bb/functions"
class funcName{
recompile=1;
}
}
};
};
"""
class_tkn = Keyword('class')
lbrace_tkn = Literal('{')
rbrace_tkn = Literal('}')
semicolon_tkn = Keyword(';')
assign_tkn = Keyword(';')
class_block = ( class_tkn + identifier + lbrace_tkn + \
OneOrMore(class_block | ZeroOrMore(assignment)) + \
rbrace_tkn + semicolon_tkn \
)
def test_parser(test):
try:
results = class_block.parseString(test)
print test, ' -> ', results
except ParseException, s:
print "Syntax error:", s
def main():
test_parser(test_code)
return 0
if __name__ == '__main__':
main()
Also, this code is only the parser - it does not generate any output. As you can see in the PyParsing docs, you can later add the actions you want. But the first step would be to recognize the what you want to translate.
And a last note: Do not underestimate the complexities of parsing code... Even with a library like PyParsing, which takes care of much of the work, there are many ways to get mired in infinite loops and other amenities of parsing. Implement things step-by-step!
EDIT: A few sources for information on PyParsing are:
http://werc.engr.uaf.edu/~ken/doc/python-pyparsing/HowToUsePyparsing.html
http://pyparsing.wikispaces.com/
(Particularly interesting is http://pyparsing.wikispaces.com/Publications, with a long list of articles - several of them introductory - on PyParsing)
http://pypi.python.org/pypi/pyparsing_helper is a GUI for debugging parsers
There is also a 'tag' Pyparsing here on stackoverflow, Where Paul McGuire (the PyParsing author) seems to be a frequent guest.
* NOTE: *
From PaulMcG in the comments below: Pyparsing is no longer hosted on wikispaces.com. Go to github.com/pyparsing/pyparsing