We Keep Coding

Error 'rlp: expected List' when calling a smart contract function using web3py

I'm trying to call a function on a smart contract deployed on SmartBCH.
This is the function ABI:
{
"inputs": [],
"name": "startStake",
"outputs": [],
"stateMutability": "nonpayable",
"type": "function"
}
This is the Python code:
from web3 import Web3
w3 = Web3(Web3.HTTPProvider('https://smartbch.greyh.at'))
if not w3.isConnected():
w3 = Web3(Web3.HTTPProvider('https://smartbch.fountainhead.cash/mainnet'))
def start_celery_stake():
import server_settings
ABI = open("ABIs/CLY-ABI.json", "r") # ABI for CLY token
abi = json.loads(ABI.read())
contract = w3.eth.contract(address="0x7642Df81b5BEAeEb331cc5A104bd13Ba68c34B91", abi=abi)
nonce = w3.eth.get_transaction_count(portfolio_address)
stake_cly_tx = contract.functions.startStake().buildTransaction({'chainId': 10000, 'gas': 64243, 'maxFeePerGas': w3.toWei('2', 'gwei'), 'maxPriorityFeePerGas': w3.toWei('2', 'gwei'), 'nonce': nonce})
private_key = server_settings.PORTFOLIO_PRIV_KEY
signed_txn = w3.eth.account.sign_transaction(stake_cly_tx, private_key=private_key)
signed_txn.rawTransaction
w3.eth.send_raw_transaction(signed_txn.rawTransaction)
The private key is stored as a string in server_settings.PORTFOLIO_PRIV_KEY.
The error I got is:
Traceback (most recent call last):
File "/usr/lib/python3.8/code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 13, in <module>
File "/home/administrador/Descargas/BCH/transparency_portal/venv/lib/python3.8/site-packages/web3/eth.py", line 722, in send_raw_transaction
return self._send_raw_transaction(transaction)
File "/home/administrador/Descargas/BCH/transparency_portal/venv/lib/python3.8/site-packages/web3/module.py", line 57, in caller
result = w3.manager.request_blocking(method_str,
File "/home/administrador/Descargas/BCH/transparency_portal/venv/lib/python3.8/site-packages/web3/manager.py", line 198, in request_blocking
return self.formatted_response(response,
File "/home/administrador/Descargas/BCH/transparency_portal/venv/lib/python3.8/site-packages/web3/manager.py", line 171, in formatted_response
raise ValueError(response["error"])
ValueError: {'code': -32000, 'message': 'rlp: expected List'}
This is the raw transaction, which I got when calling signed_txn.rawTransaction:
HexBytes('0x02f87182271081fa8477359400847735940082faf3947642df81b5beaeeb331cc5a104bd13ba68c34b91808428e9d35bc080a0c5570eba5692b8beb1e1dd58907ab709f35409f95daddc8bf568fcfcbf1a4320a02250b01810c2f801fb7afec9ca3f24ffea84869f42c3c91e2c8df245af8bc2b7')
According to a Ethereum tx decoder, this raw transaction is not correct, so perhaps something isn't formatted properly.

This is the solution:
stake_cly_tx = contract.functions.startStake().buildTransaction(
{'chainId': 10000,
'gas': 108287,
'gasPrice': w3.toWei('1.05', 'gwei'),
'nonce': nonce})
The problem came from 'maxFeePerGas' and 'maxPriorityFeePerGas' parameters. These are obsolete.

'S3' object has no attribute 'get_object_lock_configuration'

I'm trying to implement the object lock feature but functions (get/put_object_lock_configuration) are not available :
>>> import boto3
>>> boto3.__version__
'1.17.64'
>>> client = boto3.client('s3')
>>> client.get_object_lock_configuration
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/botocore/client.py", line 553, in __getattr__
self.__class__.__name__, item)
AttributeError: 'S3' object has no attribute 'get_object_lock_configuration'
>>> client.get_object_lock_configuration(Bucket='tst', ExpectedBucketOwner='tst')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/botocore/client.py", line 553, in __getattr__
self.__class__.__name__, item)
AttributeError: 'S3' object has no attribute 'get_object_lock_configuration'
Edit:
object lock functions not showing in python (tab tab) :
>>> client.get_object_
client.get_object_acl( client.get_object_tagging( client.get_object_torrent(
>>> client.put_object
client.put_object( client.put_object_acl( client.put_object_tagging(

get_object_lock_configuration is a function not a property.
You need to call it like that:
response = client.get_object_lock_configuration(
Bucket='string',
ExpectedBucketOwner='string'
)

The syntanx to call function client.get_object_lock_configuration:
response = client.get_object_lock_configuration(
Bucket='string',
ExpectedBucketOwner='string'
)
Syntax to call function client.put_object_lock_configuration:
response = client.put_object_lock_configuration(
Bucket='string',
ObjectLockConfiguration={
'ObjectLockEnabled': 'Enabled',
'Rule': {
'DefaultRetention': {
'Mode': 'GOVERNANCE'|'COMPLIANCE',
'Days': 123,
'Years': 123
}
}
},
RequestPayer='requester',
Token='string',
ContentMD5='string',
ExpectedBucketOwner='string'
)
To know more please refer to this: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_object_lock_configuration
Edit:
Example Code:
import json
import boto3
client = boto3.client('s3')
response = client.get_object_lock_configuration(
Bucket='anynewname')
print(response)
Output sytanx:
{
'ObjectLockConfiguration': {
'ObjectLockEnabled': 'Enabled',
'Rule': {
'DefaultRetention': {
'Mode': 'GOVERNANCE'|'COMPLIANCE',
'Days': 123,
'Years': 123
}
}
}
}
Note: It will throw an error if object lock config is not set on bucket.
{
"errorMessage": "An error occurred (ObjectLockConfigurationNotFoundError) when calling the GetObjectLockConfiguration operation: Object Lock configuration does not exist for this bucket",
"errorType": "ClientError",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 7, in lambda_handler\n response = client.get_object_lock_configuration(\n",
" File \"/var/runtime/botocore/client.py\", line 357, in _api_call\n return self._make_api_call(operation_name, kwargs)\n",
" File \"/var/runtime/botocore/client.py\", line 676, in _make_api_call\n raise error_class(parsed_response, operation_name)\n"
]
}

Unable to deserialize to object: type, KeyError: ' key: int; value: str '

I am writing a python script to create a user in azure devops using the python client library for azure-devops-rest-api.
I am using the add_user_entitlement() function of MemberEntitlementManagementClient.
Link to the code of that client:
https://github.com/microsoft/azure-devops-python-api/blob/dev/azure-devops/azure/devops/v5_0/member_entitlement_management/member_entitlement_management_client.py
Corresponding REST API documentation:
https://learn.microsoft.com/en-us/rest/api/azure/devops/memberentitlementmanagement/user%20entitlements/add?view=azure-devops-rest-4.1
I wrote the code:
from azure.devops.connection import Connection
from azure.devops.v5_0.member_entitlement_management.models import *
import pprint
personal_access_token = <my token>
credentials = BasicAuthentication('', personal_access_token)
connection = Connection(base_url=organization_url, creds=credentials)
member_ent_mngmnt_client = connection.clients_v5_0.get_member_entitlement_management_client()
# List member entitlements
resp = member_ent_mngmnt_client.get_user_entitlements()
print(resp)
# ------ Add user entitlement -----------------
access_level = AccessLevel("express", None, None, None, None, None, None)
print(access_level)
graph_user = GraphUser(None, None, None, None, None, None, None, "user", None, None, "anaya.john#dynactionize.onmicrosoft.com", None, None, None)
print(graph_user)
user_entitlement = UserEntitlement(None, None, None, None, None, None, graph_user)
print(user_entitlement)
# This is to check what is sent as the request body of REST API POST request
content = member_ent_mngmnt_client._serialize.body(user_entitlement, 'UserEntitlement')
print("\n Content : \n")
print(content)
# Add user entitlement
resp = member_ent_mngmnt_client.add_user_entitlement(user_entitlement)
print("\n Result: \n")
print(resp)
But I got the output with an error msrest.exceptions.DeserializationError::
{'additional_properties': {}, 'account_license_type': 'express', 'assignment_source': None, 'license_display_name': None, 'licensing_source': None, 'msdn_license_type': None, 'status': None, 'status_message': None}
{'additional_properties': {}, '_links': None, 'descriptor': None, 'display_name': None, 'url': None, 'legacy_descriptor': None, 'origin': None, 'origin_id': None, 'subject_kind': 'user', 'domain': None, 'mail_address': None, 'principal_name': 'anaya.john#mydomain.com', 'is_deleted_in_origin': None, 'metadata_update_date': None, 'meta_type': None}
{'additional_properties': {}, 'access_level': None, 'extensions': None, 'group_assignments': None, 'id': None, 'last_accessed_date': None, 'project_entitlements': None, 'user': <azure.devops.v5_0.member_entitlement_management.models.GraphUser object at 0x000002147F444FD0>}
Content :
{'user': {'subjectKind': 'user', 'principalName': 'anaya.john#dynactionize.onmicrosoft.com'}}
Traceback (most recent call last):
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1294, in _deserialize
value = self.deserialize_data(raw_value, attr_desc['type'])
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1447, in deserialize_data
return self.deserialize_type[iter_type](data, data_type[1:-1])
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1478, in deserialize_iter
return [self.deserialize_data(a, iter_type) for a in attr]
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1478, in <listcomp>
return [self.deserialize_data(a, iter_type) for a in attr]
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1447, in deserialize_data
return self.deserialize_type[iter_type](data, data_type[1:-1])
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1494, in deserialize_dict
return {k: self.deserialize_data(v, dict_type) for k, v in attr.items()}
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1494, in <dictcomp>
return {k: self.deserialize_data(v, dict_type) for k, v in attr.items()}
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1449, in deserialize_data
obj_type = self.dependencies[data_type]
KeyError: ' key: int; value: str '
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:/Users/Anjana/Desktop/scripts_O365/az_devops_clientAPI_PAT.py", line 152, in <module>
resp = member_ent_mngmnt_client.add_user_entitlement(user_entitlement)
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\azure\devops\v5_0\member_entitlement_management\member_entitlement_management_client.py", line 184, in add_user_entitlement
return self._deserialize('UserEntitlementsPostResponse', response)
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1228, in __call__
return self._deserialize(target_obj, data)
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1294, in _deserialize
value = self.deserialize_data(raw_value, attr_desc['type'])
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1460, in deserialize_data
return self._deserialize(obj_type, data)
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1298, in _deserialize
raise_with_traceback(DeserializationError, msg, err)
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\exceptions.py", line 51, in raise_with_traceback
raise error.with_traceback(exc_traceback)
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1294, in _deserialize
value = self.deserialize_data(raw_value, attr_desc['type'])
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1447, in deserialize_data
return self.deserialize_type[iter_type](data, data_type[1:-1])
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1478, in deserialize_iter
return [self.deserialize_data(a, iter_type) for a in attr]
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1478, in <listcomp>
return [self.deserialize_data(a, iter_type) for a in attr]
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1447, in deserialize_data
return self.deserialize_type[iter_type](data, data_type[1:-1])
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1494, in deserialize_dict
return {k: self.deserialize_data(v, dict_type) for k, v in attr.items()}
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1494, in <dictcomp>
return {k: self.deserialize_data(v, dict_type) for k, v in attr.items()}
File "C:\Users\Anjana\AppData\Local\Programs\Python\Python37\lib\site-packages\msrest\serialization.py", line 1449, in deserialize_data
obj_type = self.dependencies[data_type]
msrest.exceptions.DeserializationError: Unable to deserialize to object: type, KeyError: ' key: int; value: str '
Can anyone help me to solve this error?

The deserialization error says that python cannot deserialize the response got to an object of type UserEntitlementsPostResponse. This happens when the response is not the expected one.
I edited the add_user_entitlement() function of MemberEntitlementManagementClient class of the python client by adding a line to print the response of the POST request. The response was:
{'operationResult': {'isSuccess': False, 'errors': [{'key': 5032, 'value': 'Access Denied: This user needs the following permission(s) on the resource Users to perform this action: Add Users'}], 'userId': '261d25ad091b', 'result': None}, 'isSuccess': False, 'userEntitlement': None}
Obviously, this was not the expected result. And this cannot be converted to a UserEntitlementsPostResponse type object. Hence, the error occurred.
As the error suggests, after adding the user (who runs the script) to a group 'project collection administrators' in our azure devops organization, the script worked correctly.
This step was important as the microsoft documentation for azure devops REST API ( https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/add-organization-users?view=azure-devops#prerequisites ) says:
To access and manage users, you must have Azure DevOps project
collection administrator or organization owner permissions.

How do you create a Adwords BigQuery Transfer and Transfer Runs using the bigquery_datatransfer Python client?

I have been able to successfully authenticate, and list transfers and transfer runs. But I keep running into the issue of not being able to create a transfer because the transfer config is incorrect.
Here's the Transfer Config I have tried:
transferConfig = {
'data_refresh_window_days': 1,
'data_source_id': "adwords",
'destination_dataset_id': "AdwordsMCC",
'disabled': False,
'display_name': "TestR",
'name': "TestR",
'schedule': "every day 07:00",
'params': {
"customer_id": "999999999" -- Changed Number
}
}
response = client.create_transfer_config(parent, transferConfig)
print(response)
And this is the error I get:
Traceback (most recent call last):
File "./create_transfer.py", line 84, in <module>
main()
File "./create_transfer.py", line 61, in main
response = client.create_transfer_config(parent, transferConfig)
File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/bigquery_datatransfer_v1/gapic/data_transfer_service_client.py", line 438, in create_transfer_config
authorization_code=authorization_code)
ValueError: Protocol message Struct has no "customer_id" field.
DDIS:bigquery siddharthsudheer$ ./create_transfer.py
Traceback (most recent call last):
File "./create_transfer.py", line 84, in <module>
main()
File "./create_transfer.py", line 61, in main
response = client.create_transfer_config(parent, transferConfig)
File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/google/cloud/bigquery_datatransfer_v1/gapic/data_transfer_service_client.py", line 438, in create_transfer_config
authorization_code=authorization_code)
ValueError: Protocol message Struct has no "customer_id" field.

I managed to set up a Data Transfer through the API by defining the params as class google.protobuf.struct_pb2.Struct.
Try if the adding the following works for you:
from google.protobuf.struct_pb2 import Struct
params = Struct()
params["customer_id"] = "999999999"
And then changing your transferConfig to:
transferConfig = {
'data_refresh_window_days': 1,
'data_source_id': "adwords",
'destination_dataset_id': "AdwordsMCC",
'disabled': False,
'display_name': "TestR",
'name': "TestR",
'schedule': "every day 07:00",
'params': params
}
}

MongoDB doesn't handle aggregation with allowDiskUsage:True

the data structure is like:
way: {
_id:'9762264'
node: ['253333910', '3304026514']
}
and I'm trying to count the frequency of nodes' appearance in ways. Here is my code using pymongo:
node = db.way.aggregate([
{'$unwind': '$node'},
{
'$group': {
'_id': '$node',
'appear_count': {'$sum': 1}
}
},
{'$sort': {'appear_count': -1}},
{'$limit': 10}
],
{'allowDiskUse': True}
)
it will report an error:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File ".../OSM Wrangling/explore.py", line 78, in most_passed_node
{'allowDiskUse': True}
File ".../pymongo/collection.py", line 2181, in aggregate
**kwargs)
File ".../pymongo/collection.py", line 2088, in _aggregate
client=self.__database.client)
File ".../pymongo/pool.py", line 464, in command
self.validate_session(client, session)
File ".../pymongo/pool.py", line 609, in validate_session
if session._client is not client:
AttributeError: 'dict' object has no attribute '_client'
However, if I removed the {'allowDiskUse': True} and test it on a smaller set of data, it works well. It seems that the allowDiskUse statement brings something wrong? And there is no information about this mistake in the docs of MongoDB
How should I solve this problem and get the answer I want?

How should I solve this problem and get the answer I want?
This is because in PyMongo v3.6 the method signature for collection.aggregate() has been changed. An optional parameter for session has been added.
The method signature now is :
aggregate(pipeline, session=None, **kwargs)
Applying this to your code example, you can specify allowDiskUse as below:
node = db.way.aggregate(pipeline=[
{'$unwind': '$node'},
{'$group': {
'_id': '$node',
'appear_count': {'$sum': 1}
}
},
{'$sort': {'appear_count': -1}},
{'$limit': 10}
],
allowDiskUse=True
)
See also pymongo.client_session if you would like to know more about session.

js is case sensitive, please use lowercase boolean true
{'allowDiskUse': true}