We Keep Coding

How to use python-cloudbuild to run a build trigger

How to use python-cloudbuild library to run a build trigger with correctly passing data from SourceRepo?
UPDATE 1:
I have a build trigger set up and I am trying to run that trigger by changing the substitutions and the repo branch
UPDATE 2:
Actual code result:
Traceback (most recent call last): File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 67, in error_remapped_callable return callable_(*args, **kwargs) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/grpc/_channel.py", line 946, in call return _end_unary_response_blocking(state, call, False, None) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
credentials, project_id = google.auth.default()
client = cloudbuild_v1.services.cloud_build.CloudBuildClient()
trigger_id = '2f1erbc4-asdf-1234-qwery-c4bc74d16d62'
repo_source = cloudbuild_v1.RepoSource()
repo_source.branch_name = 'develop'
repo_source.substitutions = {
"_ENVIRONMENT":"dev",
"NAMESPACE":"dev"
}
operation = client.run_build_trigger(
project_id=project_id,
trigger_id=trigger_id,
source=repo_source
)

I am facing the same issue when using the Cloud Build Client Library for Python (google-cloud-build). However, it does work properly when calling the REST API directly, so the library seems to be at cause here. As an alternative, you can achieve the same using the Google API Python client library (google-api-python-client):
from googleapiclient.discovery import build
project_id = "my-project-id"
trigger_id = "00000000-1111-2222-aaaa-bbbbccccdddd"
with build("cloudbuild", "v1") as cloudbuild:
run_build_trigger = cloudbuild.projects().triggers().run(
projectId = project_id,
triggerId = trigger_id,
body = {
"branchName": "dev",
"substitutions": {
"_TEST": "FOO"
}
}
)
run_build_trigger.execute()
Make sure that all substitutions are already declared on the existing trigger.

Using Python to Manage AWS

I’m trying to use Python to create EC2 instances but I keep getting these errors.
Here is my code:
#!/usr/bin/env python
import boto3
ec2 = boto3.resource('ec2')
instance = ec2.create_instances(
ImageId='ami-0922553b7b0369273',
MinCount=1,
MaxCount=1,
InstanceType='t2.micro')
print instance[0].id
Here are the errors I'm getting
Traceback (most recent call last):
File "./createinstance.py", line 8, in <module>
InstanceType='t2.micro')
File "/usr/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action
response = action(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__
response = getattr(parent.meta.client, operation_name)(**params)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 320, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 623, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAMIID.NotFound) when calling the RunInstances operation: The image id '[ami-0922553b7b0369273]' does not exist
I also get an error when trying to create a key pair
Here's my code for creating the keypair
import boto3
ec2 = boto3.resource('ec2')
# create a file to store the key locally
outfile = open('ec2-keypair.pem','w')
# call the boto ec2 function to create a key pair
key_pair = ec2.create_key_pair(KeyName='ec2-keypair')
# capture the key and store it in a file
KeyPairOut = str(key_pair.key_material)
print(KeyPairOut)
outfile.write(KeyPairOut)
response = ec2.instance-describe()
print response
Here's are the error messages
./createkey.py: line 1: import: command not found
./createkey.py: line 2: syntax error near unexpected token `('
./createkey.py: line 2: `ec2 = boto3.resource('ec2')'
What I am I missing?

For your first script, one of two possibilities could be occurring:
1. The AMI you are referencing by the ID is not available because the key is incorrect or the AMI doesn't exist
2. AMI is unavailable in the region that your machine is setup for
You most likely are running your script from a machine that is not configured for the correct region. If you are running your script locally or on a server that does not have roles configured, and you are using the aws-cli, you can run the aws configure command to configure your access keys and region appropriately. If you are running your instance on a server with roles configured, your server needs to be ran in the correct region, and your roles need to allow access to EC2 AMI's.
For your second question (which in the future should probably be posted separate), your syntax error in your script is a side effect of not following the same format for how you wrote your first script. It is most likely that your python script is not in fact being interpreted as a python script. You should add the shebang at the top of the file and remove the spacing preceding your import boto3 statement.
#!/usr/bin/env python
import boto3
# create a file to store the key locally
outfile = open('ec2-keypair.pem','w')
# call the boto ec2 function to create a key pair
key_pair = ec2.create_key_pair(KeyName='ec2-keypair')
# capture the key and store it in a file
KeyPairOut = str(key_pair.key_material)
print(KeyPairOut)
outfile.write(KeyPairOut)
response = ec2.instance-describe()
print response

How to extract variable from within a function in Python?

I'm making a program which scrapes bus information from a server and sends it to a user via Facebook messenger. It works fine, but I'm trying to add functionality which splits really long timetables into separate messages. To do this, I had to make an if statement that detects really long timetables, splits them and calls the send_message function from my main file, app.py
Here is the part of the function in app with the variable I need to extract:
for messaging_event in entry["messaging"]:
if messaging_event.get("message"): # someone sent us a message
sender_id = messaging_event["sender"]["id"] # the facebook ID of the person sending you the message
recipient_id = messaging_event["recipient"]["id"] # the recipient's ID, which should be your page's facebook ID
message_text = messaging_event["message"]["text"] # the message's text
tobesent = messaging_event["message"]["text"]
send_message(sender_id, fetch.fetchtime(tobesent))
and here is the if statement in fetch which detects long messages, splits them and calls the send_message function from the other file, app.py:
if len(info["results"]) > 5:
for i, chunk in enumerate(chunks(info, 5), 1):
app.send_message((USER ID SHOULD BE HERE, 'Listing part: {}\n \n{}'.format(i, chunk)))
I'm trying to call the send_message function from app.py, but it requires two arguments, sender_id and the message text. How can I go about getting the sender_id variable from this function and using it in fetch? I've tried returning it and calling the function, but it doesn't work for me.
EDIT:error
Traceback (most recent call last):
File "bus.py", line 7, in <module>
print fetch.fetchtime(stopnum)
File "/home/ryan/fb-messenger-bot-master/fetch.py", line 17, in fetchtime
send_message((webhook(),'Listing part: {}\n \n{}'.format(i, chunk)))
File "/home/ryan/fb-messenger-bot-master/app.py", line 30, in webhook
data = request.get_json()
File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 343, in __getattr__
return getattr(self._get_current_object(), name)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 302, in _get_current_object
return self.__local()
File "/usr/local/lib/python2.7/dist-packages/flask/globals.py", line 37, in _lookup_req_object
raise RuntimeError(_request_ctx_err_msg)
RuntimeError: Working outside of request context.
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.

python amazon api error connection

I wrote a simple script with the python amazon api. I copied the example but when I run the script I received the following error:
for book in api.item_search('Books', Publisher='Galileo Press'):
return paginator(self.call, **operators)
File "/usr/local/lib/python2.7/dist-packages/amazonproduct/processors/__init__.py",
line 88, in __init__
self.page(kwargs.get(self.counter, 1))
root = self.fun(*self.args, **self.kwargs) mazonproduct.errors.InvalidClientTokenId: InvalidClientTokenId: The
AWS Access Key Id you provided does not exist in our records. File
"/usr/local/lib/python2.7/dist-packages/amazonproduct/api.py", line
529, in item_search
return paginator(self.call, **operators) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/processors/__init__.py",
line 88, in __init__
self.page(kwargs.get(self.counter, 1)) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/processors/__init__.py",
line 121, in page
root = self.fun(*self.args, **self.kwargs) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/api.py", line
334, in call
return self._parse(e.fp) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/api.py", line
277, in _parse
raise _e(errors[e.code]) amazonproduct.errors.InvalidClientTokenId: InvalidClientTokenId: The
AWS Access Key Id you provided does not exist in our records.

AWS Access Key Id you provided does not exist in our records.
Go to aws.amazon.com and either create a new account or login to your account. When in the dashboard, click on your name on the menu bar located at the upper right corner of the screen, then select Security Credentials. Expand the line item that says Access Keys (Access Key ID and Secret Access Key), and create a new Access Key ID. This will also supply you with your Secret Key, make sure to record both.
Create a file ~/.amazon-product-api containing the following data:
[Credentials]
access_key = <your access key>
secret_key = <your secret key>
associate_tag = <your associate id>
Then your program should run.

Generating SSH keypair with paramiko in Python

I am trying to generate a SSH key pair with the python module paramiko. There doesn't seem to be much info about key generation. I've read through the paramiko docs but can't figure out whats wrong. I can generate a private and public key without password encryption. However, when I try to encrypt the private key I get the following error.
ValueError: IV must be 8 bytes long
I believe the above error is from pycrypto. I've looked through the relevant code in paramiko.pkey and pycrypto without any luck.
Here is a small example.
import paramiko
def keygen(filename,passwd=None,bits=1024):
k = paramiko.RSAKey.generate(bits)
#This line throws the error.
k.write_private_key_file(filename,password = 'cleverpassword')
o = open(fil+'.pub' ,"w").write(k.get_base64())
traceback
Traceback (most recent call last):
File "/var/mobile/Applications/149E4C21-2F92-4712-BAC6-151A171C6687/Documents/test.py", line 14, in keygen
k.write_private_key_file(filename,password = 'cleverpassword')
File "/var/mobile/Applications/149E4C21-2F92-4712-BAC6-151A171C6687/Pythonista.app/pylib/site-packages/paramiko/rsakey.py", line 127, in write_private_key_file
self._write_private_key_file('RSA', filename, self._encode_key(), password)
File "/var/mobile/Applications/149E4C21-2F92-4712-BAC6-151A171C6687/Pythonista.app/pylib/site-packages/paramiko/pkey.py", line 323, in _write_private_key_file
self._write_private_key(tag, f, data, password)
File "/var/mobile/Applications/149E4C21-2F92-4712-BAC6-151A171C6687/Pythonista.app/pylib/site-packages/paramiko/pkey.py", line 341, in _write_private_key
data = cipher.new(key, mode, salt).encrypt(data)
File "/var/mobile/Applications/149E4C21-2F92-4712-BAC6-151A171C6687/Pythonista.app/pylib/site-packages/Crypto/Cipher/DES3.py", line 114, in new
return DES3Cipher(key, *args, **kwargs)
File "/var/mobile/Applications/149E4C21-2F92-4712-BAC6-151A171C6687/Pythonista.app/pylib/site-packages/Crypto/Cipher/DES3.py", line 76, in __init__
blockalgo.BlockAlgo.__init__(self, _DES3, key, *args, **kwargs)
File "/var/mobile/Applications/149E4C21-2F92-4712-BAC6-151A171C6687/Pythonista.app/pylib/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
self._cipher = factory.new(key, *args, **kwargs)
ValueError: IV must be 8 bytes long

The Problem
This looks like a bug in paramiko.
If you look at the line that threw the error in pkey.py, it is the following line:
data = cipher.new(key, mode, salt).encrypt(data)
Let us now look at the lines before it, which set the mode by first selecting a cipher_name.
# since we only support one cipher here, use it
cipher_name = list(self._CIPHER_TABLE.keys())[0]
cipher = self._CIPHER_TABLE[cipher_name]['cipher']
keysize = self._CIPHER_TABLE[cipher_name]['keysize']
blocksize = self._CIPHER_TABLE[cipher_name]['blocksize']
mode = self._CIPHER_TABLE[cipher_name]['mode']
Here are the contents of _CIPHER_TABLE.
_CIPHER_TABLE = {
'AES-128-CBC': {'cipher': AES, 'keysize': 16, 'blocksize': 16, 'mode': AES.MODE_CBC},
'DES-EDE3-CBC': {'cipher': DES3, 'keysize': 24, 'blocksize': 8, 'mode': DES3.MODE_CBC},
}
Observe how the comment contradicts the code. Two ciphers are available, and the line above which selects the cipher_name assumes there is only one.
Based on the error, it appears that 'DES-EDE3-CBC' is selected. If we look at the comment in DES3.py, we see the following requirement for an IV.
IV : byte string
The initialization vector to use for encryption or decryption.
It is ignored for `MODE_ECB` and `MODE_CTR`.
For `MODE_OPENPGP`, IV must be `block_size` bytes long for encryption
and `block_size` +2 bytes for decryption (in the latter case, it is
actually the *encrypted* IV which was prefixed to the ciphertext).
It is mandatory.
From paramiko's source, we observe that no IV is passed, and hence the error we saw.
Workaround
Change the following line in pkey.py to hardcode the 'AES-128-CBC' cipher instead.
# cipher_name = list(self._CIPHER_TABLE.keys())[1]
cipher_name = 'AES-128-CBC'