We Keep Coding

pika rabbitmq python 3.6

I am trying to use pika to connect with rabbitmq
def get_connection():
credentials = pika.PlainCredentials(MQ_USER, MQ_PASS)
connection = pika.BlockingConnection(pika.ConnectionParameters(MQ_SERVER, 5672, '/', credentials))
return connection
I can use those credentials with rabbitmqctl, the output is something like this:
# rabbitmqctl authenticate_user user pass
Authenticating user "user" ...
Success
I have also tried to just use strings with the values inside the function and get the same error. I also have telnet access on the rabbitmq port and the user has access to the channel.
When execute the python code I get this error:
Internal Server Error: /api/analysis/stream/finish/
Traceback (most recent call last):
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/path/to/api/core/views.py", line 2465, in record_finsh
inform_process(video.filename)
File "/path/to/api/core/views.py", line 702, in inform_process
con = get_connection()
File "/path/to/api/base/rabitmq.py", line 7, in get_connection
connection = pika.BlockingConnection(pika.ConnectionParameters(host=MQ_SERVER, port=5672, virtual_host='/', credentials=credentials))
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__
self._impl = self._create_connection(parameters, _impl_class)
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
pika.exceptions.AMQPConnectionError
It looks to me like something happens on this line credentials = pika.PlainCredentials(MQ_USER, MQ_PASS) even when the error in on the next line. What does this function do exactly? Any ideas of what I am doing wrong?
EDIT:
I said I think the error is on this line credentials = pika.PlainCredentials(MQ_USER, MQ_PASS) because if I add something like:
def get_connection():
credentials = pika.PlainCredentials(MQ_USER, MQ_PASS)
exit()
connection = pika.BlockingConnection(pika.ConnectionParameters(MQ_SERVER, 5672, '/', credentials))
return connection
I still get more or less the same error:
Internal Server Error: /api/analysis/stream/finish/
Traceback (most recent call last):
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/path/to/api/core/views.py", line 2465, in record_finsh
inform_process(video.filename)
File "/path/to/api/core/views.py", line 702, in inform_process
con = get_connection()
File "/path/to/api/base/rabitmq.py", line 7, in get_connection
return 0
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__
self._impl = self._create_connection(parameters, _impl_class)
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
pika.exceptions.AMQPConnectionError
Because of this I also tried replacing with actual values like credentials = pika.PlainCredentials('user', 'mq#pass') and also get the same result.
EDIT2: Answering to the comments bellow.
def get_connection():
credentials = pika.PlainCredentials('user', 'mq#passwd')
connection = pika.BlockingConnection(pika.ConnectionParameters('172.x.y.z', 5672, '/', credentials))
return connection
Returns the same issue. Rabbit MQ runs on remote IP. I already tested and I can telnet to the IP.

pika.exceptions.AMQPConnectionError is raised when the host is not reachable by pika.
In case of invalid credentials, pika raises:
pika.exceptions.ConnectionClosedByBroker: (403, 'ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.')
for invalid virtual host:
pika.exceptions.ConnectionClosedByBroker: (530, 'NOT_ALLOWED - vhost / not found')
Check if the host or port value provided is valid.
Reference: pika docs

Not able to run google sheet api from composer

I'm running a google sheet extract python api from compute engine and its works fine
and I'm running the same from the composer, but its not working
In composer, I'm login to the vm with the same user and running ssh command and I'm using the same service account in google sheet & vm
The following is the error message I'm getting
File "/home/utetwork_multiplier.py", line 15, in sheet_reade
result = service.spreadsheets().values().get(spreadsheetId=spreadsheet_id, range=range_name).execute(
File "/hsite-packages/googleapiclient/_helpers.py", line 130, in positional_wrappe
return wrapped(*args, **kwargs
File "/on3.5/site-packages/googleapiclient/http.py", line 849, in execut
raise HttpError(resp, content, uri=self.uri
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://sheets.googleapis.com/v4/spreadsheets/14r0cQ1RCXhLyd7i0VCrmddXOFiugFnfioRb6cYI_BWQ/values/Master%21A%3AJ?alt=json returned "Request had insufficient authentication scopes."
Traceback (most recent call last)
File "/usr/local/lib/airflow/airflow/contrib/operators/ssh_operator.py", line 164, in execut
.format(self.command, error_msg)
airflow.exceptions.AirflowException: error running cmd: set -e;cd src/digital_platform && ../../venvs/bdp/bin/python -m.marketing.adnetwork_multiplier, error: Traceback (most recent call last)
File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_mai
"__main__", mod_spec
File "/usr/lib/python3.5/runpy.py", line 85, in _run_cod
exec(code, run_globals
File "/home/tt/src/digital_platform//marketing/adnetwork_multiplier.py", line 74, in <module
main(
File "/home/tt/src/digital_platform//marketing/adnetwork_multiplier.py", line 28, in mai
data = sheet_reader(range_name
File "/home/tt/src/digital_platform//marketing/adnetwork_multiplier.py", line 15, in sheet_reade
result = service.spreadsheets().values().get(spreadsheetId=spreadsheet_id, range=range_name).execute(
File "/home//venvs/bdp/lib/python3.5/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrappe
return wrapped(*args, **kwargs
File "/home//venvs/bdp/lib/python3.5/site-packages/googleapiclient/http.py", line 849, in execut
raise HttpError(resp, content, uri=self.uri
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://sheets.googleapis.com/v4/spreadsheets/14r0cQ1RCXhLydcYI_BWQ/values/Master%21A%3AJ?alt=json returned "Request had insufficient authentication scopes."
During handling of the above exception, another exception occurred
Scopes
scope = [
'https://www.googleapis.com/auth/bigquery',
'https://www.googleapis.com/auth/spreadsheets.readonly',
'https://www.googleapis.com/auth/spreadsheets'
]
home = os.path.expanduser('~')
csf = os.path.join(home, '.client_secret.json')
token_filename = os.path.join(home, '.google_auth.dat')
flow = flow_from_clientsecrets(csf, scope=scope, message="%s is missing" % csf)
storage = Storage(token_filename)
credentials = storage.get()
if credentials is None or credentials.invalid:
flags = tools.argparser.parse_args(args=[])
flags.noauth_local_webserver = True
credentials = tools.run_flow(flow, storage, flags=flags)

From the error message you apear to be running Method: spreadsheets.values.get
That method requires one of the following scopes to access
https://www.googleapis.com/auth/drive
https://www.googleapis.com/auth/drive.readonly
https://www.googleapis.com/auth/drive.file
https://www.googleapis.com/auth/spreadsheets
https://www.googleapis.com/auth/spreadsheets.readonly
The error message states
403 when requesting https://sheets.googleapis.com/v4/spreadsheets/14r0cQ1RCXhLydcYI_BWQ/values/Master%21A%3AJ?alt=json returned "Request had insufficient authentication scopes.
Which means that the user you have authenticated with was not authenticated with one of the required scopes above. I suggest that you log out the user delete the credentials for that user and run your code again making sure that when it pops up the consent screen it is asking for one of the required scopes. I suspect you are running your script with outdated grants from the user.

Using Google API Refresh Token in Python

I made a personal use app that just queries my gmail for a certain type of email. The app is successfully running on a pi night and day making requests to the Gmail service every thirty seconds.
However every week or so I get this error:
messages = gmail.users().messages().list(userId='me').execute() #
q='is:unread'
File "C:\Users\rexma\Anaconda3\envs\stan_env\lib\site-
packages\oauth2client\_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "C:\Users\rexma\Anaconda3\envs\stan_env\lib\site-
packages\googleapiclient\http.py", line 839, in execute
method=str(self.method), body=self.body, headers=self.headers)
File "C:\Users\rexma\Anaconda3\envs\stan_env\lib\site-
packages\googleapiclient\http.py", line 166, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "C:\Users\rexma\Anaconda3\envs\stan_env\lib\site-
packages\oauth2client\transport.py", line 186, in new_request
credentials._refresh(orig_request_method)
File "C:\Users\rexma\Anaconda3\envs\stan_env\lib\site-
packages\oauth2client\client.py", line 761, in _refresh
self._do_refresh_request(http)
File "C:\Users\rexma\Anaconda3\envs\stan_env\lib\site-
packages\oauth2client\client.py", line 819, in _do_refresh_request
raise HttpAccessTokenRefreshError(error_msg, status=resp.status)
oauth2client.client.HttpAccessTokenRefreshError: invalid_grant: Token has
been expired or revoked.
Which forces me to go back in, delete my old credentials and rerun the authorization. I've tried to do the other fixes online like setting
flow.authorization_url(access_type = 'offline', approval_prompt='force')
But the 'OAuth2WebServerFlow' object has no attribute 'authorization_url'.
Here is my full authorization code:
def authenticate(CLIENT_SECRET_FILE, APPLICATION_NAME, SCOPE, credential_name):
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir, credential_name)
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPE)
flow.user_agent = APPLICATION_NAME
# flow.authorization_url(access_type='offline', approval_prompt='force')
credentials = run_flow(flow, store)
return credentials
I'm definitely still a scrub with OAuth2, even having read this. I just want to know how I can have my app use the refresh token in the json credentials to keep running without expiring.
Any help is much appreciated.

Can't get users from G Suite in Python 3

I'm trying to solve a task using Admin SDK API
- Get the user-list from domain
- Сhange users password
For my case i created a service user in G Suite and wrote the script from the example below:
from oauth2client.service_account import ServiceAccountCredentials
from httplib2 import Http
from apiclient.discovery import build
def main():
scopes = ['https://www.googleapis.com/auth/admin.directory.user']
credentials = ServiceAccountCredentials.from_json_keyfile_name('paswd.json', scopes=scopes)
http_auth = credentials.authorize(Http())
service = build('admin', 'directory_v1', http=http_auth)
print('Getting the first 10 users in the domain')
results = service.users().list(customer='my_customer', maxResults=10, orderBy='email', domain='nnn.nn').execute()
print(results)
if __name__ == '__main__':
main()
When I execute script, I got the next exception:
Getting the first 10 users in the domain
Traceback (most recent call last):
File "./run-2.py", line 25, in <module>
main()
File "./run-2.py", line 21, in main
results = service.users().list(customer='my_customer', maxResults=10, orderBy='email', domain='nnn.nn').execute()
File "/usr/local/lib/python3.4/dist-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.4/dist-packages/googleapiclient/http.py", line 840, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/admin/directory/v1/users?customer=my_customer&domain=g.nsu.ru&alt=json&maxResults=10&orderBy=email returned "Not Authorized to access this resource/api">
What could be the problem?

As far as I have been able to figure out, what you are missing is:
credentials = ServiceAccountCredentials.from_json_keyfile_name('paswd.json', scopes=scopes)
delegated_credentials = credentials.create_delegated(DELEGATED_ACCOUNT)
http_auth = delegated_credentials.authorize(Http())
Where DELEGATED_ACCOUNT is an admin for your domain.

Obtaining access token error 401 in Twython

I am trying to veify Twitter account of user via Twython
def twitter_view(request):
twitter = Twython(APP_KEY, APP_SECRET)
auth = twitter.get_authentication_tokens(callback_url='http://127.0.0.1:8000/confirm/', force_login=True)
request.session['oauth_token'] = auth['oauth_token']
request.session['oauth_token_secret'] = auth['oauth_token_secret']
return HttpResponseRedirect(auth['auth_url'])
def redirect_view(request):
oauth_verifier = request.GET['oauth_verifier']
twitter = Twython(APP_KEY, APP_SECRET)
final_step = twitter.get_authorized_tokens(oauth_verifier)
request.user.twitter_oauth_token = final_step['oauth_token']
request.user.twitter_oauth_token_secret = final_step['oauth_token_secret']
request.user.save()
return redirect('twitterapp:homepage')
I am getting
Twitter API returned a 401 (Unauthorized), Invalid / expired Token
Traceback (most recent call last):
File
"/Users/bharatagarwal/my-venv/lib/python2.7/site-packages/django/core/handlers/base.py",
line 149, in get_response
response = self.process_exception_by_middleware(e, request)
File
"/Users/bharatagarwal/my-venv/lib/python2.7/site-packages/django/core/handlers/base.py",
line 147, in get_response response = wrapped_callback(request,
*callback_args, **callback_kwargs)
File
"/Users/bharatagarwal/projects/twitterproject/mysite/twitterapp/views.py",
line 100, in redirect_view
final_step = twitter.get_authorized_tokens(str(oauth_verifier))
File
"/Users/bharatagarwal/my-venv/lib/python2.7/site-packages/twython/api.py",
line 379, in get_authorized_tokens
ken'), error_code=response.status_code)
TwythonError: Twitter API returned a 401 (Unauthorized), Invalid /
expired To ken

On the second Twython instantiation, you have to include the OAUTH_TOKEN and the OAUTH_SECRET_TOKEN obtained in the first step.
twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
It is returning Invalid Token because the instantiation you're using didn't include the tokens you received.