I am trying to use pika to connect with rabbitmq
def get_connection():
credentials = pika.PlainCredentials(MQ_USER, MQ_PASS)
connection = pika.BlockingConnection(pika.ConnectionParameters(MQ_SERVER, 5672, '/', credentials))
return connection
I can use those credentials with rabbitmqctl, the output is something like this:
# rabbitmqctl authenticate_user user pass
Authenticating user "user" ...
Success
I have also tried to just use strings with the values inside the function and get the same error. I also have telnet access on the rabbitmq port and the user has access to the channel.
When execute the python code I get this error:
Internal Server Error: /api/analysis/stream/finish/
Traceback (most recent call last):
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/path/to/api/core/views.py", line 2465, in record_finsh
inform_process(video.filename)
File "/path/to/api/core/views.py", line 702, in inform_process
con = get_connection()
File "/path/to/api/base/rabitmq.py", line 7, in get_connection
connection = pika.BlockingConnection(pika.ConnectionParameters(host=MQ_SERVER, port=5672, virtual_host='/', credentials=credentials))
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__
self._impl = self._create_connection(parameters, _impl_class)
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
pika.exceptions.AMQPConnectionError
It looks to me like something happens on this line credentials = pika.PlainCredentials(MQ_USER, MQ_PASS) even when the error in on the next line. What does this function do exactly? Any ideas of what I am doing wrong?
EDIT:
I said I think the error is on this line credentials = pika.PlainCredentials(MQ_USER, MQ_PASS) because if I add something like:
def get_connection():
credentials = pika.PlainCredentials(MQ_USER, MQ_PASS)
exit()
connection = pika.BlockingConnection(pika.ConnectionParameters(MQ_SERVER, 5672, '/', credentials))
return connection
I still get more or less the same error:
Internal Server Error: /api/analysis/stream/finish/
Traceback (most recent call last):
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/path/to/api/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/path/to/api/core/views.py", line 2465, in record_finsh
inform_process(video.filename)
File "/path/to/api/core/views.py", line 702, in inform_process
con = get_connection()
File "/path/to/api/base/rabitmq.py", line 7, in get_connection
return 0
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__
self._impl = self._create_connection(parameters, _impl_class)
File "/path/to/api/venv/lib/python3.6/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
pika.exceptions.AMQPConnectionError
Because of this I also tried replacing with actual values like credentials = pika.PlainCredentials('user', 'mq#pass') and also get the same result.
EDIT2: Answering to the comments bellow.
def get_connection():
credentials = pika.PlainCredentials('user', 'mq#passwd')
connection = pika.BlockingConnection(pika.ConnectionParameters('172.x.y.z', 5672, '/', credentials))
return connection
Returns the same issue. Rabbit MQ runs on remote IP. I already tested and I can telnet to the IP.
pika.exceptions.AMQPConnectionError is raised when the host is not reachable by pika.
In case of invalid credentials, pika raises:
pika.exceptions.ConnectionClosedByBroker: (403, 'ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.')
for invalid virtual host:
pika.exceptions.ConnectionClosedByBroker: (530, 'NOT_ALLOWED - vhost / not found')
Check if the host or port value provided is valid.
Reference: pika docs
Related
I am building a python client-side application that uses Firestore. I have successfully used Google Identity Platform to sign up and sign in to the Firebase project, and created a working Firestore client using google.cloud.firestore.Client which is authenticated as a user:
import json
import requests
import google.oauth2.credentials
from google.cloud import firestore
request_url = f"https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key={self.__api_key}"
headers = {"Content-Type": "application/json; charset=UTF-8"}
data = json.dumps({"email": self.__email, "password": self.__password, "returnSecureToken": True})
response = requests.post(request_url, headers=headers, data=data)
try:
response.raise_for_status()
except (HTTPError, Exception):
content = response.json()
error = f"error: {content['error']['message']}"
raise AuthError(error)
json_response = response.json()
self.__token = json_response["idToken"]
self.__refresh_token = json_response["refreshToken"]
credentials = google.oauth2.credentials.Credentials(self.__token,
self.__refresh_token,
client_id="",
client_secret="",
token_uri=f"https://securetoken.googleapis.com/v1/token?key={self.__api_key}"
)
self.__db = firestore.Client(self.__project_id, credentials)
I have the problem, however, that when the token has expired, I get the following error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
return callable_(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 826, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/usr/local/lib/python3.7/dist-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAUTHENTICATED
details = "Missing or invalid authentication."
debug_error_string = "{"created":"#1613043524.699081937","description":"Error received from peer ipv4:172.217.16.74:443","file":"src/core/lib/surface/call.cc","file_line":1055,"grpc_message":"Missing or invalid authentication.","grpc_status":16}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/home/my_app/src/controllers/im_alive.py", line 20, in run
self.__device_api.set_last_updated(utils.device_id())
File "/home/my_app/src/api/firestore/firestore_device_api.py", line 21, in set_last_updated
"lastUpdatedTime": self.__firestore.SERVER_TIMESTAMP
File "/home/my_app/src/api/firestore/firestore.py", line 100, in update
ref.update(data)
File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/document.py", line 382, in update
write_results = batch.commit()
File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/batch.py", line 147, in commit
metadata=self._client._rpc_metadata,
File "/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/gapic/firestore_client.py", line 1121, in commit
request, retry=retry, timeout=timeout, metadata=metadata
File "/usr/local/lib/python3.7/dist-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
return wrapped_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
on_error=on_error,
File "/usr/local/lib/python3.7/dist-packages/google/api_core/retry.py", line 184, in retry_target
return target()
File "/usr/local/lib/python3.7/dist-packages/google/api_core/timeout.py", line 214, in func_with_timeout
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.Unauthenticated: 401 Missing or invalid authentication.
I have tried omitting the token and only specifying the refresh token, and then calling credentials.refresh(), but the expires_in in the response from the https://securetoken.googleapis.com/v1/token endpoint is a string instead of a number (docs here), which makes _parse_expiry(response_data) in google.oauth2._client.py:257 raise an exception.
Is there any way to use the firestore.Client from either google.cloud or firebase_admin and have it automatically handle refreshing tokens, or do I need to switch to the manually calling the Firestore RPC API and refreshing tokens at the correct time?
Note: There are no users interacting with the python app, so the solution must not require user interaction.
Can't you just pass the string cast as integer _parse_expiry(int(float(response_data))) ?
If it is not working you could try to make a call and refresh token after getting and error 401, see my answer for the general idea on how to handle tokens.
As mentioned by #Marco, it is recommended that you use a service account if it's going to be used in an environment without user. When you use service account, you can just set GOOGLE_APPLICATION_CREDENTIALS environment variable to location of service account json file and just instantiate the firestore Client without any credentials (The credentials will be picked up automatically):
import firestore
client = firestore.Client()
and run it as (assuming Linux):
$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json
$ python file.py
Still, if you really want to use user credentials for the script, you can install the Google Cloud SDK, then:
$ gcloud auth application-default login
This will open browser and for you to select account and login. After logging in, it creates a "virtual" service account file corresponding to your user account (that will also be loaded automatically by clients). Here too, you don't need to pass any parameters to your client.
See also: Difference between “gcloud auth application-default login” and “gcloud auth login”
I'm unable to call a SOAP request from a simple Python script in a Windows Server 2016 environment with WinPython/VSCode:
from requests import Session
from zeep import Client
from zeep.transports import Transport
wsdl = "http://www.dneonline.com/calculator.asmx?wsdl"
#wsdl = "calculator.xml"
client = Client(wsdl=wsdl)
request_data={'intA' : 1 ,
'intB' : 2}
response=client.service.Add(**request_data)
print("response: " + response)
The output I get:
Traceback (most recent call last):
File ".\zeeptest.py", line 31, in <module>
client = Client(wsdl=wsdl)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\client.py", line 73, in __init__
self.wsdl = Document(wsdl, self.transport, settings=self.settings)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\wsdl\wsdl.py", line 92, in __init__
self.load(location)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\wsdl\wsdl.py", line 95, in load
document = self._get_xml_document(location)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\wsdl\wsdl.py", line 155, in _get_xml_document
return load_external(
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\loader.py", line 79, in load_external
content = transport.load(url)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\transports.py", line 122, in load
content = self._load_remote_data(url)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\zeep\transports.py", line 134, in _load_remote_data
response = self.session.get(url, timeout=self.load_timeout)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "E:\Projects\test\zeeptest\zeeptest\lib\site-packages\requests\adapters.py", line 414, in send
raise InvalidURL(e, request=request)
requests.exceptions.InvalidURL: Not supported proxy scheme None
I tried to set the proxy manually with the following commands without success:
set http_proxy="http://<ip>:<port>"
set https_proxy="http://<ip>:<port>"
I resolved my problem by issuing the following command in a Powershell shell as an admin user:
netsh winhttp set proxy <ip>:<port>
Here is my python code:
def check(request):
import redis
R = redis.StrictRedis(host='127.0.0.1', port=6379, db=0)
R.set("name","srk")
r = R.HSET("name","srk")
print(r)
It encounters an error with this error message:
Internal Server Error: /get_user/
Traceback (most recent call last):
File "/home/soubhagya/.local/share/virtualenvs/pipenv-r-zifbiy/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/soubhagya/.local/share/virtualenvs/pipenv-r-zifbiy/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/soubhagya/.local/share/virtualenvs/pipenv-r-zifbiy/lib/python3.5/site-packages/django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/soubhagya/Desktop/Dev/rango/backend/access/views.py", line 103, in get_user
r = R.HSET("name","srk")
AttributeError: 'StrictRedis' object has no attribute 'HSET'
In py-redis, how can I check if a value exists or not in redis database.
A raw query will be helpful.
Please have a look into my code.
StrictRedis has no HSET function, but it does have the hset function for setting fields in Redis Hashes. That is the cause for the error you're getting.
To check whether a key exists in Redis with redis-py, use r = R.exists("name").
I have meet a porblem when I want to run my spiders in Djanjo.Some months ago.This method work for me:
def crawllist(self,lists):
runner = CrawlerRunner(get_project_settings())
for topic in lists:
logging.error("topic name is %s" % topic.name)
runner.crawl(topic.type,author = topic.author,links = topic.base_url)
d = runner.join()
d.addBoth(lambda _: reactor.stop())
logging.error("start crawl")
reactor.run(installSignalHandlers=False)
But it doesn't work now.Get these errors:
Internal Server Error: /CreateTopicServlet
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.5/dist-packages/django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/zdc/Push/job/views.py", line 150, in CreateTopicServlet
sp.crawllist([item])
File "/home/zdc/Push/job/SpiderManager.py", line 59, in crawllist
reactor.run(installSignalHandlers=False)
File "/usr/local/lib/python3.5/dist-packages/twisted/internet/base.py", line 1260, in run
self.startRunning(installSignalHandlers=installSignalHandlers)
File "/usr/local/lib/python3.5/dist-packages/twisted/internet/base.py", line 1240, in startRunning
ReactorBase.startRunning(self)
File "/usr/local/lib/python3.5/dist-packages/twisted/internet/base.py", line 746, in startRunning
raise error.ReactorAlreadyRunning()
I have read all answers about it. But doesn't work for me.The spider can run successfully when I run it locally without djanjo. But meet the Reactor not Restartable in Djanjo.
I have tried a method like that
def crawl(self,type,url,author):
print('crawl11')
module_name="Spidermanager.spiders.{}".format(type+'spider')
scrapy_var = importlib.import_module(module_name) #do some dynamic import of selected spider
spiderObj=scrapy_var.zhihuSpider(author = author,links = url)
print(spiderObj.start_urls)
runner = CrawlerRunner(get_project_settings())
runner.crawl(spiderObj)
print('crawl finished')
It solves the reactor problem。But the spider seems not run and crawl nothin.
I have written the following Pythin script to try and gather TFS (Team Foundation Server) work item data. Below is my script:-
from tfs import TFSAPI
user="andrew.xxxx"
password="xxxxxxxx"
tfsAddress = "http://man-tfsmig-1:8080/"
print(tfsAddress)
client = TFSAPI(tfsAddress, project="DefaultCollection/xxxxxxxx", user=user, password=password)
work_item = client.get_workitem(28274)
When I run the code I get the following error, but I do not know what the problem is:-
Traceback (most recent call last):
File ".\TFS_Release_Notes_app.py", line 13, in <module>
work_item = client.get_workitem(28274)
File "C:\Python\lib\site-packages\tfs\connection.py", line 70, in get_workitem
return self.get_workitems(id_, fields)[0]
File "C:\Python\lib\site-packages\tfs\connection.py", line 80, in get_workitems
work_items_batch_info = self.__get_workitems(work_items_batch, fields=fields, expand=expand)
File "C:\Python\lib\site-packages\tfs\connection.py", line 65, in __get_workitems
object_class=Workitem)
File "C:\Python\lib\site-packages\tfs\connection.py", line 46, in get_tfs_object
raw = self.rest_client.send_get(uri=uri, payload=payload, project=project)
File "C:\Python\lib\site-packages\tfs\connection.py", line 323, in send_get
return self.__send_request('GET', uri, None, payload=payload, project=project, json=json)
File "C:\Python\lib\site-packages\tfs\connection.py", line 360, in __send_request
response.raise_for_status()
File "C:\Python\lib\site-packages\requests\models.py", line 939, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://xxxx-xxxxx-1:8080/DefaultCollection/_apis/wit/workitems?ids=28274&$expand=all&api-version=1.0
It's telling you the error: 404, not found.
That means the URI you're providing for the API is incorrect.
It's generating this URI: http://xxxx-xxxxx-1:8080/DefaultCollection/_apis/wit/workitems. Validate if that is correct.
Usually, when TFS is running on port 8080 over HTTP, there's a /tfs/ virtual directory.
I needed to add the following line to authenticate:-
# Use NTLM authorization
from requests_ntlm import HttpNtlmAuth
client = TFSAPI("https://tfs.tfs.ru/tfs/", user=user, password=password, auth_type=HttpNtlmAuth)