I am using VM, Ubuntu 16.04, Python 2.7.12. I learn an example from here. I am using demo account and TWS.
from ib.ext.Contract import Contract
from ib.ext.Order import Order
from ib.opt import Connection, message
def error_handler(msg):
"""Handles the capturing of error messages"""
print "Server Error: %s" % msg
def reply_handler(msg):
"""Handles of server replies"""
print "Server Response: %s, %s" % (msg.typeName, msg)
def create_contract(symbol, sec_type, exch, prim_exch, curr):
"""Create a Contract object defining what will
be purchased, at which exchange and in which currency.
symbol - The ticker symbol for the contract
sec_type - The security type for the contract ('STK' is 'stock')
exch - The exchange to carry out the contract on
prim_exch - The primary exchange to carry out the contract on
curr - The currency in which to purchase the contract"""
contract = Contract()
contract.m_symbol = symbol
contract.m_secType = sec_type
contract.m_exchange = exch
contract.m_primaryExch = prim_exch
contract.m_currency = curr
return contract
def create_order(order_type, quantity, action):
"""Create an Order object (Market/Limit) to go long/short.
order_type - 'MKT', 'LMT' for Market or Limit orders
quantity - Integral number of assets to order
action - 'BUY' or 'SELL'"""
order = Order()
order.m_orderType = order_type
order.m_totalQuantity = quantity
order.m_action = action
return order
if __name__ == "__main__":
# Connect to the Trader Workstation (TWS) running on the
# usual port of 7496, with a clientId of 100
# (The clientId is chosen by us and we will need
# separate IDs for both the execution connection and
# market data connection)
tws_conn = Connection.create("127.0.0.1", port=7496, clientId=100)
tws_conn.connect()
# Assign the error handling function defined above
# to the TWS connection
tws_conn.register(error_handler, 'Error')
# Assign all of the server reply messages to the
# reply_handler function defined above
tws_conn.registerAll(reply_handler)
# Create an order ID which is 'global' for this session. This
# will need incrementing once new orders are submitted.
order_id = 1
# Create a contract in GOOG stock via SMART order routing
goog_contract = create_contract('GOOG', 'STK', 'SMART', 'SMART', 'USD')
# Go long 100 shares of Google
goog_order = create_order('MKT', 100, 'BUY')
# Use the connection to the send the order to IB
print(tws_conn.placeOrder(order_id, goog_contract, goog_order))
# Disconnect from TWS
tws_conn.disconnect()
Here is my TWS configuration:
I can connect to the TWS since from the log I can see that:
2019-06-02 10:57:03.974 [GS] INFO [JTS-EServerSocket-153] - [0:62:76:1:0:0:0:SYS] Server version is 76
2019-06-02 10:57:03.974 [GS] INFO [JTS-EServerSocket-153] - [0:62:76:1:0:0:0:SYS] Client version is 62
2019-06-02 10:57:03.974 [GS] INFO [JTS-EServerSocket-153] - [0:62:76:1:0:0:0:SYS] is 3rdParty false
2019-06-02 10:57:03.974 [GS] INFO [JTS-EServerSocketNotifier-154] - Starting async queue thread
2019-06-02 10:57:03.977 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:0:0:SYS] Starting new conversation with client{100} at 127.0.0.1
2019-06-02 10:57:03.977 [GS] INFO [AWT-EventQueue-0] - MDConnectionsModel: Updated [127.0.0.1:34076 CLIENT ACCEPTED 100]
But the I cannot make an order, and I get errors from log:
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:9:1:INFO] Sending next valid order id.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:9:1:DET] [9;1;1]
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Sending error.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] [4;2;-1;2104;Market data farm connection is OK:usfarm.nj]
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Error sent.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Sending error.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] [4;2;-1;2104;Market data farm connection is OK:cashfarm]
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Error sent.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Sending error.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] [4;2;-1;2104;Market data farm connection is OK:usfarm]
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Error sent.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Sending error.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] [4;2;-1;2106;HMDS data farm connection is OK:hkhmds]
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Error sent.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Sending error.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] [4;2;-1;2106;HMDS data farm connection is OK:ushmds]
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:4:2:DET] Error sent.
2019-06-02 10:57:03.978 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:0:0:INFO] Start processing incoming messages for client {100}.
2019-06-02 10:57:03.980 [GS] INFO [JTS-EServerSocket-153] - [100:62:76:1:0:0:0:INFO] Handling incoming PlaceOrder(3) message.
2019-06-02 10:57:03.982 [GS] INFO [JTS-EWriter2-155] - Broken pipe (Write failed)
2019-06-02 10:57:03.982 [GS] INFO [JTS-EWriter2-155] - [100:62:76:1:0:0:0:ERR] Unable write to socket client{100} -
2019-06-02 10:57:03.982 [GS] INFO [JTS-EWriter2-155] - Broken pipe (Write failed)
2019-06-02 10:57:03.982 [GS] INFO [JTS-EWriter2-155] - [100:62:76:1:0:0:0:INFO] Close call made for client{100} socket connection.
2019-06-02 10:57:04.000 [GS] INFO [JTS-EWriter2-155] - Cleaning up [serverId: 100]...
2019-06-02 10:57:04.000 [GS] INFO [JTS-EWriter2-155] - Cleaning up [serverId: 100]...
2019-06-02 10:57:04.000 [GS] INFO [JTS-EWriter2-155] - Cleaning up [serverId: 100]...
2019-06-02 10:57:04.000 [GS] INFO [JTS-EWriter2-155] - [100:62:76:1:0:0:0:DET] closePrim called. Stopping all mkt data and HMDS requests for client{100}.
Broken pipe, what does it means? How can I fix it and make order through python API? Thank you very much.
You call disconnect at the end of your program. I'm guessing that's why it gets disconnected;)
A few random thoughts.
There is a new python API available from IB so unless you want to use python 2.7 you should use the newer API as it has more features and probably more users as well by now.
It's not a problem with your program but you call connect and don't wait to make sure you're connected. Notice that next valid id is the first thing sent when the connection is established, you should use this as a signal to start your interaction with TWS/Gateway. Usually you implement the nextValidId callback and put a call to your startup code there, like placeAllOrders() or something.
Most people put a sleep() in their program to wait for it to finish before disconnecting. This is a bad idea since you never know when it's finished. In your case, imagine you want to disconnect after the order is received. You can implement the orderStatus callback and disconnect after it says it got your order. But you can put a sleep(5) or something just before your disconnect() call for testing.
I don't know what this line does print(tws_conn.placeOrder(order_id, goog_contract, goog_order)) placeOrder doesn't return anything, after TWS receives the order it will send an orderStatus callback.
The 'errors' saying that the market data is working aren't really errors, just information. If it says 'connection is broken' then you know you're not getting data from that particular farm.
It won't matter for goog but primary exchange is never SMART. That's an actual exchange where the contract you're using primarily trades. This is only used for disambiguation in case goog trades in USD in some other country.
Related
hey i tried making this connection work... i first installed the updated version of python-socketio[client]==5.7.1 but apparently this version supports EIO=4 and the socket.io client i'm trying to open is EIO=3.. so i had to uninstall it and install an old version python-socketio==4.6.1 . after that i tried running this code
import socketio
sio = socketio.Client(ssl_verify=False,logger=True,engineio_logger=True)
#sio.event
def connect():
print("Connected!")
sio.emit("authorization",{"session":"2ot3i7..................3h","isDemo":1})
#sio.event
def connect_error(data):
print("The connection failed!\n\n" + data)
#sio.event
def disconnect():
print("Disconnected!")
#sio.on('*')
def catch_all(event,data):
pass
#sio.on('handshake')
def on_message(data):
print('Handshake',data)
sio.emit("instruments/update",{"asset":"GBPUSD","period":60})
sio.emit("indicator/list")
sio.emit("drawing/load")
sio.emit("pending/list")
sio.emit("chart_notification/get")
sio.emit("depth/follow","GBPUSD")
sio.emit("tick")
sio.emit("settings/store",{"chartId":"graph","settings":{"chartId":"graph","chartType":2,"currentExpirationTime":0,"isFastOption":False,"isFastAmountOption":False,"isIndicatorsMinimized":False,"isIndicatorsShowing":True,"isShortBetElement":False,"chartPeriod":4,"currentAsset":{"symbol":"GBPUSD"},"dealValue":1000,"dealPercentValue":1,"isVisible":True,"timePeriod":60,"gridOpacity":8,"isAutoScrolling":1,"isOneClickTrade":True,"upColor":"#0FAF59","downColor":"#FF6251"}})
if __name__ == '__main__' :
sio.connect('https://ws.quotex.io/socket.io/', transports=['websocket'])
sio.wait()
and here is what came back...
Attempting WebSocket connection to wss://ws.quotex.io/socket.io/?transport=websocket&EIO=3
WebSocket connection accepted with {'sid': '5AeKOf4oS0U-pHnaB1-2', 'upgrades': [], 'pingInterval': 25000, 'pingTimeout': 5000}
Engine.IO connection established
Sending packet PING data None
Received packet MESSAGE data 0
Received packet MESSAGE data 1
Namespace / is connected
Connected!
Emitting event "authorization" [/]
Sending packet MESSAGE data 2["authorization",{"session":"2ot......................3h","isDemo":1}]
Disconnected!
Received packet NOOP data None
WebSocket connection was closed, aborting
Waiting for write loop task to end
Exiting write loop task
Waiting for ping loop task to end
PONG response has not been received, aborting
Exiting ping task
Engine.IO connection dropped
Connection failed, new attempt in 1.41 seconds
Exiting read loop task
Attempting WebSocket connection to wss://ws.quotex.io/socket.io/?transport=websocket&EIO=3
WebSocket connection accepted with {'sid': 'sQvjGss3KsxnBzQ2B2gI', 'upgrades': [], 'pingInterval': 25000, 'pingTimeout': 5000}
Engine.IO connection established
Sending packet PING data None
Reconnection successful
Received packet MESSAGE data 0
Namespace / is connected
Received packet MESSAGE data 1
Connected!
Emitting event "authorization" [/]
Received packet NOOP data None
Sending packet MESSAGE data 2["authorization",{"session":"2ot.......................3h","isDemo":1}]
WebSocket connection was closed, aborting
Waiting for write loop task to end
Exiting write loop task
Waiting for ping loop task to end
PONG response has not been received, aborting
Exiting ping task
Engine.IO connection dropped
Disconnected!
Disconnected!
Connection failed, new attempt in 0.90 seconds
Exiting read loop task
i had to kill the terminal cause it was trying to reconnect
please if you have any questions to make my explanations clear you can comment and i'll answer
thank you if you can give an advice
I am using Python 3.8.2 on Linux and the azure iot python sdk version 2. I have setup device provisioning in the Azure Portal and can successfully provision my device using the following code.
Provisioning code:
# Imports for Azure
from azure.iot.device import ProvisioningDeviceClient, X509
from azure.iot.device import IoTHubDeviceClient, Message
def runAzureProvisioning(azureConfig):
# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------
provisioning_host = azureConfig["PROVISIONING_HOST"]
id_scope = azureConfig["PROVISIONING_IDSCOPE"]
registration_id = azureConfig["DPS_X509_REGISTRATION_ID"]
x509 = X509(
cert_file=azureConfig["X509_CERT_FILE"],
key_file=azureConfig["X509_KEY_FILE"],
pass_phrase=azureConfig["PASS_PHRASE"],
)
provisioning_device_client = ProvisioningDeviceClient.create_from_x509_certificate(
provisioning_host=provisioning_host,
registration_id=registration_id,
id_scope=id_scope,
x509=x509,
)
registration_result = provisioning_device_client.register()
# The result can be directly printed to view the important details.
app.config['LOGGER'].info(registration_result)
if registration_result.status == "assigned":
app.config['LOGGER'].info("Will send telemetry from the provisioned device")
# Create device client from the above result
device_client = IoTHubDeviceClient.create_from_x509_certificate(
x509=x509,
hostname=registration_result.registration_state.assigned_hub,
device_id=registration_result.registration_state.device_id,
)
# Connect the client.
device_client.connect()
for i in range(1, 2):
app.config['LOGGER'].info("sending message #" + str(i))
device_client.send_message("test payload message " + str(i))
time.sleep(1)
return 0, f"{registration_result.registration_state.device_id}"
else:
app.config['LOGGER'].info("Can not send telemetry from the provisioned device")
return 1,"Can not send telemetry from the provisioned device"
Provisioning Output:
2021-02-10 19:59:58,313 - azure.iot.device.common.mqtt_transport:137 - INFO - creating mqtt client
2021-02-10 19:59:58,317 - azure.iot.device.common.mqtt_transport:150 - INFO - Creating client for connecting using MQTT over TCP
2021-02-10 19:59:58,412 - azure.iot.device.provisioning.provisioning_device_client:70 - INFO - Registering with Provisioning Service...
2021-02-10 19:59:58,416 - azure.iot.device.provisioning.provisioning_device_client:91 - INFO - Enabling reception of response from Device Provisioning Service...
2021-02-10 19:59:58,428 - azure.iot.device.common.pipeline.pipeline_stages_base:955 - INFO - ReconnectStage(ConnectOperation): State changes LOGICALLY_DISCONNECTED->LOGICALLY_CONNECTED. Adding to wait list and sending new connect op down
2021-02-10 19:59:58,435 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:171 - INFO - MQTTTransportStage(ConnectOperation): connecting
2021-02-10 19:59:58,443 - azure.iot.device.common.mqtt_transport:374 - INFO - connecting to mqtt broker
2021-02-10 19:59:58,447 - azure.iot.device.common.mqtt_transport:385 - INFO - Connect using port 8883 (TCP)
2021-02-10 20:00:01,583 - azure.iot.device.common.mqtt_transport:180 - INFO - connected with result code: 0
2021-02-10 20:00:01,589 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:275 - INFO - _on_mqtt_connected called
2021-02-10 20:00:01,599 - azure.iot.device.common.pipeline.pipeline_stages_base:552 - INFO - ConnectionLockStage(ConnectOperation): processing 0 items in queue
2021-02-10 20:00:01,607 - azure.iot.device.common.pipeline.pipeline_stages_base:1141 - INFO - ReconnectStage: completing waiting ops with error=None
2021-02-10 20:00:01,612 - azure.iot.device.common.pipeline.pipeline_stages_base:411 - INFO - AutoConnectStage(MQTTSubscribeOperation): Connected. Sending down and adding callback to check result
2021-02-10 20:00:01,621 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:225 - INFO - MQTTTransportStage(MQTTSubscribeOperation): subscribing to $dps/registrations/res/#
2021-02-10 20:00:01,625 - azure.iot.device.common.mqtt_transport:474 - INFO - subscribing to $dps/registrations/res/# with qos 1
2021-02-10 20:00:01,706 - azure.iot.device.common.mqtt_transport:234 - INFO - suback received for 1
2021-02-10 20:00:01,718 - azure.iot.device.provisioning.provisioning_device_client:98 - INFO - Successfully subscribed to Device Provisioning Service to receive responses
2021-02-10 20:00:01,736 - azure.iot.device.common.pipeline.pipeline_stages_base:411 - INFO - AutoConnectStage(MQTTPublishOperation): Connected. Sending down and adding callback to check result
2021-02-10 20:00:01,741 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:211 - INFO - MQTTTransportStage(MQTTPublishOperation): publishing on $dps/registrations/PUT/iotdps-register/?$rid=aa969347-7e41-41e4-b874-54b995f129ef
2021-02-10 20:00:01,747 - azure.iot.device.common.mqtt_transport:533 - INFO - publishing on $dps/registrations/PUT/iotdps-register/?$rid=aa969347-7e41-41e4-b874-54b995f129ef
2021-02-10 20:00:01,817 - azure.iot.device.common.mqtt_transport:248 - INFO - payload published for 2
2021-02-10 20:00:02,066 - azure.iot.device.common.mqtt_transport:255 - INFO - message received on $dps/registrations/res/202/?$rid=aa969347-7e41-41e4-b874-54b995f129ef&retry-after=3
2021-02-10 20:00:02,074 - azure.iot.device.provisioning.pipeline.pipeline_stages_provisioning_mqtt:129 - INFO - Received payload:b'{"operationId":"4.bd46492d8689531e.a04c1182-f5d8-4ea7-b2fd-4902abb2f789","status":"assigning"}' on topic:$dps/registrations/res/202/?$rid=aa969347-7e41-41e4-b874-54b995f129ef&retry-after=3
2021-02-10 20:00:02,086 - azure.iot.device.provisioning.pipeline.pipeline_stages_provisioning:399 - WARNING - RegistrationStage(RequestAndResponseOperation): Op will transition into polling after interval 2. Setting timer.
2021-02-10 20:00:04,099 - azure.iot.device.provisioning.pipeline.pipeline_stages_provisioning:384 - INFO - RegistrationStage(RequestAndResponseOperation): polling
2021-02-10 20:00:04,111 - azure.iot.device.common.pipeline.pipeline_stages_base:411 - INFO - AutoConnectStage(MQTTPublishOperation): Connected. Sending down and adding callback to check result
2021-02-10 20:00:04,116 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:211 - INFO - MQTTTransportStage(MQTTPublishOperation): publishing on $dps/registrations/GET/iotdps-get-operationstatus/?$rid=04db5bb9-5e70-45cf-91a6-6d9df8a34478&operationId=4.bd46492d8689531e.a04c1182-f5d8-4ea7-b2fd-4902abb2f789
2021-02-10 20:00:04,120 - azure.iot.device.common.mqtt_transport:533 - INFO - publishing on $dps/registrations/GET/iotdps-get-operationstatus/?$rid=04db5bb9-5e70-45cf-91a6-6d9df8a34478&operationId=4.bd46492d8689531e.a04c1182-f5d8-4ea7-b2fd-4902abb2f789
2021-02-10 20:00:04,190 - azure.iot.device.common.mqtt_transport:248 - INFO - payload published for 3
2021-02-10 20:00:04,206 - azure.iot.device.common.mqtt_transport:255 - INFO - message received on $dps/registrations/res/200/?$rid=04db5bb9-5e70-45cf-91a6-6d9df8a34478
2021-02-10 20:00:04,213 - azure.iot.device.provisioning.pipeline.pipeline_stages_provisioning_mqtt:129 - INFO - Received payload:b'{"operationId":"4.bd46492d8689531e.a04c1182-f5d8-4ea7-b2fd-4902abb2f789","status":"assigned","registrationState":{"x509":{"enrollmentGroupId":"New-Gateway-Jack"},"registrationId":"New-Gateway-Jack","createdDateTimeUtc":"2021-02-10T20:00:02.2501194Z","assignedHub":"New-Gateway-Test-Jack.azure-devices.net","deviceId":"New-Gateway-Jack","status":"assigned","substatus":"initialAssignment","lastUpdatedDateTimeUtc":"2021-02-10T20:00:02.4704038Z","etag":"IjM1MDA2OGJjLTAwMDAtMDEwMC0wMDAwLTYwMjQzYjQyMDAwMCI="}}' on topic:$dps/registrations/res/200/?$rid=04db5bb9-5e70-45cf-91a6-6d9df8a34478
2021-02-10 20:00:04,231 - azure.iot.device.provisioning.abstract_provisioning_device_client:221 - INFO - Successfully registered with Provisioning Service
2021-02-10 20:00:04,235 - /usr/bin/company/cloudcredentials/app.py:442 - INFO - New-Gateway-Jack
New-Gateway-Test-Jack.azure-devices.net
initialAssignment
null
assigned
2021-02-10 20:00:04,239 - /usr/bin/company/cloudcredentials/app.py:445 - INFO - Will send telemetry from the provisioned device
2021-02-10 20:00:04,273 - azure.iot.device.common.mqtt_transport:137 - INFO - creating mqtt client
2021-02-10 20:00:04,277 - azure.iot.device.common.mqtt_transport:150 - INFO - Creating client for connecting using MQTT over TCP
2021-02-10 20:00:04,303 - azure.iot.device.iothub.sync_clients:112 - INFO - Connecting to Hub...
2021-02-10 20:00:04,315 - azure.iot.device.common.pipeline.pipeline_stages_base:955 - INFO - ReconnectStage(ConnectOperation): State changes LOGICALLY_DISCONNECTED->LOGICALLY_CONNECTED. Adding to wait list and sending new connect op down
2021-02-10 20:00:04,321 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:171 - INFO - MQTTTransportStage(ConnectOperation): connecting
2021-02-10 20:00:04,329 - azure.iot.device.common.mqtt_transport:374 - INFO - connecting to mqtt broker
2021-02-10 20:00:04,333 - azure.iot.device.common.mqtt_transport:385 - INFO - Connect using port 8883 (TCP)
2021-02-10 20:00:04,951 - azure.iot.device.common.mqtt_transport:180 - INFO - connected with result code: 0
2021-02-10 20:00:04,957 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:275 - INFO - _on_mqtt_connected called
2021-02-10 20:00:04,966 - azure.iot.device.common.pipeline.pipeline_stages_base:552 - INFO - ConnectionLockStage(ConnectOperation): processing 0 items in queue
2021-02-10 20:00:04,969 - azure.iot.device.iothub.sync_clients:86 - INFO - Connection State - Connected
2021-02-10 20:00:04,975 - azure.iot.device.common.pipeline.pipeline_stages_base:1141 - INFO - ReconnectStage: completing waiting ops with error=None
2021-02-10 20:00:04,986 - azure.iot.device.iothub.sync_clients:118 - INFO - Successfully connected to Hub
2021-02-10 20:00:04,990 - /usr/bin/company/cloudcredentials/app.py:457 - INFO - sending message #1
2021-02-10 20:00:04,995 - azure.iot.device.iothub.sync_clients:166 - INFO - Sending message to Hub...
2021-02-10 20:00:05,004 - azure.iot.device.common.pipeline.pipeline_stages_base:411 - INFO - AutoConnectStage(MQTTPublishOperation): Connected. Sending down and adding callback to check result
2021-02-10 20:00:05,009 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:211 - INFO - MQTTTransportStage(MQTTPublishOperation): publishing on devices/New-Gateway-Jack/messages/events/
2021-02-10 20:00:05,013 - azure.iot.device.common.mqtt_transport:533 - INFO - publishing on devices/New-Gateway-Jack/messages/events/
2021-02-10 20:00:05,295 - azure.iot.device.common.mqtt_transport:248 - INFO - payload published for 1
2021-02-10 20:00:05,306 - azure.iot.device.iothub.sync_clients:172 - INFO - Successfully sent message to Hub
Now, I am trying to connect ( not provision ) with the credentials I used to successfully provision and connect above using the following code.
Upon successful provisioning, I save the following data ( keys, certs, and strings ):
# Now use the credentials to provision this device
# NOTE: If the credentials were accepted by Azure, the message returned here is the 'device id'
(status, message) = runAzureProvisioning(azureConfig)
if status != 0:
responseJson = {"response": f"error: Could not Run Azure provisioning script: {message}"}
return make_response(jsonify(responseJson), 550)
else:
# Write the meta information out to a JSON file as well
metaData = {}
metaData['provisioning_host'] = f"{provisioninghost}"
metaData['provisioning_id_scope'] = f"{provisioningidscope}"
metaData['x509_registration_id'] = f"{x509registrationid}"
metaData['device_id'] = f"{message}"
metaData['key'] = f"{path.join(app.config['AZURE_CREDS_UPLOAD_PATH'], azureConfig['X509_KEY_FILE'])}"
metaData['cert'] = f"{path.join(app.config['AZURE_CREDS_UPLOAD_PATH'], azureConfig['X509_CERT_FILE'])}"
with open(f"{app.config['AZURE_CREDS_UPLOAD_PATH']}/meta.json", 'w') as outfile:
json.dump(metaData, outfile)
responseJson = {"response": f"success"}
return make_response(jsonify(responseJson), 200)
Then, later I read that saved information, and try to connect:
# Ensure we have everything we need to connect
if not metaData.get("provisioning_host") or not metaData.get("provisioning_id_scope") \
or not metaData.get("x509_registration_id") or not metaData.get("key") or not metaData.get("cert") \
or not metaData.get("device_id"):
responseJson = {"response": f"error: Could not find all required meta data in JSON store: {metaData}"}
return make_response(jsonify(responseJson), 550)
# Define the required credentials to establish a connection
# Get the x.509 certificate path from the meta information
x509 = X509(
cert_file=metaData.get("cert"),
key_file=metaData.get("key"),
pass_phrase=None
)
# TODO: Get a proper hostname here
client = IoTHubDeviceClient.create_from_x509_certificate(hostname=metaData.get("provisioning_host"), device_id=metaData.get("device_id"), x509=x509)
# Try to connect
try:
# Attempt to connect
app.config['LOGGER'].info(
f"Connecting with: {metaData.get('x509_registration_id')} | {metaData.get('key')} | {metaData.get('cert')}")
client.connect()
client.disconnect()
responseJson = {"response": f"success"}
return make_response(jsonify(responseJson), 200)
except Exception as err:
responseJson = {"response": f"error: Could not connect to AWS: {err}"}
return make_response(jsonify(responseJson), 550)
But I get 'Unauthorised' and 'Connection Refused' errors as shown below:
2021-02-10 20:00:13,073 - azure.iot.device.common.mqtt_transport:137 - INFO - creating mqtt client
2021-02-10 20:00:13,078 - azure.iot.device.common.mqtt_transport:150 - INFO - Creating client for connecting using MQTT over TCP
2021-02-10 20:00:13,109 - /usr/bin/company/cloudcredentials/app.py:1190 - INFO - Connecting with: New-Gateway-Jack | /persistent-storage/azure-creds/device.key | /persistent-storage/azure-creds/device.crt
2021-02-10 20:00:13,114 - azure.iot.device.iothub.sync_clients:112 - INFO - Connecting to Hub...
2021-02-10 20:00:13,122 - azure.iot.device.common.pipeline.pipeline_stages_base:955 - INFO - ReconnectStage(ConnectOperation): State changes LOGICALLY_DISCONNECTED->LOGICALLY_CONNECTED. Adding to wait list and sending new connect op down
2021-02-10 20:00:13,128 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:171 - INFO - MQTTTransportStage(ConnectOperation): connecting
2021-02-10 20:00:13,139 - azure.iot.device.common.mqtt_transport:374 - INFO - connecting to mqtt broker
2021-02-10 20:00:13,143 - azure.iot.device.common.mqtt_transport:385 - INFO - Connect using port 8883 (TCP)
2021-02-10 20:00:16,240 - azure.iot.device.common.mqtt_transport:180 - INFO - connected with result code: 5
2021-02-10 20:00:16,248 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:300 - INFO - MQTTTransportStage: _on_mqtt_connection_failure called: UnauthorizedError('Connection Refused: not authorised.')
2021-02-10 20:00:16,254 - azure.iot.device.common.pipeline.pipeline_ops_base:107 - ERROR - ConnectOperation: completing with error UnauthorizedError('Connection Refused: not authorised.')
2021-02-10 20:00:16,258 - azure.iot.device.common.mqtt_transport:206 - INFO - disconnected with result code: 5
2021-02-10 20:00:16,262 - azure.iot.device.common.pipeline.pipeline_stages_base:518 - ERROR - ConnectionLockStage(ConnectOperation): op failed. Unblocking queue with error: UnauthorizedError('Connection Refused: not authorised.')
2021-02-10 20:00:16,274 - azure.iot.device.common.pipeline.pipeline_stages_base:552 - INFO - ConnectionLockStage(ConnectOperation): processing 0 items in queue
2021-02-10 20:00:16,279 - azure.iot.device.common.pipeline.pipeline_stages_base:1141 - INFO - ReconnectStage: completing waiting ops with error=UnauthorizedError('Connection Refused: not authorised.')
2021-02-10 20:00:16,283 - azure.iot.device.common.pipeline.pipeline_ops_base:107 - ERROR - ConnectOperation: completing with error UnauthorizedError('Connection Refused: not authorised.')
2021-02-10 20:00:16,290 - azure.iot.device.common.evented_callback:48 - ERROR - Callback completed with error UnauthorizedError('Connection Refused: not authorised.')
2021-02-10 20:00:16,295 - azure.iot.device.common.evented_callback:49 - ERROR - NoneType: None
2021-02-10 20:00:16,383 - azure.iot.device.common.mqtt_transport:297 - INFO - Forcing paho disconnect to prevent it from automatically reconnecting
2021-02-10 20:00:16,390 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:322 - INFO - MQTTTransportStage: _on_mqtt_disconnect called: ConnectionFailedError('The connection was refused.')
2021-02-10 20:00:16,398 - azure.iot.device.common.pipeline.pipeline_stages_mqtt:360 - INFO - MQTTTransportStage: disconnection was unexpected
2021-02-10 20:00:16,400 - azure.iot.device.iothub.sync_clients:90 - INFO - Connection State - Disconnected
2021-02-10 20:00:16,409 - azure.iot.device.iothub.sync_clients:92 - INFO - Cleared all pending method requests due to disconnect
2021-02-10 20:00:16,404 - azure.iot.device.common.handle_exceptions:52 - INFO - Unexpected disconnection. Safe to ignore since other stages will reconnect.
2021-02-10 20:00:16,422 - azure.iot.device.common.handle_exceptions:53 - INFO - azure.iot.device.common.transport_exceptions.ConnectionFailedError: ConnectionFailedError('The connection was refused.')
So, how come I can provision successfully, but cannot used those same credentials to connect with later?
The answer is that MS does not know how to solve this issue. There are two tickets on the same issue as you are having on github and microsoft docs.
First it is important to know exact version of your
azure-iot-sdk-python. The reporter had these:
Software versions: Python==3.7.8 azure-iot-device==2.3.0
I presume there is an regression in the azure-iot-sdk-python. Currently Microsoft does not know the cause of the issue.
What to do?
I would report it to Microsoft:
Hello #VarunVenkatesh-7647, Could you please send us an email
attaching your python app code. Please remove your IoT Central's ID
scope, Device ID, and Primary key from the .py file before attaching
it to the email.
Please send an email to azcommunity#microsoft.com with the below
details, so that we can work on this matter.
Thread URL: Link to this thread. Subject : Attn Satish Boddu
Attachment: your environmental_sensor.py file.
Next to try is to downgrade your azure-iot-sdk-python gradually and see if the problem is resolved. You could then do a diff of the changes and send it to Microsoft too.
I have a problem with communication via Modbus RTU. First of all sorry for my English ;):
I'm using RS485 Pi SN65HVD72 to connect with MultiCONT PRD-28-2. I made everything like is in this instruction: https://www.abelectronics.co.uk/kb/article/1035/raspberry-pi-3-serial-port-usage
config.txt changes:
toverlay=pi3-miniuart-bt
enable_uart=1
After this steps the config of my /dev looks like this:
lrwxrwxrwx 1 root root 7 gru 30 15:03 serial0 -> ttyAMA0
lrwxrwxrwx 1 root root 5 gru 30 15:03 serial1 -> ttyS0
In MultiCONT PRD-28-2 I change RS485 options to:
Protocol: Modbus
Parity: ODD
Stop bit: 1 bit
Python code:
from pymodbus.client.sync import ModbusSerialClient as ModbusClient
import logging
FORMAT = ('%(asctime)-15s %(threadName)-15s'
' %(levelname)-8s %(module)-15s:%(lineno)-8s %(message)s')
logging.basicConfig(format=FORMAT)
log = logging.getLogger()
log.setLevel(logging.DEBUG)
#multicont = ModbusClient(method='rtu',port='/dev/serial0',stopbits=1,bytsize=8,parity='N',baudrate=9600,timeout=1,strict=False)
multicont = ModbusClient(method='rtu',port='/dev/serial0',stopbits=1,bytsize=8,parity='O',baudrate=9600,timeout=1,strict=False)
connection = multicont.connect()
print(connection)
value = multicont.read_coils(1,15,unit=1)
log.debug(value)
Error message:
Modbus Error: [Input/Output] Modbus Error: [Invalid Message]
Incomplete message received, expected at least 2 bytes (0 received)
Logs:
2020-12-30 16:24:51,992 MainThread DEBUG transaction :118
Current transaction state - IDLE 2020-12-30 16:24:51,992
MainThread DEBUG transaction :123 Running transaction
1 2020-12-30 16:24:51,992 MainThread DEBUG transaction
:230 SEND: 0x1 0x1 0x0 0x1 0x0 0xf 0x2d 0xce 2020-12-30
16:24:51,993 MainThread DEBUG sync :75 New
Transaction state 'SENDING' 2020-12-30 16:24:51,993 MainThread
DEBUG transaction :239 Changing transaction state from
'SENDING' to 'WAITING FOR REPLY' 2020-12-30 16:24:52,995
MainThread DEBUG transaction :253 Transaction failed.
(Modbus Error: [Invalid Message] Incomplete message received, expected
at least 2 bytes (0 received)) 2020-12-30 16:24:52,995
MainThread DEBUG rtu_framer :241 Frame - [b''] not
ready 2020-12-30 16:24:52,996 MainThread DEBUG
transaction :409 Getting transaction 1 2020-12-30
16:24:52,996 MainThread DEBUG transaction :204
Changing transaction state from 'PROCESSING REPLY' to
'TRANSACTION_COMPLETE' 2020-12-30 16:24:52,996 MainThread
DEBUG communication :20 Modbus Error: [Input/Output] Modbus
Error: [Invalid Message] Incomplete message received, expected at
least 2 bytes (0 received)
PyModbus is not receiving a response.
If you have access to an oscilloscope, this is the moment to use it. There are soooo many things that can go wrong with Modbus-RTU.
Make sure you know what you are doing with the electrical wiring, familiarize yourself with RS485 biasing and termination.
Possible causes:
request sent to wrong serial device/interface
device uses different baudrate or serial settings
device can hear you allright but has a different slave ID configured, so it doesn't respond
wrong wiring, grounding issues
RS485 electrical problems, most notably the master must be biasing the bus, usually via a resistor that you have to solder/enable
Bus termination resistors are required for correct voltage levels (sometimes they are already built-in, more often you need to enable them, with some luck may work without)
Master still driving the RS485 bus while the slave is responding. (If you do not have a dedicated RS485 driver, the bus may end up being asserted/deasserted by unreliable software timers. Some devices allow you to configure a response delay as a workaround.)
Master not driving the bus at all. (Working in RS232 mode instead of RS485 mode.)
Also, Linux might have ttyAMA0 configured as a serial console for boot/kernel messages. (This is unlikely to cause your current problem, just something else to watch out for.)
Final piece of advice: if you have the option to use Modbus-TCP instead of Modbus-RTU, do that instead. This replaces all the RS485 electrical headaches which IP-configuration headaches, for which you can use over-the-counter medication.
I have an app that's outputting a JSON through a socket which was working fine in the past, however recently the socket doesn't seem to be establishing and it continuously POST & GET's with the lines:
<user_ip>,<client_ip> - - [17/Jul/2018 12:48:17]"GET /socket.io/?EIO=3&transport=polling......HTTP/1.1" 200 221 0.000000
<user_ip>,<client_ip> - - [17/Jul/2018 12:48:17]"POST /socket.io/?EIO=3&transport=polling......HTTP/1.1" 200 243 0.517600
Additionally, when the WebSocket first tries, I can see from Chromes console the message:
WebSocket connection to ..... failed: Establishing a tunnel via proxy server failed.
I've also had a look through what I think is a similar issue based on socketIO-client but wasn't able to resolve my problem.
Can anyone help with overcoming this connection issue?
Included a full log on running the app below:
Server initialized for eventlet.
* Debugger is active!
* Debugger PIN: 129-744-633
(7616) wsgi starting up on http://0.0.0.0:6328
(7616) accepted (<client_ip>, 50548)
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "GET /<app_url> HTTP/1.1" 200 1664 0.015000
3602c46fa50247eb9d397fda82f3eae8: Sending packet OPEN data {'sid': '3602c46fa50247eb9d397fda82f3eae8', 'upgrades': ['websocket'], 'pingTimeout': 60000, 'pingInterval': 25000}
3602c46fa50247eb9d397fda82f3eae8: Sending packet MESSAGE data 0
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "GET /socket.io/?EIO=3&transport=polling&t=1531829903367-0 HTTP/1.1" 200 381 0.000000
(7616) accepted (<client_ip>, 50560)
3602c46fa50247eb9d397fda82f3eae8: Received packet MESSAGE data 0/testnamespace
3602c46fa50247eb9d397fda82f3eae8: Sending packet MESSAGE data 0/testnamespace
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "POST /socket.io/?EIO=3&transport=polling&t=1531829903377-1&sid=3602c46fa50247eb9d397fda82f3eae8 HTTP/1.1" 200 221 0.000000
<user_ip>,<client_ip> - - [17/Jul/2018 13:18:23] "GET /socket.io/?EIO=3&transport=polling&t=1531829903379-2&sid=3602c46fa50247eb9d397fda82f3eae8 HTTP/1.1" 200 226 0.000000
(7616) accepted (<client_ip>, 50562)
3602c46fa50247eb9d397fda82f3eae8: Received packet MESSAGE data 2/testnamespace,["event_1",{"data":"Web app connection successful."}]
3602c46fa50247eb9d397fda82f3eae8: Sending packet MESSAGE data 2/testnamespace,["client_event",{"json":[{"Date&Time":"........}]
I am having difficulty in establishing a connection with XMPP (Prosody) . But if I use PSI it works fine and request for your suggestions.
Here is the code snippet of my python :
client = xmpp.Client(host)
client.connect(server=(host,port))
client.auth(username, passwd,resource='', sasl=1)
client.sendInitPresence()
In debug mode :
DEBUG: dispatcher warn Registering protocol "error" as <class 'xmpp.protocol.Protocol'>(http://etherx.jabber.org/streams)
DEBUG: socket sent <?xml version='1.0'?><stream:stream xmlns="jabber:client" to="localhost" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" >
DEBUG: socket error Socket error while receiving data
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/xmpp/transports.py", line 161, in receive
try: received = self._recv(BUFLEN)
error: [Errno 104] Connection reset by peer
DEBUG: client stop Disconnect detected
DEBUG: socket error Socket operation failed
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/xmpp/transports.py", line 161, in receive
try: received = self._recv(BUFLEN)
error: [Errno 104] Connection reset by peer
DEBUG: socket error Socket error while receiving data
DEBUG: client stop Disconnect detected
**Prosody logs :**
Oct 20 23:46:39 c2s94ba718 info Client connected
Oct 20 23:46:39 c2s94ba718 info Client disconnected: ssl handshake failed
Oct 20 23:46:39 c2s94ba718 info Destroying session for (unknown) ((unknown)#(unknown))
Oct 20 23:48:57 c2s96052a0 info Client connected
Oct 20 23:48:57 c2s96052a0 info Client disconnected: ssl handshake failed
Oct 20 23:48:57 c2s96052a0 info Destroying session for (unknown) ((unknown)#(unknown))
Any suggestions on this would be much appreciated. Aim is to connect to a XMPP server and join a MUC through python.
As I see that you are using Python 2.7, be sure you updated to the latest version (2.7.11) because there are some recently SSL changes that could break a LOT of code.
I had this error in Pidgin (libpurple 2.10.11) when I did not provide a certificate chain on the server-side.
I solved this by adding the CA-bundle (i.e. intermediate certificate) at the end of the certificate on the server:
cat /path/to/ica.crt >> /path/to/prosody.crt
Also see https://prosody.im/doc/certificates#certificate_chains for further information