HIGHESTMODSEQ response from gmail - python

I have a python script that uses a twisted imap4.IMAP4Client to check for new emails in a gmail account. It does so every ~30 seconds. It has been running fine for months.
Starting June 11, I began to see this message after the search("(UNSEEN)") call:
Unhandled unsolicited response: ['OK', ['HIGHESTMODSEQ', '1234567']]
This message appears at every check for new messages now.
What does this message mean? Is it anything to worry about?

HIGHESTMODSEQ is a protocol extension you can ignore. It is documented in RFC4551.

It's just silly log noise. The IMAP4 client parser is telling you the server sent it something it didn't expect and doesn't really know how to handle. So it is handling it by logging that message.
Off the top of my head, I have no idea what HIGHESTMODSEQ is for, but if you don't care about that information either, then you can just ignore this. If you do care about it, you can contribute a patch to Twisted to add support for this kind of unsolicited response. :)

Related

How to save the information in Telegram notification by python ? (possible?)

I want when someone sends a message in Telegram, the information of the message is stored in the telegram bot or something else without seeing the other party's message.
I assume you want to do what is known as "ghost mode" which means you read messages but they're not double-checked.
Before I answer your question, I should warn you that Telegram's founder is against such use (see https://twitter.com/durov/status/891227609706536960).
The fun fact is you don't have to do anything to avoid double-checking a message. In fact there is an extra step for that. Therefore, all you need to do is to read the message. You can do so in telethon as simple as this:
#app.on(events.NewMessage())
def read(event):
print(event.message.message)

Gathering bug reports?

Premise: I am a beginner in search for an easy way to send bug reports from users over sea.
I've made a script for some friends that are living on the other side of the sea (US - EUROPE)... I will like to gather automatic bug reports whenever they happen. So my first idea was to send myself an email with the smtplib module. It works fine when testing home, but as soon as the sender "sends", my email provider (gmail) blocks the connection because of course, its from an "unknown device". I've already enabled "Allow less secure apps" as someone suggested but with no avail.
What I am searching its a simple way of dealing with this.
Yes I could make the script to ignore the error if the email its not being sent, and then go into my google account and enable those devices so at least it will work from the second run..
But it doesn't seem what a programmer would do in this case. I am learning so a solution withing the language is what I am after.
A different provider that has no restriction its also a good start but I tried Yahoo, Live, Yandex but I couldn't make them work. Are there any?
So my question is: how others do? what is the best solution for some one like me?
I've read about sentry or other error/bug tracking but its obviously way too much for want I need
You should certainly not incorporate e.g. Gmail credentials in the code that is remotely executed on devices you do not control, given I understand correctly the Gmail less secure device issue happens as every "user" is running this code and using your credentials. This holds true for any other provider.
Now this won't exactly be simple but one way to go about it would be to create a server side API endpoint that can accept HTTP(s) or any other protocol requests that then will authenticate in a little more secure way on the server side with Gmail.
The concept for emails is:
Bug > Python Script > API call > Email
This could be implemented using Python on the API side (Flask e.g.) using an AWS Lambda Function with Amazon API Gateway, but again that is something to get through and understand by itself which will take a good chunk of time.
You need to touch a lot of concepts, like auth tokens to make this really secure.
Could you elaborate a little on where the code needs to run and if you are willing to try AWS or any other cloud provider, or would have access to an internet connected server ? This makes it easier to provide you with a full example on the solution in a hackish way while I would highlight the problems you could face on the security side.
I understand that this is not the way to go but as for my needs and my level of experience it works for me!
Yandex allows you to send email from different ip so Yandex is the way to go. What I was doing wrong in the first place was to use the wrong port (587 instead of 465)

WebSocket Safety

I decided to write a websocket chat that he supported text messaging [unlimited number of characters, not like Twitter :)] and file transfer.
Ask this question. How to make banal identification data? That is, when the connection is open, the server is just a set of bytes, and even if there is json, it can be easy to substitute: mark as "message" and send confusing file(since the number of characters is not limited, it will sent to all people users).
That is, briefly, how to discern what comes from the user?
Thanks in advance!
P.S. Transfer files via jquery is not accepted, as the websocket is not protected.
UPD
Any wrote that did not understand a question essence. Explain: 1. There is a chat on tornado WS. 2. In chat can send as text messages and files. 3. I control it with javascript BUT if someone will make a connection and send a file for example, as a message? Clients receives a huge number of "unknown byte". The situation is like this - {'msg': 'Hello, world!!1'}; {'file': a file}. All is well, but it would be a shame if {'msg': a file}. It should be clear ;)
Welcome to SO.
Before I answer your question, allow me to clarify a few points about this site and our community. This is your first question, so maybe you didn't know:
SO (StackOverflow) is not a forum, it's more like both a chat and a library. If you ask a question, you are expected to stay on line for at least an hour or two to check in and give more information if somebody asks you.
If somebody asks for more information, edit the question - don't answer inside the comments (you can add a comment saying that you updated the question, but the question should be edited).
If you are not here to clarify your question, it WILL be closed because there are many people wanting answers and we give priority to the people who are here to respond.
It is important that your questions include information about what you already tried. It is better if your questions include some of the code you wrote when you tried your own solution.
Otherwise, it feels like you are trying to outsource your work to the community - which is a very big No No.
It is important that your questions are clear and that you write in the best English that you can manage.
Please read the comments to see what I mean about the community's expectations.
As for Websocket security:
I do not know what language you work with, so I will write in Ruby because it's easier for me and I think the code will be easier to read.
Websockets start as an HTTP connection which is then "upgraded" to a websocket connection.
Since most applications already have HTTP security / authentication logic that they wrote for the website, it is best to use this same security / authentication logic for websockets.
This is usually done BEFORE the connection is upgraded.
It is true that sometimes people write a new security / authentication logic layer for websockets, but that is not the most effective way to deal with this issue (mainly because messages could be sent to the websocket while authentication is still being processed, which starts a new world of issues and considerations).
Here is an example, using the Plezi framework, that requires authentication before the upgrade. test this example by installing the plezi gem (terminal: $ [sudo] gem install plezi) and running the following code in irb (terminal : $ irb):
require 'plezi'
class WebsocketSecDemo
# pre_connect is called before the upgrade, while still in HTTP mode.
def pre_connect
# return false (refuse the connection) unless the auth logic is okay
return false unless auth_logic
true
end
def on_message data
puts "got #{data}"
response << data
broadcast :ws_write, data
end
def index
"Check this using http://www.websocket.org/echo.html
\r\nConnect to the websocket at ws://localhost:3000/
\r\nTo authenticate, visit http://localhost:3000/login
\r\nTo un-authenticate visit http://localhost:3000/logout"
end
def login
cookies[:my_auth] = true
redirect_to :index
end
def logout
cookies[:my_auth] = nil
redirect_to :index
end
# protected methods aren't available to the HTTP router.
protected
def ws_write data
response << data
end
# The worst auth logic in the history of man kind...
def auth_logic
cookies[:my_auth] == true
end
end
Plezi.listen
Plezi.route '/', WebsocketSecDemo
# exit the terminal to start the demo
exit
Now visit:
localhost:3000 - for instructions
localhost:3000/login - to authenticate
localhost:3000/logout - to remove authentication
Also use www.websocket.org/echo.html a few times to try and connect to the websocket at ws://localhost:3000/ - try this:
before authentication;
after authentication (login); and
after you 'logout'...
Good Luck!

Gmail IMAP is sometimes returning bad results for fetch

I'm using python to connect to gmail via IMAP.
When I do a fetch for a bunch of uids, the results sometimes contain a couple weird ones tacked on to the end.
This bug appears intermittent and started appearing sometime in the last week or so.
For example I do a fetch like:
>>>import imaplib
>>>conn = imaplib.IMAP4_SSL('imap.gmail.com')
>>># authenticate etc
>>>conn.uid('fetch', '12,13', '(X-GM-THRID)')
I sometimes get a result like:
>>>['1 (X-GM-THRID 123123123123 UID 12)',
'2 (X-GM-THRID 123123123123 UID 13)',
'365022 (UID 601722 FLAGS (\\Seen))']
(Line breaks added for readability, ids changed from original, normally I'm fetching far more than two.)
This is pretty weird. I haven't requested anything at all about the extra message. Sometimes it's there, sometimes not. No matter what fields I fetch, the extra result (I've only ever seen one or two) never contains them, and only contains the FLAGS info.
Any ideas why this might be happening?
The following was posted in the IMAP Protocol mailing list earlier this week:
"As of yesterday [Monday October 7th], Gmail now supports reporting flags changes anywhere we would report new/expunged messages (ie, most places its allowed by the protocol, but definitely during IDLE). Its only enabled for gmail.com users at the moment, Google Apps users will follow in a week or so, assuming we don't find any issues."
However, there are apparently issues with the new functionality because today they said it is being rolled back:
"We're seeing several reports of programs not expected extra FETCH responses, which we rolled out on Monday. We're rolling back soon, since older versions of a very popular client are having issues (though not to our knowledge with this part of things)."
-Rick
Gmail is sending you unsolicited FLAGS updates (because someone changed the message remotely). These aren't in response to your request, but IMAP allows the server to send you any* information at any time. Many servers will save these unsolicited responses for IDLE or NOOP responses, but apparently Gmail does not wish to wait.
However, until recently (apparently?) Gmail did not send flag updates at all, only EXPUNGEs.
*: There are a few rules about which responses can be sent when to avoid race conditions, but this isn't one of those.

QuickFIX logon trouble: multiple rapid fire logon attempts being sent

QuickFIX logon trouble: (using QuickFIX, with FIX 4.4 in Python 2.7)
Once I do initiator.start() a connection is made, and logon message is sent. However, I don't ever see the ACK and session status message that the broker is sending back (all the overloaded Application methods are just supposed to print out what they receive).
QuickFIX immediately re-tries the logon (according to the broker log files), and the same thing happens, but according to the server, I am already logged in.
QuickFIX then issues a Logout command, which the server complies with.
I have tried enter Timeout values in the settings file, but to no avail. (Do I need to explicitly reference these values in the code to have the utilized, or will the engine see them and act accordingly automatically?)
Any ideas what is going on here?
Sounds like you do not have message logs enabled. If your app rejects messages below the application level (such as if the seq no is wrong, or the message is malformed), then it'll be rejected before your custom message handlers even see it.
If you are starting your Initiator with a ScreenLogStore, change it to a FileLogStore. This will create a log file that will contain every message sent and received on the session, valid or not. Dollars to donuts you'll see your Logon acks in there as well as some Transport-layer rejections.
Solved! I think there was something wrong with my datadictionary (FIX44.xml) file. I had seen a problem in it before, but thought I fixed it. I got a new copy online and dropped it in and now everything seems to be working. Maybe the bad dictionary was not letting FIX accept the logon response?

Categories