We Keep Coding

Retrieve all emails from Gmail i did but only got 3000 email not all

What is the way to pull out all emails from Gmail?
I did full_sync, but that didn't return all of my email - only about 3000 emails, while I know I have more. In the documentation they did not mention about this.
My code snippet:
history = service.users().history().list(
userId='me',
startHistoryId=start_history_id,
maxResults=500,
labelId='INBOX'
).execute()
if "history" in history:
try:
for message in history["history"]:
batch.add(
service.users().messages().get(userId='me', id=message["messages"][0]["id"]),
callback="somecallbak",
request_id=request_id
)
batch.execute()
while 'nextPageToken' in history:

If you are doing a full sync, you should refer to this documentation, that recommends two steps:
listing all the messages with the users.messages.list method
for each of the entry get the required information using the users.messages.get method
So you don't need use the users.history.list as you will have an hard time finding the startHistoryId from which to start.
You can achieve this with a snipped similar to the one below (tested and working on my python 3.x console). As suggested by others I used the python client pagination and batch request functionalities.
from httplib2 import Http
from googleapiclient.discovery import build
from oauth2client import client, tools, file
# callback for the batch request (see below)
def print_gmail_message(request_id, response, exception):
if exception is not None:
print('messages.get failed for message id {}: {}'.format(request_id, exception))
else:
print(response)
# Scopes
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly', ]
# where do we store our credentials?
creds_store = file.Storage('gmail-list.json')
start_creds = creds_store.get()
# standard oauth2 authentication flow
if not start_creds or start_creds.invalid:
# client_id.json is exported from your gcp project
start_flow = client.flow_from_clientsecrets('client_id.json', SCOPES)
start_creds = tools.run_flow(start_flow, creds_store)
# Gmail SDK
http = Http()
gmail_sdk = build('gmail', 'v1', http=start_creds.authorize(http))
# messages.list parameters
msg_list_params = {
'userId': 'me'
}
# messages.list API
message_list_api = gmail_sdk.users().messages()
# first request
message_list_req = message_list_api.list(**msg_list_params)
while message_list_req is not None:
gmail_msg_list = message_list_req.execute()
# we build the batch request
batch = gmail_sdk.new_batch_http_request(callback=print_gmail_message)
for gmail_message in gmail_msg_list['messages']:
msg_get_params = {
'userId': 'me',
'id': gmail_message['id'],
'format': 'full',
}
batch.add(gmail_sdk.users().messages().get(**msg_get_params), request_id=gmail_message['id'])
batch.execute(http=http)
# pagination handling
message_list_req = message_list_api.list_next(message_list_req, gmail_msg_list)

As suggested in this link, you may use batch requests.
Use batch and request 100 messages at a time. You will need to make 1000 requests but the good news is that's quite fine and it'll be easier for everyone (no downloading 1GB response in a single request!).
Also based from this thread, you could save the next page token on every request and use it in your next request. If there is no next page token in the response, you know that you have gotten all messages.

Receive and Print a Twilio SMS with Python

For clarification, I don't want to reply to the SMS. Every tutorial or document I've looked at is about setting up a port to listen on.
What I'm trying to do is just get the SMS and print it. I can send them fine and without problems.
Here is my sending function, and it works.
def send():
message = client.messages \
.create(
body=sendMSG,
from_='MY_TWILIO_NUMBER',
to='MY_PERSONAL_NUMBER'
)
print(message.sid)
How would you receive an SMS without Flask? Is there a way to do something similar to this method below just for receiving?
def receive():
message = client.messages \
.recieve(
from_='MY_PERSONAL_NUMBER',
to='MY_TWILIO_NUMBER'
)
print(message.sid)

I have not personally tried to get SMS messages from the logs before, always getting it directly through a webhook, but from what I see, it appears the command you might be looking for is list(). You can add filters, as shown in the API docs, and there are three filtering options. You can filter by DateSent, To, or From.
I have not tried this, but it would seem that the way to use this would be the following (adjusted from the code they supply):
# Download the helper library from https://www.twilio.com/docs/python/install
from twilio.rest import Client
# Your Account Sid and Auth Token from twilio.com/console
account_sid = 'ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
auth_token = 'your_auth_token'
client = Client(account_sid, auth_token)
messages = client.messages.list(from='MY_PERSONAL_NUMBER', to='MY_TWILIO_NUMBER')
for record in messages:
print(record.sid)
If that doesn't work, the variables they use are actually capitalized "To" and "From", so you might try that.
After looking at that a bit, you might be looking more for this:
received = client.messages.list(to='MY_TWILIO_NUMBER')
sent = client.messages.list(from='MY_PERSONAL_NUMBER')
That will separate out those sent to you, and those sent from you

How implament fieldMask in python grpc updating request?

I have not understood completely how protobuf FieldMask works in grpc request, I've created this message from client, but I do not know how to receive and show the values in the server site.
Client.py
fm = FieldMask(
paths=[ 'f.a',
'f2.a'
])
msg = proto_pb2.Message(
f=proto_pb2.F(a=a_value),
f2=proto_pb2.F2(a=a_value)
)
new_message = proto_pb2.Message()
fm.MergeMessage(msg, new_message)
# Create message to send server
request = server_manager_pb2.UpdateMessageRequest(
id=proto_pb2.Message(id=message_id),
update_mask=fm)
# make the call
response = stub.UpdateMessage(request)
but in server.py
def UpdateMessage(self, request, context):
print(request.update_mask)
is displaying the follow value:
paths: "f.a"
paths: "f2.a"
how can I display the field values sent by client in this server method or how FieldMask works?

Alexa and Python Lambda Function to retrieve Device Address

I'm trying to get the device address from Alexa.
I've followed this tutorial but I'm having some problems.
The code I'm using is this one:
def test_intent(event, context):
from botocore.vendored import requests
device_id = event["context"]["System"]["device"]["deviceId"]
api_access_token = event["context"]["System"]["apiAccessToken"]
api_uri = "https://api.amazonalexa.com/v1/devices/{}/settings/address".format(device_id)
api_headers = {"Accept": "application/json", "Authorization": "Bearer {}".format(api_access_token)}
api_response = requests.get(api_uri, headers=api_headers)
data = api_response.json()
I put some location (US location) in my alexa app and I allowed the permission the skill ask me (Full Address in this case).
The api_response.status_code is 204, so it should be ok. But when I run the last line "api_response.json()" it crashes (alexa replays that there is a problem with the skill).
I think the problem is the api_response is None and the json is not working, but I don't know why this could happend.
Any guess about this?
Edit:
After some test, I realised that I've set the phone's location, not the echo dot location in the Alexa app, so that's the reason the app responded with 204 (no data)

Waiting for websocket.receive consumer in django-channels

How to wait for a response from client after sending the client something using django-channels?
Whenever Group.send() is called from function send_to_client() and upon receiving the message by client, send_to_client() is expecting a response back from client which is getting received on websocket.receive channel.
Is there a way to return the response_msg in send_to_client() function?
Now I have reached here
Sample Code for consumers.py:
def ws_receive(message):
response_msg = message.content['text']
return response_msg
def send_to_client(param1, param2, param3):
Group.send({
"text" : json.dumps({
"First" : param1,
"Second" : param2,
})
})
So once the message reaches at the client side, the client will send a response back to the server which will be received by the ws_receive(message) function through the websocket.receive channel which is defined in the urls.py file,
channel_patterns = [
route("websocket.receive", ws_receive),
...
]
Is there a way to do this so that my function would look like this?
def send_to_client(...):
Group.send(...)
response_msg = #response message from client

Since you are recieving via a websocket, I am not sure if you would be able to even tell if the recieving thing is directly as a response for your request. I would rather put an id variable or something in the ongoing request, and maybe ask the client to put that id in the response as well. That might require both the sender and reciever to know the value of id as well (probably store in the db?)
Also, it do not seem logical to be blocked waiting for the response from websocket as well.