We Keep Coding

give an example using GrpcHook and GrpcOperator in Airflow

i am new in airflow and gRPC
i use airflow running in docker with default setting
https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html
when i try to do in this link
https://airflow.apache.org/docs/apache-airflow-providers-grpc/stable/_api/airflow/providers/grpc/index.html
channel = grpc.insecure_channel('localhost:50051')
number = calculator_pb2.Number(value=25)
con = GrpcHook(grpc_conn_id='grpc_con',
interceptors=[UnaryUnaryClientInterceptor]
)
run = GrpcOperator(task_id='square_root',
stub_class=calculator_pb2_grpc.CalculatorStub(channel),
call_func='SquareRoot',
grpc_conn_id='grpc_con',
data=number,
log_response=True,
interceptors=[UnaryUnaryClientInterceptor]
)
no response in DAG log even server is shut down or server port is wrong, but it works if i call with simple client

What you're looking for I guess is the GrpcOperator example.
In your example, the wrong parameter is data.
The data parameter should be data={'request':calculator_pb2.Number(value=25)}, if you don't modify generated protof files.
This is an example.
from airflow.providers.grpc.operators.grpc import GrpcOperator
from some_pb2_grpc import SomeStub
from some_pb2 import SomeRequest
GrpcOperator(task_id="task_id", stub_class=SomeStub, call_func='Function', data={'request': SomeRequest(var='data')})

How can i do print string by time while i running http_serve_forever() in python

I am tring to run the httpserver on the raspberrypi to connect and send data via cellphone. I can send the data and run other function, but now I would like to print the string at 12 every day. I've tried sockettime and date_time_string
Here is my code:
from http.server import BaseHTTPRequestHandler, HTTPServer
import random
import os
from datetime import datetime, timedelta
class RequestHandler_httpd(BaseHTTPRequestHandler):
def do_GET(self):
global Request, test, data, case
messagetosend = bytes('test', "utf")
self.send_response(200)
self.send_header('Content-Type', 'text/plain')
self.send_header('Content-Length', len(messagetosend))
self.end_headers()
self.wfile.write(messagetosend)
Request = self.requestline
Request = Request[5: int(len(Request)-9)]
return
def date_time_string(self, timestamp=None):
if __name__ == '__main__':
server_address_httpd = ('192.168.66.19', 8080)
httpd = HTTPServer(server_address_httpd, RequestHandler_httpd)
print('start')
httpd.serve_forever()
Any help would be appreciated.

I see five options:
1.) run your web server and add on the same machine a cronjob, that accesses the required url (for example with wget)
2.) run your web server and add a cronjob on another machine
3.) don't use a web server at all, but just use a cron job
4.) Depending on the framework you're using for the web server you might add a thread programming a timer, executing the job and reprogramming the timer.
In general However I try to avoid adding threads to a web server. You have to be careful to not do things, that are not thread safe and this can be tricky depending on your framework. but for some use cases it could be a simple solution.
5.) almost the same like for, but simulating an http request to your own url, which will probably avoid any race condition, which you might encounter with 4.

Multiple instances of tor.exe, each with own identity via python

I'm testing my own ddos protection feature implemented in my server (this is necessary). Currently I have a terrible loop for making multiple tor requests, each with it's own identity.
os.system("taskkill /f /im tor.exe")
os.startfile("C:/Tor/Browser/TorBrowser/Tor/tor.exe")
session = requests.session()
session.proxies = {}
session.proxies['http'] = 'socks5h://localhost:9050'
session.proxies['https'] = 'socks5h://localhost:9050'
Now I want to multithread this for faster speeds, since each tor connection takes ages to load.
If I google how to run multiple tor instances, I get info on how to do this from within the tor browser itself, never how to do it programmatically, Is there a way to do this on windows python3 specifically?
Any help appreciated

The key point to understand about running multiple separate Tor processes is that each one will need to listen on it's own ControlPort and SocksPort so that your clients can issue requests through each individual instance.
If you use Stem, stem.process.launch_tor_with_config would be the recommended way to launch multiple Tor processes. By using this method, you can pass the necessary config options dynamically to each client without having to create individual files, and you'll have better process management over the Tor instances.
If you want to use os, you will need to create one config file per instance and pass that to tor when you start it.
At minimum, create one torrc config file for each instance you want to run with the following:
torrc.1
ControlPort 9800
SocksPort 9801
torrc.2
ControlPort 9802
SocksPort 9803
Each individual client will connect on the different socks ports to issue requests.
To start them, use:
os.system("C:/Tor/Browser/TorBrowser/Tor/tor.exe -f C:/path/to/torrc.1")
os.system("C:/Tor/Browser/TorBrowser/Tor/tor.exe -f C:/path/to/torrc.2")
Then create one or more clients per instance:
session1 = requests.session()
session1.proxies = {}
session1.proxies['http'] = 'socks5h://localhost:9801'
session1.proxies['https'] = 'socks5h://localhost:9801'
session2 = requests.session()
session2.proxies = {}
session2.proxies['http'] = 'socks5h://localhost:9803'
session2.proxies['https'] = 'socks5h://localhost:9803'

first of All, install stem like this in terminal
>>>pip install stem
write this code in a text file and rename the file like this myfile.py
include stem and requests first like this in start of file and write following code
import requests
import stem.process
x = 6
for i in range(1, x):
cp = str(10000+i)
sp = str(11000+i)
tp1 = stem.process.launch_tor_with_config(tor_cmd = 'C:\\Users\\<Tor Directory>\\Browser\\TorBrowser\\Tor\\tor.exe', config = {
'ControlPort': cp,
'SocksPort' : sp,
'DataDirectory': 'C:/<Any Path for data directories>/proxies/'+str(i)+'/',
'Log': [
'NOTICE stdout',
'ERR file C:/<Any Path for Error file>/tor_error_log.txt',
],
},
)
proxies = {
'http': 'socks5h://127.0.0.1:'+str(sp),
'https': 'socks5h://127.0.0.1:'+str(sp)
}
r1 = requests.get('http://ipinfo.io/json', proxies=proxies)
print('\n')
print(r1.content)
print('\n')
now go into the folder that contains myfile.py and run command prompt(cmd) or any terminal there and launch the file like this.
>>>python myfile.py
this will launch 5 tor processes on these ports 11001,11002,11003,11004,11005
you can access the tor proxy(socks5) by using ip address 127.0.0.1 and any of the above ports from any program.
if you open task manager you will see 5 tor processes running that consumes 10-20mb of ram each process
if you get an error like this while running myfile.py in terminal,
can not bind listening port. working with config files left us in broken state. Dying
then just close all processes of tor and launch myfile.py again. this error happens because you have already a tor process running on the port.
to create more tor processes, close all tor instances from task manager, change the value of variable x in start of file like this
x = any integer like 10,20,30,50
save myfile.py and run this file again.
cheers!

Python ping list of IPv6 hosts and make a dictionary of reachable and unreachable hosts with time

the following script is working perfectly fine for IPv4 but however my job is to do same for IPv6..
#!/usr/bin/python
import pyping
response = pyping.ping('8.8.8.8')
if response.ret_code == 0:
print("reachable")
else:
print("unreachable")
is there any way.. i tried to install aioping or aio_ping.. but didnt work,.. is there any workaround to run the same as above for IPv6 in linux machines

Using the example from the multi-ping (pip install multiping) documentation:
from multiping import MultiPing
# Create a MultiPing object
mp = MultiPing(["2a00:1450:4005:803::200e"])
# Send the pings to those addresses
mp.send()
# With a 1 second timout, wait for responses (may return sooner if all
# results are received).
responses, no_responses = mp.receive(1)
if responses:
print("reachable: %s" % responses)
if no_responses:
print("unreachable: %s" % no_responses)
Have a look at the documentation to see how responses/no_responses are structured and how to ping multiple addresses simultaneously.

ZeroRPC between nodejs and Python running only for the first time

We are trying to call a python module from NodeJS using ZeroRPC.
Nodejs Code:
app.get('/topList', function(req, res){
var zerorpc = require("zerorpc");
var client = new zerorpc.Client({ timeout: 3000, heartbeatInterval: 300000 });
console.log('Here');
client.connect("tcp://127.0.0.1:4212");
console.log('And Here');
client.invoke("toplist", '2015-02-13T05:46:29.800Z', '2015-02-13T08:46:29.800Z', function(error, tlist, more)
{
console.log('And Here also');
if(error) {
console.error(error);
}
console.log(tlist);
res.send(tlist);
client.close();
});
});
Python code:
import zerorpc
import logging
import gevent
logging.basicConfig()
class HelloRPC(object):
def toplist(self, fromtime, tilltime):
print 'Python Hello'
# Doing some some manipulations and placing the result in the variable top_20
return top_20
p = HelloRPC()
s = zerorpc.Server(p)
s.bind("tcp://127.0.0.1:4212")
s.run()
First we ran the python code to up the server. Next we start the nodejs code.
Now in the browser for the first time if we try for localhost:8080/topList, nodejs issues an request to python, then python program is running and returning the result to nodejs, then in the browser we are able to see the result.
The problem comes if we try for the second time localhost:8080/topList, nodejs not able to invoke the python code. We didn't see 'Python Hello' in python console.
We are not able to fix this scenario. It is only running for the first time, then nodejs not able to invoke python. Is the server is getting closed after runnig first time.
We also tried to use While True or gevent like below. But still same issue.
While True:
p = HelloRPC()
s = zerorpc.Server(p)
s.bind("tcp://127.0.0.1:4212")
s.run()
#gevent.spawn(s.run)
Can anyone will provide some idea to fix this ?