I'm writing a simple code in python 2.7 that is consuming messaging from an apache kafka topic. The code is the following:
from kafka import SimpleConsumer,KafkaClient
group = "my_group_test"
client = KafkaClient('localhost:9092')
cons = SimpleConsumer(client, group, "my_topic")
messages = cons.get_messages(count=1000,block=False)
But is raising this exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/simple.py", line 285, in get_messages
update_offset=False)
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/simple.py", line 320, in _get_message
self._fetch()
File "/usr/local/lib/python2.7/dist-packages/kafka/consumer/simple.py", line 425, in _fetch
raise ConsumerFetchSizeTooSmall()
kafka.errors.ConsumerFetchSizeTooSmall
How could i modify this parameter (ConsumerFetchSize) in order to make this code work?
I found a solution,
using the parameter max_buffer_size in the SimpleConsumer.
The working code is:
#the size is adherent with my need but is totally arbitrary
cons = SimpleConsumer(client, group, "my_topic",max_buffer_size=9932768)
Related
Iam trying to remotely control the trace32 and trying to add commands from pyhton, in this process i have to set break point using python but i cannot set as its saying the function name is not defined (i have already flashed my hex and elf) other functions such as symbol name is working well
import lauterbach.trace32.rcl as t32
dbg = t32.autoconnect()
self = dbg.symbol.query_by_name(name='ComMainFunctionTx')
print(self)
bp = dbg.breakpoint.set(*"ComComMainFunctionTx ",**())
This is my output iam not understanding what to keep here for mapping
ComComMainFunctionTx P:0x801bacf0 930
Traceback (most recent call last):
File "c:\Users\Desktop\test hello.py", line 24, in <module>
bpt=dbg.breakpoint.set(*"ComComMainFunctionTx ",**())
TypeError: lauterbach.trace32.rcl._rc._breakpoint.BreakpointService.set() argument after ** must be a mapping, not tuple
update
this is what iam trying to do , iam adding dbg.go_return() to go to the next breakpoint but its throwing timedout error please help if i manually press go button its going to the next breakpoint in trace32 but giving bp.go() its giving timedout error
data=pd.read_csv("useR1.csv")
# converting column data to list
ARG = data['Arguments'].tolist()
for i in ARG:
mysys = dbg.symbol.query_by_name(name=i)
bp = dbg.breakpoint.set(address=mysys.address)
dbg.go_return()
Break
File "c:\Users\Desktop\IT_Script\test hello.py", line 70, in <module>
dbg.go_return()
File "C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\lauterbach\trace32\rcl\rcl.py", line 429, in go_return
self.cmd("Go.Return")
File "C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\lauterbach\trace32\rcl\_rc\_command.py", line 16, in __call__
self.__conn._cmd(cmd)
File "C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\lauterbach\trace32\rcl\rcl.py", line 284, in _cmd
self.__library.t32_executecommand(cmd.encode(), 4096)
File "C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\lauterbach\trace32\rcl\_rc\_library.py", line 429, in t32_executecommand
return self.generic_api_call(
File "C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\lauterbach\trace32\rcl\_rc\_library.py", line 313, in generic_api_call
recv_data = self._link.receive()
File "C:\Users\AppData\Local\Programs\Python\Python310\lib\site-packages\lauterbach\trace32\rcl\_rc\hlinknet.py", line 649, in receive
raise ApiConnectionTimeoutError(str(e)) from None
lauterbach.trace32.rcl._rc._error.ApiConnectionTimeoutError: timed out
This should work:
import lauterbach.trace32.rcl as t32
dbg = t32.autoconnect()
sym = dbg.symbol.query_by_name(name='ComMainFunctionTx')
print(sym)
bp = dbg.breakpoint.set(address=sym.address)
print(bp)
I'm trying to call the remote procedure DisplayFolderAndSelect() of Thunar file manager from my own program:
import dbus
bus = dbus.SessionBus()
obj = bus.get_object('org.xfce.Thunar', '/org/xfce/FileManager')
iface = dbus.Interface(obj, 'org.xfce.FileManager')
_thunar_display_folder_and_select = iface.get_dbus_method('DisplayFolderAndSelect')
_thunar_display_folder_and_select('~/Downloads/', 'doc.pdf', '', '')
However I've encountered the following exception at runtime:
Traceback (most recent call last): File "", line 1, in
File "/usr/lib/python2.7/dist-packages/dbus/proxies.py",
line 70, in call
return self._proxy_method(*args, **keywords) File "/usr/lib/python2.7/dist-packages/dbus/proxies.py", line 145, in
call
**keywords) File "/usr/lib/python2.7/dist-packages/dbus/connection.py", line 651, in
call_blocking
message, timeout) dbus.exceptions.DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name :1.576 was not
provided by any .service files
I'm unable to understand what does this exception mean. And what's the cause behind the exception.
Any thoughts?
I think it is an OS-related issue, restarting D-Bus service solved the problem
I am trying to do parallel ssh on servers. While doing this, i am getting "TypeError: 'NoneType' object is not iterable" this error. Kindly help.
My script is below
from pssh import ParallelSSHClient
from pssh.exceptions import AuthenticationException, UnknownHostException, ConnectionErrorException
def parallelsshjob():
client = ParallelSSHClient(['10.84.226.72','10.84.226.74'], user = 'root', password = 'XXX')
try:
output = client.run_command('racadm getsvctag', sudo=True)
print output
except (AuthenticationException, UnknownHostException, ConnectionErrorException):
pass
#print output
if __name__ == '__main__':
parallelsshjob()
And the Traceback is below
Traceback (most recent call last):
File "parallelssh.py", line 17, in <module>
parallelsshjob()
File "parallelssh.py", line 10, in parallelsshjob
output = client.run_command('racadm getsvctag', sudo=True)
File "/Library/Python/2.7/site-packages/pssh/pssh_client.py", line 520, in run_command
raise ex
TypeError: 'NoneType' object is not iterable
Help me with the solution and also suggest me to use ssh-agent in this same script. Thanks in advance.
From reading the code and debugging a bit on my laptop, I believe the issue is that you don't have a file called ~/.ssh/config. It seems that parallel-ssh has a dependency on OpenSSH configuration, and this is the error you get when that file is missing.
read_openssh_config returns None here: https://github.com/pkittenis/parallel-ssh/blob/master/pssh/utils.py#L79
In turn, SSHClient.__init__ blows up when trying to unpack the values it expects to receive: https://github.com/pkittenis/parallel-ssh/blob/master/pssh/ssh_client.py#L97.
The fix is presumably to get some sort of OpenSSH config file in place, but I'm sorry to say I know nothing about that.
EDIT
After cleaning up some of parallel-ssh's exception handling, here's a better stack trace for the error:
Traceback (most recent call last):
File "test.py", line 11, in <module>
parallelsshjob()
File "test.py", line 7, in parallelsshjob
output = client.run_command('racadm getsvctag', sudo=True)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/pssh_client.py", line 517, in run_command
self.get_output(cmd, output)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/pssh_client.py", line 601, in get_output
(channel, host, stdout, stderr, stdin) = cmd.get()
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/gevent/greenlet.py", line 480, in get
self._raise_exception()
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/gevent/greenlet.py", line 171, in _raise_exception
reraise(*self.exc_info)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/gevent/greenlet.py", line 534, in run
result = self._run(*self.args, **self.kwargs)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/pssh_client.py", line 559, in _exec_command
channel_timeout=self.channel_timeout)
File "/Users/smarx/test/pssh/venv/lib/python2.7/site-packages/pssh/ssh_client.py", line 98, in __init__
host, config_file=_openssh_config_file)
TypeError: 'NoneType' object is not iterable
This was seemingly a regression in the 0.92.0 version of the library which is now resolved in 0.92.1. Previous versions also work. OpenSSH config should not be a dependency.
To answer your SSH agent question, if there is one running and enabled in the user session it gets used automatically. If you would prefer to provide a private key programmatically can do the following
from pssh import ParallelSSHClient
from pssh.utils import load_private_key
pkey = load_private_key('my_private_key')
client = ParallelSSHClient(hosts, pkey=pkey)
Can also provide an agent with multiple keys programmatically, per below
from pssh import ParallelSSHClient
from pssh.utils import load_private_key
from pssh.agent import SSHAgent
pkey = load_private_key('my_private_key')
agent = SSHAgent()
agent.add_key(pkey)
client = ParallelSSHClient(hosts, agent=agent)
See documentation for more examples.
I am new to python. I am trying to use track_from_file method in pyechonest to create a track object. This is the code I wrote:
from pyechonest import config
from pyechonest.track import track_from_file
config.ECHO_NEST_API_KEY = "KRZG4XOVUQRKI45DP"
f = open("ShayneWard-NoPromises.mp3")
t = track_from_file(f, 'mp3')
But this is not working. I got an EchoNestIOError. This is the full error Traceback that i got.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.linux-x86_64/egg/pyechonest/track.py", line 243, in track_from_file
File "build/bdist.linux-x86_64/egg/pyechonest/track.py", line 331, in track_from_md5
File "build/bdist.linux-x86_64/egg/pyechonest/track.py", line 209, in _profile
File "build/bdist.linux-x86_64/egg/pyechonest/util.py", line 257, in callm pyechonest.util.
EchoNestIOError: Echo Nest Unknown Error
Can anyone tell me what have I done wrong?
I finally figured it out. It seems that the upload does not take place through the proxy server of my university network which uses NAT.
I'am playing around with scapy but i cant get it to work. I tried different code's but all gave me the same output:
Traceback (most recent call last):
File "<module1>", line 7, in <module>
File "C:\Python26\lib\site-packages\scapy\sendrecv.py", line 357, in srp
s = conf.L2socket(iface=iface, filter=filter, nofilter=nofilter, type=type)
File "C:\Python26\lib\site-packages\scapy\arch\pcapdnet.py", line 313, in __init__
self.outs = dnet.eth(iface)
File "dnet.pyx", line 112, in dnet.eth.__init__
OSError: Result too large
Iam using python 2.6 with all dependencies installed for scapy.
How to fix this?