I am trying to setup commonsearch and i am the point where backend imports alexa1m data to rocksdb, but it die with error( error below ).
Traceback (most recent call last):
File "urlserver/import.py", line 21, in <module>
ds.import_dump()
File "./urlserver/datasources/__init__.py", line 62, in import_dump
for i, row in self.iter_dump():
File "./urlserver/datasources/__init__.py", line 102, in iter_dump
f = self.open_dump()
File "./urlserver/datasources/__init__.py", line 144, in open_dump
return GzipStreamFile(f)
File "/cosr/back/venv/src/gzipstream/gzipstream/gzipstreamfile.py", line 62, in __init__
super(GzipStreamFile, self).__init__(self._gzipstream)
File "/usr/lib64/python2.6/io.py", line 921, in __init__
raw._checkReadable()
AttributeError: '_GzipStreamFile' object has no attribute '_checkReadable'
have been fighting with this without progress for 2 days now.. if somebody could givme somekind of insight or advice i would be more than happy!
Found the solution, it was problem in the data parsed to the tool, not the tool itself.
Related
I am currently coding pyspark pipelines using databricks connect library. The steps I followed are given here. This library has been installed in a virtual environment.
When I try to execute this code
spark.read.load(path).first()
I get this error
<class 'TypeError'>, 'JavaPackage' object is not callable, <traceback object at 0x0000017AB70ECF88>
Traceback (most recent call last):
File "D:/Friendsurance/Repository/data-ingestion/job/main.py", line 83, in <module>
run()
File "D:/Friendsurance/Repository/data-ingestion/job/main.py", line 79, in run
el_job.run()
File "D:\Friendsurance\Repository\data-ingestion\job\task\__init__.py", line 18, in run
data: DataFrame = self.extract()
File "D:\Friendsurance\Repository\data-ingestion\job\task\ELTask.py", line 14, in extract
return self.extractor.extract()
File "D:\Friendsurance\Repository\data-ingestion\job\task\extractor\BucketExtractor.py", line 26, in extract
self.spark, self.load_storage.get_path(), self.conf.partition_column
File "D:\Friendsurance\Repository\data-ingestion\job\task\extractor\__init__.py", line 14, in calculate_last_day_run
spark.read.load(path).first().show()
File "D:\Friendsurance\Repository\data-ingestion\venv\lib\site-packages\pyspark\sql\dataframe.py", line 1381, in first
return self.head()
File "D:\Friendsurance\Repository\data-ingestion\venv\lib\site-packages\pyspark\sql\dataframe.py", line 1369, in head
rs = self.head(1)
File "D:\Friendsurance\Repository\data-ingestion\venv\lib\site-packages\pyspark\sql\dataframe.py", line 1371, in head
return self.take(n)
File "D:\Friendsurance\Repository\data-ingestion\venv\lib\site-packages\pyspark\sql\dataframe.py", line 657, in take
return self.limit(num).collect()
File "D:\Friendsurance\Repository\data-ingestion\venv\lib\site-packages\pyspark\sql\dataframe.py", line 596, in collect
if self._sc._conf.get(self._sc._jvm.PythonSecurityUtils.USE_FILE_BASED_COLLECT()):
TypeError: 'JavaPackage' object is not callable
But when I am out of the virtual environment where I am using the pyspark library provided here, I am able to execute the same line and it gives me the output.
Can anyone please tell me where I am going wrong with this?
I am trying to do some basic api integration with Tableau server.
I was trying the Tableau_tools library (https://github.com/bryanthowell-tableau/tableau_tools).
I am unable to get simple create AD group or add user working. Is there something I am doing wrong?
I am able to view users/groups.
This is my code:
t = TableauServerRest37(cfg["server"], pat_name=cfg['personal_access_token_name'], pat_secret=cfg['personal_access_token_secret'], site_content_url=cfg["site_url"])
t.signin()
t.enable_logging(logger)
list_of_groups = t.groups.query_groups_json()
list_of_users = t.users.query_users_json()
#Add Users
users_to_add = ['user_1', 'user_2', 'user_3']
for user in users_to_add:
t.users.add_user(username=user, fullname=user, site_role='Publisher')
t.groups.create_group_from_ad_group( group, domain, default_site_role="ExplorerCanPublish", sync_as_background=True)
I am getting the following Error for Users:
Traceback (most recent call last):
File ".\Tableau_tools_api.py", line 117, in <module>
main()
File ".\Tableau_tools_api.py", line 82, in main
t.users.add_user(username=user, fullname=user, site_role='Publisher')
File "C:\Users\ABCD\AppData\Roaming\Python\Python37\site-packages\tableau_tools\tableau_rest_api\methods\user.py", line 164, in add_user
update_if_exists=update_if_exists, auth_setting=auth_setting)
File "C:\Users\ABCD\AppData\Roaming\Python\Python37\site-packages\tableau_tools\tableau_rest_api\methods\user.py", line 89, in add_user_by_username
if site_role not in self.rest.site_roles:
AttributeError: 'TableauServerRest37' object has no attribute 'site_roles'
I am getting the following Error for Groups:
Traceback (most recent call last):
File ".\Tableau_tools_api.py", line 117, in <module>
main()
File ".\Tableau_tools_api.py", line 88, in main
t.groups.create_group_from_ad_group( group, domain, default_site_role="ExplorerCanPublish", sync_as_background=True)
File "C:\Users\ABCD\AppData\Roaming\Python\Python37\site-packages\tableau_tools\tableau_rest_api\methods\group.py", line 74, in create_group_from_ad_group
if default_site_role not in self.rest._site_roles:
AttributeError: 'TableauServerRest37' object has no attribute '_site_roles'
I was wondering if anyone has gotten this to work for api version 3.7?
I am trying to generate an error log html by “rebot” package of robot framework and its getting generated successfully.
But if I use the rebot function in my module then its affect default log and report html which gets generated after script execution.
[ ERROR ] Unexpected error: AttributeError: 'NoneType' object has no attribute 'encode'
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/robot/utils/application.py", line 83, in _execute
rc = self.main(arguments, **options)
File "/usr/local/lib/python3.5/dist-packages/robot/run.py", line 445, in main
result = suite.run(settings)
File "/usr/local/lib/python3.5/dist-packages/robot/running/model.py", line 248, in run
self.visit(runner)
File "/usr/local/lib/python3.5/dist-packages/robot/model/testsuite.py", line 161, in visit
visitor.visit_suite(self)
File "/usr/local/lib/python3.5/dist-packages/robot/model/visitor.py", line 87, in visit_suite
suite.tests.visit(self)
File "/usr/local/lib/python3.5/dist-packages/robot/model/itemlist.py", line 76, in visit
item.visit(visitor)
File "/usr/local/lib/python3.5/dist-packages/robot/model/testcase.py", line 74, in visit
visitor.visit_test(self)
File "/usr/local/lib/python3.5/dist-packages/robot/running/runner.py", line 159, in visit_test
self._output.end_test(ModelCombiner(test, result))
File "/usr/local/lib/python3.5/dist-packages/robot/output/output.py", line 59, in end_test
LOGGER.end_test(test)
File "/usr/local/lib/python3.5/dist-packages/robot/output/logger.py", line 183, in end_test
logger.end_test(test)
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 51, in end_test
self._writer.status(test.status, clear=True)
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 114, in status
self._clear_status()
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 124, in _clear_status
self._write_info()
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/verbose.py", line 90, in _write_info
self._stdout.write(self._last_info)
File "/usr/local/lib/python3.5/dist-packages/robot/output/console/highlighting.py", line 51, in write
self._write(console_encode(text, stream=self.stream))
File "/usr/local/lib/python3.5/dist-packages/robot/utils/encoding.py", line 60, in console_encode
return string.encode(encoding, errors).decode(encoding)
The error message is rather clear. It looks like you're trying to encode a variable which contains a None instead of a string. You need to make sure that this variable always contains a string and handle cases where something else is inside. You can do it for example using the try ... except statement.
I am trying to use this python code here, without success:
https://github.com/debajyotiguha11/PyCrypto
I run the script, but I get these errors:
Do you want to (E)ncrypt or (D)ecrypt? *Case Sensitive*
E
Enter the password:
abcd
Traceback (most recent call last):
File "pycrypto3.py", line 84, in <module>
encrypt(SHA256.new(password).digest(), str(Tfiles))
File "/usr/local/lib/python3.7/dist-packages/Crypto/Hash/SHA256.py", line 88, in new
return SHA256Hash().new(data)
File "/usr/local/lib/python3.7/dist-packages/Crypto/Hash/SHA256.py", line 75, in new
return SHA256Hash(data)
File "/usr/local/lib/python3.7/dist-packages/Crypto/Hash/SHA256.py", line 72, in __init__
HashAlgo.__init__(self, hashFactory, data)
File "/usr/local/lib/python3.7/dist-packages/Crypto/Hash/hashalgo.py", line 51, in __init__
self.update(data)
File "/usr/local/lib/python3.7/dist-packages/Crypto/Hash/hashalgo.py", line 69, in update
return self._hash.update(data)
TypeError: Unicode-objects must be encoded before hashing
How do I fix this?
Traceback (most recent call last):
File "E:\blahblahblah\emailsend.py", line 26, in <module>
msg.attach(MIMEText(file))
File "E:\blahblahblah\Python 2.7.11\lib\email\mime\text.py", line 30, in __init__
self.set_payload(_text, _charset)
File "E:\blahblahblah\Python 2.7.11\lib\email\message.py", line 226, in set_payload
self.set_charset(charset)
File "E:\blahblahblah\Python 2.7.11\lib\email\message.py", line 268, in set_charset
cte(self)
File "E:\blahblahblah\Python 2.7.11\lib\email\encoders.py", line 73, in encode_7or8bit
orig.encode('ascii')
AttributeError: 'file' object has no attribute 'encode'https://stackoverflow.com/questions/ask#
I've been looking this up a lot but I haven't found an answer.
The only important parts of the code is this:
file = open('newfile.txt')
msg.attach(MIMEText(file))
There are other parts but I've debugged it and I get the error at the 'msg.attach(MIMEText(file))' line.
Any help?
MIMEText takes the content of the file, not the file object.
msg.attach(MIMEText(open("newfile.txt").read()))