Okay so I am making a program to test if certain pages in a website are offline or online.
import urllib2
u = 'http://www.google.com/'
pages = open('pages.txt', 'r').readlines()
for page in pages:
url = u + page
try:
req = urllib2.urlopen(url)
except urllib2.HTTPError as e:
if e.code == 404:
print url + " does not exists"
else:
print url + " exists"
and the "Pages.txt" contains something like this:
search
page
plus
signin
account
security
lol
about
contactus
someotherpage.html
Now the program is working fine but I want it to store the available pages in a txt file. Can someone help me with that? If not just listing the pages that exist and forgetting the offline pages would be also great. Thanks :)
What about :
python your_script > Pages.txt
Edit
For writing in a file
with open('Pages.txt', 'w') as f:
f.write('something')
f.close()
Just write to the file in the same way you're reading:
out = open('pages.txt', 'w')
... then in the else: tag you've already written:
out.write(url+"\n")
Making:
import urllib2
u = 'http://www.google.com/'
pages = open('pages.txt', 'r').readlines()
out = open('pages.txt', 'w')
for page in pages:
url = u+page
try:
req = urllib2.urlopen(url)
except urllib2.HTTPError as e:
if e.code == 404:
print url+" does not exists"
else:
print url+" exists"
out.write(url+"\n")
Open up a file in append mode to write to.
Redirect the print statement to print to the new file handler.
import urllib2
u = raw_input('Enter a url: ') or 'http://www.google.com/'
pages = open('pages.txt', 'r').readlines()
with open('available.txt', 'a') as available:
for page in pages:
url = u.rstrip('\n')+page
try:
req = urllib2.urlopen(url)
except urllib2.HTTPError as e:
if e.code == 404:
print url+" does not exists"
else:
print url+" exists"
print >> available, url.rstrip('\n')
Output:
(availablepages)macbook:availablepages joeyoung$ ls -al
total 16
drwxr-xr-x 4 joeyoung staff 136 Sep 7 00:23 .
drwxr-xr-x 4 joeyoung staff 136 Sep 6 23:54 ..
-rw-r--r-- 1 joeyoung staff 478 Sep 7 00:20 availablepages.py
-rw-r--r-- 1 joeyoung staff 70 Sep 6 23:56 pages.txt
(availablepages)macbook:availablepages joeyoung$ python availablepages.py
Enter a url: http://www.google.com/
http://www.google.com/search
exists
http://www.google.com/page
does not exists
http://www.google.com/plus
exists
http://www.google.com/signin
does not exists
http://www.google.com/account
exists
http://www.google.com/security
exists
http://www.google.com/lol
does not exists
http://www.google.com/about
exists
http://www.google.com/someotherpage.html
does not exists
(availablepages)macbook:availablepages joeyoung$ ls -al
total 24
drwxr-xr-x 5 joeyoung staff 170 Sep 7 00:23 .
drwxr-xr-x 4 joeyoung staff 136 Sep 6 23:54 ..
-rw-r--r-- 1 joeyoung staff 145 Sep 7 00:23 available.txt
-rw-r--r-- 1 joeyoung staff 478 Sep 7 00:20 availablepages.py
-rw-r--r-- 1 joeyoung staff 70 Sep 6 23:56 pages.txt
(availablepages)macbook:availablepages joeyoung$ cat available.txt
http://www.google.com/search
http://www.google.com/plus
http://www.google.com/account
http://www.google.com/security
http://www.google.com/about
(availablepages)macbook:availablepages joeyoung$ python availablepages.py
Enter a url: http://www.bing.com/
http://www.bing.com/search
exists
http://www.bing.com/page
does not exists
http://www.bing.com/plus
does not exists
http://www.bing.com/signin
does not exists
http://www.bing.com/account
exists
http://www.bing.com/security
does not exists
http://www.bing.com/lol
does not exists
http://www.bing.com/about
does not exists
http://www.bing.com/someotherpage.html
does not exists
(availablepages)macbook:availablepages joeyoung$ ls -al
total 24
drwxr-xr-x 5 joeyoung staff 170 Sep 7 00:23 .
drwxr-xr-x 4 joeyoung staff 136 Sep 6 23:54 ..
-rw-r--r-- 1 joeyoung staff 200 Sep 7 00:24 available.txt
-rw-r--r-- 1 joeyoung staff 478 Sep 7 00:20 availablepages.py
-rw-r--r-- 1 joeyoung staff 70 Sep 6 23:56 pages.txt
(availablepages)macbook:availablepages joeyoung$ cat available.txt
http://www.google.com/search
http://www.google.com/plus
http://www.google.com/account
http://www.google.com/security
http://www.google.com/about
http://www.bing.com/search
http://www.bing.com/account
Related
I wrote a python code like:
#!/usr/bin/python
import os
import sys
f=open("/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement/guru99.txt","w+")
i tried to create file on mounted drive (on AIX)
node mounted
-------- --------------- --------------- ------ ------------ ---------------
clsfs038 /Bank_statements /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements
...and i got error:
OSError: [Errno 22] Invalid argument: '/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement/guru99.txt'
same code for local path is working ok!
I create file on mounted drive by bash without any problem.
It's df:
[inform#xxx/inform ] $ df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/hd10opt 1966080 380432 81% 15329 24% /opt
/dev/livedump 65536 64864 2% 4 1% /var/adm/ras/livedump
/dev/informlv 1257766912 662083576 48% 425081 1% /inform
clsfs038:/Bank_statements 2576863224 393165352 85% 0 0% /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements
permission on folder:
[inform#xxx /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements ] $ ls -la
total 184
drwxrwxr-x 1 inform inform 16384 Dec 19 16:44 .
drwxr-xr-x 19 inform inform 32768 Dec 18 17:16 ..
drwxrwxr-x 1 inform inform 16384 Dec 19 14:08 load
drwxrwxr-x 1 inform inform 16384 Dec 19 10:37 statement
-rwxrwxr-x 1 inform inform 0 Nov 22 12:26 testfile
-rwxrwxr-x 1 inform inform 0 Dec 18 13:06 testfile2
Mount:
[inform#xxx /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements ] $ mount
node mounted mounted over vfs date options
-------- --------------- --------------- ------ ------------ ---------------
/dev/hd4 / jfs2 Aug 26 17:39 rw,log=/dev/hd8
/dev/hd2 /usr jfs2 Aug 26 17:39 rw,log=/dev/hd8
/dev/hd9var /var jfs2 Aug 26 17:39 rw,log=/dev/hd8
/dev/hd3 /tmp jfs2 Aug 26 17:40 rw,log=/dev/hd8
/dev/hd1 /home jfs2 Aug 26 17:40 rw,log=/dev/hd8
/dev/hd11admin /admin jfs2 Aug 26 17:40 rw,log=/dev/hd8
/proc /proc procfs Aug 26 17:40 rw
/dev/hd10opt /opt jfs2 Aug 26 17:40 rw,log=/dev/hd8
/dev/livedump /var/adm/ras/livedump jfs2 Aug 26 17:40 rw,log=/dev/hd8
/dev/informlv /inform jfs2 Aug 26 17:40 rw,log=INLINE
/dev/zabbixapplv /opt/zabbixapp jfs2 Aug 26 17:40 rw,log=INLINE
clsfs042 /sap_employee /inform/app/Informatica/10.1.0/server/infa_shared/SrcFiles/SAP cifs Aug 26 17:40 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF,noprompt
clsfs042 /Applications /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/Popcorn/Informatica cifs Aug 26 17:40 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF,noprompt
clsfs041 /AFS /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/afs cifs Aug 26 17:40 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF,noprompt
clsfs038 /DetOtchet /inform/app/Informatica/10.1.0/server/infa_shared/SrcFiles/CB_PORTF cifs Aug 26 17:40 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF,noprompt
clsfs038 /BALANS /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/BALANS cifs Aug 26 17:40 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF,noprompt
clsfs040 /Workgroups /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/Metodologiya cifs Aug 26 17:40 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF,noprompt
clsfs038 /Workgroups3 /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/ALM_REPORT cifs Aug 26 17:40 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF,noprompt
clsfs042 /Applications /inform/app/Informatica/10.1.0/server/infa_shared/SrcFiles/rccf_applications cifs Nov 12 12:20 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF
clsfs038 /Bank_statements /inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements cifs Nov 22 12:27 rw,uid=482,gid=482,fmode=775,wrkgrp=RCCF
Result of command (stackoverflow asked me some more details, without this text i cant edit my post with this code):
[inform#xxx /inform/app/Informatica/10.1.0/server/infa_shared/Scripts/STATEMENT ] $ /inform/app/Informatica/10.1.0/server/infa_shared/Scripts/STATEMENT/test/stack.py
False
READ False
WRITE False
/inform True
/inform READ True
/inform WRITE True
/inform/app True
/inform/app READ True
/inform/app WRITE True
/inform/app/Informatica True
/inform/app/Informatica READ True
/inform/app/Informatica WRITE True
/inform/app/Informatica/10.1.0 True
/inform/app/Informatica/10.1.0 READ True
/inform/app/Informatica/10.1.0 WRITE True
/inform/app/Informatica/10.1.0/server True
/inform/app/Informatica/10.1.0/server READ True
/inform/app/Informatica/10.1.0/server WRITE True
/inform/app/Informatica/10.1.0/server/infa_shared True
/inform/app/Informatica/10.1.0/server/infa_shared READ True
/inform/app/Informatica/10.1.0/server/infa_shared WRITE True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles READ True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles WRITE True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements READ True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements WRITE True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement READ True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement WRITE True
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement/guru99.txt False
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement/guru99.txt READ False
/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement/guru99.txt WRITE False
Help me please)
Regards,
Alex
Can you paste result of following script ? Just in case it helps find out anything
#!/usr/bin/python
import os
file_name = "/inform/app/Informatica/10.1.0/server/infa_shared/TgtFiles/bank_statements/statement/guru99.txt"
dirs = file_name.split("/")
for i in range(len(dirs)):
dir_to_check = "/".join(dirs[:i+1])
print(dir_to_check + " " + str(os.path.exists(dir_to_check)))
print(dir_to_check + " READ " + str(os.access(dir_to_check, os.R_OK)))
print(dir_to_check + " WRITE " + str(os.access(dir_to_check, os.W_OK)))
import os
import shutil
from collections import Counter as cnt
curr_path = os.getcwd()
ls = lambda path: [x for x in os.listdir(path) if x[0] != '.']
raw_files = [x for x in ls(curr_path) if '_' not in x]
filenames = []
folders = []
for f in raw_files: # sweeping out the '.', only having filenames
try:
i = f.index('.')
f_new = f[0:i]
filenames.append(f_new)
except ValueError:
filenames.append(f)
fname_freq = cnt(filenames)
for fname, freq in fname_freq.items():
if freq > 1:
folders.append(fname)
for fldr in folders:
print(fldr+'\n')
try:
os.makedirs(fldr+'_')
# adding a '_' to make the true folder name, i.e. foldername_ instead of foldername
except OSError:
pass
for f in raw_files:
print("File being analyzed is: {fn} \n".format(fn=f))
for fldr in folders:
print("Folder to move stuff to is {fold}\n".format(fold=fldr))
print("File {f} being checked . . . ".format(f=f))
if fldr in f and f[-1] != '_':
print("\t Moving file {fn} \n".format(fn=f))
shutil.move(f, fldr+'_')
The above is a program FileGrouper.py, which will look inside a directory for whether there are multiple files with a shared name (i.e. random.java, random.class). If there are, then they will be moved to a directory named [insert shared name]_ to organize them.
This code in particular refuses to work for a specific set of files.
Note: I replaced my actual Username, as well as the Timestamps with placeholders
-rwxr-xr-x 1 UserName staff 8480 Month 20 Time:Time another_ptr_func
-rw-r--r--# 1 UserName staff 324 Month 20 Time:Time another_ptr_func.c
-rwxr-xr-x 1 UserName staff 8572 Month 20 Time:Time arrays1
-rw-r--r-- 1 UserName staff 321 Month 20 Time:Time arrays1.c
-rw-r--r--# 1 UserName staff 2058 Month 20 Time:Time file_grouper.py
-rwxr-xr-x 1 UserName staff 8432 Month 20 Time:Time forloop
-rw-r--r-- 1 UserName staff 119 Month 20 Time:Time forloop.c
-rwxr-xr-x 1 UserName staff 4248 Month 20 Time:Time gen_a
-rw-r--r-- 1 UserName staff 53 Month 20 Time:Time gen_a.c
-rwxr-xr-x 1 UserName staff 8432 Month 20 Time:Time hello
-rw-r--r-- 1 UserName staff 65 Month 20 Time:Time hello.c
-rwxr-xr-x 1 UserName staff 8432 Month 20 Time:Time hello2
-rw-r--r-- 1 UserName staff 140 Month 20 Time:Time hello2.c
-rw-r--r--# 1 UserName staff 343 Month 20 Time:Time pointer.c
-rwxr-xr-x 1 UserName staff 8472 Month 20 Time:Time pointer_func
-rw-r--r-- 1 UserName staff 352 Month 20 Time:Time pointer_func.c
-rwxr-xr-x# 1 UserName staff 19340 Month 20 Time:Time pointer_hex
-rwxr-xr-x 1 UserName staff 8432 Month 20 Time:Time switch
-rw-r--r--# 1 UserName staff 219 Month 20 Time:Time switch.c
-rwxr-xr-x 1 UserName staff 8480 Month 20 Time:Time void_another_ptr_func
-rw-r--r-- 1 UserName staff 335 Month 20 Time:Time void_another_ptr_func.c
The above is the specific set of files that this code refuses to work for. I have tested this same code on the following, consisting of dummy files with zero size (used touch to create them):
-rw-r--r-- 1 UserName staff 1183 Month 20 Time:Time README.md
-rw-r--r--# 1 UserName staff 2067 Month 20 Time:Time file_grouper.py
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random1
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random1.txt
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random2
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random2.txt
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random3
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random3.txt
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random4
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random4.txt
-rw-r--r-- 1 UserName staff 0 Month 20 Time:Time random5.txt
The program worked successfully on these files, and generated the following:
.
├── README.md
├── file_grouper.py
├── random1_
│ ├── random1
│ └── random1.txt
├── random2_
│ ├── random2
│ └── random2.txt
├── random3_
│ ├── random3
│ └── random3.txt
├── random4_
│ ├── random4
│ └── random4.txt
└── random5.txt
As you can see, the program successfully ignored the lone random5.txt, as no other files with a shared name existed. However, the files where others with a shared name did exist were successfully grouped into a folder of [insert shared name]_
If by "refusing to work" you mean certain files are skipped, then that could be because of this line:
raw_files = [x for x in ls(curr_path) if '_' not in x]
All files with an underscore are removed.
Another issue is that file systems will not allow files and directories with the same name, which you cover by appending an underscore.
I'd strongly suggest to use the pathlib library, it makes your code more maintainable and readable:
from pathlib import Path
def file_grouper():
# Path to group
path = Path('.')
# List all files
ls = path.glob('*')
# Map all stems (file names without extension) to their file names
names = {}
for x in ls:
if x.is_file():
if x.stem not in names:
names[x.stem] = []
names[x.stem].append(x)
# Create and move
for stem, values in names.items():
if len(values) > 1:
(path / (stem + '_')).mkdir(exist_ok=True)
for value in values:
value.rename(path / (stem + '_') / value.name)
file_grouper()
I'm trying to read a specific file from a compressed file bz2 using python.
tar = tarfile.open(filename, "r|bz2", bufsize=57860311)
for tarinfo in tar:
print tarinfo.name, "is", tarinfo.size, "bytes in size and is",
if tarinfo.isreg():
print "a regular file."
# read the file
f = tar.extractfile(tarinfo)
#print f.read()
elif tarinfo.isdir():
print "a directory."
else:
print "something else."
tar.close()
But at the end I got the error:
/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.pyc in read(self, size)
577 buf = "".join(t)
578 else:
--> 579 buf = self._read(size)
580 self.pos += len(buf)
581 return buf
/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.pyc in _read(self, size)
594 break
595 try:
--> 596 buf = self.cmp.decompress(buf)
597 except IOError:
598 raise ReadError("invalid compressed data")
EOFError: end of stream was already found
I also tried to list the files within the tar through 'tar.list()' and again ...
-rwxr-xr-x lindauer/or3uunp 0 2013-05-21 00:58:36 r3.2/
-rw-r--r-- lindauer/or3uunp 6057 2012-01-05 14:41:00 r3.2/readme.txt
-rw-r--r-- lindauer/or3uunp 44732 2012-01-04 10:08:54 r3.2/psychometric.csv
-rw-r--r-- lindauer/or3uunp 57860309 2012-01-04 09:58:20 r3.2/logon.csv
/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.pyc in _read(self, size)
594 break
595 try:
--> 596 buf = self.cmp.decompress(buf)
597 except IOError:
598 raise ReadError("invalid compressed data")
EOFError: end of stream was already found
I listed the files inside the archive using the tar command. Here is the result:
tar -tvf r3.2.tar.bz2
drwxr-xr-x 0 lindauer or3uunp 0 May 21 2013 r3.2/
-rw-r--r-- 0 lindauer or3uunp 6057 Jan 5 2012 r3.2/readme.txt
-rw-r--r-- 0 lindauer or3uunp 44732 Jan 4 2012 r3.2/psychometric.csv
-rw-r--r-- 0 lindauer or3uunp 57860309 Jan 4 2012 r3.2/logon.csv
-rw-r--r-- 0 lindauer or3uunp 12494829865 Jan 5 2012 r3.2/http.csv
-rw-r--r-- 0 lindauer or3uunp 1066622500 Jan 5 2012 r3.2/email.csv
-rw-r--r-- 0 lindauer or3uunp 218962503 Jan 5 2012 r3.2/file.csv
-rw-r--r-- 0 lindauer or3uunp 29156988 Jan 4 2012 r3.2/device.csv
drwxr-xr-x 0 lindauer or3uunp 0 May 20 2013 r3.2/LDAP/
-rw-r--r-- 0 lindauer or3uunp 140956 Jan 4 2012 r3.2/LDAP/2011-01.csv
-rw-r--r-- 0 lindauer or3uunp 147370 Jan 4 2012 r3.2/LDAP/2010-05.csv
-rw-r--r-- 0 lindauer or3uunp 149221 Jan 4 2012 r3.2/LDAP/2010-02.csv
-rw-r--r-- 0 lindauer or3uunp 141717 Jan 4 2012 r3.2/LDAP/2010-12.csv
-rw-r--r-- 0 lindauer or3uunp 148931 Jan 4 2012 r3.2/LDAP/2010-03.csv
-rw-r--r-- 0 lindauer or3uunp 147370 Jan 4 2012 r3.2/LDAP/2010-04.csv
-rw-r--r-- 0 lindauer or3uunp 149793 Jan 4 2012 r3.2/LDAP/2009-12.csv
-rw-r--r-- 0 lindauer or3uunp 143979 Jan 4 2012 r3.2/LDAP/2010-09.csv
-rw-r--r-- 0 lindauer or3uunp 145591 Jan 4 2012 r3.2/LDAP/2010-07.csv
-rw-r--r-- 0 lindauer or3uunp 139444 Jan 4 2012 r3.2/LDAP/2011-03.csv
-rw-r--r-- 0 lindauer or3uunp 142347 Jan 4 2012 r3.2/LDAP/2010-11.csv
-rw-r--r-- 0 lindauer or3uunp 138285 Jan 4 2012 r3.2/LDAP/2011-04.csv
-rw-r--r-- 0 lindauer or3uunp 149793 Jan 4 2012 r3.2/LDAP/2010-01.csv
-rw-r--r-- 0 lindauer or3uunp 146008 Jan 4 2012 r3.2/LDAP/2010-06.csv
-rw-r--r-- 0 lindauer or3uunp 144711 Jan 4 2012 r3.2/LDAP/2010-08.csv
-rw-r--r-- 0 lindauer or3uunp 137967 Jan 4 2012 r3.2/LDAP/2011-05.csv
-rw-r--r-- 0 lindauer or3uunp 140085 Jan 4 2012 r3.2/LDAP/2011-02.csv
-rw-r--r-- 0 lindauer or3uunp 143420 Jan 4 2012 r3.2/LDAP/2010-10.csv
-r--r--r-- 0 lindauer or3uunp 3923 Jan 4 2012 r3.2/license.txt
I think this is due to the fact the archive has subfolders and for some reason python libraries have problems in dealing with subfolders extractions?
I also tried to open the tar file manually and I have no problems so I don't think the file is corrupted. Any help appreciated.
Comment: I tried the debug=3 and I get : ReadError: bad checksum
Found the following related Infos:
tar: directory checksum error
Cause
This error message from tar(1) indicates that the checksum of the directory and the files it has read from tape does not match the checksum advertised in the header block. Usually this message indicates the wrong blocking factor, although it could indicate corrupt data on tape.
Action
To resolve this problem, make certain that the blocking factor you specify on the command line (after -b) matches the blocking factor originally specified. If in doubt, leave out the block size and let tar(1) determine it automatically. If that remedy does not help, the tape data could be corrupted.
SE:tar-ignore-or-fix-checksum
I'd try the -i switch to see if you can just ignore and messages regarding EOF.
-i, --ignore-zeros ignore zeroed blocks in archive (means EOF)
Example
$ tar xivf backup.tar
bugs.python.org:tarfile-headererror
The comment in tarfile.py reads (Don't know the date of the file!):
- # We shouldn't rely on this checksum, because some tar programs
- # calculate it differently and it is merely validating the
- # header block.
ReadError: unexpected end of data
From the tarfile Documentation
The tarfile module defines the following exceptions:
exception tarfile.ReadError
Is raised when a tar archive is opened, that either cannot be handled by the tarfile module or is somehow invalid.
First, try with another tar archiv file to verify your python environent.
Second, check if your tar archiv file match the following format:
tarfile.DEFAULT_FORMAT
The default format for creating archives. This is currently GNU_FORMAT.
Third, instead of using tarfile.open(...), to create a tarfile instance, try to use the following, to set debug=3.
tar = tarfile.TarFile(name=filename, debug=3)
tar.open()
...
class tarfile.TarFile(name=None, mode='r', fileobj=None, format=DEFAULT_FORMAT, tarinfo=TarInfo, dereference=False, ignore_zeros=False, encoding=ENCODING, errors='surrogateescape', pax_headers=None, debug=0, errorlevel=0)
I'm using paramiko.SSHClient.exec_command() to run commands on remote servers.
Does someone know if it's possible to change the width, like with invoke_shell(width=150) ?
When I exec_command("ls -la") I get :
drwx------. 6 myuser myuser 4096 25 avril 15:59
.
drwxr-xr-x. 14 root root 4096 5 mai 15:05
..
-rw-------. 1 myuser myuser 2818 28 avril 11:09
.bash_history
-rw-r--r--. 1 myuser myuser 340 14 avril 14:16
.bashrc
and I want :
drwx------. 6 myuser myuser 4096 25 avril 15:59 .
drwxr-xr-x. 14 root root 4096 5 mai 15:05 ..
-rw-------. 1 myuser myuser 2818 28 avril 11:09 .bash_history
-rw-r--r--. 1 myuser myuser 340 14 avril 14:16 .bashrc
(using exec_command, not invoke_shell)
My code :
ssh_client = paramiko.SSHClient()
mykey = paramiko.RSAKey.from_private_key_file("/path/to/my/key", password="passphrase")
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect("myserver.mydomain.com", username="myuser", pkey=mykey)
transport = ssh_client.get_transport()
agent_channel = transport.open_session()
agent_handler = paramiko.agent.AgentRequestHandler(agent_channel)
stdin, stdout, stderr = ssh_client.exec_command("ls -la")
Ok, found it: it was just a pprint strange behavior. I put the received lines in a list, and then pprint this list. If I do :
for line in received:
print(line)
Then It's ok.
pprint printed :
['-rw-r----- 1 myuser mygroup 23228063744 06 mai 11:41 '
'my_file.txt-rw-r----- 1 '
...
I don't know why. Well, I'll stop using pprint.
I've got a directory full of JPG photographs. I want to take the file names of those photographs and end up with the following being printed:
<description>Test. <![CDATA[<img src='.
/files/fantaWP.jpg]>]]></description>
The file name is a variable. I've tried my very best below and i'm nearly there, but I end up with the following output:
<description>Test. <![CDATA[<img src='.
/files/['fantaWP.jpg', 'icon', 'p1.JPG', 'p2.JPG', 'p3.jpg', 'p4.jpg']>]]></description>
Here is my code:
photofileName = []
path='C:\Users\Simon\Desktop\Dir\pics'
dirList=os.listdir(path)
for fname in dirList:
photofileName.append(fname)
print photofileName
photoVar = [x for x in photofileName]
itemsInListOne = 3
iterations = itemsInListOne
num = 0
while num < iterations:
num = num+1
print ("\<description>Test. <![CDATA[<img src='./files/{}'>]]></description>\n".format(photoVar))
Thank you in advance.
The following should be enough if I understand you correctly.
for fname in os.listdir(path):
print("\<description>Test. <![CDATA[<img src='./files/{}'>]]>=</description>\n".format(fname))
Example:
>>> path = "/home/msvalkon/Pictures/Sample Album"
>>> for fname in os.listdir(path):
... print("\<description>Test. <![CDATA[<img src='./files/{}'>]]>=</description>\n".format(fname))
...
...
\<description>Test. <![CDATA[<img src='./files/Costa Rican Frog.jpg'>]]>=</description>
\<description>Test. <![CDATA[<img src='./files/Pensive Parakeet.jpg'>]]>=</description>
\<description>Test. <![CDATA[<img src='./files/Boston City Flow.jpg'>]]>=</description>
>>>
And the content of the path..
msvalkon#Lunkwill:~/Pictures/Sample Album$ ll
total 1208
drwxrwxr-x 2 msvalkon msvalkon 4096 Apr 19 2012 ./
drwxr-xr-x 7 msvalkon msvalkon 28672 Jan 3 18:27 ../
-rw-rw-r-- 1 msvalkon msvalkon 339773 Dec 13 2009 Boston City Flow.jpg
-rw-rw-r-- 1 msvalkon msvalkon 354633 Dec 13 2009 Costa Rican Frog.jpg
-rw-rw-r-- 1 msvalkon msvalkon 480098 Dec 13 2009 Pensive Parakeet.jpg