Escaping Special Characters (*) in Python subprocess.call() - python

I can't seem to be able to escape these special characters - *.
I'm trying to copy some files using a Python script with the following line
subprocess.call(r"mkdir E:\CONTENT\Full1",shell=True)
subprocess.call(r"copy","E:\DATA FOR CONTENT\*.*","E:\CONTENT\Full1",shell=True)
subprocess.call(r"fsutil file createnew ","E:\CONTENT\versionfile.txt","6500000",shell=True)
But I get the following error at the 2nd line
Traceback (most recent call last):
File "basic1.py", line 9, in <module>
subprocess.call(r"copy","E:\DATA FOR CONTENT\*.*","E:\CONTENT\Full1",shell=True)
File "C:\Python34\lib\subprocess.py", line 537, in call
with Popen(*popenargs, **kwargs) as p:
File "C:\Python34\lib\subprocess.py", line 767, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integer
Also I hate passing it using the CSV Style ("arg1","arg2"...). Is there any way of passing it as a raw string?

args should be a sequence of program arguments or else a single string.
subprocess.call(["copy", r"E:\DATA FOR CONTENT\*.*", r"E:\CONTENT\Full1"], shell=True)

Better don't do this via subprocess. Believe me it's a bad idea.
For simple filesystem copy operations take a look on python shutil module.
For creating new directories use os.makedirs.
For creating files of certain size see this answers.

Related

why is the error thinking Its not string?

path of file is:
"C:\Users\deana\OneDrive\Marlon's files\Programming\Python\PITT\PITT_LIbrary\Lists\test.txt"
lines of code are:
import os
os.chdir("C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists")
exec(open('test.txt'))
the error is this:
Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
exec(open('test.txt'))
TypeError: exec() arg 1 must be a string, bytes or code object
also if I try on one line as such:
exec(open(r"C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists/test.txt"))
i'ts the same error. (with and without r)
super frustrationg as it reads like i'm not inputting string... but it is string!?!
also I've done this litteraly the same way before, restarted IDLE shell, no difference.
ugh! I always get stupid errors with file paths.
I should have been using os.startfile() to open this.
It was confusing by using .open(). as I was attempting to open in default app.
before, i've used exec.open() to open .py files and guess I got them confused.
exec is just used to open other scripts... need stronger coffee next time.
Try this:
import os
os.chdir("C:/Users/deana/OneDrive/Marlon's files/Programming/Python/PITT/PITT_LIbrary/Lists")
exec(open('test.txt', 'rb'))
You can convert the txt file to bytes by opening it with rb (read bytes).

Pytorch Cityscapes Dataset, train_distribute problem - "Typeerror: path should be string, bytes, pathlike or integer, not NoneType"

I'm very unfamiliar with Machine Learning, python, and such, so forgive my oblivious errors. I'm trying to use machine learning systems on a dataset of streetscapes I have. I found a lot or resources, and I'm working off of this package which has a lot of examples and seems straightforward.
When I attempted to run the train_distribute.py file, I received this error:
(base) corey#corona:~/Desktop/pycity/GALD-Net-master$ python train_distribute.py
Traceback (most recent call last):
File "train_distribute.py", line 261, in <module>
main()
File "train_distribute.py", line 136, in main
if not os.path.exists(args.save_dir):
File "/home/corey/anaconda3/lib/python3.7/genericpath.py", line 19, in exists
os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
Looking in the code, it's coming from these lines:
def main():
# make save dir
if args.local_rank == 0:
if not os.path.exists(args.save_dir):
os.makedirs(args.save_dir)
# launch the logger
Log.init(
log_level=args.log_level,
I'm guessing this means I need a more exact file structure, and to point the code at the right location. I am in no way a computer scientist and have close to zero understanding of what does what and how things like this work. Any advice for what I'm doing wrong and how I can approach fixing things?
From the error message, my guess would be that args.save_dir is None. os.path.exists cannot deal with None as a path:
>>> import os
>>> os.path.exists(None)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/genericpath.py", line 19, in exists
os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
Looking at the script you cited, the save_dir argument has a default value of None. It might be useful to make this a required argument and remove the default value, since the main function depends on it.
You have to specify the save_dir as an argument.
Run the code like this
python train_distribute.py --save_dir=SAVE_PATH
Here SAVE_PATH will take the path where you want to get the outputs saved.
Also, note that if the folder specified by the given path does not exist then that folder will be created.

PyPDF2: writing output to stdout fails with python3

I am trying to use Python 3.7.2 with PyPDF2 1.26 to select some pages of an input PDF file and write the output to stdout (the actual code is more complicated, this is just a MCVE):
import sys
from PyPDF2 import PdfFileReader, PdfFileWriter
input = PdfFileReader("example.pdf")
output = PdfFileWriter()
output.addPage(input.getPage(0))
output.write(sys.stdout)
This fails with the following error:
UserWarning: File <<stdout>> to write to is not in binary mode. It may not be written to correctly. [pdf.py:453]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.7/site-packages/PyPDF2/pdf.py", line 487, in write
stream.write(self._header + b_("\n"))
TypeError: write() argument must be str, not bytes
The problem seems to be that sys.stdout is not open in binary mode. As some of the answers suggest, I have tried the following:
output.write(sys.stdout.buffer)
This fails with the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.7/site-packages/PyPDF2/pdf.py", line 491, in write
object_positions.append(stream.tell())
OSError: [Errno 29] Illegal seek
I have also tried the answer from Changing the way stdin/stdout is opened in Python 3:
sout = open(sys.stdout.fileno(), "wb")
output.write(sout)
This fails with the same error as above.
How can I use the PyPDF2 library to output a PDF to standard output?
More generally, how do I correctly switch sys.stdout to binary mode (akin to Perl's binmode STDOUT)?
Note: There is no need to tell me that I can open a file in binary mode and write the PDF to that file. That works; however, I specifically want to write the PDF to stdout.
From the documentation:
write(stream)
Writes the collection of pages added to this object out as a PDF file.
Parameters: stream – An object to write the file to. The object must support the write method and the tell method, similar to a file object.
It turns out that sys.stdout.buffer is not tellable if not redirected to a file, hence you can't use it as a stream for PdfFileWriter.write.
Say your script is called myscript. If you call just myscript, then you'll get this error, but if you use it with a redirection, as in:
myscript > myfile.pdf
then Python understands it's a seekable stream, and you won't get the error.

python dictionary from config parser if value and "%(" in value: argument of type 'int' is not iterable

Getting this error in as random, sometimes appear, sometimes not:
for i in range(1,32):
sezione = "GRP"+str(i)
dizionarioGRP = dict(config.items(sezione))
print int(dizionarioGRP['r'])
and this is the error
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/pi/gigi.py", line 436, in run
self.esegui()
File "/home/pi/gigi.py", line 307, in esegui
L.gruppo(idgruppo)
File "/home/pi/gigi1.py", line 282, in gruppo
dizionarioGRP = dict(config.items(sezione))
File "/usr/lib/python2.7/ConfigParser.py", line 655, in items
for option in options]
File "/usr/lib/python2.7/ConfigParser.py", line 663, in _interpolate
if value and "%(" in value:
TypeError: argument of type 'int' is not iterable
why it is referring as int, if I've converted it in a string?
Somewhere in your code you're doing this:
config.set('something', 'something', 0) # or some integer
Or optionally you're passing in a non-string value in your ConfigParser constructor dict.
This is subtly against what the docs say:
If the given section exists, set the given option to the specified
value; otherwise raise NoSectionError. While it is possible to use
RawConfigParser (or ConfigParser with raw parameters set to true) for
internal storage of non-string values, full functionality (including
interpolation and output to files) can only be achieved using string
values.
So whenever you call ConfigParser.set, make sure you wrap what you're setting in str. Otherwise it throws an error when it runs through all the values in _interpolate. (I consider some part of this behavior a bug, but it's probably kept as-is for backwards compatibility. Or something.)
Another good thing to do is use SafeConfigParser which will throw a TypeError if you try to set a non-string value, so you know this won't happen again.

Python can not compile that regex. sre_constants.error: nothing to repeat [duplicate]

This question already has answers here:
regex error - nothing to repeat
(6 answers)
Closed 3 years ago.
I'm converting a C# function to Python. It's should bug for bug compatible with exist function.
This is a regex in that function: http://[a-zA-Z0-9]+(\.[a-zA-Z0-9]+)+([-A-Z0-9a-z_$.+!*()/\\\,:;#&=?~#%]*)*. But Python can't compile it:
>>> re.compile(r"http://[a-zA-Z0-9]+(\.[a-zA-Z0-9]+)+([-A-Z0-9a-z_$.+!*()/\\\,:;#&=?~#%]*)*")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.3/re.py", line 214, in compile
return _compile(pattern, flags)
File "/usr/lib/python3.3/re.py", line 281, in _compile
p = sre_compile.compile(pattern, flags)
File "/usr/lib/python3.3/sre_compile.py", line 498, in compile
code = _code(p, flags)
File "/usr/lib/python3.3/sre_compile.py", line 483, in _code
_compile(code, p.data, flags)
File "/usr/lib/python3.3/sre_compile.py", line 75, in _compile
elif _simple(av) and op is not REPEAT:
File "/usr/lib/python3.3/sre_compile.py", line 362, in _simple
raise error("nothing to repeat")
sre_constants.error: nothing to repeat
Note: There is a JavaScript version of that regex: /http:\/\/[a-zA-Z0-9]+(\.[a-zA-Z0-9]+)+([-A-Z0-9a-z\$\.\+\!\_\*\(\)\/\,\:;#&=\?~#%]*)*/gi.
I searched about nothing to repeat Error, but got nothing.
Sorry, this is a duplicate post.
Where is the problem?
I've reproduced the error with:
re.compile(r"([A]*)*")
The problem is that [A]* can potentially match an empty string. Guess what happens when it tries to match ([A]*)* when [A]* is empty? "nothing to repeat". The regex engine won't wait around for that to actually happen, though. It fails because it is even remotely possible for the scenario to happen.
This should work for you:
r"http://[a-zA-Z0-9]+(\.[a-zA-Z0-9]+)+([-A-Z0-9a-z_$.+!*()/\\\,:;#&=?~#%]*)"
I just removed the last *.
Had the same error come up with the following regex:
re.compile(r'(?P<term>[0-9]{1,2})-(?P<features>[A-Za-z\:]*)?')
It was the '?' at the end that caused the error. Strictly-speaking, this is NOT a repeat and, in fact, this works just fine (as it should) as of python 2.7.9. However, the bug was present as of python 2.7.3.

Categories