Apologies in advance since I'm new to Django (and I've also had to freshen up my Python skills). I'm trying to make a simple example of uploading a file through a form and then printing the rows in my terminal (as a test before doing some actual processing). My views.py contains the following:
def upload_csv(request):
if "GET" == request.method:
return render(request, "portal/csvupload.html")
csv_file = request.FILES["csv_file"]
handle_files(csv_file)
return HttpResponseRedirect(reverse("portal:csvupload"))
def handle_files(csvfile):
csv_reader = csv.reader(csvfile)
for line in csv_reader:
print(line)
Now this returns an error message saying "expected str, bytes or os.PathLike object, not InMemoryUploadedFile", and I'm unsure what's wrong with the code based on the error message? From a Python perspective it looks fine I think, but perhaps it's something to do with the re-direct? Apperciate all answers
request.FILES["csv_file"] is returning an InMemoryUploadedFile object and csv.reader does not know how to handle such an object. I believe you need to call the object's read method: handle_files(csv_file.read()). Note the warning in the documentation: "Be careful with this method: if the uploaded file is huge it can overwhelm your system if you try to read it into memory. You’ll probably want to use chunks() instead; see below."
Related
I am currently working on getting a basis on how to write a password generator for a final project in my programming class. The area of code that's currently giving me problems looks like:
`if not isfile('words.txt'):
print('Downloading words.txt ...')
url=str('https://raw.githubusercontent.com/dwyl/english-words/master/words.txt')
with open('words.txt', 'w') as f:
f.write(urlopen(url)).read()`
With the help of a friend, we decided to 'brute force' url to have str(). The error message I get implies that I need to put write() into str() but in doing that I get an error about,
'io.TextIOWrapper' object has no attribute 'str'
. This was originally written in Python 2, however, I managed to get everything working in Python 3.8.0 excluding the block of code above. Thank you in advance for the assistance.
urlopen returns a httpresponse object, not a string. You can only write strings or bytes to a file- not an arbitrary object. You use that object like so (read() returns bytes- if you want a string, have to decode it)
with urlopen(url) as resp:
print(resp.read().decode('utf-8'))
So, to get your code working do-
with open('words.txt', 'w') as f:
with urlopen(url) as resp:
f.write(resp.read().decode('utf-8'))
Using
f.write(urlopen(url)).read()
You are trying to read the file f
Try using
f.write(urlopen(url).read())
Here is my code and does anyone have any ideas what is wrong? I open my JSON content directly by browser and it works,
data = requests.get('http://ws.audioscrobbler.com/2.0/?method=library.getartists&api_key=4c22bd45cf5aa6e408e02b3fc1bff690&user=joanofarctan&format=json').text
data = json.load(data)
print type(data)
return data
thanks in advance,
Lin
This error raised because the data is a unicode/str variable, change the second line of your code to resolve your error:
data = json.loads(data)
json.load get a file object in first parameter position and call the read method of this.
Also you can call the json method of the response to fetch data directly:
response = requests.get('http://ws.audioscrobbler.com/2.0/?method=library.getartists&api_key=4c22bd45cf5aa6e408e02b3fc1bff690&user=joanofarctan&format=json')
data = response.json()
requests.get(…).text returns the content as a single (unicode) string. The json.load() function however requires a file-like argument.
The solution is rather simple: Just use loads instead of load:
data = json.loads(data)
An even better solution though is to simply call json() on the response object directly. So don’t use .text but .json():
data = requests.get(…).json()
While this uses json.loads itself internally, it hides that implementation detail, so you can just focus on getting the JSON response.
I am trying to read only one file from a tar.gz file. All operations over tarfile object works fine, but when I read from concrete member, always StreamError is raised, check this code:
import tarfile
fd = tarfile.open('file.tar.gz', 'r|gz')
for member in fd.getmembers():
if not member.isfile():
continue
cfile = fd.extractfile(member)
print cfile.read()
cfile.close()
fd.close()
cfile.read() always causes "tarfile.StreamError: seeking backwards is not allowed"
I need to read contents to mem, not dumping to file (extractall works fine)
Thank you!
The problem is this line:
fd = tarfile.open('file.tar.gz', 'r|gz')
You don't want 'r|gz', you want 'r:gz'.
If I run your code on a trivial tarball, I can even print out the member and see test/foo, and then I get the same error on read that you get.
If I fix it to use 'r:gz', it works.
From the docs:
mode has to be a string of the form 'filemode[:compression]'
...
For special purposes, there is a second format for mode: 'filemode|[compression]'. tarfile.open() will return a TarFile object that processes its data as a stream of blocks. No random seeking will be done on the file… Use this variant in combination with e.g. sys.stdin, a socket file object or a tape device. However, such a TarFile object is limited in that it does not allow to be accessed randomly, see Examples.
'r|gz' is meant for when you have a non-seekable stream, and it only provides a subset of the operations. Unfortunately, it doesn't seem to document exactly which operations are allowed—and the link to Examples doesn't help, because none of the examples use this feature. So, you have to either read the source, or figure it out through trial and error.
But, since you have a normal, seekable file, you don't have to worry about that; just use 'r:gz'.
In addition to the file mode, I attempted to seek on a network stream.
I had the same error when trying to requests.get the file, so I extracted all to a tmp directory:
# stream == requests.get
inputs = [tarfile.open(fileobj=LZMAFile(stream), mode='r|')]
t = "/tmp"
for tarfileobj in inputs:
tarfileobj.extractall(path=t, members=None)
for fn in os.listdir(t):
with open(os.path.join(t, fn)) as payload:
print(payload.read())
I have followed the tutorial posted here in order to get AJAX file uploads on my Django app. The thing is that it doesn't work, and the closest I could get to the issue is finding out that the save_upload() method raises the following exception: 'WSGIRequest' object has no attribute 'read'. Any ideas on what I am doing wrong?
EDIT: I figured out that this is only works in Django 1.3. Any ideeas on how to make it work in Django 1.2?
I think I have gotten to the bottom of your problem.
1) You are trying to run .read() on a request object, which is not allowed. Instead, you need to run it on request.raw_post_data.
2) Before you can run .read(), which takes a file-like object, you need to convert it from a str to a file-like object.
Try this:
import StringIO
output = StringIO.StringIO()
output.write(request.raw_post_data)
...now you'll be able to run output.read() and get the data you want.
#loop through, writing more of the file each time
file_so_far = output.read( 1024 ) #Get ready....
while file_so_far: #..get set...
dest.write( file_so_far ) #Go.
file_so_far = output.read( 1024
I know there is a StringIO stream in Python, but is there such a thing as a file stream in Python? Also is there a better way for me to look up these things? Documentation, etc...
I am trying to pass a "stream" to a "writer" object I made. I was hoping that I could pass a file handle/stream to this writer object.
I am guessing you are looking for open(). http://docs.python.org/library/functions.html#open
outfile = open("/path/to/file", "w")
[...]
outfile.write([...])
Documentation on all the things you can do with streams (these are called "file objects" or "file-like objects" in Python): http://docs.python.org/library/stdtypes.html#file-objects
There is a builtin file() which works much the same way. Here are the docs: http://docs.python.org/library/functions.html#file and http://python.org/doc/2.5.2/lib/bltin-file-objects.html.
If you want to print all the lines of the file do:
for line in file('yourfile.txt'):
print line
Of course there is more, like .seek(), .close(), .read(), .readlines(), ... basically the same protocol as for StringIO.
Edit: You should use open() instead of file(), which has the same API - file() goes in Python 3.
In Python, all the I/O operations are wrapped in a hight level API : the file likes objects.
It means that any file likes object will behave the same, and can be used in a function expecting them. This is called duck typing, and for file like objects you can expect the following behavior :
open / close / IO Exceptions
iteration
buffering
reading / writing / seeking
StringIO, File, and all the file like objects can really be replaced with each others, and you don't have to care about managing the I/O yourself.
As a little demo, let's see what you can do with stdout, the standard output, which is a file like object :
import sys
# replace the standar ouput by a real opened file
sys.stdout = open("out.txt", "w")
# printing won't print anything, it will write in the file
print "test"
All the file like objects behave the same, and you should use them the same way :
# try to open it
# do not bother with checking wheter stream is available or not
try :
stream = open("file.txt", "w")
except IOError :
# if it doesn't work, too bad !
# this error is the same for stringIO, file, etc
# use it and your code get hightly flexible !
pass
else :
stream.write("yeah !")
stream.close()
# in python 3, you'd do the same using context :
with open("file2.txt", "w") as stream :
stream.write("yeah !")
# the rest is taken care automatically
Note that a the file like objects methods share a common behavior, but the way to create a file like object is not standard :
import urllib
# urllib doesn't use "open" and doesn't raises only IOError exceptions
stream = urllib.urlopen("www.google.com")
# but this is a file like object and you can rely on that :
for line in steam :
print line
Un last world, it's not because it works the same way that the underlying behavior is the same. It's important to understand what you are working with. In the last example, using the "for" loop on an Internet resource is very dangerous. Indeed, you know is you won't end up with a infinite stream of data.
In that case, using :
print steam.read(10000) # another file like object method
is safer. Hight abstractions are powerful, but doesn't save you the need to know how the stuff works.