I have a large (>1000) number of files in which there are fields containing numbers that are defined as text fields. I need to have a fields containing these values as numbers. I can add the new fields, but when I'm failing to populate them.
I'm using ArcGis 10.1. Rows may have values ranging from 0-10, and including up to one decimal place, or they may be empty for a variable (actually blank, no placeholder).
Below is the python script I'm using for two of the variables (N_CT and N_CFY), and the error I get. It looks like my problem is in how to transfer the text value into the Decimal conversion.
I'm new to scripting, so please excuse me if my description or word choices are unclear.
import arcpy, os, sys
from arcpy import env
from decimal import *
arcpy.env.overwriteOutput = True
env.workspace = "C:\Users\OuelletteMS\Desktop\Ice_Data_testarea"
listFCs = arcpy.ListFeatureClasses("*")
for fc in listFCs:
print str("processing " + fc) # displays the file that is currently being handled
strNCT = "N_CT" # the current, text version of the field
newNCT = "NCT" # the new, number version I want to create
strNCFY = "N_CFY" # the current, text version of the field
newNCFY = "NCFY" # the new, number version I want to create
arcpy.AddField_management(fc,newNCT,"DOUBLE")
arcpy.AddField_management(fc,newNCFY,"DOUBLE")
cursor = arcpy.UpdateCursor(fc)
for row in cursor:
row.setValue(newNCT, row.getValue(Decimal(strNCT)))
row.setValue(newNCFY, row.getValue(Decimal(strNCFY)))
cursor.updateRow(row)
Error mesage:
Runtime error Traceback (most recent call last): File "",
line 23, in File "C:\Python27\ArcGIS10.1\Lib\decimal.py",
line 548, in new
"Invalid literal for Decimal: %r" % value) File "C:\Python27\ArcGIS10.1\Lib\decimal.py", line 3844, in _raise_error
raise error(explanation) InvalidOperation: Invalid literal for Decimal: 'N_CT'
You could convert a string value to an integer of float by using:
stringA = '12'
# Convert string to integer:
int(stringA)
# Convert string to float
float(stringA)
Related
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 1 year ago.
Improve this question
I am trying to read a text file line by line as integers. I did every suggestion I saw here but none works for me. here is the code I'm using. It reads some seismic data from the datadir and evaluates the SNR ratio to decide whether keep the data or remove it. To do so, I need to calculate the distance between stations and the earthquake which the info comes from input files.
from obspy import UTCDateTime
import os
datadir = "/home/alireza/Desktop/Saman/Eqcomplete"
homedir = "/home/alireza/Desktop/Saman"
eventlist = os.path.join (homedir, 'events.dat')
stationlist = os.path.join (homedir, 'all_st')
e = open (eventlist, "r")
for event in e.readlines():
year, mon, day, time, lat, lon = event.split (" ")
h = str (time.split (":")[0]) # hour
m = str (time.split (":")[1]) # minute
s = str (time.split (":")[2]) # second
s = open (stationlist, "r")
for station in s.readlines():
stname, stlo, stla = station.split (" ")
OafterB = UTCDateTime (int(year), int(mon), int(day), int(h), int(m), int(s))
print (OafterB) # just to have an output!
s.close ()
e.close ()`
Update:
There are two input files:
events.dat which is like:
2020 03 18 17:45:39 -11.0521 115.1378
all_st which is like:
AHWZ 48.644 31.430
AFRZ 59.015 33.525
NHDN 60.050 31.493
BDRS 48.881 34.054
BMDN 48.825 33.772
HAGD 49.139 34.922
Here is the output:
Traceback (most recent call last):
File "SNR.py", line 21, in <module>
OafterB = UTCDateTime (int(year), int(mon), int(day), int(h), int(m), int(s))
TypeError: int() argument must be a string, a bytes-like object or a number, not '_io.TextIOWrapper'
Here to test the code you need to install the obspy package.
pip install obspy may work.
You define s here:
s = str (time.split (":")[2]) # second
But then, immediately afterward, you refine it:
s = open (stationlist, "r")
Now s points to a file object, so int(s) fails with the above error. Name your station list file object something different, and the problem goes away.
Other tips which you may find helpful:
split() will automatically split on whitespace unless you tell it otherwise, so there's no need to specify " ".
You can use multiple assignment to assign h, m, and s the same way you did with the previous line. Currently, you're performing the same split operation three different times.
It's recommended to open files using the with keyword, which will automatically handle closing the file, even if an exception occurs.
You can iterate over a file object directly, without creating a list with readlines().
Using pathlib can make it much simpler and cleaner to deal with filesystem paths and separators.
It's considered bad form to put spaces between the name of a function and the parentheses.
There's also a convention that variable names (other than class names) are usually all lowercase, with underscores between words as needed. (See PEP 8 for a helpful rundown of all such style conventions. They're not hard and fast rules, but they can help make code more consistent and readable.)
With those things in mind, here's a slightly spruced up version of your above code:
from pathlib import Path
from obspy import UTCDateTime
data_dir = Path('/home/alireza/Desktop/Saman/Eqcomplete')
home_dir = Path('/home/alireza/Desktop/Saman')
event_list = home_dir / 'events.dat'
station_list = home_dir / 'all_st'
with open(event_list) as e_file:
for event in e_file:
year, mon, day, time, lat, lon = event.split()
h, m, s = time.split(':')
with open(station_list) as s_file:
for station in s_file:
stname, stlo, stla = station.split()
o_after_b = UTCDateTime(
int(year), int(mon), int(day), int(h), int(m), int(s)
)
print(o_after_b)
Im trying to make the program to identify a number in a NETCDF file name, I altered the code, but is still giving me the same error and I can't identify why.
The section of the code creating the error is:
Band = int((listofallthefiles[number][listofallthefiles[number].find("M3C" or "M4C" or "M6C")+3:listofallthefiles[number].find("_G16")]))
The path and name of the NETCDF file is:
/Volumes/Anthonys_backup/Hurricane_Dorian/August_28/Channel_13/OR_ABI-L2-CMIPF-M6C13_G16_s20192400000200_e20192400009520_c20192400010004.nc
Im trying to extract the "13" between "M6C" and "_G16" to save the value, but its giving me the error message:
ValueError: invalid literal for int() with base 10: 'olumes/Anthonys_backup/Hurricane_Dorian/August_28/Channel_13/OR_ABI-L2-CMIPF-M6C13'
First extract the number of your string, so that int can properly convert it, see here.
It might be easier to use regex to do so, e.g.:
import re
...
str = listofallthefiles[number]
num = re.findall('.*M6C(.*)_G16', str)[0]
Now you can convert that to an integer:
val = int(num)
I have this code that will allow me to count the number of missing rows of numbers within the csv for a script in Python 3.6. However, these are the following errors in the program:
Error:
Traceback (most recent call last):
File "C:\Users\GapReport.py", line 14, in <module>
EndDoc_Padded, EndDoc_Padded = (int(s.strip()[2:]) for s in line)
File "C:\Users\GapReport.py", line 14, in <genexpr>
EndDoc_Padded, EndDoc_Padded = (int(s.strip()[2:]) for s in line)
ValueError: invalid literal for int() with base 10: 'AC-SEC 000000001'
Code:
import csv
def out(*args):
print('{},{}'.format(*(str(i).rjust(4, "0") for i in args)))
prev = 0
data = csv.reader(open('Padded Numbers_export.csv'))
print(*next(data), sep=', ') # header
for line in data:
EndDoc_Padded, EndDoc_Padded = (int(s.strip()[2:]) for s in line)
if start != prev+1:
out(prev+1, start-1)
prev = end
out(start, end)
I'm stumped on how to fix these issues.Also, I think the csv many lines in it, so if there's a section that limits it to a few numbers, please feel free to update me on so.
CSV Snippet (Sorry if I wasn't clear before!):
The values you have in your CSV file are not numeric.
For example, FMAC-SEC 000000001 is not a number. So when you run int(s.strip()[2:]), it is not able to convert it to an int.
Some more comments on the code:
What is the utility of doing EndDoc_Padded, EndDoc_Padded = (...)? Currently you are assigning values to two variables with the same name. Either name one of them something else, or just have one variable there.
Are you trying to get the two different values from each column? In that case, you need to split line into two first. Are the contents of your file comma separated? If yes, then do for s in line.split(','), otherwise use the appropriate separator value in split().
You are running this inside a loop, so each time the values of the two variables would get updated to the values from the last line. If you're trying to obtain 2 lists of all the values, then this won't work.
Summary
In PsychoPy v1.85.1 I am configuring the 'polygon' stimuli object. I am trying to set the height attribute via a csv conditions file. I get an error message saying "Invalid parameter. Single numbers are not accepted"
Details
PsychoPy v1.85.1 on Windows.
In the polygon's pop-up UI 'size' box I enter:
$height
In the csv-file I have a 'height' column. Each row has values such as:
(1.5, 0)
PsychoPy gives the error message:
File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy \visual\basevisual.py", line 1312, in pos
self.__dict__['pos'] = val2array(value, False, False)
File "C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy\tools\arraytools.py", line 176, in val2array
raise ValueError(msg % str(length))
ValueError: Invalid parameter. Single numbers are not accepted. Should be tuple/list/array of length 2
Troubleshooting - Misc
Scalar variables work in other csv columns work fine so PsychoPy connects with the csv-file.
Tried xlsx format.
Tried entering without parenthesis and with square bracket parenthesis
Troubleshooting - Running the code outside of PsychoPy
I go to the arraytools.py file and find the relevant code snippet. I paste it into a Python notebook (although it is python 3.3) and add some print rows for debugging:
# Copied code snippet from
# C:\Program Files (x86)\PsychoPy2\lib\site-packages\psychopy\tools\arraytools.py
import numpy
def val2array(value, withNone=True, withScalar=True, length=2):
"""Helper function: converts different input to a numpy array.
Raises informative error messages if input is invalid.
withNone: True/False. should 'None' be passed?
withScalar: True/False. is a scalar an accepted input?
Will be converted to array of this scalar
length: False / 2 / 3. Number of elements input should have or be
converted to. Might be False (do not accept arrays or convert to such)
"""
if value is None:
if withNone:
return None
else:
raise ValueError('Invalid parameter. None is not accepted as '
'value.')
value = numpy.array(value, float)
print ("value:", value) #I ADDED
print ("value.shape:", value.shape) #I ADDED
print ("numpy.product(value.shape):", numpy.product(value.shape)) #I ADDED
if numpy.product(value.shape) == 1: #MY COMMENT: WHY DOES THIS EVALUTE TRUE?
if withScalar:
# e.g. 5 becomes array([5.0, 5.0, 5.0]) for length=3
return numpy.repeat(value, length)
else:
msg = ('Invalid parameter. Single numbers are not accepted. '
'Should be tuple/list/array of length %s')
raise ValueError(msg % str(length))
elif value.shape[-1] == length:
return numpy.array(value, float)
else:
msg = 'Invalid parameter. Should be length %s but got length %s.'
raise ValueError(msg % (str(length), str(len(value))))
I test it by entering a value and then run the function.
# Run the function
value = (1.5,0.0)
val2array(value, False, False, length =2)
Results below. Seems to work fine:
value: [ 1.5 0. ]
value.shape: (2,)
numpy.product(value.shape): 2
Out: array([ 1.5, 0. ])
Debugging in Coder View
Thank you Michael. It seems the input value becomes a unicode string which the numpy arrary function cannot convert
print "position: ", position
print "type(position): ", type(position)
print "numpy.array(position, float): ", np.array(position, float)
#Results:
position: (0, 0.5)
type(position): <type 'unicode'>
numpy.array(position, float):
Traceback (most recent call last):
File "C:\Users\nikla\Documents\PsychoPy\test2.py", line 127, in <module>
print "numpy.array(position, float): ", np.array(position, float)
ValueError: could not convert string to float: (0, 0.5)
Any idea what I am doing wrong?
The problem was fixed by entering the attribute values (in the GUI pop-up) using square brackets:
[1.5, 0]
The error has to do with the position attribute rather than the height. Do you set the position to be a scalar? It has to be a coordinate. Try writing the position in your csv file as (1.5, 0.0) and then enter $eval(position) in Builder (or whatever you called the position column).
I want to rename a file called decon.out using two variables in my program. So far I have
gwf = input ("Enter value: ")
myList = os.listdir('.')
for myFile in myList:
if re.match("^HHEMQZ", myFile):
numE = myFile
elif re.match("^HHNMQZ", myFile):
numN = myFile
else:
den = myFile
os.rename('decon.out', 'RF'+gwf+''+numE+'')
For example, gwf = 2.5 and numE = HHEMQZ20010101
I would then want decon.out to be renamed as RF2.5HHEMQZ20010101 where RF will always be the same.
Currently when I run the script I get an error:
Traceback (most recent call last):
File "RunDeconv.py", line 77, in <module>
os.rename('decon.out', 'RF'+gwf+''+numE+'')
TypeError: cannot concatenate 'str' and 'float' objects
Any suggestions?
Use raw_input() instead, input() interprets the input values as Python code turning your 2.5 input into a float number.
About the error: in the string concatenation
'RF'+gwf+''+numE+''
all the members must be strings.
You can use
type(gwf)
type(numE)
to check which is a number.
You then just need to
str(gwf)
or
str(numE)
depending on which may be the case. Or probably both gwf and numE need the str() treatment, so your last line of code should look like this:
os.rename('decon.out', 'RF'+str(gwf)+''+str(numE)+'')