We Keep Coding

Conver string from `ABCD0011` to `ABCD_11`

I have the following CSV file:
ABCD0011
ABCD1404
ABCD1255
There are many such rows in the CSV file which I want to convert as follows:
Input
Expected Output
Actual Output
ABCD0011
ABCD_11_0
ABCD_0011_0
ABCD1404
ABCD_1404_0
ABCD_144_0
ABCD1255
ABCD_1255_0
ABCD_1255_0
Basically, it takes the zeros after the letters and replace it with an underscore ("_").
Code
import numpy as np
import pandas as pd
df = pd.read_csv('Book1.csv')
df.A = df.A.str.replace('[0-9]+', '')+'_'+df.A.str.replace('([A-Z])+', '')+'_0'
Actual Output and Issues
I got the values that are without leading zeros correctly converted like
from ABCD1255 to ABCD_1255_0.
But for values with leading zeros it failed, example:
from ABCD0011 to ABCD_0011_0. Did not change the format.
Even for values with zeros inside it failed, like
from ABCD1404 to ABCD_144_0. It deleted the zero in the middle.
Question
How can I fix this issue?

If we know the input strings will always be eight characters, with the first four being letter and the second set of four being a number, we could:
>>> s = "ABCD0011"
>>> f"{s[:4]}_{int(s[4:])}_0"
'ABCD_11_0'
If we don't know the lengths for sure, we can use re.sub with a lambda to transform two different matching groups.
>>> import re
>>> re.sub(r'([a-zA-Z]+)(\d+)', lambda m: f"{m.group(1)}_{int(m.group(2))}_0", s)
'ABCD_11_0'
>>> re.sub(r'([a-zA-Z]+)(\d+)', lambda m: f"{m.group(1)}_{int(m.group(2))}_0", 'A709')
'A_709_0'

Ignoring the apparent requirement for a dataframe, this is how you could parse the file and generate the strings you need. Uses re. Does not use numpy and/or pandas
import re
FILENAME = 'well.csv'
PATTERN = re.compile(r'^([a-zA-Z]+)(\d+)$')
with open(FILENAME) as csv_data:
next(csv_data) # skip header(s)
for line in csv_data:
if m := PATTERN.search(line):
print(f'{m.group(1)}_{int(m.group(2))}_0')
This will work for the data shown in the question. Other data structures may cause this to fail.
Output:
ABCD_11_0
ABCD_1404_0
ABCD_1255_0

How to split a comma-separated line if the chunk contains a comma in Python?

I'm trying to split current line into 3 chunks.
Title column contains comma which is delimiter
1,"Rink, The (1916)",Comedy
Current code is not working
id, title, genres = line.split(',')
Expected result
id = 1
title = 'Rink, The (1916)'
genres = 'Comedy'
Any thoughts how to split it properly?

Ideally, you should use a proper CSV parser and specify that double quote is an escape character. If you must proceed with the current string as the starting point, here is a regex trick which should work:
inp = '1,"Rink, The (1916)",Comedy'
parts = re.findall(r'".*?"|[^,]+', inp)
print(parts) # ['1', '"Rink, The (1916)"', 'Comedy']
The regex pattern works by first trying to find a term "..." in double quotes. That failing, it falls back to finding a CSV term which is defined as a sequence of non comma characters (leading up to the next comma or end of the line).

lets talk about why your code does not work
id, title, genres = line.split(',')
here line.split(',') return 4 values(since you have 3 commas) on the other hand you are expecting 3 values hence you get.
ValueError: too many values to unpack (expected 3)
My advice to you will be to not use commas but use other characters
"1#\"Rink, The (1916)\"#Comedy"
and then
id, title, genres = line.split('#')

Use the csv package from the standard library:
>>> import csv, io
>>> s = """1,"Rink, The (1916)",Comedy"""
>>> # Load the string into a buffer so that csv reader will accept it.
>>> reader = csv.reader(io.StringIO(s))
>>> next(reader)
['1', 'Rink, The (1916)', 'Comedy']

Well you can do it by making it a tuple
line = (1,"Rink, The (1916)",Comedy)
id, title, genres = line

Python put string into dictionary

I want to convert a string into a dictionary. I saved this dictionary previously in a text file.
The problem is now, that I am not sure, how the structure of the keys are. The values are generated with Counter(dictionaryName). The dictionary is really large, so I cannot check every key to see how it would be possible.
The keys can contain simple quotes like ', double quotes ", commas and maybe other characters. So is there any possibility to convert it back into a dictionary?
For example this is stored in the file:
Counter({'element0':512, "'4,5'element1":50, '4:55foobar':23,...})
I found previous solutions with for example json, but I have problems with the double quotes and I cannot simply split for the commas.

If you trust the source, load from collections import Counter and eval() the string

How about something like:
>> from collections import Counter
>> line = '''Counter({'element0':512, "'4,5'element1":50, '4:55foobar':23})'''
>> D = eval(line)
>> D
Counter({"'4,5'element1": 50, '4:55foobar': 23, 'element0': 512})

You could remove the Counter( and ) parts, then parse the rest with ast.literal_eval as long as it only involves basic Python data types:
import ast
def parse_Counter_string(s):
s = s.strip()
if not (s.startswith('Counter(') and s.endswith(')')):
raise ValueError('String does not match expected format')
# Counter( is 8 characters
# 12345678
s = s[8:-1]
return Counter(ast.literal_eval(s))
In the future, I recommend picking a different way to serialize your data.

you can use demjson library for doing this, you can have the text directly in your program
import demjson
counter = demjson.decode("enter your text here")
if it is in the file ,you can do the following steps :
WD = dirname(realpath(__file__))
file = open(WD, "filename"), "r")
counter = demjson.decode(file.read())
file.close()

Python scraping from a txt web-page

I have a large txt file from a website
https://en90.tribalwars.net/map/village.txt
These are the first few lines:
1,Barbarian+village,508,538,10342642,4208,0
2,ckouta+village,507,542,11001011,9761,0
3,Bonus+village,464,449,0,1513,1
4,Revenge+Will+Be+Sweet,501,532,9202536,9835,0
5,OFF,515,501,11158923,5644,0
I would now like to extract the the first figure from the line that matches with a given third and fourth column. For example: given I'm looking for x = 464 and y = 449 I would want my script to return 3.
I tried parsing the html page with beautifulsoup and then matching the correct line using regex but I cannot make this work.

You can use brackets and groups() in re module.
The following code will enable you to access to the 1st, 3rd and 4th number.
import re
pattern = r'(.+),.+,(.+),(.+),.+,.+,.+'
string = '3,Bonus+village,464,449,0,1513,1'
foo = re.match(pattern, string).groups()
print(foo)
which leaves you only to compare the 2nd of foo to'464', 3rd of foo to '449'.
I saw one of the comments recommending using csv and I believe that is a very rational idea. The equivalent to using csv can be done by using string.split(',')

On that particular case, I would not use regex. This data looks like CSV data (comma separated values) and is very consistent.
My suggestion:
from urllib import urlopen
from collections import namedtuple
text = 'https://en90.tribalwars.net/map/village.txt'
content = urlopen(text).read()
lines = content.split('\n')[0:-1] # last character is an empty string
village = namedtuple('village', ['id', 'name', 'x', 'y', 'z', 'whatever'])
def create_item(line):
item = village(
id=line.split(',')[0],
name=line.split(',')[1],
x=line.split(',')[2],
y=line.split(',')[3],
z=line.split(',')[4],
whatever=line.split(',')[5]
)
return item
lines = [create_item(line) for line in lines]
sample = lines[0]
print sample.id
print sample.name
print sample.x # 512
print sample.y # 529
I added a namedtuple too to make it even cooler. The lines contains all the data, and you should be able to write a function to filter based on x and y values.

Python: How split data into different data types into 2D array

I’m trying to split downloaded data to an 2D array into different datatypes. The downloaded data looks like this:
000|17:40
000|17:45
010|17:50
025|17:55
056|18:00
178|18:05
202|18:10
203|18:15
190|18:20
072|18:25
013|18:30
002|18:35
000|18:40
000|18:45
000|18:50
000|18:55
000|19:00
000|19:05
000|19:10
000|19:15
000|19:20
000|19:25
000|19:30
000|19:35
000|19:40
I’m using the following code to parse this into a two dimensional array:
#!/usr/bin/python
import urllib2
response = urllib2.urlopen('http://gps.buienradar.nl/getrr.php?lat=52&lon=4')
html = response.read()
htmlsplit = []
for record in html.split("\r\n"):
htmlsplit.append(record.split("|"))
print htmlsplit
This is working great, but as expected, it treats it as a string. I’ve found some examples that splits into integers. That’s great if both sides where integers. But in my case it’s an integer | string (or maybe some kind of Python time format)
How can I split this directly into different data types?

Something like this?
for record in html.split("\r\n"): # beware, newlines are treacherous!
s = record.split("|")
htmlsplit.append((int(s[0]), s[1]))
Just write a parser for each record, if you have data this simple. However, I would add some try/except clause to catch errors for non-conforming lines, empty lines, etc. which may be present in the data. The code above is very fragile. Also, you might want to break at only \n and then clean your strings by strip() (i.e. replace s[1] by s[1].strip()). The integer conversion takes care of it automatically.

Use str.splitlines instead of splitting on \r\n
Use the csv module to iterate over the lines:
import csv
txt = '000|17:40\n000|17:45\n000|17:50\n000|17:55\n000|18:00\n000|18:05\n000|18:10\n000|18:15\n000|18:20\n000|18:25\n000|18:30\n000|18:35\n000|18:40\n000|18:45\n000|18:50\n000|18:55\n000|19:00\n000|19:05\n000|19:10\n000|19:15\n000|19:20\n000|19:25\n000|19:30\n000|19:35\n000|19:40\n'
reader = csv.reader(txt.splitlines(), delimiter='|')
column1 = []
column2 = []
for c1, c2 in reader:
column1.append(c1)
column2.append(c2)
You can also use the DictReader
import StringIO
reader2 = csv.DictReader(StringIO.StringIO(txt),
fieldnames=['int', 'time'],
delimiter='|')
column1 = []
column2 = []
for row in reader2:
column1.append(row['time'])
column2.append(row['int'])