We Keep Coding

Python list(?) sending incorrect number of values to other program

I have been trying to upload close to a thousand SVG files to a particular program (FontForge) and as such searched for a way to automate it. Unfortunately I am really unfamiliar with Python, to the extent that I'm not sure if what I changed about the code ended up changing something fundamental.
The original code you were meant to individually continue the table that the original coder left with the file name and glyph names of the SVG files. This would require doing it manually, as I realized quickly it wouldn't allow loops within the brackets itself. The original code was as follows, albeit with more items:
select = [
('null', 'null'),
('bs-info-circle', 'infoCircle'),
]
Looking at it, and with a lot of googling and experimentation, I guessed that it was a list of tuples. As such, I created various loops adding onto a toSelect list that I created. Since they are rougly the same I'll just show one here:
for x in svc:
icon="svc-"+x+"con_lang"
glyph=x
#print((icon, glyph))
toSelect.extend(((icon, glyph),)) #comma necessary to force it to add in a pair, rather than individually
The variable svc is a list of strings: ['ba', 'be', 'bi', 'bo'...] pulled from a TXT file. The variable toSelect, when printed, looks as follows:
[('svc-bacon_lang', 'ba'), ('svc-becon_lang', 'be'), ('svc-bicon_lang', 'bi'), ...]
Long story short, I now have a list that seems to be formatted the same as the contents of the original code. As such, I set it equal in a simple manner:
select = toSelect
However, running the build program that pulls from this code is giving the following error message:
Traceback (most recent call last):
File "C:\Users\*****\Downloads\ff-batch-main\ff-batch-main\build.py", line 392, in <module>
run_fontforge(config)
File "C:\Users\*****\Downloads\ff-batch-main\ff-batch-main\build.py", line 148, in run_fontforge
for key, name in config.select:
ValueError: not enough values to unpack (expected 2, got 1)
I have tried every variation of declaring select that I can, including a few that cause the following error message:
Traceback (most recent call last):
File "C:\Users\*****\Downloads\ff-batch-main\ff-batch-main\build.py", line 392, in <module>
run_fontforge(config)
File "C:\Users\*****\Downloads\ff-batch-main\ff-batch-main\build.py", line 148, in run_fontforge
for key, name in config.select:
ValueError: too many values to unpack (expected 2)
Printing the value of select[0] does seem to tell me that that error is caused by all of the entries being considered one entry on the list? So at least I know that.
Still, I can't figure out why it doesn't take my original attempt, as select[0] = ('null', 'null'). I'm worried that select isn't supposed to be a list at all, but is something different that I'm simply unfamiliar with since I do not know python. Is this some sort of function that I broke by adding items sequentially instead of all at once?
I will also show the code from the 'build' program that is flagged as the problem. The only thing that I edited was the 'config' program, as instructed by the coder, but hopefully this at least will give context?
def run_fontforge(config):
from icon_map import maps
try:
from select_cache import maps as icons
except:
icons = {}
print(f"Generating fonts => {config.build_dir}/{config.font_name}.ttf")
with open('select_cache.py', 'w') as f:
f.write("# SVG to Font Icon mapping\n")
f.write(f"# Generated: {datetime.now()}\n")
f.write("maps = {\n")
last = config.font_start_code - 1
for _,m in icons.items():
last = max(last, m['code'])
last += 1
for key, name in config.select:
icon = icons.get(key)
src = maps.get(key)
So yeah. Um, any advice or explanations would be greatly appreciated, and I'll do my best to give additional information when possible? Unfortunately I started trying to understand Python yesterday and am coming at this from a rusty knowledge of java so I am not really fluent, might not know terms and whatnot. I just wanted to import some files man...

You've gotten quite far just 1 day into Python.
My hypothesis for the bug is config.select not containing the toSelect data.
Ways to investigate:
Interactively run this to verify that toSelect is a list of pairs:
for k, n in toSelect: pass
print both variables, or interactively evaluate config.select == toSelect to compare them, or set breakpoints in the PyCharm or VSCode debugger and examine these variables.
Is this some sort of function that I broke by adding items sequentially instead of all at once?
No.
BTW, you can make:
toSelect.extend(((icon, glyph),))
easier to understand by writing it as:
toSelect.append((icon, glyph))
Bonus: The most "Pythonic" way to write that for loop is as a list comprehension:
toSelect = [("svc-"+x+"con_lang", x) for x in svc]

Python- Insert new values into 'nested' list?

What I'm trying to do isn't a huge problem in php, but I can't find much assistance for Python.
In simple terms, from a list which produces output as follows:
{"marketId":"1.130856098","totalAvailable":null,"isMarketDataDelayed":null,"lastMatchTime":null,"betDelay":0,"version":2576584033,"complete":true,"runnersVoidable":false,"totalMatched":null,"status":"OPEN","bspReconciled":false,"crossMatching":false,"inplay":false,"numberOfWinners":1,"numberOfRunners":10,"numberOfActiveRunners":8,"runners":[{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":2.8,"size":34.16},{"price":2.76,"size":200},{"price":2.5,"size":237.85}],"availableToLay":[{"price":2.94,"size":6.03},{"price":2.96,"size":10.82},{"price":3,"size":33.45}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832765}...
All I want to do is add in an extra field, containing the 'runner name' in the data set below, into each of the 'runners' sub lists from the initial data set, based on selection_id=selectionId.
So initially I iterate through the full dataset, and then create a separate list to get the runner name from the runner id (I should point out that runnerId===selectionId===selection_id, no idea why there are multiple names are used), this works fine and the code is shown below:
for market_book in market_books:
market_catalogues = trading.betting.list_market_catalogue(
market_projection=["RUNNER_DESCRIPTION", "RUNNER_METADATA", "COMPETITION", "EVENT", "EVENT_TYPE", "MARKET_DESCRIPTION", "MARKET_START_TIME"],
filter=betfairlightweight.filters.market_filter(
market_ids=[market_book.market_id],
),
max_results=100)
data = []
for market_catalogue in market_catalogues:
for runner in market_catalogue.runners:
data.append(
(runner.selection_id, runner.runner_name)
)
So as you can see I have the data in data[], but what I need to do is add it to the initial data set, based on the selection_id.
I'm more comfortable with Php or Javascript, so apologies if this seems a bit simplistic, but the code snippets I've found on-line only seem to assist with very simple Python lists and nothing 'nested' (to me the structure seems similar to a nested array).
As per the request below, here is the full list:
{"marketId":"1.130856098","totalAvailable":null,"isMarketDataDelayed":null,"lastMatchTime":null,"betDelay":0,"version":2576584033,"complete":true,"runnersVoidable":false,"totalMatched":null,"status":"OPEN","bspReconciled":false,"crossMatching":false,"inplay":false,"numberOfWinners":1,"numberOfRunners":10,"numberOfActiveRunners":8,"runners":[{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":2.8,"size":34.16},{"price":2.76,"size":200},{"price":2.5,"size":237.85}],"availableToLay":[{"price":2.94,"size":6.03},{"price":2.96,"size":10.82},{"price":3,"size":33.45}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832765},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":20,"size":3},{"price":19.5,"size":26.36},{"price":19,"size":2}],"availableToLay":[{"price":21,"size":13},{"price":22,"size":2},{"price":23,"size":2}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832767},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":11,"size":9.75},{"price":10.5,"size":3},{"price":10,"size":28.18}],"availableToLay":[{"price":11.5,"size":12},{"price":13.5,"size":2},{"price":14,"size":7.75}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832766},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":48,"size":2},{"price":46,"size":5},{"price":42,"size":5}],"availableToLay":[{"price":60,"size":7},{"price":70,"size":5},{"price":75,"size":10}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832769},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":18.5,"size":28.94},{"price":18,"size":5},{"price":17.5,"size":3}],"availableToLay":[{"price":21,"size":20},{"price":23,"size":2},{"price":24,"size":2}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832768},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":4.3,"size":9},{"price":4.2,"size":257.98},{"price":4.1,"size":51.1}],"availableToLay":[{"price":4.4,"size":20.97},{"price":4.5,"size":30},{"price":4.6,"size":16}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832771},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":24,"size":6.75},{"price":23,"size":2},{"price":22,"size":2}],"availableToLay":[{"price":26,"size":2},{"price":27,"size":2},{"price":28,"size":2}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":12832770},{"status":"ACTIVE","ex":{"tradedVolume":[],"availableToBack":[{"price":5.7,"size":149.33},{"price":5.5,"size":29.41},{"price":5.4,"size":5}],"availableToLay":[{"price":6,"size":85},{"price":6.6,"size":5},{"price":6.8,"size":5}]},"sp":{"nearPrice":null,"farPrice":null,"backStakeTaken":[],"layLiabilityTaken":[],"actualSP":null},"adjustmentFactor":null,"removalDate":null,"lastPriceTraded":null,"handicap":0,"totalMatched":null,"selectionId":10064909}],"publishTime":1551612312125,"priceLadderDefinition":{"type":"CLASSIC"},"keyLineDescription":null,"marketDefinition":{"bspMarket":false,"turnInPlayEnabled":false,"persistenceEnabled":false,"marketBaseRate":5,"eventId":"28180290","eventTypeId":"2378961","numberOfWinners":1,"bettingType":"ODDS","marketType":"NONSPORT","marketTime":"2019-03-29T00:00:00.000Z","suspendTime":"2019-03-29T00:00:00.000Z","bspReconciled":false,"complete":true,"inPlay":false,"crossMatching":false,"runnersVoidable":false,"numberOfActiveRunners":8,"betDelay":0,"status":"OPEN","runners":[{"status":"ACTIVE","sortPriority":1,"id":10064909},{"status":"ACTIVE","sortPriority":2,"id":12832765},{"status":"ACTIVE","sortPriority":3,"id":12832766},{"status":"ACTIVE","sortPriority":4,"id":12832767},{"status":"ACTIVE","sortPriority":5,"id":12832768},{"status":"ACTIVE","sortPriority":6,"id":12832770},{"status":"ACTIVE","sortPriority":7,"id":12832769},{"status":"ACTIVE","sortPriority":8,"id":12832771},{"status":"LOSER","sortPriority":9,"id":10317013},{"status":"LOSER","sortPriority":10,"id":10317010}],"regulators":["MR_INT"],"countryCode":"GB","discountAllowed":true,"timezone":"Europe\/London","openDate":"2019-03-29T00:00:00.000Z","version":2576584033,"priceLadderDefinition":{"type":"CLASSIC"}}}

i think i understand what you are trying to do now
first hold your data as a python object (you gave us a json object)
import json
my_data = json.loads(my_json_string)
for item in my_data['runners']:
item['selectionId'] = [item['selectionId'], my_name_here]
the thing is that my_data['runners'][i]['selectionId'] is a string, unless you want to concat the name and the id together, you should turn it into a list or even a dictionary
each item is a dicitonary so you can always also a new keys to it
item['new_key'] = my_value

So, essentially this works...with one exception...I can see from the print(...) in the loop that the attribute is updated, however what I can't seem to do is then see this update outside the loop.
mkt_runners = []
for market_catalogue in market_catalogues:
for r in market_catalogue.runners:
mkt_runners.append((r.selection_id, r.runner_name))
for market_book in market_books:
for runner in market_book.runners:
for x in mkt_runners:
if runner.selection_id in x:
setattr(runner, 'x', x[1])
print(market_book.market_id, runner.x, runner.selection_id)
print(market_book.json())
So the print(market_book.market_id.... displays as expected, but when I print the whole list it shows the un-updated version. I can't seem to find an obvious solution, which is odd, as it seems like a really simple thing (I tried messing around with indents, in case that was the problem, but it doesn't seem to be, its like its not refreshing the market_book list post update of the runners sub list)!

Type error that shows on console, but not on pythontutor.com

Ok, so I have a block of code that I am trying to debug, and I usually use Pythontutor.com to step through the code to see where it is going wrong. Problem is, the exact code works on the website, but not in my console.
row = []
row.append("Acid Arrow")
testList = ['Detect', 'Discern', 'Summon', 'Call', 'Binding']
nameList = row[0].split(' ')
print testList, nameList
a = list(set(testList) & set(nameList))
The error I am getting is this:
C:\Users\User\Dropbox\D&D\SpellBag>livingSpell.py
['Detect', 'Discern', 'Summon', 'Call', 'Binding'] ['Acid', 'Arrow']
Traceback (most recent call last):
File "C:\Users\User\Dropbox\D&D\SpellBag\livingSpell.py", line 121, in <module>
sb = spellBook(r'allSpells.csv')
File "C:\Users\User\Dropbox\D&D\SpellBag\livingSpell.py", line 27, in __init__
a = list(set(testList) & set(nameList))
TypeError: 'str' object is not callable
The above code works flawlessly on PythonTutor, but fails when I run it in the console. What it is intended to do is check if a word from the list is in the spell name, which if any of them are, the spell is passed over and it moves on. It should be returning an empty list, but instead I get the error.

The line that has the error is a = list(set(testList) & set(nameList)), and the error says "'str' object is not callable." This means the Python interpreter tried to call a function and found out it wasn't actually a function. This is the same error you would get if you typed "bad_code"(), since the string "bad_code" is not a function.
It's impossible to say exactly which of the two is having an issue, but either list or set has been overwritten and is now a string rather than the default functions provided in Python. That snippet of code works fine by itself in pythontutor.com because the offending line of code happens somewhere before it in your file (the error says you have 22 lines of code beforehand). In fact, if you started a blank file and only had the snippet you posted here on StackOverflow, it would run perfectly. Check for anything like list = ... or set = ... in your original source code.
It is a somewhat common convention in Python to avoid naming conflicts with reserved words (list, set, or, if, with, while, etc...) by appending an underscore to the name. In this case, that would mean writing either list_ = ... or set_ = .... A good coding practice in general though would be to come up with a specific name for your variable that describes it exactly. For example, you might use used_spell_list instead of list (just guessing...I have no idea how this was overwritten).

BioPython Pubmed Eutils url?

I'm trying to run some queries against Pubmed's Eutils service. If I run them on the website I get a certain number of records returned, in this case 13126 (link to pubmed).
A while ago I bodged together a python script to build a query to do much the same thing, and the resultant url returns the same number of hits (link to Eutils result).
Of course, not having any formal programming background, it was all a bit cludgy, so I'm trying to do the same thing using Biopython. I think the following code should do the same thing, but it returns a greater number of hits, 23303.
from Bio import Entrez
Entrez.email = "A.N.Other#example.com"
handle = Entrez.esearch(db="pubmed", term="stem+cell[All Fields]",datetype="pdat", mindate="2012", maxdate="2012")
record = Entrez.read(handle)
print(record["Count"])
I'm fairly sure it's just down to some subtlety in how the url is being generated, but I can't work out how to see what url is being generated by Biopython. Can anyone give me some pointers?
Thanks!
EDIT:
It's something to do with how the url is being generated, as I can get back the original number of hits by modifying the code to include double quotes around the search term, thus:
handle = Entrez.esearch(db='pubmed', term='"stem+cell"[ALL]', datetype='pdat', mindate='2012', maxdate='2012')
I'm still interested in knowing what url is being generated by Biopython as it'll help me work out how i have to structure the search term for when i want to do more complicated searches.

handle = Entrez.esearch(db="pubmed", term="stem+cell[All Fields]",datetype="pdat", mindate="2012", maxdate="2012")
print(handle.url)

You've solved this already (Entrez likes explicit double quoting round combined search terms), but currently the URL generated is not exposed via the API. The simplest trick would be to edit the Bio/Entrez/__init__.py file to add a print statement inside the _open function.
Update: Recent versions of Biopython now save the URL as an attribute of the returned handle, i.e. in this example try doing print(handle.url)

Find Hyperlinks in Text using Python (Follow-up to another post)

In regards to (Extracting a URL in Python) I have a follow-up question. Note: I'm new to SO and Python, so feel free to correct me on etiquette.
I pulled the regex from the above post and this works fine for me:
myString = """ <iframe width="640" height="390" src="http://www.youtube.com/embed/24WIANESD7k?rel=0" frameborder="0" allowfullscreen></iframe> """
print re.search("(?P<url>https?://[^\s]+)", myString).group("url")
However what I really need to do is loop through a data set that I have previously retrieved from a database. So I did the below, which gives me a strange error, also below.
# Note: "data" here is actually a list of strings, not a data set
for pseudo_url in data:
print re.search("(?P<url>https?://[^\s]+)", str(pseudo_url)).group("url")
Error:
Traceback (most recent call last):
File "find_and_email_bad_press_urls.py", line 136, in <module>
main()
File "find_and_email_bad_press_urls.py", line 14, in main
scrubbed_urls = extract_urls_from_raw_data(raw_url_data)
File "find_and_email_bad_press_urls.py", line 47, in extract_urls_from_raw_data
print re.search("(?P<url>https?://[^\s]+)", str(pseudo_url)).group("url")
AttributeError: 'NoneType' object has no attribute 'group'
When I Google this I find tons of irrelevant posts, so I was hoping SO could shed some light. My hunch is that the regex is blowing up on some null data, special character, etc., but I don't know enough about Python to figure it out. Casting to a string didn't help either.
Any ideas or workarounds to power through this would be much appreciated!

Your regex is not finding a url in every string in data. You should check to make sure you have a match before making the call to group:
for pseudo_url in data:
m = re.search("(?P<url>https?://[^\s]+)", pseudo_url)
if m:
print m.group("url")
You don't need the call to str() either if pseudo_url is already a string.
And as #Blender suggested in his comment, if data is really lines read from an HTML file, you may want to consider using Beautiful Soup instead of regex for this.