We Keep Coding

f-string script conversion to earlier version of Python

I have a question in regards to comparing files using python. For context, the problem I am having is that I have two firewalls with different configurations (over 14000 lines each on Notepad++) and I need to find the differences and annotate them.
Quick Example -
Firewall 1:
Version: 123
set policy EXPLICIT_DENY ALL
IP address allow 1.2.3.4
IP address allow 4.3.2.1
set policy EXPLICIT_ALLOW NONE
Firewall 2:
Version: 321
set policy EXPLICIT_ALLOW NONE
IP address allow 4.3.2.1
IP address allow 1.2.3.4
set policy EXPLICIT_DENY ALL
A line-by-line comparison would show that all of those lines are incorrect because they do not match side by side, however, the configuration is the same and would not need to be annotated. The only difference would be the Version # in the example. The script below was able to work for my purposes.
Current Script I ran -
'file1 = open("OLD FW.txt",'r')
'file2 = open("NEW FW.txt",'r')
'file3 = open("Results.txt",'r+')
'file1_lines = file1.readlines()
'file2_lines = file2.readlines()
'for position, a in enumerate(file1_lines):
'linematch = False
'for b in file2_lines:
'if a == b:
'linematch = True
'if linematch == False:
'file.3write(f"{position+1}: {a.strip()}\")
'file1.close()
'file2.close()'
The output would show every line from the OLD firewall that does not appear on the NEW firewall. This would effectively let me see what configurations are missing and/or different AND show me what line is is on the original FW.
The issue I figured out after coming up with this is that my current software version at work is only Python 2.7.16 which doesn't support f-strings. I did a little bit of research but am far to novice currently to figure this out in the short time window I have.
Mai question: How do I convert this Python f-string script to something that would work the same in an older version of Python that doesn't support f-strings?
Thanks in advance to anyone who can help me figure this out!

For simple cases like you show, you can use the .format() method.
file.write("{}: {}\n".format(position+1, a.strip()))

Retrieve domain name from email address

I am working on a huge email-address dataset in Python and need to retrieve the organization name.
For example, email#organizationName.com is easy to extract, but what about email#info.organizationName.com or even email#organizationName.co.uk?
I need a universal extractor that should be able to handle all different possibilities accordingly.

If organisationName is always before .com or other ending - this may work -
email_str.split('#')[1].split('.')[-2]

A regex won't work well here. In order to be able to reliably do this, you need to use a lib that has knowledge on what constitutes a valid suffix.
Otherwise, how would the extractor be able distinguish email#info.organizationName.com from email#organizationName.co.uk?
This can be done using tldextract:
Example:
import tldextract
emails = ['email#organizationName.com',
'email#info.organizationName.com',
'email#organizationName.co.uk',
'email#info.organizationName.co.uk',
]
for addr in emails:
print(tldextract.extract(addr))
Output:
ExtractResult(subdomain='', domain='organizationName', suffix='com')
ExtractResult(subdomain='info', domain='organizationName', suffix='com')
ExtractResult(subdomain='', domain='organizationName', suffix='co.uk')
ExtractResult(subdomain='info', domain='organizationName', suffix='co.uk')
To access just the domain, use tldextract.extract(addr).domain.

Is there a way to include an implicit "yes" in ssh commands? [duplicate]

This question already has answers here:
Passing arguments to an interactive program non-interactively
(5 answers)
Closed 2 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Is it possible to have a bash script automatically handle prompts that would normally be presented to the user with default actions? Currently I am using a bash script to call an in-house tool that will display prompts to the user (prompting for Y/N) to complete actions, however the script I'm writing needs to be completely "hands-off", so I need a way to send Y|N to the prompt to allow the program to continue execution. Is this possible?

A simple
echo "Y Y N N Y N Y Y N" | ./your_script
This allow you to pass any sequence of "Y" or "N" to your script.

This is not "auto-completion", this is automation. One common tool for these things is called Expect.
You might also get away with just piping input from yes.

If you only have Y to send :
$> yes Y |./your_script
If you only have N to send :
$> yes N |./your_script

I found the best way to send input is to use cat and a text file to pass along whatever input you need.
cat "input.txt" | ./Script.sh

In my situation I needed to answer some questions without Y or N but with text or blank. I found the best way to do this in my situation was to create a shellscript file. In my case I called it autocomplete.sh
I was needing to answer some questions for a doctrine schema exporter so my file looked like this.
-- This is an example only --
php vendor/bin/mysql-workbench-schema-export mysqlworkbenchfile.mwb ./doctrine << EOF
`#Export to Doctrine Annotation Format` 1
`#Would you like to change the setup configuration before exporting` y
`#Log to console` y
`#Log file` testing.log
`#Filename [%entity%.%extension%]`
`#Indentation [4]`
`#Use tabs [no]`
`#Eol delimeter (win, unix) [win]`
`#Backup existing file [yes]`
`#Add generator info as comment [yes]`
`#Skip plural name checking [no]`
`#Use logged storage [no]`
`#Sort tables and views [yes]`
`#Export only table categorized []`
`#Enhance many to many detection [yes]`
`#Skip many to many tables [yes]`
`#Bundle namespace []`
`#Entity namespace []`
`#Repository namespace []`
`#Use automatic repository [yes]`
`#Skip column with relation [no]`
`#Related var name format [%name%%related%]`
`#Nullable attribute (auto, always) [auto]`
`#Generated value strategy (auto, identity, sequence, table, none) [auto]`
`#Default cascade (persist, remove, detach, merge, all, refresh, ) [no]`
`#Use annotation prefix [ORM\]`
`#Skip getter and setter [no]`
`#Generate entity serialization [yes]`
`#Generate extendable entity [no]` y
`#Quote identifier strategy (auto, always, none) [auto]`
`#Extends class []`
`#Property typehint [no]`
EOF
The thing I like about this strategy is you can comment what your answers are and using EOF a blank line is just that (the default answer). Turns out by the way this exporter tool has its own JSON counterpart for answering these questions, but I figured that out after I did this =).
to run the script simply be in the directory you want and run 'sh autocomplete.sh' in terminal.
In short by using << EOL & EOF in combination with Return Lines you can answer each question of the prompt as necessary. Each new line is a new answer.
My example just shows how this can be done with comments also using the ` character so you remember what each step is.
Note the other advantage of this method is you can answer with more then just Y or N ... in fact you can answer with blanks!
Hope this helps someone out.

There is a special build-in util for this - 'yes'.
To answer all questions with the same answer, you can run
yes [answer] |./your_script
Or you can put it inside your script have specific answer to each question

Snippets vs. Abbreviations in Vim

What advantages and/or disadvantages are there to using a "snippets" plugin, e.g. snipmate, ultisnips, for VIM as opposed to simply using the builtin "abbreviations" functionality?
Are there specific use-cases where declaring iabbr, cabbr, etc. lack some major features that the snippets plugins provide? I've been unsuccessful in finding a thorough comparison between these two "features" and their respective implementations.
As #peter-rincker pointed out in a comment:
It should be noted that abbreviations can execute code as well. Often via <c-r>= or via an expression abbreviation (<expr>). Example which expands ## to the current file's path: :iabbrev ## <c-r>=expand('%:p')<cr>
As an example for python, let's compare a snipmate snippet and an abbrev in Vim for inserting lines for class declaration.
Snipmate
# New Class
snippet cl
class ${1:ClassName}(${2:object}):
"""${3:docstring for $1}"""
def __init__(self, ${4:arg}):
${5:super($1, self).__init__()}
self.$4 = $4
${6}
Vimscript
au FileType python :iabbr cl class ClassName(object):<CR><Tab>"""docstring for ClassName"""<CR>def __init__(self, arg):<CR><Tab>super(ClassName, self).__init__()<CR>self.arg = arg
Am I missing some fundamental functionality of "snippets" or am I correct in assuming they are overkill for the most part, when Vim's abbr and :help template templates are able to do all most of the stuff snippets do?
I assume it's easier to implement snippets, and they provide additional aesthetic/visual features. For instance, if I use abbr in Vim and other plugins for running/testing python code inside vim--e.g. syntastic, pytest, ropevim, pep8, etc--am I missing out on some key features that snippets provide?

Everything that can be done with snippets can be done with abbreviations and vice-versa. You can have (mirrored or not) placeholders with abbreviations, you can have context-sensitive snippets.
There are two important differences:
Abbreviations are triggered when the abbreviation text has been typed, and a non word character (or esc) is hit. Snippets are triggered on demand, and shortcuts are possible (no need to type while + tab. w + tab may be enough).
It's much more easier to define new snippets (or to maintain old ones) than to define abbreviations. With abbreviations, a lot of boiler plate code is required when we want to do neat things.
There are a few other differences. For instance, abbreviations are always triggered everywhere. And seeing for expanded into for(placeholder) {\n} within a comment or a string context is certainly not what the end-user expects. With snippets, this is not a problem any more: we can expect the end-user to know what's he's doing when he asks to expand a snippet. Still, we can propose context-aware snippets that expand throw into #throw {domain::exception} {explanation} within a comment, or into throw domain::exception({message}); elsewhere.

Snippets
Rough superset of Vim's native abbreviations. Here are the highlights:
Only trigger on key press
Uses placeholders which a user can jump between
Exist only for insert mode
Dynamic expansions
Abbreviations
Great for common typos and small snippets.
Native to Vim so no need for plugins
Typically expand on whitespace or <c-]>
Some special rules on trigger text (See :h abbreviations)
Can be used in command mode via :cabbrev (often used to create command aliases)
No placeholders
Dynamic expansions
Conclusion
For the most part snippets are more powerful and provide many features that other editors enjoy, but you can use both and many people do. Abbreviations enjoy the benefit of being native which can be useful for remote environments. Abbreviations also enjoy another clear advantage which is can be used in command mode.

Snippets are more powerful.
Depending on the implementation, snippets can let you change (or accept defaults for) multiple placeholders and can even execute code when the snippet is expanded.
For example with ultisnips, you can have it execute shell commands, vimscript but also Python code.
An (ultisnips) example:
snippet hdr "General file header" b
# file: `!v expand('%:t')`
# vim:fileencoding=utf-8:ft=`!v &filetype`
# ${1}
#
# Author: ${2:J. Doe} ${3:<jdoe#gmail.com>}
# Created: `!v strftime("%F %T %z")`
# Last modified: `!v strftime("%F %T %z")`
endsnippet
This presents you with three placeholders to fill in (it gives default values for two of them), and sets the filename, filetype and current date and time.
After the word "snippet", the start line contains three items;
the trigger string,
a description and
options for the snippet.
Personally I mostly use the b option where the snippet is expanded at the beginning of a line and the w option that expands the snippet if the trigger string starts at the beginning of a word.
Note that you have to type the trigger string and then input a key or key combination that actually triggers the expansion. So a snippet is not expanded unless you want it to.
Additionally, snippets can be specialized by filetype. Suppose you want to define four levels of headings, h1 .. h4. You can have the same name expand differently between e.g. an HTML, markdown, LaTeX or restructuredtext file.

snippets are like the built-in :abbreviate on steroids, usually with:
parameter insertions: You can insert (type or select) text fragments in various places inside the snippet. An abbreviation just expands once.
mirroring: Parameters may be repeated (maybe even in transformed fashion) elsewhere in the snippet, usually updated as you type.
multiple stops inside: You can jump from one point to another within the snippet, sometimes even recursively expand snippets within one.
There are three things to evaluate in a snippet plugin: First, the features of the snippet engine itself, second, the quality and breadth of snippets provided by the author or others; third, how easy it is to add new snippets.

does distinct work with GAE Python SDK - 1.9.4 release

Code:
pages = Line.gql("where projectNAME='%s'"%(projectName)).run(projection=["projectNAME","pageno"],distinct=True)
This show the following error,
TypeError: Unknown configuration option ('distinct')
Why so?

It looks like the documentation for run may be incorrect. It isn't possible to provide the distinct option there. Instead, try specifying what you want inside the GQL string:
pages = db.GqlQuery('SELECT DISTINCT pageno FROM Line WHERE projectNAME=:1', projectName).run()
Note that here the only projection specified is pageno. It is not possible to specify a projection that is used in an equality (note you know what the projectNAME is already because you specified it).