We Keep Coding

How to edit CPython's Lib/importlib/_bootstrap.py?

I would like to play with CPython's Lib/importlib/_bootstrap.py file, but my edits in my current CPython installation on MacOS (installed with the Homebrew package manager) have no effect when I import a module.
For instance if I add the print("hello", file=sys.stderr) statement at the beginning of the body of the _init_module_attrs function:
$ vi /usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/_bootstrap.py
and then import a module:
$ python3 -c "import pathlib"
nothing is printed.

importlib._bootstrap is a frozen module. To update it, the importlib module should be recompiled:
$ cd /usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/Python/
$ make regen-importlib
$ make

#!/usr/bin/env not working as environment

I am working on MacOS to learn the environment
#map_it.py - launches a map in the browser using an address from the command line or clipborad
#!/usr/bin/env python
import webbrowser, sys
if len(sys.argv) > 1:
#Get the address from the command line
address = "".join(sys.argv[1:])
print(address)
#todo:get the address from the clipboard
I set shebang at top of the script which did not work as I intended
$ map_it.py test
-bash: map_it.py: command not found
$ map_it test
-bash: map_it: command not found
After I changed #!/usr/bin/env python to #!/usr/bin/env python3 and #!/usr/bin/env,
It report the identical error.
How could I fix such a bug?

You need to do either:
$ ./map_it.py test
or
$ python map_it.py test
...otherwise you are indicating the script is a system installed command.

Sending python command line arguments to IPython

How can we start ipython REPL and instruct it to pass some command-line arguments to the underlying python interpreter?
For example, we can open a python REPL with increased verbosity by using
python -v
But I could not see how to pass that flag through when opening IPython.

I'd say the best way to do that would be to explicitly launch ipython with python:
python /usr/bin/ipython
as the ipython executable is just a python script ; or you can launch ipython by telling python to load the ipython library:
python -m IPython.frontend.terminal.ipapp
and then you can add all the native python arguments:
python -v /usr/bin/ipython
python -v -m IPython.frontend.terminal.ipapp
HTH

You could write your own ipython shebang script.
Here I copied my ipython script and added the -v
#!/usr/local/bin/python3.5 -v
# -*- coding: utf-8 -*-
import re
import sys
from IPython import start_ipython
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(start_ipython())
Now when I execute ./vipython I get many pages of import information upon startup and shutdown.
I gather from other SO questions that I might not be able to add multiple options to such a shebang line.
How to use multiple arguments with a shebang (i.e. #!)?
So for example
#!/usr/local/bin/python3.5 -vv
works, but
#!/usr/local/bin/python3.5 -v -v
doesn't - it gives me
1008:~/mypy$ ./vipython
Unknown option: -
usage: /usr/local/bin/python3.5 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.

That -v option affects the behavior of the interpreter itself, not just the REPL. You get the extra import information regardless of whether you add the -i option.
Here's the default script that launches ipython (or at least one version of that)
1522:~/mypy$ cat /usr/local/bin/ipython3.5
#!/usr/local/bin/python3.5
# -*- coding: utf-8 -*-
import re
import sys
from IPython import start_ipython
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(start_ipython())
and from within an ipython session:
In [1153]: from IPython import start_ipython
In [1154]: start_ipython??
String form: <function start_ipython at 0xb697edac>
File: /usr/lib/python3/dist-packages/IPython/__init__.py
Definition: start_ipython(argv=None, **kwargs)
Source:
def start_ipython(argv=None, **kwargs):
"..."
from IPython.terminal.ipapp import launch_new_instance
return launch_new_instance(argv=argv, **kwargs)
Eventually the argv are passed to an argparse parser. Ipython populates that parser with arguments derived from its config files. So you have several options for setting parameters - default config, profile config, and commandline. But all of this is after the interpreter has been launched. Some things are acted on in the same was a with an interpreter REPL, but not all (-m, but not -v).
When -v is used as zmo suggests, we see all the imports of ipython code - which are quite a few. Are you interested in those, or are you more interested in imports related to your own script?
I use ipython a lot to test answers, especially for numpy. In fact my default ipython call has the --pylab flag. But to test stand alone scripts I use plain python (often called from a terminal window in my editor). Sometimes though I'll %run a script from within ipython. That loads the module into the main workspace, making it easy to perform %timeit tests on functions.
Other ipython scripts use
#!/usr/bin/python
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point(...)
I don't write much code using this style, but I don't see how that kind initiation can pass argv on through to the interpreter. By the time the module has been loaded and start running, the interpreter is already running.
In general it looks like ipython handles options like -i, -m, -c in basically the same way as the regular python. It may doing so with its own code, rather than delegating to the interpreter. But things like -v, -O, -t apply to the interpreter, not the REPL, and aren't handled by ipython code.

python sys library command prompt

I am trying out the sys library in python. In command prompt I am using this.
>>>import sys
>>>sys.ps1 ='$'
#my own input 'print 'test print''
$print 'test print'
test print
That worked in CLI, however, when I tried to do it in a python file and run it in CLI ( python file.py did not return anything.
#!/usr/bin/python
import sys
sys.ps1= '$'
Am I missing anything in the file?

What are you trying to do? sys.ps1 will just set the prompt for the python interpreter, so as soon as it exits it's effect is gone again. When running a file, there is no command-line.
If you want to see this have an effect, try running this with python -i file.py - this will run the command, and then drop you into the interactive shell, which should have PS1 set to $ now

How to launch an EDITOR (e. g. vim) from a python script?

I want to call up an editor in a python script to solicit input from the user, much like crontab e or git commit does.
Here's a snippet from what I have running so far. (In the future, I might use $EDITOR instead of vim so that folks can customize to their liking.)
tmp_file = '/tmp/up.'+''.join(random.choice(string.ascii_uppercase + string.digits) for x in range(6))
edit_call = [ "vim",tmp_file]
edit = subprocess.Popen(edit_call,stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=True )
My problem is that by using Popen, it seems to keep my i/o with the python script from going into the running copy of vim, and I can't find a way to just pass the i/o through to vim. I get the following error.
Vim: Warning: Output is not to a terminal
Vim: Warning: Input is not from a terminal
What's the best way to call a CLI program from python, hand control over to it, and then pass it back once you're finished with it?

Calling up $EDITOR is easy. I've written this kind of code to call up editor:
import sys, tempfile, os
from subprocess import call
EDITOR = os.environ.get('EDITOR', 'vim') # that easy!
initial_message = '' # if you want to set up the file somehow
with tempfile.NamedTemporaryFile(suffix=".tmp") as tf:
tf.write(initial_message)
tf.flush()
call([EDITOR, tf.name])
# do the parsing with `tf` using regular File operations.
# for instance:
tf.seek(0)
edited_message = tf.read()
The good thing here is the libraries handle creating and removing the temporary file.

In python3: 'str' does not support the buffer interface
$ python3 editor.py
Traceback (most recent call last):
File "editor.py", line 9, in <module>
tf.write(initial_message)
File "/usr/lib/python3.4/tempfile.py", line 399, in func_wrapper
return func(*args, **kwargs)
TypeError: 'str' does not support the buffer interface
For python3, use initial_message = b"" to declare the buffered string.
Then use edited_message.decode("utf-8") to decode the buffer into a string.
import sys, tempfile, os
from subprocess import call
EDITOR = os.environ.get('EDITOR','vim') #that easy!
initial_message = b"" # if you want to set up the file somehow
with tempfile.NamedTemporaryFile(suffix=".tmp") as tf:
tf.write(initial_message)
tf.flush()
call([EDITOR, tf.name])
# do the parsing with `tf` using regular File operations.
# for instance:
tf.seek(0)
edited_message = tf.read()
print (edited_message.decode("utf-8"))
Result:
$ python3 editor.py
look a string

Package python-editor:
$ pip install python-editor
$ python
>>> import editor
>>> result = editor.edit(contents="text to put in editor\n")
More details here: https://github.com/fmoo/python-editor

click is a great library for command line processing and it has some utilities, click.edit() is portable and uses the EDITOR environment variable. I typed the line, stuff, into the editor. Notice it is returned as a string. Nice.
(venv) /tmp/editor $ export EDITOR='=mvim -f'
(venv) /tmp/editor $ python
>>> import click
>>> click.edit()
'stuff\n'
Check out the docs https://click.palletsprojects.com/en/7.x/utils/#launching-editors My entire experience:
/tmp $ mkdir editor
/tmp $ cd editor
/tmp/editor $ python3 -m venv venv
/tmp/editor $ source venv/bin/activate
(venv) /tmp/editor $ pip install click
Collecting click
Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Installing collected packages: click
Successfully installed click-7.0
You are using pip version 19.0.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(venv) /tmp/editor $ export EDITOR='=mvim -f'
(venv) /tmp/editor $ python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import click
>>> click.edit()
'stuff\n'
>>>

The PIPE is the problem. VIM is an application that depends on the fact that the stdin/stdout channels are terminals and not files or pipes. Removing the stdin/stdout paramters worked for me.
I would avoid using os.system as it should be replaced by the subprocess module.

The accepted answer does not work for me. edited_message stays the same as initial_message. As explained in the comments, this is caused by vim saving strategy.
There are possible workarounds, but they are not portable to other editors. Instead, I strongly recommend to use click.edit function. With it, your code will look like this:
import click
initial_message = "edit me!"
edited_message = click.edit(initial_message)
print(edited_message)
Click is a third-party library, but you probably should use it anyway if you are writing a console script. click to argparse is the same as requests to urllib.