We Keep Coding

How to export result from Unix executable?

I'm running deep learning inference from this great image quality assessment library:
./predict \
--docker-image nima-cpu \
--base-model-name MobileNet \
--weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 \
--image-source $(pwd)/src/tests/test_images
Running predict, which is a Unix Executable, returns the following sample output:
[
{
"image_id": "42039",
"mean_score_prediction": 5.154480201676051
},
{
"image_id": "42040",
"mean_score_prediction": 4.297615758532629
},
{
"image_id": "42041",
"mean_score_prediction": 5.450399260735139
},
{
"image_id": "42042",
"mean_score_prediction": 5.163813116261736
},
{
"image_id": "42044",
"mean_score_prediction": 5.728919437354534
}
]
I'm trying to save this output, ideally to json, but don't know how. Appending --output $(pwd)/src/out.json, or any variants of output, doesn't return anything.
Do Unix exe's have a default flag for exporting data? Or is there a way to view all the flag options in this exe?

Looks like predict is a bash script. There's no rule on how the arguments should work. I would just pipe your output to a file like this:
./predict \
--docker-image nima-cpu \
--base-model-name MobileNet \
--weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 \
--image-source $(pwd)/src/tests/test_images
> output.json
The last part tells your terminal to send stdout (what would normally show in your screen) to the mentioned file, either creating it or overwriting it.

How to configure VS Code to use Cmder?

I'm using VS code as my IDE for Python (anaconda environment). So I configured cmder as my terminal and I tried to test a simple print statement by clicking on the run button (green triangle). I got the following error:
My JSON settings looks like this:
{
"terminal.integrated.shell.windows": "Cmder.exe",
"terminal.integrated.env.windows": {
"CMDER_ROOT": "C:\\Users\\EK\\Cmder.exe"
},
"terminal.integrated.shellArgs.windows": [
"/k",
"%CMDER_ROOT%\\vendor\\bin\\vscode_init.Cmder"
],
"python.pythonPath": "C:\\ProgramData\\Anaconda3\\python.exe",
"atomKeymap.promptV3Features": true,
"editor.multiCursorModifier": "ctrlCmd",
"editor.formatOnPaste": true,
"kite.showWelcomeNotificationOnStartup": false,
"workbench.colorCustomizations": {
"terminal.background": "#1D2021",
"terminal.foreground": "#09c0f8",
"terminalCursor.background": "#A89984",
"terminalCursor.foreground": "#A89984",
"terminal.ansiBlack": "#1D2021",
"terminal.ansiBlue": "#0D6678",
"terminal.ansiBrightBlack": "#665C54",
"terminal.ansiBrightBlue": "#0D6678",
"terminal.ansiBrightCyan": "#8BA59B",
"terminal.ansiBrightGreen": "#95C085",
"terminal.ansiBrightMagenta": "#8F4673",
"terminal.ansiBrightRed": "#FB543F",
"terminal.ansiBrightWhite": "#FDF4C1",
"terminal.ansiBrightYellow": "#FAC03B",
"terminal.ansiCyan": "#fbfdfc",
"terminal.ansiGreen": "#95C085",
"terminal.ansiMagenta": "#8F4673",
"terminal.ansiRed": "#FB543F",
"terminal.ansiWhite": "#A89984",
"terminal.ansiYellow": "#FAC03B"
},
"workbench.colorTheme": "Monokai Dimmed",
"workbench.editor.decorations.colors": true,
"workbench.preferredHighContrastColorTheme": "Visual Studio Light",
"python.formatting.provider": "none",
"terminal.integrated.automationShell.linux": "",
"terminal.integrated.scrollback": 100000,
"workbench.editorAssociations": [
{
"viewType": "jupyter.notebook.ipynb",
"filenamePattern": "*.ipynb"
}
],
"terminal.integrated.env.windows": {}
}
I have no clue why things are going wrong? Can someone please help me with this?

Please find the absolute path of the executable file of "cmder", and use this path in "settings.json":
"settings.json":
"terminal.integrated.shell.windows": "..:\\cmder\\Cmder.exe",
Run:
In addition, please check whether the "cmder" terminal can be used outside of VS Code.
Reference: Integrated Terminal in VS Code.

Permission error when initialising RotatingFileHandler logging in Flask app factory

There are a lot of similar questions on SO, but I couldn't one that exactly fit my situation.
I'm setting up logging in my app factory like so:
__init__.py
import os
from flask import Flask
from logging.config import dictConfig
LOG_FOLDER = f'{os.path.dirname(os.path.abspath(__file__))}/logs'
def create_app(test_config=None):
# Setup logging
# Make log folder if it doesn't exist
try:
os.makedirs(LOG_FOLDER)
print("created logs folder")
except OSError:
print("log folder already exists")
pass
dictConfig({
"version": 1,
"handlers": {
"fileHandler": {
"class": "logging.handlers.RotatingFileHandler",
"formatter": "myFormatter",
"filename": f"{LOG_FOLDER}/flask.log",
"maxBytes": 500,
"backupCount": 5
},
"werkzeugFileHandler": {
"class": "logging.handlers.RotatingFileHandler",
"formatter": "myFormatter",
"filename": f"{LOG_FOLDER}/werkzeug.log",
"maxBytes": 500,
"backupCount": 5
},
"console": {
"class": "logging.StreamHandler",
"formatter": "myFormatter"
}
},
"loggers": {
APP_NAME: {
"handlers": ["fileHandler", "console"],
"level": "INFO",
},
"werkzeug": {
"level": "INFO",
"handlers": ["werkzeugFileHandler", "console"],
}
},
"formatters": {
"myFormatter": {
"format": "[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s"
}
}
})
# create and configure the app
app = Flask(__name__, instance_relative_config=True)
<remainder omitted>
And accessing the logger in my other classes like so:
foo.py
from flask import Flask
from definitions import APP_NAME
app = Flask(APP_NAME)
app.logger.info("blah")
But when it comes time for RotatingFileHandler to rename flask.log to flask.log.1, I get this error I've seen in numerous SO posts:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\user\\project_root\\logs\\flask.log' -> 'C:\\Users\\user\\project_root\\logs\\flask.log.1'
I am running the flask server locally in development mode, using the flask run cli command.
Another thing to note is, when the flask server is running, I am unable to modify (i.e. delete or rename) the log files manually, so it seems the mere act of having the server running is locking the files from modification? Is it wrong to initialise the logging in __init__.py, or is there something I'm missing?

I think this is a duplicated question as PermissionError when using python 3.3.4 and RotatingFileHandler. But just reposting my answer:
Spent half a day on this as non previous answer resolved my issue.
My working solution is to use https://pypi.org/project/concurrent-log-handler/ instead of RotatingFileHandler. In multiple thread scenarios like Flask app, PermissionError will be raised when we rotate the log file that reaches maximum size.
Install pypiwin32 to get rid of No Module name win32con error.
Thanks go to https://www.programmersought.com/article/43941158027/

Try changing the delay parameter to True in your handler (in my case I used TimeRotatingFileHandler).
https://stackoverflow.com/a/69378029/15036810

FLASK : plotly : Error: module 'index' has no attribute 'app.server'

the context
I am debugging an application using Visual Studio Code (VSCode).
The application relies mainly on https://plot.ly, https://palletsprojects.com/p/flask, https://pandas.pydata.org/ and https://numpy.org/
Breakpoints ARE NOT hit!
The breakpoints ARE NOT hit when I am using the launch.json (See [1])
I can debug with this launch.json (See [2]) but the debugger does not stops at the breakpoint !
I would like VSCode to stop on my breakpoints when necessary
**What is the correct configuration for launch.json to hit the breakpoints? **
Thank you for the time you are investing helping me!
the hierarchy of the project
launch.json
index.py See [4]
app.py See [3]
pages
index.py
transactions.py
launch.json is described here below [1]
the issue : Error: module 'index' has no attribute 'app.server'
The Error message is displayed after clicking on 'Start debugging > F5' = Error: module 'index' has no attribute 'app.server'
I tried dozens of ways to set the "FLASK_APP": "index:app.server" but they generate diverse error messages :
"FLASK_APP": "index:app.server" generates this error Error: A valid Flask application was not obtained from "index:app".
"FLASK_APP": "index.py" generates this error Error: Failed to find Flask application or factory in module "index". Use "FLASK_APP=index:name to specify one.
for information : gunicorn command (working)
here is a command available in azure-pipelines.yml running the plotly app :
gunicorn --bind=0.0.0.0 --timeout 600 index:app.server
attached files
[1] launch.json - non working
{
"version": "0.2.0",
"configurations": [
{
"name": "Flask",
"type": "python",
"request": "launch",
"module": "flask",
"env": {
"FLASK_APP": "index:app.server",
"FLASK_ENV": "development",
"FLASK_DEBUG": "1",
"FLASK_RUN_PORT": "8052"
},
"args": [
"run",
"--no-debugger",
"--no-reload"
],
"jinja": true
}
]
}
[2] launch.json - working but breakpoints are not hit
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File",
"type": "python",
"request": "launch",
"program": "${workspaceRoot}\\index.py",
"console": "integratedTerminal"
}
]
}
[3] webapp.py
# -*- coding: utf-8 -*-
import dash
app = dash.Dash(
__name__, meta_tags=[{"name": "viewport",
"content": "width=device-width, initial-scale=1"}]
)
server = app.server
app.config.suppress_callback_exceptions = True
index.py - root of the application
# -*- coding: utf-8 -*-
import dash_html_components as html
import dash_core_components as dcc
from webapp import app
from dash.dependencies import Input, Output
from pages import (
transactions, index)
# Describe the layout/ UI of the app
app.layout = html.Div([
dcc.Location(id="url", refresh=False),
html.Div(id="page-content")
])
# Update page
#app.callback(Output("page-content", "children"),
[Input("url", "pathname")])
def display_page(pathname):
if pathname == "/dash/index":
return index.layout
if pathname == "/dash/transactions":
return transactions.layout
else:
return index.layout
if __name__ == "__main__":
app.run_server(debug=True, port=8051)

Your [1] example isn't working because you set FLASK_APP to index:app.server which tries to find an attribute named app.server on the index module. Attribute names can't have a dot (you can verify this by importing that module and trying out getattr(index, "app.server")). You should be able to make FLASK_APP simply say index to have it work.
See the Flask documentation on app discovery for more details.

Replace string in Python and open browser with result

I am trying to build a plugin for the program Sublimetext2.
It uses plugins coded with Python. I have no Python knowledge at all but from looking at existing plugins and my PHP knowledge here is what I need help with...
this is the start of the Python file so far
import sublime, sublime_plugin
import webbrowser
settings = sublime.load_settings('openonserver.sublime-settings')
settings.get('file_path_prefix')
settings.get('server_url')
class OpenonServerCommand(sublime_plugin.TextCommand):
def run(self,edit):
file_path = self.view.file_name()
What I need to do though take the value of the settings
file_path will be the path to the file I am running this on so lets say...
E:\Server\htdocs\mytest_project_\some\folder_\test.php
The settings
file_path_prefix will be E:\Server\htdocs\ and
server_url will be http://localhost/
I need to see if file_path_prefix exist in file_path if it does,
I need to replace the E:\Server\htdocs\ with the http://localhost/ and replace all \ to / and then store this new path in a variable
so...
E:\Server\htdocs\mytest_project_\some\folder_\test.php would become
http://localhost/mytest_project_/some/folder_/test.php
I then need to send this to the browser.
Any help is greatly appreciated

Use
os.system("path_to_browser url")
To run any external program. I also recomend to take a look at this comment

Ok after many hours (I hate Python now) my solution (i'm very not impressed) but it partially works
#Context.sublime-menu
[
{ "command": "openserver", "caption": "Open on Server" }
]
#Default (Windows).sublime-keymap
[
{ "keys": ["ctrl+shift+b"], "command": "openserver" }
]
#Main.sublime-menu
[
{
"caption": "Tools",
"mnemonic": "T",
"id": "tools",
"children":
[
{ "command": "openserver", "caption": "Open on Server" }
]
}
]
#Openserver.sublime-commands
[
{
"caption": "Open file on Server in Browser",
"command": "openserver"
}
]
#Openserver.sublime-settings
{
"file_path_prefix": "E:/Server/htdocs",
"url_prefix": "http://localhost"
}
Main file
#openserver.py
import sublime, sublime_plugin
import os
import webbrowser
import re
import os2emxpath
import logging
import sys
class OpenserverCommand(sublime_plugin.TextCommand):
def run(self,edit):
file_path = self.view.file_name()
settings = sublime.load_settings('Openserver.sublime-settings')
file = os2emxpath.normpath(file_path)
url = re.sub(settings.get('file_path_prefix'), settings.get('url_prefix'), file)
#logging.warning(url)
#webbrowser.open_new(url)
if sys.platform=='win32':
os.startfile(url)
elif sys.platform=='darwin':
subprocess.Popen(['open', url])
else:
try:
subprocess.Popen(['xdg-open', url])
except OSError:
logging.warning(url)
Now when I say it works but it partially doesn't, it does take the file name, replaces my path and server URL from a settings file and then does launch a browser with the proper URL
Except, in Sublimetext2 when you run this on a .py file or any file that you do not have set to be able to open in a web browser, then instead of opening the file in the web browser it will give the windows popup asking to set a default program to open the file, very annoying!