How to export result from Unix executable? - python

I'm running deep learning inference from this great image quality assessment library:
./predict \
--docker-image nima-cpu \
--base-model-name MobileNet \
--weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 \
--image-source $(pwd)/src/tests/test_images
Running predict, which is a Unix Executable, returns the following sample output:
[
{
"image_id": "42039",
"mean_score_prediction": 5.154480201676051
},
{
"image_id": "42040",
"mean_score_prediction": 4.297615758532629
},
{
"image_id": "42041",
"mean_score_prediction": 5.450399260735139
},
{
"image_id": "42042",
"mean_score_prediction": 5.163813116261736
},
{
"image_id": "42044",
"mean_score_prediction": 5.728919437354534
}
]
I'm trying to save this output, ideally to json, but don't know how. Appending --output $(pwd)/src/out.json, or any variants of output, doesn't return anything.
Do Unix exe's have a default flag for exporting data? Or is there a way to view all the flag options in this exe?

Looks like predict is a bash script. There's no rule on how the arguments should work. I would just pipe your output to a file like this:
./predict \
--docker-image nima-cpu \
--base-model-name MobileNet \
--weights-file $(pwd)/models/MobileNet/weights_mobilenet_technical_0.11.hdf5 \
--image-source $(pwd)/src/tests/test_images
> output.json
The last part tells your terminal to send stdout (what would normally show in your screen) to the mentioned file, either creating it or overwriting it.

Related

How to configure VS Code to use Cmder?

I'm using VS code as my IDE for Python (anaconda environment). So I configured cmder as my terminal and I tried to test a simple print statement by clicking on the run button (green triangle). I got the following error:
My JSON settings looks like this:
{
"terminal.integrated.shell.windows": "Cmder.exe",
"terminal.integrated.env.windows": {
"CMDER_ROOT": "C:\\Users\\EK\\Cmder.exe"
},
"terminal.integrated.shellArgs.windows": [
"/k",
"%CMDER_ROOT%\\vendor\\bin\\vscode_init.Cmder"
],
"python.pythonPath": "C:\\ProgramData\\Anaconda3\\python.exe",
"atomKeymap.promptV3Features": true,
"editor.multiCursorModifier": "ctrlCmd",
"editor.formatOnPaste": true,
"kite.showWelcomeNotificationOnStartup": false,
"workbench.colorCustomizations": {
"terminal.background": "#1D2021",
"terminal.foreground": "#09c0f8",
"terminalCursor.background": "#A89984",
"terminalCursor.foreground": "#A89984",
"terminal.ansiBlack": "#1D2021",
"terminal.ansiBlue": "#0D6678",
"terminal.ansiBrightBlack": "#665C54",
"terminal.ansiBrightBlue": "#0D6678",
"terminal.ansiBrightCyan": "#8BA59B",
"terminal.ansiBrightGreen": "#95C085",
"terminal.ansiBrightMagenta": "#8F4673",
"terminal.ansiBrightRed": "#FB543F",
"terminal.ansiBrightWhite": "#FDF4C1",
"terminal.ansiBrightYellow": "#FAC03B",
"terminal.ansiCyan": "#fbfdfc",
"terminal.ansiGreen": "#95C085",
"terminal.ansiMagenta": "#8F4673",
"terminal.ansiRed": "#FB543F",
"terminal.ansiWhite": "#A89984",
"terminal.ansiYellow": "#FAC03B"
},
"workbench.colorTheme": "Monokai Dimmed",
"workbench.editor.decorations.colors": true,
"workbench.preferredHighContrastColorTheme": "Visual Studio Light",
"python.formatting.provider": "none",
"terminal.integrated.automationShell.linux": "",
"terminal.integrated.scrollback": 100000,
"workbench.editorAssociations": [
{
"viewType": "jupyter.notebook.ipynb",
"filenamePattern": "*.ipynb"
}
],
"terminal.integrated.env.windows": {}
}
I have no clue why things are going wrong? Can someone please help me with this?
Please find the absolute path of the executable file of "cmder", and use this path in "settings.json":
"settings.json":
"terminal.integrated.shell.windows": "..:\\cmder\\Cmder.exe",
Run:
In addition, please check whether the "cmder" terminal can be used outside of VS Code.
Reference: Integrated Terminal in VS Code.

How to extract "$" by jsonpath?

I'd like to extract an elememt by "$". But it retrieves nothing (the 1st call of main.py). Does anybody know what is wrong? Thanks.
$ cat data.json
{
"id": {
"$": {
"view": "all",
"id": "sec4",
"role": "materials-methods"
}
}
}
$ cat main.py
#!/usr/bin/env python
# vim: set noexpandtab tabstop=2 shiftwidth=2 softtabstop=-1 fileencoding=utf-8:
import sys
import json
json_data = json.load(sys.stdin)
import jsonpath_rw_ext
res = jsonpath_rw_ext.match(sys.argv[1], json_data)
for match in res:
print match.keys()
$ < data.json ./main.py '$."$"'
$ < data.json ./main.py '$."id"'
[u'$']
The right jsonpath expression to use would be '$.id.$'.
Note: Please don't use Python2 for new code and migrate existing Python2 code to Python3
Try this one
$[?(!#.$)]
This ignores all the nodes containing $

How to debug Django custom management command using VS Code

I am trying to debug a custom management command using Visual Studio Code.
For this I have gone through the official VS Code tutorials for working with Python and Django, and I have managed to get debugging to work while following these tutorials.
VS Code Python tutorial /
VS Code Django tutorial
The problem is that for a Python script (no Django), the debugger works because I run a specific file (by pressing f5) while the file's tab is open. Django debugging works because VS Code knows when a browser request causes my Django app to hit a break-point I entered in VS Code.
But a custom management command is run differently. For this I type the following in the console:
python manage.py name_of_management_command
How do I debug this in VS Code?
While writing this question I came up with a solution myself.
In the VS Code launch.json file (which contains the settings for the VS Code Django debugger) contains the following entry, by default:
"args": ["runserver", "--noreload", "--nothreading"]
I changed this to:
"args": ["name_of_management_command"]
Then start the debugger (press f5), and I am debugging my custom management command
Rik's answer is correct, but requires changing the launch config for every management command, or creating multiple launch config entries.
To create one entry that can debug all management commands, even the ones you still have to write, add the following config to your launch.json:
{
"name": "Django MGMT Command",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/manage.py",
"args": [
"${fileBasenameNoExtension}"
]
}
This works because the name of the management command is the name of the file in which it's defined, without the .py extension. Or ${fileBasenameNoExtension} for short.
See https://code.visualstudio.com/docs/editor/variables-reference for other variables you can use in your launch config.
#Rik and #jrial are missing an important argument, which is justMyCode, suppose you want to do a custom command that use startprojectapp and you want to see the code behind it to understand how to pass the arguments. You need to specify that argument for being able to enter there.
Also, I prefer to select the custom command from a list, instead of having to create one configuration to each file (#Rik solution), or require having the file with focus (#jrial solution).
A complete configuration can be found here.
Configuration to debug any command added to option list:
{
"name": "Django Command",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/manage.py",
"args": [
"${input:variableID}",
"restaurants"
],
"justMyCode": false
}
],
"inputs": [{
"id": "variableID",
"description": "Select client or server",
"type": "pickString",
"options": ["createsuperuser", "startapp", "A CUSTOM COMMAND"],
"default": "createsuperuser"
}]
Configuration to debug the open command file: (#jrial solution modified)
{
"name": "Django Current Custom Command",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/manage.py",
"args": [
"${fileBasenameNoExtension}"
],
"justMyCode": false
}
if you don't want to go through third party libraries code, omit the variable.
Configuration to debug the open specific custom_command.py file:(#Rik solution modified)
{
"name": "Django Current Custom Command",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/manage.py",
"args": [
"custom_command",
"THIS CAN BE A REQUIRED ARGUMENT",
],
"justMyCode": false
}

Why is python assuming my path is the project root, which is two directory levels up?

I'm doing some learning python tutorials on Lynda. The tutorial has the script creating the files in the same directory as the script, as one would expect.
For whatever reason, though, my installation is creating the file and looking to read the file (using the path object) in the project root, or two directories up.
The script I am running is C:\Users\user\Development\Ex_Files_Learning_Python\Exercise Files\Ch4
The script looks like this:
import os
from os import path
import datetime
from datetime import date, time, timedelta
import time
def main():
# Print the name of the OS
print(os.name)
# Check for item existence and type
print("Item exists: " + str(path.exists("textfile.txt")))
print("Item is a file: " + str(path.isfile("textfile.txt")))
print("Item is a directory: " + str(path.isdir("textfile.txt")))
# Work with file paths
print("Item path" + str(path.realpath("textfile.txt")))
print("Item path and name: " + str(path.split(path.realpath("textfile.txt"))))
# Get the modification time
# Calculate how long ago the item was modified
if __name__ == "__main__":
main()
Its output is
nt
Item exists: False
Item is a file: False
Item is a directory: False
Item pathC:\Users\user\Development\Ex_Files_Learning_Python\textfile.txt
Item path and name: ('C:\\Users\\user\\Development\\Ex_Files_Learning_Python', 'textfile.txt')
So as you can see, it assumes its path is the project root, two directories up. I had the same problem in my previous exercise, creating a file. When i used open() on a file object, it created the file two directories up in the root of the project.
Any pointers are appreciated.
Update: I've established that this is happening because I'm using the VSCode terminal. How do I instruct the VSCode terminal to run the program from the cwd of the file i'm editing and debugging, rather than the project root?
For the record, here's my launch.json for the debugger
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File (Integrated Terminal)",
"type": "python",
"request": "launch",
"program": "${file}",
"console": "integratedTerminal"
},
{
"name": "Python: Remote Attach",
"type": "python",
"request": "attach",
"port": 5678,
"host": "localhost",
"pathMappings": [
{
"localRoot": ".",
"remoteRoot": "."
}
]
},
{
"name": "Python: Module",
"type": "python",
"request": "launch",
"module": "enter-your-module-name-here",
"console": "integratedTerminal"
},
{
"name": "Python: Django",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/manage.py",
"console": "integratedTerminal",
"args": [
"runserver",
"--noreload",
"--nothreading"
],
"django": true
},
{
"name": "Python: Flask",
"type": "python",
"request": "launch",
"module": "flask",
"env": {
"FLASK_APP": "app.py"
},
"args": [
"run",
"--no-debugger",
"--no-reload"
],
"jinja": true
},
{
"name": "Python: Current File (External Terminal)",
"type": "python",
"request": "launch",
"program": "${file}",
"console": "externalTerminal"
}
]
}
According to https://code.visualstudio.com/docs/python/settings-reference :
python.terminal.executeInFileDir
false
Indicates whether to run a file in the file's directory instead of the current folder.
So presumably just set python.terminal.executeInFileDir to true.
If you're running the script from the terminal, you're probably running:
python "Exercise Files\Ch4\my_script.py"
Or something similar.
Instead, change folders first and then run it on the terminal:
cd "Exercise Files\Ch4"
python my_script.py
Or, to avoid all this manual directory switching, explicitly change the working directory in your script.
This is my fault, as I did not clarify that I was trying to modify the debugger terminal in VSCode. #Adam Smith's answer is right for the majority of cases dealing with the standard terminal.
If, however, you're trying to run the file from the debugger, the answer is to set "cwd" to an empty string, ie "" in launch.json (current debugger configuration)
https://code.visualstudio.com/docs/python/debugging#_cwd
What you need to use is the __file__ built-in variable that Python associates with the current Python file name.
see what does the __file__ variable mean/do?
This should work if you're on Python 3.
pathlib.Path is on Python 3 and serves to manipulate file paths.
from pathlib import Path
myfilepath = Path(Path(__file__).resolve()).parent / "textfile.txt"
^ ^
find the currrent script location
^ get its parent directory
^ add your file name
and this the script
import os
from os import path
import datetime
from datetime import date, time, timedelta
import time
def main():
# Print the name of the OS
print(os.name)
from pathlib import Path
myfilepath = Path(Path(__file__).resolve()).parent / "textfile.txt"
# Check for item existence and type
print("Item exists: " + str(path.exists(myfilepath)))
print("Item is a file: " + str(path.isfile(myfilepath)))
print("Item is a directory: " + str(path.isdir(myfilepath)))
# Work with file paths
print("Item path" + str(path.realpath(myfilepath)))
print("Item path and name: " + str(path.split(path.realpath(myfilepath))))
# Get the modification time
# Calculate how long ago the item was modified
main()
output:
posix
Item exists: True
Item is a file: True
Item is a directory: False
Item path/Users/jluc/kds2/wk/explore/textfile.txt
Item path and name: ('/Users/jluc/kds2/wk/explore', 'textfile.txt')
If you're on Python 2, you get the same results by using this instead:
(os.path.join("foo", "bar") will return foo\bar on Windows or foo/bar on Linux/macOs.
#from pathlib import Path
myfilepath = os.path.join(os.path.dirname(__file__), "textfile.txt")
^ get parent directory
^ append your filename

JSON output via CGI

I'm trying to execute a Python script via CGI. There is no problem with the execution of the script. It's the output or display where I am not getting the desired format. I'm running this CGI script with Apache2.
/usr/lib/cgi-bin/info.cgi:
#!/usr/bin/python
print "content-type: text/html\n\n";
import json
from napalm import get_network_driver
driver = get_network_driver('ios')
hub2 = driver('10.0.0.120', 'admin', 'admin')
hub2.open()
ios_output = hub2.get_facts();
print json.dumps(ios_output, indent=5)
hub2.close()
Output looks like this:
{ "os_version": "Linux Software (ADVENTERPRISEK9-M), Version 15.5(3)S3, SOFTWARE", "uptime": 10080, "interface_list": [ "Ethernet0/0", "Ethernet0/1", "Ethernet0/2", "Ethernet0/3" ], "vendor": "Cisco", "serial_number": "67109072", "model": "Unknown", "hostname": "R13", "fqdn": "R13.lab1.com" }
But the desired output after running this script via CLI should look like this:
"os_version": "Linux Software (ADVENTERPRISEK9-M), Version 15.5(3)S3,
SOFTWARE",
"uptime": 10920,
"interface_list": [
"Ethernet0/0",
"Ethernet0/1",
"Ethernet0/2",
"Ethernet0/3"
],
"vendor": "Cisco",
"serial_number": "67109072",
"model": "Unknown",
"hostname": "R13",
"fqdn": "R13.lab1.com"
Any suggestions how to get the desired output while executing info.cgi?
Fix
Changed the content type "/Json"

Categories