We Keep Coding

How to include the current timestamp in spacy's projects.yml scripts?

I'm using spacy projects for a pipeline for training a dependancy parser pipeline for a custom Bulgarian language model. I want to retrain and reevaluate mutiple times with different datasets and tokenizer rulesets. For that, I need to put a timestamp on the evaluation metrics.
I do this by running spacy evaluate ./models/model-best test.spacy -dp ./visualizations/ --output metrics-$(date +"%FT%H%M").json to get a metrics file with the current time included in the name.
This scrtipt, however, does not work when it's part of the project.yml file. I tried different ways to escape the " and % characters, but nothing led to success. Any help will be much appriciated.

In theory, you could run this command from the project with sh -c if you can figure out the YAML escaping (I couldn't quickly), so if I wanted to do this, I'd just move the command into a shell script and call that from the project instead.

Couldn't open file yolov3_custom_last.weights when trying to run darknet detection

I've been trying to use YOLO (v3) to implement and train an object detection of Tank with OpenImage dataset.
I have tried to get help from this tutorial and my code pretty much looks like it.
Also I'm using Google Colab and Google Drive services.
everything is going fine through my program. But I hit an error at the final step when I'm running the darknet to train detection.
!./darknet detector train "data/obj.data" cfg/yolov3_custom.cfg "darknet53.conv.74" -dont_show
after 100 iterations, when it's trying to save the progress in the backup folder I've addressed in obj.data file, I get the following error:
Saving weights to /content/drive/My\Drive/YOLOv3/backup/yolov3_custom_last.weights
Couldn't open file: /content/drive/My\Drive/YOLOv3/backup/yolov3_custom_last.weights
at first, I thought I made a mistake in using the address; So, I tried checking the address by using ls command
!ls /content/drive/My\Drive/YOLOv3/backup/
and the result was an empty folder (However, not an error meaning I've written the address correctly and that it is accessable in my google drive).
Here are contents of data.object file:
classes = 1
train = data/train.txt
valid = data/test.txt
names = data/obj.names
backup = /content/drive/My\ Drive/YOLOv3/backup
also I've made required changes in config file so I don't think that the problem is about that. But just to make sure here are the changes I've made in my yolov3.cfg file:
Fist of all we will comment lines 3 and 4 (batch, subdivisions) to unset testing mode
We will uncomment lines 6 and 7(batch, subdivisions) to set to training mode
We change our max_batches value to 2000 * number_of_classes (if there's one class like our case, set to 4000)
We change our step tuple-like values to 80%, 90% value of our max_baches value. In this case it will be 3200, 3600.
For all YOLO layers and convoloutional layers before them, changed the classes value to number of classes, In this case 1, and change the value of filters according to following formula(In this case, 18)
Formula for conv layers filters value: (number_of_classes + 5)*3
I searched the error and found this issue on Github.
However, I tried the following methods recommended there and the problem is still the same:
Removing and recreating the backup folder
Tried to adding the line backup = backup in my yolo.data file in folder .cfg but there was no such file in cfg folder.
Creating an empty yolov3_custom_last.weights in backup folder
The other solutions mentioned in this issue was about when you are running YOLO on your PC and not google Colab.
Also, here my tree structure of the folder YOLOv3 which is stored in my Google Drive My Drive(main folder).
YOLOv3
darknet53.conv.74
obj.data
obj.names
Tank.zip
yolov3.weights
yolov3_custom.cfg
yolov3_custom1.cfg.txt
So, I'm kinda stuck and I have no idea what could fix this. I'd appreciate some help.

I have solved the problem with changing the local drive address with ln command.
The problem wasn't from my code rather from the way way yolov3 developers were handling space in directory addresses! Apparently as much as I figured in their docs, they are not handling space in directory quite well.
So I created a virtual address which does not have space like My Drive has.
P.S: As you know My Drive folder is already there in your google drive so you can't actually rename it.
Here is the code you can use to achieve this:
!ln -s /content/drive/My\ Drive/ /mydrive

I was getting the same error on my Windows 10, and I think I managed to fix it. All I had to do was move my "weights" folder (with my yolov3.weights inside), closer to the .exe file, which was in the "x64" folder, in my case. After that, the error stopped appearing and the app was able to predict the test image normally.

I want to transfer the image file stored in the oscilloscope to the pc and read it on the pc with Python

I want to transfer images stored in an oscilloscope to a PC with Python.
We have saved the image, but we are inquiring about how to transfer the image file to the PC.
attach my code
inst=rm.open_resource("blahblah::INSTR")
inst.write("*IDN?")
print(inst.read())
# image capture (Success)
inst.write(':DISK:SIM "IMAGE",BMP,SCR,ON,INVERT')
#???????????????????????????????????????????
inst.write(':MMEM:STOR:SCR "myscreen.png"')
inst.write('MMEM:DATA? "C:\\Workspace\\", B')
inst.query("*OPC?")
data=inst.read_raw() ## problem ##
#???????????????????????????????????????????
There seems to be a problem in the code between the question marks.
Especially in the read_raw section time error occurs.
There is also a question as to whether the command is supported.
The user guide does not seem to provide the export command.
I did not find any command when I searched for the word transfer or export.
The oscilloscope I use is the 54831D MOS.
If you know anybody, please help me.
Thank you for reading.

Creating log directory in tensorboard

I am trying to learn how to use tensorboard and I would like to have it run in my program. I do not understand how to create a log directory. These are the lines I have for running tensorboard.
summary_writer = tf.train.SummaryWriter('/tensorflow/logdir', sess.graph_def)
tensorboard --logdir=tensorflow/logdir
The error message that I got was
Cannot assign to operator

This line needs to be in your code (the python script), as it seems you put it:
summary_writer = tf.train.SummaryWriter('/tensorflow/logdir',
sess.graph_def)
This line, however, you have to call from linux (and not from within the script):
tensorboard --logdir=tensorflow/logdir
However, there is a lot more you need to do, before tensorboard really runs:
How to create a Tensorflow Tensorboard Empty Graph

The tutorial might disclose not very clear on the TensorFlow official website
I have stuck in the same issue before
But in order not to confuse you, i still use it as a guide here
First Part (lines of codes in .py file)
Just skip to class tf.train.SummaryWriter in official guide
First, you need this lines of code in your .py file to create a dataflow graph
In tensorflow, a session is where the graph been created
#...create a graph...
# Launch the graph in a session.
sess = tf.Session()
Then, you also need to type in these lines into your code
# Create a summary writer, add the 'graph' to the event file.
writer = tf.train.SummaryWriter(< directory name you create>, sess.graph)
The logs folder will be generated in the directory you assigned after the .py file you created is executed
Here is the sample code you can use
Second Part (lines of code in your linux terminal)
In your Linux terminal window, type in
tensorboard --logdir="path of your log file"
It will link to your log file automatically
Last step (key in link into your browser)
After key in
tensorboard --logdir="path of your log file"
It will generate a http link ,ex http://666.6.6.6:6006
Copy the http link into your web browser
Enjoy it!
Be careful
Do not go to the directory where the log file is before key in the line of code above
It might miss the log file
This youtube video will explain more explicitly about this at 9:40
You also can take a look of how to launch tensorboard on official guide
Hope you can show your data graph ASAP~

for --colab
%tensorflow_version 2.x
from torch.utils.tensorboard import SummaryWriter
tensorboard_logdir = '/content/tensorboard/'
writer = SummaryWriter(tensorboard_logdir)

Multiple Event files on Tensorboard

I am using Tensorboard to show the training results of the code using Tensorflow (0.7). The previous Tensorflow version had issue for multiple event files: when I run my local server using $ tensorboard --logdir=./tmp/, it shows up error if there are more than 1 event files. It seems that the latest version (0.7) does not show up the same error for multiple event files, but it still shows the overlapped curves for multiple event files on Tensorboard. I wonder how to solve this problem. Thanks!

Training my own networks, I am writing summaries in different subfolders like /tmp/project/train and /tmp/project/eval. If you start TensorBoard using
tensorboard --logdir=/tmp/project/
you will still receive multiple graphs from each event file in the subfolders at once, as you mentioned. To see seperate graphs you can start TensorBoard from the desired subfolder:
tensorboard --logdir=/tmp/project/train/

I second ArnoXf's answer. You should use different subfolders for each experiment and assuming the logging root is /tmp start tensorboard using:
tensorboard --logdir=/tmp/
If you want to display just a single graph you can either pass that directory to your tensorboard call as described in ArnoXf's answer.
However, with the above call you can also select your graph directly in tensorboard, i.e., deactivate all others. The same way you can also compare individual runs as shown in the following screenshot. In my opinion, this is usually the preferred option, as it gives you more flexibility.
A detailed example can be found here.