I am trying to learn how to use tensorboard and I would like to have it run in my program. I do not understand how to create a log directory. These are the lines I have for running tensorboard.
summary_writer = tf.train.SummaryWriter('/tensorflow/logdir', sess.graph_def)
tensorboard --logdir=tensorflow/logdir
The error message that I got was
Cannot assign to operator
This line needs to be in your code (the python script), as it seems you put it:
summary_writer = tf.train.SummaryWriter('/tensorflow/logdir',
sess.graph_def)
This line, however, you have to call from linux (and not from within the script):
tensorboard --logdir=tensorflow/logdir
However, there is a lot more you need to do, before tensorboard really runs:
How to create a Tensorflow Tensorboard Empty Graph
The tutorial might disclose not very clear on the TensorFlow official website
I have stuck in the same issue before
But in order not to confuse you, i still use it as a guide here
First Part (lines of codes in .py file)
Just skip to class tf.train.SummaryWriter in official guide
First, you need this lines of code in your .py file to create a dataflow graph
In tensorflow, a session is where the graph been created
#...create a graph...
# Launch the graph in a session.
sess = tf.Session()
Then, you also need to type in these lines into your code
# Create a summary writer, add the 'graph' to the event file.
writer = tf.train.SummaryWriter(< directory name you create>, sess.graph)
The logs folder will be generated in the directory you assigned after the .py file you created is executed
Here is the sample code you can use
Second Part (lines of code in your linux terminal)
In your Linux terminal window, type in
tensorboard --logdir="path of your log file"
It will link to your log file automatically
Last step (key in link into your browser)
After key in
tensorboard --logdir="path of your log file"
It will generate a http link ,ex http://666.6.6.6:6006
Copy the http link into your web browser
Enjoy it!
Be careful
Do not go to the directory where the log file is before key in the line of code above
It might miss the log file
This youtube video will explain more explicitly about this at 9:40
You also can take a look of how to launch tensorboard on official guide
Hope you can show your data graph ASAP~
for --colab
%tensorflow_version 2.x
from torch.utils.tensorboard import SummaryWriter
tensorboard_logdir = '/content/tensorboard/'
writer = SummaryWriter(tensorboard_logdir)
Related
I've been trying to use YOLO (v3) to implement and train an object detection of Tank with OpenImage dataset.
I have tried to get help from this tutorial and my code pretty much looks like it.
Also I'm using Google Colab and Google Drive services.
everything is going fine through my program. But I hit an error at the final step when I'm running the darknet to train detection.
!./darknet detector train "data/obj.data" cfg/yolov3_custom.cfg "darknet53.conv.74" -dont_show
after 100 iterations, when it's trying to save the progress in the backup folder I've addressed in obj.data file, I get the following error:
Saving weights to /content/drive/My\Drive/YOLOv3/backup/yolov3_custom_last.weights
Couldn't open file: /content/drive/My\Drive/YOLOv3/backup/yolov3_custom_last.weights
at first, I thought I made a mistake in using the address; So, I tried checking the address by using ls command
!ls /content/drive/My\Drive/YOLOv3/backup/
and the result was an empty folder (However, not an error meaning I've written the address correctly and that it is accessable in my google drive).
Here are contents of data.object file:
classes = 1
train = data/train.txt
valid = data/test.txt
names = data/obj.names
backup = /content/drive/My\ Drive/YOLOv3/backup
also I've made required changes in config file so I don't think that the problem is about that. But just to make sure here are the changes I've made in my yolov3.cfg file:
Fist of all we will comment lines 3 and 4 (batch, subdivisions) to unset testing mode
We will uncomment lines 6 and 7(batch, subdivisions) to set to training mode
We change our max_batches value to 2000 * number_of_classes (if there's one class like our case, set to 4000)
We change our step tuple-like values to 80%, 90% value of our max_baches value. In this case it will be 3200, 3600.
For all YOLO layers and convoloutional layers before them, changed the classes value to number of classes, In this case 1, and change the value of filters according to following formula(In this case, 18)
Formula for conv layers filters value: (number_of_classes + 5)*3
I searched the error and found this issue on Github.
However, I tried the following methods recommended there and the problem is still the same:
Removing and recreating the backup folder
Tried to adding the line backup = backup in my yolo.data file in folder .cfg but there was no such file in cfg folder.
Creating an empty yolov3_custom_last.weights in backup folder
The other solutions mentioned in this issue was about when you are running YOLO on your PC and not google Colab.
Also, here my tree structure of the folder YOLOv3 which is stored in my Google Drive My Drive(main folder).
YOLOv3
darknet53.conv.74
obj.data
obj.names
Tank.zip
yolov3.weights
yolov3_custom.cfg
yolov3_custom1.cfg.txt
So, I'm kinda stuck and I have no idea what could fix this. I'd appreciate some help.
I have solved the problem with changing the local drive address with ln command.
The problem wasn't from my code rather from the way way yolov3 developers were handling space in directory addresses! Apparently as much as I figured in their docs, they are not handling space in directory quite well.
So I created a virtual address which does not have space like My Drive has.
P.S: As you know My Drive folder is already there in your google drive so you can't actually rename it.
Here is the code you can use to achieve this:
!ln -s /content/drive/My\ Drive/ /mydrive
I was getting the same error on my Windows 10, and I think I managed to fix it. All I had to do was move my "weights" folder (with my yolov3.weights inside), closer to the .exe file, which was in the "x64" folder, in my case. After that, the error stopped appearing and the app was able to predict the test image normally.
I have a ipynb file (a jupyter notebook) which I am opening in vscode with python extension. I receive the error in the title
Unexpected token # in JSON at position 0
which I dont understand at all, since the file is supposed to be interpreted as a python file.
I can change the extension to .py and its opened fine by vscode, but I dont have the decorators to run/debug cells like define here (https://code.visualstudio.com/docs/python/jupyter-support-py).
I know the file is correct because I have use it in another vscode installation in another computer and works fine.
I have no idea what might be misconfigured in my environment... Any tops would be really helpful.
Here is the actual python code I have that its producing the mentioned error my actual environment.
issue.ipynb
# %%
import world as w
import world_eg as weg
import world_case1 as wc1
import simulator_static as simulation
import numpy as np
from scipy.optimize import minimize
import matplotlib.pyplot as plt```
From the error, I understand that is parsing the file as a JSON file and the first line, which contains the #, fails.
I had a similar problem and when I opened the notebook with an editor I saw I had merge markings that git had put into the file. e.g.
<<<<<<< HEAD
...
=======
...
>>>>>>> ...
Cleaning up these, allowed jupyter to parse the file and run the notebook.
This happens when you make a request to the server and parse the response as JSON, but it’s not JSON. JSON should start with a valid JSON value – an object, array, string, number, or false/true/null. The root cause is that the server returned HTML or some other non-JSON string.
I've tried your code in my project and nothing wrong. everything looks fine. Check the Jupyter Server network, try to restart vscode and recreate a new juypter file, and see if the problem goes away.
[edit]
like the above screenshot shows, type # %% will add a new cell. Equally, when you open a .ipynb file, if python extension distinguishes the # %%, button run cell | debug cell will be displayed automatically for you to do further test.
you can copy your code without # %% to a new created blank juypter file, then
click the button export as and select Python Script to got button Run Cell | Debug Cell .
OR reinstall python extension and try again.
I had the same issue and for me that problem was solved simply by deleting the underscore (_) from the file name. I don't know why but it works.
.ipynb files aren't actually Python source code files - they're encoded as JSON files. If you create a new notebook, then rename the file extension or open it in some text editor, you'll see the structure of the underlying JSON file.
When VS Code tries to interpret your file, it's trying to parse Python source as a JSON object, which will obviously fail, and lead to the otherwise cryptic error of an unexpected token.
In other words, it's not possible to convert a Python script into a notebook just by changing the file extension. Manually copying & pasting the code around will work, or you could try googling for some tool, e.g. https://github.com/remykarem/python2jupyter
"Unable to open 'XXX.ipynb'"
"Unexpected token < in JSON at position XXX"
For me, I have similar issues when I am using git and reopen ipynb files in vscode.
To fix it, pretty easy!
(1) Open and edit the file in json format and ACCEPT the current
changes or incoming changes.
(2) Save and Close the edited file and reopen the file. Everything
works fine!
Good luck!
I had a similar issue when creating a new file in VS Code which I saved as .ipynb. After closing the file, I was not able to reopen and I received the same error message as above.
For me, simply closing and restart VS Code did the trick. Afterwards, the .ipynb-file opened as expected.
notepad encoding format
Resaving the document using another encoding format solved my problem. A regular.ipynb file (left image) is saved using Unix(LF) format, but the file that couldn't open was saved using UTF (right image)
I have an issue where I am training a model, but tensorboard isn't displaying any data from the event files. I've done everything in my knowledge. Yes I read the tensorboard ReadMe FAQ. https://github.com/tensorflow/tensorboard/blob/master/README.md#frequently-asked-questions
I start the train process with the folowing command (cmd started in folder with the directories "tf_files" and "flower_photos" in it):
python retrain.py --bottleneck_dir=tf_files/bottlenecks --model_dir=tf_files/models/"mobilenet_0.50_224" --summaries_dir=tf_files/training_summaries/"mobilenet_0.50_224" --output_graph=tf_files/retrained_graph.pb --output_labels=tf_files/retrained_labels.txt --architecture="mobilenet_0.50_224" --image_dir=flower_photos
https://www.mediafire.com/view/22orxlkdfleiw4u/retrain_start_1.png
https://www.mediafire.com/file/hfexqjci64o6aqw/retrain_start_2.png
https://www.mediafire.com/view/3uzfyvz4hqj44wh/retrain_finish.png
The retraining finishes successfully.
Tensorboard i start with the command:
tensorboard --logdir i:/temp/learningtf/tf_files/training_summaries
https://www.mediafire.com/view/j366d29whwrdbwz/start_tensorboard.png
The command above with --inspect give me the following output:
https://www.mediafire.com/view/3s0x11zb61b8si8/tensorboard_inspect_1.png
https://www.mediafire.com/view/45k87w8h15b48ks/tensorboard_inspect_2.png
As you can see, tensorboard doesn't display anything. Im quite confused.
https://www.mediafire.com/view/v74qashzjq0wnxr/tensorboard_webinterface.png
Here are the event files I'm talking about. If these are not the files you need, post a reply and tell my which files i need to upload, please.
the_i_hope_right_files.zip
If you need eny further Info on this case, please let me know.
Thanks for any help in advance.
I'm trying to open a tensorboard using the example code. I ran the example locally and then tried to open in tensorboard using the following command tensorboard --logdir=F:\mnist_tmp\tensorflow\mnist\logs\mnist_with_summaries\.
Tensorboard starts, however I am not able to see any data ie, scalars, graphs,
or histograms.
Below is a picture of what I see when I open tensorflow. If I click on the other tabs I get a similar text dialog.
Don't use '' to brace the dir , just use the
tensorboard --logdir=dir
if you use the win10 System Operation
My issue was that I my log logdir contained the F: but tensorflow uses colon as an seperator which is a known bug.
I am using Tensorboard to show the training results of the code using Tensorflow (0.7). The previous Tensorflow version had issue for multiple event files: when I run my local server using $ tensorboard --logdir=./tmp/, it shows up error if there are more than 1 event files. It seems that the latest version (0.7) does not show up the same error for multiple event files, but it still shows the overlapped curves for multiple event files on Tensorboard. I wonder how to solve this problem. Thanks!
Training my own networks, I am writing summaries in different subfolders like /tmp/project/train and /tmp/project/eval. If you start TensorBoard using
tensorboard --logdir=/tmp/project/
you will still receive multiple graphs from each event file in the subfolders at once, as you mentioned. To see seperate graphs you can start TensorBoard from the desired subfolder:
tensorboard --logdir=/tmp/project/train/
I second ArnoXf's answer. You should use different subfolders for each experiment and assuming the logging root is /tmp start tensorboard using:
tensorboard --logdir=/tmp/
If you want to display just a single graph you can either pass that directory to your tensorboard call as described in ArnoXf's answer.
However, with the above call you can also select your graph directly in tensorboard, i.e., deactivate all others. The same way you can also compare individual runs as shown in the following screenshot. In my opinion, this is usually the preferred option, as it gives you more flexibility.
A detailed example can be found here.