I am searching for a way to forward lower layer output to a higher layer with a loaded VGG16 model using CNTK.
The background of my problem is:
I reimplemented some parts of Fully Convolutional Networks for Semantic Segmentation but then I ran into some problems: Starting with this example I first replaced the fully connected layers with fully convolutional and slit the sequence in the model definition part into chunks where I could simply access pool3 and pool4 for the later usage in eg. Convolution2D((1,1), num_classes, name='score_pool4')(pool4). This works fine but after building the model I noticed, that I need to implement an own way to read batches because the build-in reader does not support 2D labels right now. Now I simply read the images using OpenCV and replaced the training_session(...).train() with a for loop and trainer.train_minibatch({model['features']: my_loaded_features, model['labels']: my_2D_labels}) this works well but because of the removed training_session part I don't know where I could apply the existing VGG16 weights.
My problem is:
I searched for transfer learning examples where those guys load models using C.load_model(...) and then clone the needed layers but now I am wondering how could I access cloned_layers->pool4 (in the middle of the loaded model) if I also want to use it in deeper layers.
I tried Convolution2D((1,1), num_classes, name='score_pool4')(cloned_layers.find_by_name('pool4'))but I ended up with some error messages while learner initialization because of "unknown shape information" in used weight variables.
So how can I access those layers within the loaded model for later (deeper) usage?
Thanks for reading (and maybe helping)!
If you are looking to read custom data. There are two tutorials on building your own readers. https://cntk.ai/pythondocs/manuals.html
Regarding cloning parts of a network - here is a link to another post on StackOverflow that has exemplar code
Related
I have a model in keras and I want to use it as a feature extractor in an autoencoder configuration. The first test I want to do is to mirror the model and use the original version as the encoder, and the second one as the decoder.
e.g. if the blocks on the original model goes
input->A->B->C->output
I want to get
input->A->B->C->(latent space)->C->B->A->reconstructed_input
However, the original model is not small and I am quite new to keras. It has a number of blocks defined with which the total model is built. What I am confused about is the order I have to follow when I reverse things. There are several convolutional, reshaping, and dense layers, as well as skip connections, and I am not sure about the strategy I should follow with each individual case. I know for a fact that I should replace Conv2d layers with Conv2DTranspose to upscale the feature maps back to the original image, but that's about it. I am baffled by the rest.
Does anyone have any recommendations or resources I could follow to make my life easier?
Cheers!
I have a pre-trained image classification model saved in caffe, the model is expected to get grayscale(one channel) images. I want to use this model in a tool that only provides input of RGB(three channels) to the model. It is not possible to change the way this tool provides images so I thought of adding a layer before the input layer that transforms the input to one channel only, is that possible in caffe? and how?
I'm looking for a solution that doesn't require to define new layers to caffe if possible.
Note that I have the ".prototxt" and the ".weights" files of the model.
I previously did a similar thing in tensorflow but I don't know if this is possible in caffe and didn't find much material online.
You can add a Python layer to do it for you.
What is a Python layer.
An example of such a layer can be found here.
I built an Autoencoder using Python and Tensorflow. To build the Autoencoder I used the Tensorflow tutorial on how to build an Autoencoder to read the MNIST Data set on handwritten digits. I used it to find features of CGRA compositions.
So far I restructured the Code for the Autoencoder in a way that I can use it on my own data. I found features and already managed to reconstruct the Input,
up to a certain error. NOW, I am trying to use the Autoencoders weights, to initialize a Neural Network with parameters similiar to the encoder part of my Autoencoder. Then, add one extra Layer with a single neuron and a linear activation function to perform Regression Analysis(or basically supervised learning).
So my question is: How do I initialize a neural network with specific weights (not random) using tensorflow?
I'd be grateful for any kind of help. Links to Tutorials or other links to other Threads.
Tahnks in Advance!
When you build tf.Variable, the first argument is an initial_value.
https://www.tensorflow.org/api_docs/python/state_ops/variables#Variable.init
You can provide any Tensor you like to initialize the variables, not just, say, random initialization.
Another option is you can assign values to the variables after construction, if you find that easier.
Hope that helps!
Is there a way to learn unsupervised features from set of images. Similar to word2vec or doc2vec, where neural network is learnt and given new document we get its features.
Expecting similar to this example shows that it can load learnt nn-model and predict features for new images.
Is there any simple example how to implement cnn over images and get their features back will help !!
Suppose in this example
If I want to get cnn features for all X_train and X_test ... is there any way?
Also, if we can get weights per layer per image, we can stack them and use as features. In that case is there a way to get the same.
Using those features for unsupervised task would be easier, if we consider them as vectors.
If I correctly understood your question, this task is quite common in a deep learning field. In case of images what I consider the best is a convolutional autoencoder. You may read about this architecture e.g. here
http://people.idsia.ch/~ciresan/data/icann2011.pdf
Previous version of Keras supported this architecture as one of core layers, though from version 1.0 I noticed that it disappeared from documentation. But - it's still quite easy to build it from a scratch :)
In noimage cases there are also another approaches like e.g. Restricted Boltzmann Machines.
UPDATE :
When it comes to what sort of activations are the best for obtaining new features from neural network activations - from my personal experience - it depends on the size of the net which you use. If you use a network which last layer is wide (has a lot of nodes) it might be useful to get only last layer (due to number of parameters if you want to consider also previous layers - it may harm the performance of learning). But - if (like in case of some MNIST networks) your last layer is not sufficient for this task - you may try using also previous layers activation or even all net activity. To be honest - I'm not expecting much of improvement in this case - but you may try. I think that you should use both approaches - starting from taking only last layer activations - and then trying to check the behaviour of your new classifier when you add activations from previous layers.
What I will strongly advise to you is also getting some insights from what sort of features network is learning - using T-SNE embeddings of it activations. In many cases I found it useful - e.g. checking if the size of a layer is sufficient. Using T-SNE you may check if the features obtained from last layer are good discriminators of your classes. It may also give you good insights about your data and what neural networks are really learning (alongside with amazing visualizations :) )
How can I load a .pb protobuf model and then tweak the network as needed (specially the outer layers) in order to train a new model for completely different classes? Effectively doing transfer learning?
I want to do something like these (i.e. train the outer layers with a bigger learning rate than the inner layers) among other things, so I need a way to not only load the graph with the variables, but to alter the network's structure and hyperparameters too.
If anyone has an example to follow with the inception model, it would be amazing!
My question is very similar to this one.
I've searched all over the internet (TF docs, Github, StackOverflow, Google...) but I can't seam to find something useful for a novice.
Thanks a lot!
This is the updated tutorial from official Tensorflow website https://www.tensorflow.org/hub/tutorials/image_retraining
They use the pre trained Inception V3 model and everything works fine. You can change the dataset folder to your own dataset
tf.import_graph_def() is the function for loading a GraphDef:
https://www.tensorflow.org/versions/0.6.0/api_docs/python/framework.html#import_graph_def
Hopefully once imported, you can make the modifications to the graph you need. It would be easier, though, to modify the Python code that generated the graph in the first place, if you have access to that.