How to interpret weight distributions of neural ne

Question

I have designed a 3 layer neural network whose inputs are the concatenated features from a CNN and RNN. The weights learned by network take very small values. What is the reasonable explanation for this? and how to interpret the weight histograms and distributions in Tensorflow? Any good resource for it?

This is the weight distribution of the first hidden layer of a 3 layer neural network visualized using tensorboard. How to interpret this? all the weights are taking up zero value?

This is the weight distribution of the second hidden layer of a 3 layer neural:

Answer 1

how to interpret the weight histograms and distributions in Tensorflow?

Well, you probably didn't realize it, but you have just asked the 1 million dollar question in ML & AI...

Model interpretability is a hyper-active and hyper-hot area of current research (think of holy grail, or something), which has been brought forward lately not least due to the (often tremendous) success of deep learning models in various tasks; these models are currently only black boxes, and we naturally feel uncomfortable about it...

Any good resource for it?

Probably not exactly the kind of resources you were thinking of, and we are well off a SO-appropriate topic here, but since you asked...:

• A recent (July 2017) article in Science provides a nice overview of the current status & research: How AI detectives are cracking open the black box of deep learning (no in-text links, but googling names & terms will pay off)

• DARPA itself is currently running a program on Explainable Artificial Intelligence (XAI)

• There was a workshop in NIPS 2016 on Interpretable Machine Learning for Complex Systems

On a more practical level:

• The Layer-wise Relevance Propagation (LRP) toolbox for neural networks (paper, project page, code, TF Slim wrapper)

• FairML: Auditing Black-Box Predictive Models, by Fast Forward Labs (blog post, paper, code)

• A very recent (November 2017) paper by Geoff Hinton, Distilling a Neural Network Into a Soft Decision Tree, with an independent PyTorch implementation

• SHAP: A Unified Approach to Interpreting Model Predictions (paper, authors' code)

These should be enough for starters, and to give you a general idea of the subject about which you asked...

UPDATE (Oct 2018): I have put up a much more detailed list of practical resources in my answer to the question Predictive Analytics - “Why” factor?