Caffe: What does "xavier" mean?

[johnny5550822]

When I am building the convolution layer, I have seen the type of weight filler can be "Gaussian" or "xavier" so far, what does xavier refer to? Also, are there any other types?

[dabilied]

see include/caffe/filler.hpp

[johnny5550822]

Great thanks!

[ghost]

/**

• @brief Fills a Blob with values @f$ x \sim U(-a, +a) @f$ where @f$ a @f$ is
• set inversely proportional to number of incoming nodes, outgoing
• nodes, or their average.
  *
• A Filler based on the paper [Bengio and Glorot 2010]: Understanding
• the difficulty of training deep feedforward neuralnetworks.
  *
• It fills the incoming matrix by randomly sampling uniform data from [-scale,
• scale] where scale = sqrt(3 / n) where n is the fan_in, fan_out, or their
• average, depending on the variance_norm option. You should make sure the
• input blob has shape (num, a, b, c) where a * b * c = fan_in and num * b * c
• = fan_out. Note that this is currently not the case for inner product layers.
  *
• TODO(dox): make notation in above comment consistent with rest & use LaTeX.
  */

[p4tr1ckc4rs0n]

great explanation: http://andyljones.tumblr.com/post/110998971763/an-explanation-of-xavier-initialization