Backpropogation with global minimum gradient descent

If a particular posterior distribution is a surface in a high dimensional space , we may ask what the dimensions of this space represent. From my understanding , In a CNN the dimensions could represent the representation encoded at each neuron in the NN as well as the filters..Where the HD surface then represents how each neurons "bias" contributes (amplitude in that dimension) to the bias of the network to classify say a cat as a cat? And if there are 4 neurons one for the legs , the head , the body and the tail then a random image must have a 4D normal distribution (surface) in this space.to classify a cat..ofcourse neurons never learn that well....because when the thing that causes a neuron to fire may be part of a leg and part of a head, with the appropiate settings even this N can identify cats , but the distribution will not be normal...if this reasoning is fine so far the real question is, is it possible to give conditions to the back propogation algorithm such that role of a particular neuron in say , classification is unambiguous...by which i mean the posterior distribution described by the dimensions (i.e. the input that causes a particular neuron to fire) is a multi dimensional normal curve....that should represent the global minimum of the cost function...so we can design the network in such a way that we compute a value for the relative values of the weights and biases needed to have the neurons explicitly encode the posterior distribution as a normal surface...then use this value as the expected value in calculating a cost function that compares this expected value to the actual relative nature of the biases and weights....then backpropagate with gradient decent to reduce the cost.......this depends on how you need to scale the weights and biases, so they maintain the same traditional cost (the usual cost)...depending on that we could adjust the algorithm...a note is that we will be backpropagating from the "side" of the network and not the last layer, meaning that the information will reach every neuron as if it belonged to the output layer,

Comments

Post a Comment

Popular posts from this blog

ABSTRACT submited

BACKPROPAGATION ON A NOVEL NETWORK (NON ANN) FOR HATESPEECH DETECTION AND MORE This paper introduces a novel type of network that can accurately classify "hate speech" and differentiate it from normal content. It develops a network centered around a unique POS (part of speech) tagging type of system whose specifics are learnt by the network. Each word is assigned variables that indicate the polarity, (sign) of the effect a particular word has on every other word in a sentence, the magnitude of that polarity and also to what extent the word filters itself from the effect from other words. When choosing a set of properties, we would like to properly understand what we want the network to do. We know that both posts classified as hate speech and those not contain samples of words from a common pool (they share words). But we still want that the results for opposite classes to polarize. On top of this I believe that this represents a complex system, in that small chang...
Read more
This involves a "hate speech" or "spam" detection algorithm. (depending on the training data). This will be based on ANN theory , and involve the backpropagation algorithm while not based on an actual neural network. The training data will be labeled into two classes, "hate speech" and "non hate speech".   Hate speech will have an expected value of 1 , representing certain hate speech, while non hate speech will have an expected value of -1.   So what we would like is for the software to be input a post and gives it a value of 1 or -1, or something much closer to one or the other. During training The program will compute a value by performing the equivalent of forward propagation through the parameters we assign to each of multiple structures we will embed in the input ...then using this value and the epected value we will compute a cost function. We will then backpropagate through the heirachical structure of parameters in the direc...
Read more