Neuro-Genetic Systems

Source: Unit 5 §8

A neuro-genetic system is an ANN and a GA hybridised. There are two kinds: a GA used for weight determination in a multilayer feed-forward network, and a GA that evolves network topologies.

⇄CompareWhy replace backpropagation

BackpropagationFollows gradient descent, so it is vulnerable to local minima and slow, needing many iterations.

GA-based learningReplaces backprop entirely. The search is guided by the GA, which samples many networks at once and uses no gradient.

The network error is the basis of fitness:

$$E = \frac{1}{2}\sum_i (TO_i - O_i)^2$$

where $TO_i$ is the target output and $O_i$ the actual output at unit $i$.

Encoding the weights

For a single-hidden-layer network with $m$ inputs, $n$ hidden and $r$ output nodes, the number of interconnection weights is $mn + rn$. Each weight is one gene, so the chromosome has $mn + rn$ genes.

≡FactsThe five-digit gene

Each weight is five digits $d_1 d_2 d_3 d_4 d_5$.

• $d_1$ is the sign - positive if $d_1$ is even, negative if odd.
• $d_2 d_3 d_4 d_5$ is the magnitude, obtained by dividing by 100.
• A whole chromosome is therefore a linear array of $(m + r) \cdot n \cdot 5$ digits.

One weight is five digits: the first decides the sign by its parity, the other four are the magnitude over 100.

The 2-3-1 worked example

A 2-3-1 network has 2×3 + 1×3 = 9 weights, and at five digits each the chromosome is 45 digits long.

→StepsSizing and decoding

1. The weights are w₁₁ w₁₂ w₁₃ w₂₁ w₂₂ w₂₃ from input to hidden, and v₁₁ v₂₁ v₃₁ from hidden to output.
2. Chromosome length is $(2 + 1) \cdot 3 \cdot 5 = \mathbf{45}$ digits.
3. Decode the gene 14345: $d_1 = 1$ is odd, so the sign is negative.
4. The magnitude is $4345 / 100 = 43.45$, so the weight is $\mathbf{-43.45}$.

Fitness

A GA maximises, but we want to minimise the error $E$, so use the reciprocal:

$$F = \frac{1}{E}$$

→StepsEvaluating one chromosome

1. Map the chromosome onto its network, decoding every gene into a weight.
2. Run the test inputs through that network.
3. Compute $E$ for each training pair.
4. Average over all pairs; the reciprocal of that average is the fitness.

◎Exam cueThe headline

No backpropagation is involved anywhere. The GA alone drives the search towards the minimum-error network, which is the whole point of the hybrid.