Remove real type
SeeWebsiteForEmail at erdani.org
Sat Apr 24 21:30:19 PDT 2010
On 04/24/2010 07:21 PM, strtr wrote:
> Andrei Alexandrescu Wrote:
>> So are you saying there are neural networks with thresholds that
>> are trained using evolutionary algorithms instead of e.g. backprop?
>> I found this:
> The moment a network is just a bit recurrent, any gradient descent
> algo will be a hell.
which does seem to support the point. I'd have to give it a closer look
>> to see whether precision would affect training.
> I would love to see your results :)
> But even in the basic 3 layer sigmoid network the question is: Will
> two outputs which are exactly the same(for a certain input) stay the
> same if you change the precision.
You shouldn't care.
> When the calculations leading up to
> the two outputs are totally different ( for instance fully dependent
> on separated subsets of the input; separated paths), changing the
> precision could influence them differently leading to different
> outputs ?
I'm not sure about that. Fundamentally all learning relies on some
smoothness assumption - at a minimum, continuity of the transfer
function (small variation in input leads to small variation in output).
I'm sure certain oddities could be derived from systems that impose
discontinuities, but by and large I think those aren't all that interesting.
The case you mention above involves a NN making a different end discrete
classification decision because numeric vagaries led to some threshold
being met or not. I have certainly seen that happening - even changing
the computation method (e.g. unrolling loops) will lead to different
individual results. But that doesn't matter; statistically the neural
net will behave the same.
More information about the Digitalmars-d