0 21 - 12 MByt -- 25 MBytE -> 0 0.1 02 03 0.4 05 0.6 Figure 5: Memory requirements for cla.ssifica. tion of test pa. tterns (in MBytes). Numbers a.re ba.sed on 4 bit/pixel for K-NN, 1 byte per pixel for Soft Ma.rgin, a.nd Ta.ngent Dista.nce, 4 byte per pixel for the rest. of the methods. Memory requirements for the neura.1 networks a.ssume 4 bytes per weight (a.nd 4 bytes per prototype component for the LeNet 4 / memory- ba.sed hybrids), but experiments show tha. t one-byte weights ca.n be used with no significa.nt cha.nge in error ra. te. Of the high-a. ccura. cy cla.ssifiers, LeNet 4 requires the lea.st memory. 5 Conclusions This pa.per is a. sna.pshot of ongoing work. Although we expect continued cha.nges in a.ll a.spects of recognition technology, there a.re some conclusions tha. t a.re likely to rema.in va.lid for some time. Performa.nce depends on ma.ny fa. ctors including high a. ccura. cy, low run time, a.nd low memory requirements. As computer technology improves, la.rger- ca.pa. city recognizers become fea.sible. La.rger recognizers in turn require la.rger tra.ining sets. LeNet 1 wa.s a.ppropria. te to the a.va.ila.ble technology five yea.rs a.go, just a.s LeNet 5 is a.ppropria. te now. Five yea.rs a.go a. recognizer a.s complex a.s LeNet 5 would ha.ve required severa.1 months' tra.ining, a.nd more da. ta. tha.n wa.s a.va.ila.ble, a.nd wa.s therefore not even considered. For quite a. long time, LeNet 1 wa.s considered the sta. te of the a.rt. The loca.1 lea.ming cla.ssifier, the optima.1 ma.rgin cla.ssifier, a.nd the ta.ngent dista.nce cla.ssifier were developed to improve upon LeNet 1 a.nd they succeeded a.t tha. t. However, they in turn motiva. ted a. sea.rch for improved neura.1 network a.rchitectures. This sea.rch wa.s guided in pa.rt by estima. tes of the ca.pa. city of va.rious lea.ming ma. chines, derived from mea.surements of the tra.ining a.nd test error a.s a. function of the number of tra.ining exa.mples. We discovered tha. t more ca.pa. city wa.s needed. Through a. series of experiments in a.rchitecture, combined with a.n a.na.lysis of the cha.ra. cteristics of recognition errors, LeNet 4 a.nd LeNet 5 were cra.fted. We find tha. t boosting gives a. substa.ntia.1 improvement in a. ccura. cy, with a. rela. tively modest pena.lty in memory a.nd computing expense. Also, distortion models ca.n be used to increa.se the effective size of a. da. ta. set without a. ctua.lly ta.king more da. ta.. >
Back to Index International Conf. on Artifical Neural Networks, Paris 1995
 Digital Library Home