資源描述：
《Very Deep Convolutional Networks》由會(huì)員上傳分享，免費(fèi)在線閱讀，更多相關(guān)內(nèi)容在學(xué)術(shù)論文-天天文庫(kù)。
1、VeryDeepConvolutionalNetworksforTextClassi?cationAlexisConneauHolgerSchwenkYannLeCunFacebookAIResearchFacebookAIResearchFacebookAIResearchaconneau@fb.comschwenk@fb.comyann@fb.comLo¨?cBarraultLIUM,UniversityofLeMans,Franceloic.barrault@univ-lemans.frAbstractterestintheresearch
2、communityandtheyaresys-tematicallyappliedtoallNLPtasks.However,ThedominantapproachformanyNLPwhiletheuseof(deep)neuralnetworksinNLPtasksarerecurrentneuralnetworks,inpar-hasshownverygoodresultsformanytasks,itticularLSTMs,andconvolutionalneuralseemsthattheyhavenotyetreachedthele
3、veltonetworks.However,thesearchitecturesoutperformthestate-of-the-artbyalargemargin,arerathershallowincomparisontotheasitwasobservedincomputervisionandspeechdeepconvolutionalnetworkswhichhaverecognition.pushedthestate-of-the-artincomputervi-Convolutionalneuralnetworks,inshort
4、Con-sion.Wepresentanewarchitecture(VD-vNets,areverysuccessfulincomputervision.InCNN)fortextprocessingwhichoperatesearlyapproachestocomputervision,handcrafteddirectlyatthecharacterlevelandusesfeatureswereused,forinstance“scale-invariantonlysmallconvolutionsandpoolingoper-featu
5、retransform(SIFT)”(Lowe,2004),followedations.Weareabletoshowthattheper-bysomeclassi?er.ThefundamentalideaofCon-formanceofthismodelincreaseswiththevNets(LeCunetal.,1998)istoconsiderfeaturedepth:usingupto29convolutionallayers,extractionandclassi?cationasonejointlytrainedwerepor
6、timprovementsoverthestate-of-task.Thisideahasbeenimprovedovertheyears,the-artonseveralpublictextclassi?cationinparticularbyusingmanylayersofconvolutionstasks.Tothebestofourknowledge,thisisandpoolingtosequentiallyextractahierarchicalthe?rsttimethatverydeepconvolutionalrepresen
7、tation(ZeilerandFergus,2014)ofthein-netshavebeenappliedtotextprocessing.put.Thebestnetworksareusingmorethan150layersasin(Heetal.,2016a;Heetal.,2016b).1IntroductionarXiv:1606.01781v2[cs.CL]27Jan2017ManyNLPapproachesconsiderwordsasba-Thegoalofnaturallanguageprocessing(NLP)issic
8、units.Animportantstepwastheintroductiontoprocesstextwithcomputersino
