資源描述：
《NEAL-MONTGOMERY NLP SYSTEM EVALUATION METHODOLOGY》由會員上傳分享，免費(fèi)在線閱讀，更多相關(guān)內(nèi)容在行業(yè)資料-天天文庫。
1、NEAL-MONTGOMERYNLPSYSTEMEVALUATIONMETHODOLOGYSharonM.WalterRomeLaboratoryRL/C3CAGriffissAFB,NY13441-5700walter@aivax.rl.af.milABSTRACTfeature.IllustrativelanguagepatternsandsamplesentencesthenguidethehumanevaluatortotheformulationofanOnwhatbasisaretheinputproc
2、essingcapabilitiesofNaturalinputthatteststhefeatureontheNLPsystemwithintheLanguagesoftwarejudged?Thatis,whatarethecapabilitiestosystem'snativedomain.bedescribedandmeasured,andwhatarethestandardsagainstwhichwemeasurethem?RomeLaboratoryiscurrentlyBasedonclearand
3、specificevaluationcriteriafortestitemsupportinganefforttodevelopaconciseterminologyforinputs,NLPsystemresponsesarescoredasfollows:describingthelinguisticprocessingcapabilitiesofNaturalLanguageSystems,andauniformmethodologyforS:Thesystemsuccessfullymetthestated
4、criteriaandappropriatelyapplyingtheterminology.Thismethodologyisdemonstratedunderstandingwithrespecttothefeatureundermeanttoproducequantitative,objectiveprofilesofNLsystemcapabilitieswithoutrequiringsystemadaptationtoanewtesttest.domainortextcorpus.Theeffortpr
5、oposestodeveloparepeatableprocedurethatproducesconsistentresultsforC:Thesystemrespondedinawaythatwascorrectindependentevaluators.(thatis,correctlyansweredthequestionposed),butthecriteriawerenotmet.1.INTRODUCTIONP:ThesystemrespondedinawaythatwasonlyAnappreciabl
6、edrawbacktocurrentcorpus-based(eg.,partiallycorrect.[BBN;1988],[Flickinger,etal;1987],[Hendrix,etal;1976],[Malhotra;1975])andtask-based(eg.,F:Thesystemrespondedinawaythatwasincorrect,["Proceedings";1991])methodologiesforevaluatingfailingtomeetthecriteria.Natur
7、alLanguageProcessingSystemsistherequirementN:Thesystemwasunabletoaccepttheinputorformfortransportationofthesystemtoatestdomain.Thearesponse(forexample,thesystemvocabularylacksexpenseandtimeconsumptionaresizableand,astheportappropriatewordstocompleteatestinpu0.
8、maybeminimalorincomplete,theevaluationmaybebasedonademonstrationoflessthanthefullpotentialofthesystem.Further,currentevaluationmethodologiesdoEachlinguisticfeatureistestedbymoretha
