博文
ALPAC 黑皮书 8/9： 附录 11-15
|||
Appendix11
TypesofErrorsCommoninMachine Translation
Twostudieshaverecentlybeenmadeofthetypesoferrorsmadeinmechanicaltranslation.ThefirststudywasverykindlymadeavailabletotheCommitteebytheIBMThomasJ.WatsonResearchCenter,YorktownHeights,NewYork.Bycountingandclassifyingthecorrectionsmadebyposteditors,thisstudydeterminedthetypesandfrequencyoferrorsfoundintheoutputof four machinetranslations(Russianto English).
GENERAL CLASSIFICATION ANDPERCENTAGEOF ERRORSOF ARTICLEI
Totalnumberof words: Approximately1,200
No. %
Transliteratedwords – –
Multiplemeaningsand ambiguities 96 8.0
Wordorder rearranged 23 2.0
Miscellaneous insertionsandcorrections 45 3.6
Total 164 13.6
GENERAL CLASSIFICATION ANDPERCENTAGEOF ERRORSOF ARTICLEII
Totalnumberof words: Approximately1,200
No. %
Transliteratedwords 6 0.5
Multiplemeaningsand ambiguities 132 11.0
Wordorder rearrarnged 17 1.4
Miscellaneous insertionsandcorrections 77 6.4
Total 232 19.3
GENERAL CLASSIFICATION ANDPERCENTAGEOF ERRORSOF ARTICLEIII
Totalnumberof words: Approximately1,700
No. %
Transliteratedwords 17 1
Multiplemeaningsand ambiguities 143 9
Wordorder rearranged 36 2
Miscellaneous insertionsandcorrections 122 7
Total 318 19
GENERAL CLASSIFICATION ANDPERCENTAGEOF ERRORSOF ARTICLEIV
Totalnumberofwords(includingindividualdigitsand symbolsinallformulas): Approximately1,600
No. %
Transliteratedwords 1 –
Multiplemeanings andambiguities 87 5.8
Wordorder rearranged 14 0.9
Miscellaneous insertionsandcorrections 436 29.0
Total 538 35.7
ThesecondstudywasmadebyArthurD.Little,Inc.,andwasdoneinamannersimilartotheIBMstudy.Thatis,machinetranslationoutputwasposteditedandtheerrorsclassifiedandcounted.Fromthestudy,theA.D.Littlegroupwasabletotellthepercentageoftotalcorrectionsmadeineachcategory.Theoriginalconsistedofapproximately200pagesofscientificRussian.One setofapproximately100pageswaseditedbytwodifferenteditors.Thesecond setcontained“approximately100pagesfromsevenMTarticleseditedbyat leastfour different editors.”*
*AnEvaluationofMachine-AidedTranslationActivitiesatF.T.D.,ContractAF33(657)-13616,Case66556, May1,1965,p.G-10.
Appendix12
Machine-AidedTranslationattheFederal ArmedForcesTranslationAgency,Mannheim,Germany
SEMIAUTOMATICTRANSLATIONAIDSYSTEM(STAGE1)
TranslatedfromGermanbytheFederalArmedForcesTranslationAgency, Annex toReport MüV - Az.:55-05(30)dated,February 18,1965.
Report onSixthTest RunOn TR4ComputerFacility
I. GENERAL
DuringtheweekofFebruary8to12,1965,asecondimprovedmodeltest runwasconductedusingtheTR4computerfacilityoftheTelefunkenCompany,Konstanz.Thetestrunwasdesignedtotestasanintegralsystemall routinesandsubroutinesdevelopedsofar.Thetest,whichrepresentsthe culminationofthedevelopmentworkdoneinStageIofthesemiautomatictranslationaidsystem,canberegardedasquitesuccessful:itconfirmedthe soundnessoftheapproach.Practicalapplicationoftheprocedure(StageI)now dependsonwhentheFederalArmedForcesComputerCenterisoperationalsothattheentirebodyoflinguisticinformationnowstoredonpunchedcardscanbetransferredtomagnetictape.Optimizationoftheprogramwillbeeffectedon the basisofpracticalexperience.
II. DESCRIPTIONOF TESTRUN
ThetestingmaterialconsistedofthreeEnglish-languagetexts(so-called partialinterrogationbatches).Thetextsboredifferentjobnumbersandwereassignedtodifferenttranslatorswhounderscoredinthetextthosetermswith which themachinewas tobepresented.Doubleortripleunderscorings of compoundsmeantthatin addition tothe translationofthecompound itselfthe translationofoneormoreofitselementswasdesiredinordertoutilizeoptimallytheinformationstoredinthemachinedictionary.Whereappropriate, theunderscoredexpressionswerereducedtothereferenceform(nominative singular,infinitive,etc.).Thetermswerethenpunchedoncardsandreadintothecomputerinthesequenceoftheiroccurrenceinthetext.Read-inofthethreepartialinterrogationbatcheswasinthesequenceofascendingjob numbers.ThedictionaryusedinthistextdidnotcontaintheentireA-to-Zstock ofvocabularybutwasamicro-glossaryspeciallycompiledforthepurposesofthistest.Thisfactalreadypointstothemodelcharacterofthetest.Theoutput unitswereprintedoutbyanOFF-LINEhigh-speedprinter.Thissecondmodeltestrundifferedfromthefirstmodeltestrun[cf.ReportüDBw-MüV-Az.:55-05(30)dtd14Oct.1964]inthatitpresupposedlargequantitiesofdata.Whileinthefirsttestsortinghadbeencircumvented,thesecondtestincludeda sorting(SORT-2)programusingfourmagnetictapes.Sincethesorting procedurehasalreadybeendiscussedinReportüDBw-MüV-Az.:55-05(30)of10Dec.1964, it neednotbe describedhere.
I. FORMATOF OUTPUTLISTS
WhathasbeensaidabouttheformatoftheoutputlistsinReportsüDBw-MüV-Az.:55-05(30)of14Oct.and10Dec.1964istruealsofortheoutput listsproducedinthepresenttestwiththeexceptionsthatinthepresenttestthelistshaveatitlelineandeachpartialinterrogationbatchbeginsonanewpage.Print-outofmorethanonepartialbatchisinthesequenceofthealphabetical order oftheabbreviated namesofthetranslators.
II. INTERPRETATIONOF SOME“MISSING” NOTATIONS
1. Themissingnotations,someofwhichwereintroducedintentionallyforreasons ofillustration, areattributableto the followingcauses:
a. Interrogation ofcompoundswith variablecontext-related elements
Examples: | freakmidgetcraft | (GRE8969034) |
| midget-typesubmarine | (GRE8969043) |
| cycliccontrolsystem | (HER8970029) |
| low-powergainrecovery | (MUL8968038) |
Insome casesinterrogation withoutthe variableelementswassuccessful.
Interrogationofwordsandwordcompoundswhichoccuras“quasi-technicalterms”incertaincontextsandwhichbecauseoftheirelusive characterare notcontained inthe dictionary.
Examples: | ASWpackage | (GRE8969025) |
| oscillatorymode | (HER8970005) |
| hydraulicallyboosted | (HER8970037) |
| distributedfashion | (MUL8968030) |
a. Spelling variants
Examples: | antisubmarineairbarrier | (GRE8969047) |
| travelling-wavemaser | (MUL8968012) |
Interrogation of the alternative spellings (anti-submarine air barrier;traveling-wave maser)was successful.
b. Interrogationofexpressionswhich,strictlyspeaking,cannotberegardedastechnicalterms
Examples: | porpoise | (GRE8969036) |
| oceanpassage | (GRE8969049) |
| stocking | (HER8970024) |
c. Uncorrectedpunchingerrors
Examples: | artificalfeedback | (HER8970040) |
| artificalfeel | (HER8970042) |
d. Inaccuraciesin theoriginaltext
Intext64/18968,line23,theletters“bL”wereinterpretedasanabbreviation.However,theyarenotanabbreviationbuttheproductofthe twoquantities“b”and“L.”Forthesakeofclaritytheproductshouldhave been written “b×L.”
e. All other “missing” notations may be interpreted as blanks in thedictionary
Examples: | advanceradarpicket | (GRE8969019) |
| missile-launcher | (GRE8969045) |
| stabilityaugmentation | (HER8970002) |
| artificialfeedbackfeel | (HER8970039) |
| maserline | (MUL8968013) |
| gainrecovery | (MUL8968039) |
Inmanycases,however,themissingequivalentscouldhavebeenderivedfrom theinformation actuallyprinted out.
2. Thejustificationofthewarningtothetranslatornottoacceptblindly everythingprintedoutbythemachineisdemonstratedbythefollowingexamples:
a. Text64/18969,line 12:“weather beacon.” The Germanequivalents“Wetterboje”and“Wetterbake”(GRE8969021)printedoutbythenachinearenotverymeaningfulinthisparticularcontext.Adestroyermayratherserveasa“Wetterstation(weatherstation)”or“Wetterschiff(weathership).”
b. Text64/18970,line18:“loop.”Whatismeanthereisa“servoloop” (“Regelkreis”);theword“loop”withoutaqualifyingadditionisnotspecific enough. The equivalents under “loop” (HER 8970 028),therefore,are notapplicable.
a. Text64/18970,line28:“displacement.”Theequivalentsprintedout underHER8970038arewronginthiscontext.Theweaknessespointed upabovearenottobeblamedonthemachineortheprocedurebutareinherent inthe language.
II. OUTLOOK
Practicalapplicationoftheproceduredevelopedsofar,aprocedureproveninasecondsuccessfulmodeltestrun,nowdependsonwhentheFederalArmed Forcescomputercanbeusedinordertotransfertheentirepunch-card informationontomagnetictape.Organizationalandprogrammingpreparatory workforthissignificantsteparealreadyunderway.Inaddition,workonthenewcomplex“processingofvocabularypassedbytheterminologyboards”hasbeeninitiated.
TEXT-RELATEDGLOSSARIES ANDMACHINE-PRODUCEDENGLISH-LANGUAGETECHNICAL TEXTS
(1) Onecommonpracticeistocreditanyshipwithahullnumber startingwithDasbeingperseanASWship.Tobesure,destroyers(DD),escorts(DE),andfrigates(DL)allhave ASWcapabilities.Sodoallothertypesofships.Thebowof
(5) anoceanliner,ifitrammedasubmarine,wouldbeamightyASWweapon.ThisdoesnotmakemerchantshipsintoanASWforce.Isaguidedmissiledestroyer(DDG),oraradarescortpicket(DER),anymoreanASWcraft?
Shipsareinherentlymulti-purpose,evenwheneffortsaremade
(10) tospecializetheirfunctions.Theversatiledestroyer,our traditionalASWsurfacecraft,cananddoesserveasanti-airscreen,advanceradarpicket,torpedoboat,weatherbeacon,andevenasanemergencypowerplantforagood-sizedcity. Itevenmakesaneffectivetransportandcargoship.
(15) Intothe“ASWpackage”(latelybroadenedintosomethingcalledunderseawarfare,orUSW)havegoneahodge-podgeofships.AndapotpourriofprojectshavebeenlabelledASW,including suchthingsasminesandminedetectors,noisemakersand deceptiondevices,submarinemachinery,testbargesand
(20) calibrationranges,hydrographicandoceanographicsurveys,long-rangebasicprograms...,bathyscaphs,freakmidgetcraft, andstudiesofthevocabularyofporpoises.
WarwilldemandseveralratherdifferentASWmissions.The tacticsofconvoyprotectiondifferfromthoseofahunter-killer
(25) groupfreetopursuesubswherevertheymaybefound.Theproblemofguardinganamphibiouslandingperimeteragainstcoastal
ormidget-typesubmarineshaslittleincommonwiththehuntingdownofsilentmissile-launchershoveringdeepinunfrequentedwaters,Maintaininganantisubmarineairbarrieracrosscritical
(30) oceanpassagesdiffersmarkedlyfromallthese.
Appendix14
TranslationVersusPosteditingofMachineTranslation
Thisstudyreportstheresultsofasmallexperimentdoneforthepurposeof obtainingsomefactsregardingtheprocessofposteditingmachine-translation outputascomparedwiththeprocessofordinarytranslation.Inparticular,informationwasdesiredconcerningtherelativespeedandease(ordifficulty)of posteditingascompared with thoseoftranslation.
Avarietyoftranslators(i.e.,commercialfree-lancetranslators,governmentin-housetranslators,governmentcontracttranslators,andbilingualpersonswhodidnotordinarilyengageintranslationwork)weresentapacket containing(1)a1,135-wordexcerptfromaRussianbookoncybernetics,MachinaiMysl',whichtheyweretotranslateandprovidetypedcopyoftheirtranslations;(2)a765-wordexcerptfromthesamebook;(3)aprint-outofthemachinetranslationof(2),whichwastobeposteditedandtyped;and(4)a questionnaire(Exhibit1, page99).
Thetranslatorsweretokeepacarefulrecordoftimespentintranslating,editing,postediting,and (for some)typing.
Thoserespondingwere:
(a) threetranslatorsemployedbycommercialtranslationagencies (Numbers2,14,and23);
(b) eleventranslatorswhoheldcontractswiththeU.S.JointPublications ResearchService (Numbers1,3,6,7,11,13,15,16,17,18,and22);
(c) sixfull-timetranslatorsemployed,in-house,byanagencyoftheU.S.
Government(Numbers4,9,10, 12, 19, and 21); and
(d) threemembersofthefacultyoftheRussiandepartmentattheDefense LanguageInstitute(Numbers5,8,and20).Thesethreearelanguageinstructorsand notprimarily translators.
EASE OFPOSTEDITING
Eighttranslatorsfoundposteditingtobemoredifficultthanordinarytranslation.Sixfoundit tobeabout thesame, and eightfound iteasier.(Onetranslatorindicatedthathefoundthedegreeofdifficultytoliebetween“easier”and “thesame.”)
Thus,fromtheanswersreceived,itcanbeseenthatthetranslatorswerealmost evenly divided intheir opinionsonthedifficulty ofpostediting.
Thepointofinterestisthatthemoreadept(rapid)translatorsfound posteditingmoredifficultthandidtheslowertranslators(seeExhibit2,page100).Theapparentparadoxthatthosepeoplewhothoughtposteditingwasmoredifficultweremoreproficientatitthanthosewhofoundittobe“thesame”or“easier”isexplainedbythefactthatthosewhofounditmoredifficultarethesame people whoare the mostadept at translation.
FromExhibit2onemayseethatsixoftheeighttranslatorswhofoundposteditingtobemoredifficultthantranslatingwereamongthefasterhalf,andthatsixoftheeighttranslatorswhofoundposteditingtobeeasierthantranslating werein theslowerhalf.
Theaveragetranslationspeedsoftranslatorswereasfollows:thosewhofoundposteditingmoredifficult,11.9wpm;thosewhofoundposteditingeasier,
6.5 wpm; and thosewho foundposteditingabout the same, 7.9 wpm.
Theaverageposteditingspeedsoftranslatorswereasfollows:thosewho foundposteditingmoredifficult,9.4wpm;thosewhofoundposteditingeasier,
8.6 wpm; and thosewho foundposteditingabout the same, 8.0 wpm.
RELIANCEONTHEORIGINAL
Onlyonetranslator(number2)indicatedthatheseldomhadtorefertotheoriginal(8a)inordertoposteditmachinetranslation.Eighttranslatorsindicatedthatitwasalmostnecessarytotranslatetheoriginal(8b),and14translatorsansweredthatthedegreeofreliancefellbetweenanswers(8a)and(8b).Itisof interesttonotethatmostofthosewhosaidtheyhadtotranslatetheoriginalwere thefastest translators (andperhaps thebest at translation).
POSTEDITINGANDTRANSLATIONSPEED
Translation Speed
Thefastesttranslationspeedwas19.5wpmbytranslatornumber1andtheslowestwas4.2wpmbytranslatornumber23.Thedifferencebetweenthe translationratesofthefastestandslowestwas15.3wpm;themeanspeedwas
8.7 wpm; themedianwas 7.6 wpm;the mode was6.3wpm(Figure2).
PosteditingSpeed
Thefastestposteditorwastranslatornumber5,witharateof12.7wpm.Theslowestwastranslatornumber23,witharateof3.9wpm.Thedifferencebetweentheposteditingratesofthefastestandslowesttranslatorswas8.8wpm;themeanposteditingspeedwas8.7wpm;themedianposteditingspeed was 9.2wpm; themode was10.2 wpm(Figure2).
FIGURE2.Speed(inwordsperminute)oftranslationandpostediting.
OBSERVATIONS
(a) Themeanspeedfor both translationand posteditingwas 8.7 wpm.
(b) Althoughthefastesttranslatorcouldtranslatealmostfivetimesasfastastheslowesttranslator,thefastesttranslatorcouldposteditonlyaboutthreetimesasfast asthe slowestposteditor.
(c) Ofthe23respondents,ten(3,6,7,11,13,14,15,16, 17,and22)indicatedthattheyhadhadpreviousexperienceatposteditingmachine-translation output (one translator said that he had postedited 93,000words). Ofthis group, halfhad slower rates forposteditingthanforordinary translation. Almost exactly the same ratio (numberslower:numberfaster)heldoverall(11/23 slower:12/23 faster).
(d) Themeanposteditingspeedoftheexperiencedposteditorswas 8.6 wpm.Themeanposteditingspeedofthosewhodidnotindicatehaving experience at posteditingwas8.8wpm.
(e)
1. Thefourfastestposteditorshadanaverageposteditingrateof11.8and anaveragetranslation rate of11.5.
2. Thefourslowestposteditorshadanaverageposteditingrateof5.3andanaveragetranslationrate of6.1.
3. Thefourfastesttranslatorshadanaverageposteditingrateof10.4and anaveragetranslation rate of16.3.
4. Thefourslowesttranslatorshadanaverageposteditingrateof8.5andanaveragetranslationrateof5.3.Thusthedifferencebetweenthefaster andslowerofthesetwogroupswasonly1.9wpmforposteditingbut11wpm fortranslation.
5. Thefastesttranslator'sposteditingratewasthemedianforpostediting(9.2 wpm).
6. Theslowesttranslatorwas also theslowest posteditor.
IMPACT OFPOSTEDITING ONOUTPUTRATES
Figure3indicatesforeachtranslatorhisspeedsforposteditingandtranslation.Itisfairlyobviousfromaglanceatthischartthatfasttranslatorswillloseproductivityifgivenposteditingtodo,whereasslowtranslatorswillgain.
If translators are given postediting to do, then, contrasted with theirtranslation rates:
Translators1-4willshowanaggregatelossof23.6wpmor34percentinoutput.
Translators5-8willshowanaggregategainof1.7wpmor5percentinoutput.
Translators9-12willshowanaggregategainof2.1wpmor3percentin output.
Translators13-15willshowanaggregategainof0.6wpmor3percentinoutput.
Translators16-19willshowanaggregategainof6.3wpmor20percentin output.
Translators20-23willshowanaggregategainof12.6wpmor37percent in output.
Thus,itmaybeseenthatposteditingmachinetranslationtendstoimpede the rapid translators and assistthe slowtranslators.
FIGURE3.Percentagegainorlossinoutputfrompostediting.
TIMESPENTPREPARINGTHE COPY
Practicevariedinproducingtypedtranslations.Somerespondentscombinedvariousprocesses.Tentranslatorsperformedtranslation,editing,and typingasseparateoperations.Thetotalamountoftimethese10spentonthevariousprocesseswas asfollows:
Translation 1,697minor63percent
Editing 365minor13percent
Typing 645minor24percent
Averagetypingspeedoftranslatorswasonly18wpm.Notalltranslatorsproduced a typedcopy.
WILLINGNESSTOPOSTEDITMACHINETRANSLATION
Twentytranslatorsansweredquestion9a.Ofthe20replies,eightwere negative,11wereaffirmative,andonewasaqualifiedaffirmative(yes,onlyifstraighttranslationisnotavailable).Ofthosewhowoulddoposteditingatalowerratethanthatreceivedfortranslation,overhalf(6/11)wouldbewillingto postedit for one halforlessthanthe ratepaidfortranslation.
No.of Translators Rate
1 1/3
1 1/3−1/2
4 1/2
1 2/3
1 2/3−3/4
1 3/4
2 4/5
Itisofconsiderableinterest(especiallyinasocietythatisallegedly materialistic)tocomparethewillingnesstoposteditatreducedrateswiththe respondents'speedsoftranslationandpostediting(seeExhibit2).Forexample,althoughtranslatornumber13indicatedthathewouldacceptarateof1/3for postediting,hispost-editingspeed(7.0wpm)isactuallylowerthanhis translationspeed(7.3wpm).Onlyonetranslator,number22,would havebrokeneven.Theother10wouldbewillingineffect,todothesamenumberofhours ofworkfor less pay.
Ofthosetranslatorswhoindicatedtheirwillingnesstoposteditatreducedrates,oneoutofthreewerecommercialtranslators,threeoutofsixweregovernmentin-housetranslators.Sevenoutof11weregovernment-contracttranslators (aneighthgavea qualified“yes”).
TRANSLATORS'REACTIONSTOPOSTEDITING
Twentyrespondentstookthetimetogivetheirreactionstotheprocessofposteditingmachine-translationoutput.Althoughtheirremarksmakeinterestingreading,forthepurposeofthisstudywewillonlysummarizesomeoftheopinionsexpressed:
Mostofthetranslatorsfoundposteditingtediousandevenfrustrating.Inparticular,theycomplainedofthecontortedsyntaxproducedbythemachine.Othercomplaintsconcernedtheexcessivenumberoflexicalalternatives providedandtheamountoftimerequiredtomakepurelymechanicalrevisions.Anumberoftheexperiencedposteditorsremarkedthat,althoughthematerialin thisstudyhadbeencarefullykeypunched,theyhadfoundintheirpreviousexperience that carelesskeypunchingwas aconsiderabledetriment.
Althoughnotranslatorcommentedthathereallylikedtoworkwiththemachineoutput,a numberstatedthat theyfoundthe output servedasanaidinthetranslationprocess,particularlywithregardtotechnical terms.
(Thedifficultyintryingtoreflectaccuratelytheopinionsofthetranslatorsmaybeappreciatedwhenonereadsthefollowingcommentmadebytranslatornumber23):“Inconclusion,theMTwasanaidandmadetranslationeasier,butwhenallthe time used isfiguredup,was notasfastorprofitable.”
TRANSLATORS'RECOMMENDATIONS
Severaloftherespondentsweremovedtosuggestpossibleimprovementsin themachineoutput:
Number21
“Ibelieveitmightdowelltoscanthecopytobetranslatedandprovidea translatorwitha vocabularyand thenallowhim totranslateitdirectly.”
Number15
“Syntax-wise,sometimeinposteditingmightbereducediftheeditorhas knowledgeofthedegree ofdissemination to begiven theend product.”
Number 3
“Amajorimprovementwouldbeamuchbolderprogrammingofword-blockswhichhavea single orat most dualword Englishequivalent.”
Number 9
“Morespaceforcorrectionswouldbeawelcomeformatmodificationandwould, incidently, helpassure accuracy ifthe text isto beretyped afterediting.”
CONCLUSIONS
Inviewofthesmallsamplethatformedthebasisforthisstudy,anyconclusionsmustbetentative.Withthisinmind,onemightdrawthefollowingconclusionsfrom thisstudy:
1. An adepttranslator's skillswill probablybewastedonpostediting.
2. Theslowerthetranslator,thegreaterthelikelihoodthathisoutputcan beincreased by having himpostedit machinetranslation.
EXHIBIT1.
QUESTIONNAIRE
1. Exactly how much time (hours and minutes) was required totranslatedocumentnumber2?
2. Exactlyhowmuchtime(hoursandminutes)wasrequiredtoeditthe
translation?
3. Exactlyhowmuchtime(hoursandminutes)wasrequiredtotypethistranslation?
4. Howmuchtimewasrequiredtoeditdocumentnumber3?
5. Howmuchtimewasrequiredtoedittheeditedcopy(ifthiswasnecessary)?
6. Howmuchtimewasrequiredtotypedocumentnumber3?
7. Howdidyoufindtheposteditingprocesstobecomparedtothe processoffulltranslationfromtheoriginal?
Easier? o
MoreDifficult?o
AbouttheSame? o
8. Checktheappropriatebox:
oa.“Itwasnecessaryalmosttotranslatetheoriginalinorderto
properlyeditthemachineoutput.”
ob.“Iseldomhadtorefertotheoriginal.”
oc.“Iplacednotsogreatrelianceontheoriginalasquestionnumber
8,butgreaterthanindicatedbyquestionnumber9.”
9.a. Wouldyoubewillingtoregularlyposteditsimilarmachine- translationoutputifyouweretobepaidatalowerratethanyouearnfor translatingfromadocumentintheoriginallanguage?
Yes o No o
9.b.
Ifyes,whatisthelowestrateyouwouldaccept?
Circle.
4/52/33/41/21/31/41/5oftheconventionaltranslationrate.
10. Yourcandidcommentsandyourreactionstotheexperienceof posteditingthemachineoutputareinvitedbelow.
Appendix15
EvaluationbyScienceEditorsofJointPublicationsResearchServiceandForeignTechnologyDivisionTranslations
FiveJointPublicationsResearchService(JPRS)translationsandfiveForeignTechnologyDivision(FTD)translations(fourposteditedmachinetranslationsandoneuneditedrough-drafthumantranslation)weresenttosixscienceeditorsoftheAmericanAssociationfortheAdvancementofScienceandtoonetranslationagencyowner.Thetranslationswererankedaccordingtotheirqualityasscientificwritings.TheJPRStranslationswere,ingeneral,rankedhigherthantheFTDtranslations.TheagreementwasalmostunanimousthattheworsttranslationofallwastheFTDuneditedrough-drafthuman translation.
WerequestedthattheClearinghouseforFederalScientificandTechnical InformationprovideuswiththesixmostrecentlyacquiredRussian-to-EnglishtranslationsfromJPRSandFTD.Whenthesearrived,weeliminatedthreetranslations–twobecauseoflengthandonebecausewewantedtoincludeanuneditedroughdrafttranslationinthesample.Thetentranslationsthatformed the samplewere keyed asfollows:
(A) AbsorptionofRadioWavesbyAirBehindaShockWave,FTDAD605883,FTD-MT-63-74,byT.V.BazhenovaandYu.S.Lobastov9/62
(B) TranslationsonSovietConstructionandBuildingMaterialsIndustryNo.65,USSR(Large-ScaleBuildingActivityinProcessThroughouttheSoviet Union) JPRS:27,267,TT: 64-5152211/6/64
(C) USSRIndustrialDevelopment,SovietChemicalIndustry,No.188 JPRS: 27,271,TT: 64-5152611/6/64
(D) Research on Heat Exchange in Vacuum by A. N. Devoyno, FTD-
MT-63-09 EditedMachineTranslation,20 Feb. 1964
(E) TestingandOzokeriteBacillusCultureLiquidforToxicitybyCh.B.
Bayriyev- USSR-JPRS:27,268, TT:64-5152311/6/64
(F) ThereisSuchaMachinebyYe.Temchin,FTD-TT-64-1170/127Oct.1964
(G) MethodofDetectionandIdentificationofRemoteExplosionsbyV.S.
Voyutskiy, FTD-MT-64-407,Edited MachineTranslation,6 Oct.1964
(H) PreventionofBrucellosisbyI.N.Ivashurova-USSR-JPRS:27,269TT: 64-5152411/6/64
(I) Investigation of Optical Oscillator on Ruby at Liquid NitrogenTemperature by V. K. Konyukhov, L. A. Kulevskiy, and A. M.
Prokhorov,FTD-MT-63-100, 21 Oct.1963
(J) TranslationsonSovietAgricultureNo.44,JPRS:27,272,TT:64-515276November1964
Thetranslationswerethenstrippedofanyidentifyingmarkersandphotoreproduced.
ThesampleswerethensenttothescienceeditorsattheAmericanAssociationfortheAdvancementofScienceandtotheownerofacommercialtranslationagencywhodidnotreadRussianbutwasexperiencedintheediting oftranslations. These editorsweregiven the followinginstructions:
Whatisneededisarank-orderingoftheenclosedmaterialswiththebestdocumentbeinggiventhenumber“1”andtheworstdocumentnumber“10.”Thebasisforjudgementwouldbethestandardswhichyouasascientificeditor normallyapply.Whatweareafterisyourratingofexcellenceorlackof excellenceofthewritinginthesedocuments.Inotherwords,howdoesthestuff read?
Inadditiontoyourrank-orderingoftheseitems(whichthusshowstheirstandingrelativetoeachother),wewouldwelcomeyourcommentsastohow theyimpressyouonanabsolutescale.Thatis,althoughnumber“1”willbethe best ofthe total group, it still maybean exampleofpoorscientificwriting.
TABLE7.RankingofFTD(lettersinparentheses)andJPRSTranslations
| Best← |
| Rating→ |
|
|
|
| Worst | |||
EditorNumber | 1 | 2 | 3 | 4 5 | 6 | 7 | 8 | 9 | 10 | ||
1 | (Commercial | H | (G) | (D) | C | (I) | E | (A) | B | (F) | J |
| firm) |
|
|
|
|
|
|
|
|
|
|
2 |
| C | H | J | (G) | E | (I) | (D) | B | (A) | (F) |
3 |
| E | H | C | (G) | (D) | B | (A) | (I) | J | (F) |
4 |
| E | H | C | B | J | (G) | (A) | (I) | (D) | (F) |
5 |
| (G) | C | H | E | (A) | (D) | (I) | J | B | (F) |
6 |
| C | H | E | (G) | B | J | (D) | (I) | (A) | (F) |
7 |
| H | E | (G) | (D) | C | B | (A) | (I) | (J) | (F) |
Resultsoftheeditors'rankingaregiveninTable7.Inordertoobtaina numericalratingofthetranslations,thoseappearingincolumn1weregivenascoreof100;eachcolumnwasscored10pointslowersothatthoseincolumn
10 were given a rating of 10. On this basis the numerical scores of thetranslationsare asfollows:
Translation | Score | TranslatingAgency |
H | 640 | JPRS |
C | 580 | JPRS |
E | 550 | JPRS |
G | 530 | FTD |
D | 360 | FTD |
B | 310 | JPRS |
I | 270 | FTD |
J | 270 | JPRS |
A | 260 | FTD |
F | 80 | FTD |
IfbothFTDandJPRShadhadequalnumbersoftranslationsoneithersideofthemedian(55),theirscoreswouldeachhavebeen1,925(halfofthetotal 3,850pointspossible).ActuallytheJPRStranslationsscored2,350pointsandthe FTDtranslationsscored1,500points.
Concerningtheabsolutemeritofthesetranslations,somecommentsofeditors mightbeinformative:
Number4.“Iconsiderthis(E-JPRS)apaperofaveragemerit,which,from thestandpointofstyleandclarity,wouldbeacceptableforpublicationinatechnicalscientificjournal.”
Number4.“‘Whatisitallabout?'sayspaperF.Whatindeed!Thisoneishopeless.”
Number3.“(EandH)couldbepublishedasisorwithverylittlerephrasing.”
Number2.“Asscientificwriting,Cisacceptable,H,J,G,andEarefairand could befixedup with alittle editing.Therestgofrompoorto verypoor.”
Althoughthesamplewastoosmalltoallowonetogeneralizewithagreatdealofconfidence,theconsensusoftheeditorsconcerningtherelativeworth (orworthlessness)ofsomeofthetranslations(e.g.,HandF)tendstoincreaseone's confidence inthefindingsofthisstudy;i.e.,theJPRStranslationsaresomewhatbetterthantheposteditedmachinetranslation,andtheuneditedrough-drafthuman translationistheworstofall.
Thisconclusion,whencoupledwiththereportfromtheGovernmentPrintingOffice(Exhibit1)concerningthegraphicartsqualityofthesesamples,wouldtendto indicatethat JPRS translationsaresuperiorto FTDtranslations.
StatisticalreliabilityfiguresbasedontheseratingshavebeencomputedbyProfessor J. B.Carroll.Theyare asfollows:
Kendall'sW.,acoefficientofconcordance,basedontheJPRS-FTDcomparisonratings,is0.724,wellbeyondthe0.001level,butnotashighas 1.00, thefigure indicatingperfectreliability.
TheapplicationoftheMann-WhitneyU-testtothesummedratingsgivesa valueofU=4.5.Forthecasewhere5valuesarebeingcomparedwith5values, thisissignificantonlywithaprobabilitybetween0.096and0.15.Thisisnotsufficientlysignificanttorejectwithanyconfidencethenullhypothesisthatthe two sets oftranslationare drawn from thesamepopulation.
ThesummedranksonwhichtheMann-Whitneytestwasbasedareas follows:
JPRS FTD
H13 (G) 24
C19 (D) 41
E22 (I)50
B46 (A) 51
J 50 (F)69
EXHIBIT1.
March24,1965
Dr.A.HoodRoberts, ExecutiveSecretary
NationalAcademyofSciences NationalResearchCouncil
2101ConstitutionAvenueWashington,D.C.20418 DearDr.Roberts:
Inanswertoyourrequestforanevaluationofthequalityoftheprintingof thetranslatedmaterialwhichyouleftwithme,wehavearrivedatthefollowingbreakdown:
Rating
1. F Satisfactory
2. B,C,H, J Fair
3. G,E, D, A, I Poor
Group1:Thisisadequateperhapsonlybecauseitisdoublespacedandseems tobeblacker thantherestofthesubmissions.
Group2:Theprintingoftheseisverypoor,althoughnotsobadbutwhatthetextcanberead.Thedifficultyhereseemstobethattherehasbeenno attempttomaintaingoodinkcoverage,orgoodqualitycameraworkandplatemaking.Thepressworkisparticularlybadwheresmudgesarepermittedto appear across the printing.
Group3:Thisgroupcontainstheillustrations.Mostofthemareevidently toomanytimesremovedfromtheoriginal,ortheyweremadefromduplicatorcopies(Xerox,Ozalid,etc.)whichalwayslosemuchofthedetail.If theoriginalcopyhadbeenusedascameracopy,Iamsuremuchbetterresults couldhavebeenobtained.Iftheoriginalcopywasused,thentheresultsare simplybadhandlingorinexperiencedpersonnel.Thereseemstobelittlereasonfor reproductionsaspoor asthislastgroup.
Sincerelyyours,JAMESL.HARRISON
PublicPrinter
By: FrankH.Mortimer
TypographyandDesignManagerUnitedStatesGovernmentPrintingOffice
https://blog.sciencenet.cn/blog-362400-730693.html
上一篇：ALPAC 黑皮书 7/9： 附录9-10
下一篇：ALPAC 黑皮书 9/9： 附录 16-20
