Regardless of the task, Speech Recognition (SR) systems make recognition errors. To evaluate technology performance, the Consortium conducts ongoing evaluations and benchmarking of various SR powered captioning and transcription systems.

dinary digital image

 

 

 

 

 

 

 

Using SR to caption and transcribe live or digitized speech is a complex and significantly different challenge than using SR for dictation. Spontaneous speech that typically occurs in traditional lectures and presentations is acoustically, linguistically, and structurally different than read speech or speech used to create written documents.Recognition accuracy for this unique task is driven by a complex interaction of both technical and non-technical variables, including the underlying speech engine, statistical models, computing hardware, microphones, audio quality, individual speech characteristics, and other myriad environmental factors.

The Consortium evaluates both “speaker dependent” and “speaker independent” systems and strives to understand and quantify variables affecting accuracy.  Speaker dependent or trained systems are typically characterized by the creation of a personal voice profile that contains data about an individual’s speech characteristics and language usage. For these systems, the quantity and quality of training data is highly correlated with recognition accuracy.  Speaker independent systems generally do not retain data about individual users and are designed to work without previous knowledge of an individual speaker.

Evaluations focus on calculating recognition Accuracy or Word Error Rates.  Accuracy is generally defined as the number of correct words transcribed divided by the total number of words spoken.  Conversely, Word Error Rate is the number of incorrect (misrecognized) words divided by the total number of words spoken (* 100).

For example:

  • 100 words spoken, 80 words transcribed correctly, 20 words transcribed incorrectly
  • Accuracy = 80%
  • Word Error Rate= 20%

SR systems are evaluated by transcribing recordings from a  “Lecture Test Set” that  includes over 120,000 words, sampled from  different speakers chosen from a wide range of academic disciplines. Separate Test Sets are used for benchmarking performance for various dialects (i.e. North American, UK, and Australian English). Word Error Rates are subsequently calculated by comparing an edited reference transcript to a raw, unedited transcript generated by the system being evaluated.  Word Errors are also classified into various categories in an effort to understand factors that affect system performance:

  • Substitutions:  Word spoken is recognized as another word or words
  • Insertions:  SR interprets sound artifact as word and inserts text.  This often occurs when breathing is interpreted as a word
  • Deletions:  When word is in fact spoken, but no word is recognized

Evaluations to date include analysis of SR systems from IBM, Nuance, Google – Youtube’s automatic captioning feature, Adobe Premiere Pro CS6 software, Microsoft speech engine, and Koemei.

Our partners:
cheapest essay writing service
custom essay
custom essay writing
  • new jerusalem bible ebook free download
  • jukebox software downloads
  • download elifoot 98 registro windows 7
  • filme rio rmvb dublado
  • oficina g3 download humanos
  • tucker and dale vs evil dvdrip free
  • router test download
  • where to download ps2 bios files
  • jkt48 tenshi no shippo clean
  • desbloqueio ps3 3.55 com multiman & bdemu
  • free offline dictionary for nokia x2
  • can you youtube videos to your iphone
  • tp link tl wn321g driver download for xp
  • seethamma vakitlo sirimalle chettu songs free southmp3
  • how to your own dvd onto itunes
  • java jre windows 7 32 bit
  • download woman in black 2012 full movie
  • jenni rivera songs free
  • rakkamma rakku song free
  • kpop video 3gp download
  • deus ex human revolution trainer steam
  • sabitri roy mp3 free
  • app statistics s
  • nfs kernel server ubuntu package
  • turbo c download for windows 7 64 bit
  • hp quality center 10.0
  • lightroom 4 release candidate
  • mp3 wawa marisa harta dan surga
  • programi za kompjuter free besplatno
  • facetime for android free download
  • download notepad for blackberry
  • criminal mp3 songs free skull
  • adobe reader 7 free for vista
  • south 3gp video songs
  • dictionary english free for mobile
  • closer inoue joe english version mp3 download
  • big b hooligan free mp3
  • vlc video player download for windows 7
  • yume no tsubasa duet mp3
  • jharkhand secretariat exam 2012 admit card download
  • need for speed 7 free full version
  • love 2d game engine download
  • bluetooth pl w komputerze download
  • homens de honra avi dublado
  • acer emachines e732z drivers for xp
  • global distribution system
  • download internet explorer for mac os x
  • revit 2009 rendering library download
  • radha soami shabad
  • old mario games free full version
  • bluetooth hack download for nokia 5800
  • civilization 3 free full version mac
  • bartpe download usb bootable
  • english movies for pc
  • ganja bus free mp3
  • naruto uzumaki mugen character download
  • avast 7 activation code
  • pokemon black 2 download free
  • download ahange sami beigi fogholade
  • beethoven piano sonata 8 download
  • cars instrumental gary numan download
  • game pc scrabble indonesia
  • bob marley one love mp3 free
  • velamma episode 13 free in hindi
  • i'm your man mp3 2pm
  • tamil film songs free with video
  • games fifa 2011 pc
  • download phim trA?i tim mA?a thu hd
  • ku wo yin yue he 2012
  • blackberry tour pictures to mac
  • hacker evolution untold deutsch
  • filme o filho da noiva legendado
  • schach download free deutsch
  • nik software color efex pro 3.0 download free
  • vat return form download rajasthan
  • mysql database server for windows xp
  • honey singh mp3 songs pk
  • lupin the third episodes
  • introduction to flight anderson kennedy's
  • 1080p hd tamil video free download
  • heroes season 4 download mp4
  • swap magic 3 download free software
  • bubble shooter game for android
  • 50 cent 21 questions mp3 free
  • download lagu istikharah cinta mp3
  • twisted metal 3 psx free
  • ubuntu 10.10 download usb
  • teamspeak 2 download mac
  • vs 2010 black theme free
  • tamil movie songs for free
  • the price is right sounds
  • waiting on the other side bruno mars download
  • war chess game download for windows 7
  • entity framework download 2010
  • virtual dj crack 6.1 free download
  • multi ip changer tibia 7.1 9.44 download
  • anh la ai dan nguyen
  • never eat alone ebook download
  • nokia n95 8gb themes free downloads on zedge
  • a free of itunes
  • burp suite windows
  • logitech harmony windows download