文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > DISCRIMINATIVE MIXTURE WEIGHT ESTIMATION FOR LARGE GAUSSIAN MIXTURE MODELS
侵权投诉

DISCRIMINATIVE MIXTURE WEIGHT ESTIMATION FOR LARGE GAUSSIAN MIXTURE MODELS

This paper describes a new approach to acoustic modeling for large vocabulary continuous speech recognition (LVCSR) systems. Each phone is modeled with a large Gaussian mixture model (GMM) whose context-dependent mixture weights are estimated with a senten

DISCRIMINATIVE MIXTURE WEIGHT ESTIMATION FOR LARGE GAUSSIAN MIXTURE MODELS Francoise Beaufays, Mitchel Weintraub, Yochai Konig

Speech Technology and Research Laboratory SRI International, Menlo Park, CA. This paper describes a new approach to acoustic modeling for large vocabulary continuous speech recognition (LVCSR) systems. Each phone is modeled with a large Gaussian mixture model (GMM) whose context-dependent mixture weights are estimated with a sentence-level discriminative training criterion. The estimation problem is casted in a neural network framework, which enables the incorporation of the appropriate constraints on the mixture weight vectors, and allows a straight-forward training procedure, based on steepest descent. Experiments conducted on the Callhome-English and Switchboard databases show a signi cant improvement of the acoustic model performance, and a somewhat lesser improvement with the combined acoustic and language models.

ABSTRACT

1. INTRODUCTION Many factors contribute to the relatively high error rates observed in LVCSR systems (e.g. diversity of speaking styles, pronunciation variants, variable degrees of articulation, noises, channel e ects). By enlarging the set of possible acoustic realizations for each phone or phone state, these factors cause the acoustic models to have broad overlapping distributions, which in turn increases the chances of misrecognition. For this same reason of acoustic diversity, the amount of training data and the e ciency with which it is used are critical factors in the performance of LVCSR speech recognizers. The modeling technique described in this paper addresses these two issues: it makes e cient use of the training data by allowing a maximum of data sharing between di erent states of each phone, and it attempts to limit the overlap between phone models by training them in a discriminative fashion. In the past few years, the trend in acoustic modeling has been to de ne triphone clusters, and model each cluster with a relatively small number of Gaussians. Such clusters were derived through acoustic-based agglomerative clustering of triphone states (e.g. 1]), or with the help of binary decision trees based on contextual linguistic features (e.g. 2]). Triphone-cluster modeling o ered a more detailed modeling of co-articulation e ects than previously used phonetically tied mixture (PTM) models, and improved recognition accuracy 1, 2]. Recently, however, large PTM systems were shown to o er an attractive alternative to triphone-cluster models

3]. N-best lists rescoring experiments on the CallhomeSpanish database showed that context-independent phone GMMs could lead to the same word error rates as contextdependent decision-tree models, provided that the number of Gaussians in each phone model was su cient, i.e. comparable to the total number of Gaussians in all the leaf nodes of the corresponding trees 3]. The main advantage of large PTM systems over small triphone-cluster models is the shari

ng of all the data aligned to a speci c phone to train its model. This advantage may however be o set by the increased acoustic space coverage of each model and hence by the increased possibility of overlap between model distributions. In this paper, we focuss on improving the discriminative power of large PTM systems by re-estimating the mixture weights of the phone GMMs for di erent contexts, using a discriminative optimization criterion. Various approaches to discriminative training of acoustic models have been proposed in the literature, e.g. 4, 5, 6, 7]. Because we wish to use the discriminative PTM models to rescore N-best lists of hypotheses, and because we believe that a global (sentence-level) training criterion will be more closely related to the recognition metric (here, the word error rate (WER)), we propose to train the models to maximize the average log-posterior probability of the correct transcriptions of the training sentences.

2. DISCRIMINATIVE MIXTURE WEIGHT ESTIMATION Each phone in the context-dependent (CD) PTM system is modeled with a Gaussian mixture model, according to N' X P iNg (xk ); p(xk j'; i)= g=1 g

(1)

where ' indicates the phone being modeled, i refers to a speci c context realization or triphone cluster of ', and Pgi and Ng (xk ) represent, respectively, the gth context-dependent mixture weight in cluster i, and the gth context-independent Gaussian distribution evaluated for the observation xk . In our implementation, triphone clusters are generated with linguistically-driven decision trees (DTs), so that the index i indicates the ith leaf node of the DT corresponding to phone '. (To be more precise, three DTs are built for each phone (one per state), and i indexes the leaf nodes across all three DTs.)

第1页

你可能喜欢

热门文档

相关文档

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email