Sign in
Author

Conference

Journal

Organization

Year

DOI
Look for results that meet for the following criteria:
since
equal to
before
between
and
Search in all fields of study
Limit my searches in the following fields of study
Agriculture Science
Arts & Humanities
Biology
Chemistry
Computer Science
Economics & Business
Engineering
Environmental Sciences
Geosciences
Material Science
Mathematics
Medicine
Physics
Social Science
Multidisciplinary
Keywords
(6)
Em Algorithm
Expectation Maximization
Gaussian Mixture
Gaussian Mixture Model
Parameter Space
Maximum Likelihood
Related Publications
(23)
Maximum likelihood from incomplete data via the em algorithm
Clustering Methodologies in Exploratory Data Analysis
Learning Mixtures of Bayesian Networks
The expectation...
Relationship between gradient and EM steps in latent variable models
Subscribe
Academic
Publications
On Convergence Properties of the EM Algorithm for Gaussian Mixtures
On Convergence Properties of the EM Algorithm for Gaussian Mixtures,10.1162/neco.1996.8.1.129,Neural Computation,Lei Xu,Michael I. Jordan
Edit
On Convergence Properties of the EM Algorithm for Gaussian Mixtures
(
Citations: 239
)
BibTex

RIS

RefWorks
Download
Lei Xu
,
Michael I. Jordan
We build up the mathematical connection between the ExpectationMaximization (EM) algorithm and gradientbased approaches for
maximum likelihood
learning of finite gaussian mixtures. We show that the EM step in
parameter space
is obtained from the gradient via a projection matrix P, and we provide an explicit expression for the matrix. We then analyze the convergence of EM in terms of special properties of P and provide new results analyzing the effect that P has on the likelihood surface. Based on these mathematical results, we present a comparative discussion of the advantages and disadvantages of EM and other algorithms for the learning of
gaussian mixture
models.
Journal:
Neural Computation  NECO
, vol. 8, no. 1, pp. 129151, 1996
DOI:
10.1162/neco.1996.8.1.129
Cumulative
Annual
View Publication
The following links allow you to view full publications. These links are maintained by other sources not affiliated with Microsoft Academic Search.
(
dx.doi.org
)
(
www.mitpressjournals.org
)
Citation Context
(119)
...Recent theoretical investigations on the asymptotic convergence properties of the EM algorithms for gaussian mixtures and the mixtures of densities from exponential families (Xu & Jordan,
1996
; Xu,
1997
; Ma, Xu, & Jordan,
2000
; Ma & Xu,
2005
; Ma & Fu,
2005
) provide a new view on the EM algorithm for ME...
Yan Yang
,
et al.
Asymptotic Convergence Properties of the EM Algorithm for Mixture of E...
...Parameter learning is usually implemented under the maximum likelihood principle by an expectationmaximization (EM) algorithm [2,
6
,7]...
...The first level is the inference of the component labels y. The second is the estimation of the parameters Θ given a fixed k ,w hich is usually implemented by maximizing the likelihood q(XN Θ )= N=1 q(xtΘ) with the help of the following well known EM algorithm [2,
6
,7]:...
Lei Shi
,
et al.
Learning Gaussian mixture with automatic model selection: A comparativ...
...It is shown in [
6
], that the EM algorithms are firstorder optimization algorithms, which provides a theoretical explanation for the slow convergence speeds...
Xudong Ma
.
LargeScale Clustering Based on Data Compression
...The computational complexity for convexNMF is of order n2p+m(2n2k+nk2) for Eq. (
25
)...
...proved to be nonincreasing, J(�(t+1)) ≤ J(�(t)). Following Xu & Jordan (
25
), we expand1...
...have a firstorder convergence rate, which is the same as the E M algorithm (
25
)...
...presented in Eqs.(26) and (
25
) depend on XTX only...
Chris H. Q. Ding
,
et al.
Convex and SemiNonnegative Matrix Factorizations
...For the example in Fig. 1(b), the task of getting an optimal classifier p(jx) is a subtask of estimating the parameters Θj = {μj, Σj ,α j} if they are unknown too, which is usually referred as parameter learning [7,
8
]...
...For Gaussian mixture introduced in Fig. 1(b), we start from the standard EM algorithm for the ML learning [7,
8
], which is here used as a benchmark for better insights on the Bayesian YingYang harmony learning and other related algorithms...
Lei Xu
.
Bayesian YingYang system, best harmony learning, and five action circ...
References
(20)
Classification using hierarchical mixtures of experts
(
Citations: 19
)
S. R. Waterhouse
,
A. J. Robinson
Published in 1994.
An Input Output HMM Architecture
(
Citations: 114
)
Yoshua Bengio
Conference:
Neural Information Processing Systems  NIPS
, 1995
Hierarchical Mixtures of Experts and the EM Algorithm
(
Citations: 1286
)
Michael I. Jordan
,
Robert A. Jacobs
Journal:
Neural Computation  NECO
, vol. 6, no. 2, pp. 181214, 1994
A New View of the EM Algorithm that Justifies Incremental and Other Variants
(
Citations: 196
)
Radford M. Neal
,
Geoffrey E. Hinton
Published in 1993.
Information geometry of the EM and em algorithms for neural networks
(
Citations: 144
)
Shunichi Amari
Journal:
Neural Networks
, vol. 8, no. 9, pp. 13791408, 1995
Sort by:
Citations
(239)
Codimensional matrix pairing perspective of BYY harmony learning: hierarchy of bilinear systems, joint decomposition of datacovariance, and applications of network biology
(
Citations: 2
)
Xu Lei
Journal:
Frontiers of Electrical and Electronic Engineering in China
, vol. 6, no. 1, pp. 86119, 2011
Asymptotic Convergence Properties of the EM Algorithm for Mixture of Experts
Yan Yang
,
Jinwen Ma
Journal:
Neural Computation  NECO
, vol. 23, no. 8, pp. 21402168, 2011
A longitudinal perspective on inductive reasoning tasks. Illuminating the probability of change
Dirk Ifenthaler
,
Norbert M. Seel
Journal:
Learning and Instruction  LEARN INSTR
, vol. 21, no. 4, pp. 538549, 2011
Robust multiple cardiac arrhythmia detection through bispectrum analysis
Antonio Lanata
,
Gaetano Valenza
,
C. Mancuso
,
Enzo Pasquale Scilingo
Journal:
Expert Systems With Applications  ESWA
, vol. 38, no. 6, pp. 67986804, 2011
Learning Gaussian mixture with automatic model selection: A comparative study on three Bayesian related approaches
Lei Shi
,
Shikui Tu
,
Lei Xu
Journal:
Frontiers of Electrical and Electronic Engineering in China
, vol. 6, no. 2, pp. 215244, 2011