Academic
Publications
Learning in the Deep-Structured Conditional Random Fields

Learning in the Deep-Structured Conditional Random Fields,Dong Yu,Li Deng,Shizhen Wang

Learning in the Deep-Structured Conditional Random Fields   (Citations: 6)
BibTex | RIS | RefWorks Download
We have proposed the deep-structured conditional random fields (CRFs) for sequential labeling and classification recently. The core of this model is its deep structure and its discriminative nature. This paper outlines the learning strategies and algorithms we have developed for the deep-structured CRFs, with a focus on the new strategy that combines the layer-wise unsupervised pre-training using entropy-based multi-objective optimization and the conditional likelihood-based back-propagation fine tuning, as inspired by the recent development in learning deep belief networks.
Cumulative Annual
View Publication
The following links allow you to view full publications. These links are maintained by other sources not affiliated with Microsoft Academic Search.
    • ...Research in speech recognition has explored layered architectures in the recognizer design for quite some time (e.g., [1][2][3][5][15][17][18][19][20]), motivated partly by the desire to capitalize on some analogous properties in the human speech generation and perception systems.,The approach we take in this paper is to consider the topmost layer of the DBN as a linear-chain conditional random field (CRF) with � / as input features from the lower layer at time 9 . This new model can be seen as a modification of the deep-structured CRF of [19][20] where the lower multiple layers of CRFs are replaced by DBNs.,Setting up the full-sequence discriminative criterion to train the DBN weights gives rise to an architecture that can be considered as a modification of the deep-structured CRF described in of [19][20]...

    Abdel-rahman Mohamedet al. Investigation of full-sequence training of deep belief networks for sp...

    • ...A recent advance in training methods for multilayer neural networks has led to renewed interest in exploring deep, multilayer networks for a number of machine learning problems including encoding [8][9], retrieval [12], as well as the problems associated with classification and regression that involves image [8][9], language [16][17] and speech [10][11][15]...

    Li Denget al. Binary coding of speech spectrograms using a deep auto-encoder

    • ...Our work extends their work in the adoption of different training criteria at different layers and the ability to learn the intermediate hidden layers in an unsupervised way [30], [31]...
    • ...Interested readers can find an effective solution in our recent work [30], [31]...
    • ...In [30], we described an approach with which the intermediate layers are first pre-trained to maximize the state occupation entropy and minimize the frame-level conditional entropy at the same time and then fine-tuned using the back-propagation algorithm...
    • ...We have recently proposed techniques to infer the intermediate layers using discriminative criteria [30], [31]...

    Dong Yuet al. Sequential Labeling Using Deep-Structured Conditional Random Fields

    • ...In the deep-structured CRF, both model parameter estimation and state sequence inference are carried out layer-by-layer in a bottom-up manner so that the computational complexity is limited to at most linear to the number of layers used [13][14]...
    • ...After the intermediate layers are pre-trained layer by layer using the unsupervised approach we just described, the model parameters are jointly fine-tuned using the back propagation to optimize the sequential conditional log-likelihood �,�:�&�· �:�; [14]...

    Dong Yuet al. Language recognition using deep-structured conditional random fields

    • ...Yu et al. proposed the deep-structured conditional random fields (DCRFs) [17] and obtained better than the state-of-the-art results on the query labeling [4] tasks.,After all layers are trained greedily layer by layer, the parameters are jointly fine-tuned following [17]...

    Dong Yuet al. Deep-structured hidden conditional random fields for phonetic recognit...

Sort by: