188宝金博页面版

DeepNorm - A Deep learning approach to TextNormalizationShaurya RohatgiPennsylvania State UniversityState College, Pennsylvaniaszr207@ist.psu.eduMaryam ZarePennsylvania State UniversityState College, Pennsylvaniamuz50@psu.eduABSTRACTThis paper presents an simple yet sophisticated approachto the challenge by Sproat and Jaitly (2016) - given a largecorpus of written text aligned to its normalized spoken form,train an RNN to learn the correct normalization function.Text normalization for a token seems very straightforwardwithout it’s context. But given the context of the used tokenand then normalizing becomes tricky for some classes. Wepresent a novel approach in which the prediction of ourclassif i cation algorithm is used by our sequence to sequencemodel to predict the normalized text of the input token. Ourapproach takes very less time to learn and perform wellunlike what has been reported by Google (5 days on theirGPU cluster). We have achieved an accuracy of 97.62 whichis impressive given the resources we use. Our approach isusing the best of both worlds, gradient boosting - state ofthe art in most classif i cation tasks and sequence to sequencelearning - state of the art in machine translation. We presentour experiments and report results with various parametersettings.KEYWORDSencoder-decoder framework, deep learning, text normaliza-tion1 INTRODUCTIONWithin the last few years a major shift has taken place inspeech and language technology: the f i eld has been takenover by deep learning approaches. For example, at a recentNAACL conference well more than half the papers related insome way to word embeddings or deep or recurrent neuralnetworks. This change is surely justif i ed by the impressiveperformance gains to be had by deep learning, somethingthat has been demonstrated in a range of areas from imageprocessing, handwriting recognition, acoustic modeling inautomatic speech recognition (ASR), parametric speech syn-thesis for text-to-speech (TTS), machine translation, parsing,IST 597-003 Fall’17, December 2017, State College, PA, USA© 2018 Copyright held by the owner/author(s).ACM ISBN 123-4567-24-567/08/06...$15.00https://doi.org/10.475/123_4and go playing to name but a few. While various approacheshave been taken and some NN architectures have surelybeen carefully designed for the specif i c task, there is alsoa widespread feeling that with deep enough architectures,and enough data, one can simply feed the data to one’s NNand have it learn the necessary function. In this paper wepresent an example of an application that is unlikely to beamenable to such a "turn- the-crank" approach. The exampleis text normalization, specif i cally in the sense of a systemthat converts from a written representation of a text into arepresentation of how that text is to be read aloud. The tar-get applications are TTS and ASR - in the latter case mostlyfor generating language modeling data from raw writtentext. This problem, while often considered mundane, is infact very important, and a major source of degradation ofperceived quality in TTS systems in particular can be tracedto problems with text normalization.We start by describing the prior work in this area, whichincludes use of RNNs in text normalization. We describe thedataset provided by Google and Kaggle and then we discussour approach and experiments1we performed with dif f erentNeural Network architectures.2 RELATED WORKText normalization has a long history in speech technology,dating back to the earliest work on full TTS synthesis (Allenetal.,1987).Sproat(1996)providedaunifyingmodelformosttext normalization problems in terms of weighted f i nite-statetransducers (WFSTs). The f i rst work to treat the problem oftext normalization as essentially a language modeling prob-lem was (Sproat et al., 2001 ) . More recent machine learningwork specif i cally addressed to TTS text normalization in-clude (Sproat, 2010; Roark and Sproat, 2014; Sproat and Hall,2014).In the last few years there has been a lot of work thatfocuses on social media (Xia et al., 2006; Choudhury et al.,2007;Kobusetal.,2008;Beaufortetal.,2010;Kaufmann,2010;Liuetal.,2011;PennellandLiu,2011;AwandLee,2012;Liuetal., 2012a; Liu et al., 2012b; Hassan and Menezes, 2013; Yangand Eisenstein, 2013). This work tends to focus on dif f erentproblems from those of TTS: on the one hand one, in social1 https://github.com/shauryr/google_text_normalizationarXiv:1712.06994v1 [cs.CL] 17 Dec 2017