Show simple item record

dc.contributor.authorTraynor, Michael
dc.date.accessioned2018-12-14T16:19:32Z
dc.date.available2018-12-14T16:19:32Z
dc.identifier.urihttp://hdl.handle.net/10222/75034
dc.description.abstractParaphrase generation is a challenging problem that requires a semantic representation of language. Language models implemented with deep neural networks (DNN) have the ability to transform text to a real valued vector space that can capture useful semantic information. In light of this, this work employs hierarchical language modeling to produce semantic representations of sentences. An encoder-decoder model is employed that uses four components: a word encoder, sentence encoder, sentence decoder, and word decoder. These components hierarchically convert a sentence from characters through word representations to a fixed-size sentence representation, then back down through words to characters. Many types of neural network are suitable for each component, and a number of them are compared in this work, including a novel architecture, the Self Attentive Recurrent Array (SARAh). The SARAh is shown to perform at least as well as Gated Recurrent Units (GRU) and Transformers on language modeling tasks, and requires fewer parameters. These language models are trained on a large and diverse dataset, but this work also shows that it is possible to fine tune such models to a particular domain, such as the works of a single author. These fine tuned models are able to leverage information learned on the larger dataset in order to perform better on the target domain. Finally, a language model is trained to produce semantic representations of sentences that are subsequently used to produce paraphrases in a completely unsupervised setting. The language model, which is trained to predict the sentence most likely to follow the input sentence, is fine tuned to instead autoencode the input sentence. Given that the sentence encoder produces a semantic representation, it is possible to use a number of techniques to encourage the decoder to generate a paraphrase rather than reconstruct the exact input sentence. These techniques include adding noise to the sentence representation, and sampling characters from the model's output layer.en_US
dc.language.isoenen_US
dc.subjectDeep Learningen_US
dc.subjectNatural Language Processingen_US
dc.subjectSemantic Representationen_US
dc.subjectLanguage Modelingen_US
dc.subjectParaphraseen_US
dc.titleUNSUPERVISED PARAPHRASE GENERATION FROM HIERARCHICAL LANGUAGE MODELSen_US
dc.date.defence2018-12-13
dc.contributor.departmentFaculty of Computer Scienceen_US
dc.contributor.degreeMaster of Computer Scienceen_US
dc.contributor.external-examinern/aen_US
dc.contributor.graduate-coordinatorDr Michael McAllisteren_US
dc.contributor.thesis-readerDr Sageev Ooreen_US
dc.contributor.thesis-readerDr Robert Beikoen_US
dc.contributor.thesis-supervisorDr Thomas Trappenbergen_US
dc.contributor.ethics-approvalNot Applicableen_US
dc.contributor.manuscriptsNoen_US
dc.contributor.copyright-releaseNot Applicableen_US
 Find Full text

Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record