Entries from 2013-05-22 to 1 day

Normalization for text tokenization (3.7)

Chapter 3.7 of the whale book. >>> raw = """'When I'M a Duchess, 'she said to herself, (not in a very hopeful ... tone though), 'I wont't have any pepper in my kitchen AT ALL. Soup does very ... well without--Maybe it's always pepper that …

Normalization of text (3.6)

Chapter 3.6 of the whale book. >>> raw = """DENNIS: Listen, stange women lying in ponds distributing swords ... is no basis for a system of government. Supreme executive power derives from ... a mandate from the masses, not from some farci…