Pre-Translation Stopword Removal
Common words don’t help retrieval much
- But mistranslations might hurt
We built a Mandarin stopword list
- Processed dictionary to identify function words
- Added the top 300 words in LDC frequency list
- Filtered by two speakers of Mandarin
Suppressed translation of stopwords