Clean a TMX for MT training data
A translation memory is parallel data — and if you fine-tune an MT or LLM on it (you will), every junk segment is a noisy training example. Clean the TM before it becomes training data.
━━ the problem
Training an MT model on segments that a previous MT produced teaches the model its own mistakes. Filtering machine-origin units by their creationid signature breaks the loop.
"café" stored as "café" from a misread encoding looks fine in a spot-check but degrades the model. A mojibake detector flags source/target with encoding corruption so you drop or fix them.
A segment repeated 400 times over-weights that example. Deduplicating to one instance per source gives the model a balanced view of your data.
━━ how TM Cleaner handles it
Run the TM through dedup + junk + MT-origin + mojibake + placeholder-mismatch classifiers, preview what's removed, and export a clean TMX — or a source⟍target TSV that drops straight into a training pipeline.
- ✓MT-origin, mojibake, placeholder-mismatch and length-outlier classifiers target exactly the noise that hurts training
- ✓deduplication balances the example distribution instead of over-weighting repeated segments
- ✓TSV export gives you a clean parallel corpus in the format most training tooling expects
- ✓audit CSV documents every dropped pair — reproducible, defensible dataset hygiene
━━ frequently asked
can I export cleaned data as plain parallel text?+
yes. alongside the cleaned TMX you can export a TSV of source⟍target pairs — the shape most MT/LLM fine-tuning pipelines expect. the audit CSV records every pair that was dropped and why.
how do you detect machine-translated units?+
by their metadata origin — the creationid/changeid stamped by MT engines and pretranslation steps. you supply or confirm the MT-signature patterns; matching units are flagged as junk_mt_origin.
does cleaning alter the parallel alignment?+
no. cleaning only removes whole units (both sides together); it never edits or re-aligns text. every kept pair is exactly as it was in your TM.
━━ also clean
free tier · pay-as-you-go after · no subscription · see pricing