The two glass dialog boxes next to the book are equipped with translucent and opaque balls respectively, reminding the reader to determine the degree of determination in the translation.

The large model research controversy: AI goes into everyday life

The child receives the English email of the communication activity, and lets the AI be translated into Chinese. You glanced at it: "Participants will receive group information before Friday." So rest assured to wait for the notification. The child pointed to the original text and asked: Is the May here only said that it is possible to receive it? This is a self-made situation, not a measured case. The first layer of answer is very simple: there are no sick sentences in Chinese, which does not mean that the original promise is still there. The receipt time is correct, and whoever sends a notice is also translated correctly, and it is possible to turn "may happen" into "has been guaranteed".

But in turn, you can't see a small word in English disappearing in Chinese, so it is judged that the translation is distorted. It is not necessarily necessary to add an additional "possible" to "is expected to be issued before Friday", and it is translated into "must be issued before Friday", which has significantly improved the degree of certainty. What I want to check today is not whether the two languages correspond verbatim, but whether people who read the translation will believe that there is one more thing that the original words are not guaranteed than those who read the original text.

In a word, what did it promise?

Let's take the three judgments apart first. Whether it is natural to read can be judged by those who are familiar with Chinese; whether the meaning has changed, it is necessary to compare two languages; One person gave a high score in the previous one, and did not complete the last two items. This is also the most difficult step when we don't understand the original text: we can feel the fluency of Chinese, but we lack the reference to discover semantic changes.

It is also necessary to distinguish what the "small words" are doing. The self-sentenced sentence "You can enter the classroom early", it may be said to allow it, or it may be that the door is open and the conditions allow; Just let the model replace all should with "should", and keep the literal, but may not mean to keep the speaker. Asking a question "who allows it, or who speculates", is often closer to the problem than the number of modal words.

So there are at least two distinct explanations. The first is that the semantics have really been strengthened: the original words are retained the possibility of not happening, and the translation blocks them. The second is that the expression has changed: the uncertainty is changed to "expected", "according to the current arrangement" or in the context, and the meaning is still there. There is another case worthy of a single column: the original sentence is vague, and the translator chooses a reading method. That needs to be supplemented, and it cannot be judged by being more beautiful in Chinese.

These explanations lead to different modifications. If it is really strengthened, it will be restored to its original reservations; if the expression has not changed, there is no need to force the "possibility" of repeating it; Translation is not to change a sentence more cautiously, the better. The original civilization has been confirmed, and the translation of "not necessarily" without authorization may also make readers misjudgment.

I have seen the problem in recent research, and I have left the account and can't figure it out.

Research published by Karachina in 2026 Specially check the modal expressions of Russian-to-English translation. The material is one hundred and sixty sentences, and forty sentences are selected according to the four types of modalities, including the selected sentences and the sentence constructed by the author; it is not a random sampling of daily translation requests. The study separates the judging of meaning, grammar, style and modality, etc. The listed mistakes include replacing one modality with another, and not having a modality. It supports "worth individual inspection", and cannot provide a common error rate for Chinese emails.

Resist the temptation to reproduce the rankings. The methods section combines five scores into percentages, while the results section speaks of “correct translations.” Without the denominator, we can’t say how many out of a hundred emails were translated correctly. The paper names ChatGPT but gives no version that identifies the underlying model, and the platforms differ in what their “improve” options do. That’s why I’ve left out the rankings. What this primary study from the past twelve months offers is a way to assess grammar and modality separately. Which service is most reliable today is a different question.

This is not a picky paper format. Assuming that a translation is very good in grammar and style, and only "may" is changed to "necessity", the comprehensive score may still appear good; while the other translation is blunt, but it is sure to ensure that it is reserved. For parents who are ready to work by mail, these two errors have different meanings. This is a hypothetical statement, no recalculation of the research, nor does it claim that some sample of it is exactly the same.

The contrast that can most change judgment: add the previous and after the same sentence

Literary translation study of Karpińska and Iyyer Using the same Text-Davinci-003 model, there are only three settings for a single sentence, a whole paragraph but translation, and a whole paragraph of translation. The artificial evaluation covers 18 language directions and 180 original paragraphs; This tests the quality of the machine translation, which is judged by people, not how many points the student can test after finishing the test.

In Figure 5 of the paper, the entire paragraph of translation has obtained about two-thirds of the preference votes compared to "look the whole paragraph but translate sentence by sentence". However, Table 2 shows at the same time that the mistranslation counts of the two settings are close, and not every single translation is the best. This control is more explanatory than simply comparing the two brands: the context can be seen, and the whole paragraph is allowed to be organized into the target language, which are two separate conditions. The research still found problems such as missed translation; the results of literary texts and old models cannot be directly moved into the results of today's Chinese notice.

Here’s a small comparison you could try with the email, though I haven’t run it. First translate “You should arrive on Friday” on its own. Then put “I’ve worked out the train times” before it. Finally, replace that lead-in with “Participants must follow these check-in arrangements.” The last two versions give the translator different information. A change in wording is no evidence of instability: a good translator would make the train version sound like a prediction and the check-in version like a requirement. To ask whether a translation unjustifiably strengthened or weakened the claim, hold both the source and its context fixed.

This also explains why "only one word" is sometimes unfair. If you first delete the maybe of the original text, and then blame the different translations twice, the test is not the degree of loyalty, but whether the model has changed the information. The really useful control is to keep the original text, put the two candidate translations together, only change the degree of determination, and then ask people who understand two languages to point out which share is promised.

The original text card is juxtaposed with the two candidate translation cards, the transparent shadow indicates that the possibility is reserved, the solid ball is indicated to be determined, and the fluency and meaning are checked separately below.

Methods: The same original text corresponds to two candidate translations, compare the intensity of the tone; the fluency and meaning are separately checked below. The transparency of the ball is only a concept symbol, and there is no probability scale; there is no cause and effect arrow in the figure, and it is not a paper measurement result.

"No" is still there, or it may have been denied the wrong place

In addition to the degree of determination, another easy-to-hide sentence is easy to hide in a smooth sentence. Look at two sentences in Chinese: "Not every student has to speak" and "every student should not speak". There are "every students" and "no", but the arrangements for readers are completely different. The first sentence does not require everyone to speak, and the latter sentence requires everyone not to speak. Just check if the negative words are left, and you will put this difference.

A classic study of Tang et al. The study is neural machine translation, not today's chat model. Its British and German manual evaluation draws 250 sentences in each direction, and the English and Chinese use NegPar materials with negative labels. The five distinctions of the table are retained in negative words, and in other words, they are still accurate, the scope is misplaced, and the meaning is wrong and omitted. Table 6 also counts as correct revisions. Therefore, if there is no corresponding "no", it is not necessarily wrong to calculate, and there is no guarantee that there is "no".

The paper distinguishes two tests. In one, the model scores a correct translation and a version someone has deliberately made wrong. In the other, the researchers examine what the model actually translates. One is like picking an answer on a multiple-choice test, the other like writing your own. Passing the first cannot stand in for passing the second. I use that distinction here. An older system’s accuracy cannot establish how today’s products perform, and the share of translations a model picks correctly tells us nothing directly about how often bilingual readers understand them correctly.

For ordinary readers, the negative range can be checked in one sentence. When you see "Not all activities require registration", you ask "Is there any activity that you can participate in without registration"; when you see "All activities do not need to register", you ask "Which one still requires registration". This is just an example of helping to find out semantic differences; if there are other conditions for truth notifications, you cannot decide to participate in eligibility by an isolated sentence. The whole paragraph is still the basis for judgment.

To add "reader understanding" in, what experiment is missing?

The above three studies did not jointly complete such an experiment: randomly give different Chinese translations of the same English email to Chinese parents to record their understanding of the sender's commitment. If you can't find a change in your tone from the translation, you can jump directly to "Parents will definitely arrange the wrong time." Some people may see that they are uncertain, some may ask the teacher separately, and some people will not act according to the email. There is still a separation between the changes of words and the consequences of behavior.

If the teacher wants to turn it into a classroom exercise, he can first ask a bilingual reader to answer two questions based on the original text: Has the matter been determined, and which conditions have not been met? Let another group of readers just read the translation to answer the same question. In the end, the bilingual reader can point out which words cause the difference. This is an unimplemented suggestion; to compare the two editions, it is best to let everyone see only one of them first, to avoid adding the meaning after reading the original text.

To be more detailed, you can ask "Whether it is natural to read" separately, and list another question "Whether it is guaranteed". Only then can we see a useful result: both versions are natural, but have different understandings of commitments. Another result is also allowed: bilingual readers themselves have different opinions on the original text. At this time, the answer should not be forcibly unified into the interpretation of a model, but should record where the dispute happened, and look for background information that can eliminate ambiguity.

Researchers also need to care who will evaluate. Translators who are familiar with two languages can locate the wrong place, and those who only understand the target language are closer to the daily users; the two have different tasks. Students, parents, and literary translators are not samples that can be exchanged at will. The evaluation of several novels neither proves that a child can misread it, nor does it prove that a certain kind of Prompt can generally improve reading comprehension.

A three-minute check that can be done today

Choose a short essay that really needs to be used and does not involve high-risk decisions. First write your own understanding next to the translation: "confirmed" "just expected" or "the original text is not explained". Go back to the original text, just look for the phrases supporting this judgment, and even read the sentences together. You can ask AI to mark the corresponding expression, but the explanation it gives needs to be checked with the original sentence, and the same explanation cannot be automatically proved to be its own proof.

Then do a counter-example check: If things didn't happen, would the original words count as violations? "May be received on Friday" and "Friday must be received", the answer to this question is different. If you don't understand the source language, it is easier to get a verifiable response to the original sentence, context and this specific question to someone who understands two languages. There is no commitment here to solve all ambiguity; the goal of the exercise is to find the most worthy of questioning.

You can also leave a very small record for this verification: what is the original sentence and what is the translation. At first, you thought that the other party had guaranteed what the other party had guaranteed, and what was changed after reviewing it. There is no need to give the whole article a total score. Next time you meet the same kind of expression, first read this record to see if the speaker, scene and premise of the new sentence are the same. If it is different, re-judgment, and don't turn the last translation into a mechanical replacement rule. The role of this record is to make one's own understanding well documented, and does not constitute a formal evaluation of a model.

In the end, only the locations that have been found have been revised. Don't sprinkle the whole paragraph "maybe" and "not necessarily", and don't delete all the reservations for Chinese simply. What needs to be reserved is the judgment intensity of the original author: he determines where you go, and you can translate it. The moment the child asks May, the most valuable next step is not to regenerate a more beautiful email, but to confirm together that what has been determined before Friday.

References


Apps by the author

I also build small tools for everyday life. Here are two of my apps:

  • Daily Expense Journal: Track your income and spending, plan a budget and set savings goals. View on the App Store.
  • One Small Action: Pick a small action that fits the time and energy you have today, then record how it felt and what you noticed. View on the App Store.

Find more articles and projects on my website: zffjzz.top. I’d love to hear how the apps work for you and what I could improve.