
The large model research controversy: AI goes into everyday life
The cover is a fictional parent-child still life scene.
When I received my schoolbag on Friday, the child asked, "It's raining tomorrow, are you still going to the park to recognize trees?" You clicked on the three lines of AI you saved yesterday. The time is right, and the activities are correct. It was not until you opened the library's activity manual that you realized that the leaf specimen in the library was changed to recognize the leaf specimens in the rainy day. This example is fictitious. The three-line summary is enough for you to decide whether to sign up or not, but not enough for you to decide where to go in the morning.
The most worthwhile is Once you add it, the words that will let you change the answer. They are sometimes an exception, and sometimes the result is only for who and when. There is no evidence that all abstracts miss the same word; the "most important" here has a very simple standard: what are you going to do with this passage? If you only want to know the theme of the event, the rainy day arrangement can be temporarily skipped. If you are already standing at the intersection, this line is the most important line in the text.
The words in the summary can be ticked, why can't I answer them?
Arrange the manual and card side by side, and you will see two different problems. If the original civilization wrote on Saturday, but the card wrote Sunday, it was a wrong. Just change it back to Saturday. If the card lists the time, activities and items, the whole section of the rainy day is omitted, and each remaining word may have a source. Check the card into the manual, and you will tick three consecutive ticks; follow the rainy days in the manual to find it in the card, but there will be a space left. These two directions should be checked, only the previous one, and no matter how serious you are, you can't find that space.
There is also a third way of writing, in between. Suppose the card says "Gather at nine o'clock in the park". It is right to read like this on a sunny day, but it will run in vain when it is raining like this. The sentence got the location from the original text, but did not get the conditions to use the location. It is easy to miss the revision method: there is no need to call the model to rearrange the entire itinerary, as long as the original two separate weather and the corresponding location are put back in the same sentence. Conversely, if the manual does not have a rainy day clause at all, then you should write "the manual is not explained", and you cannot make it up for the organizer just because the reader needs this answer.

Methods: The leaves represent the reserved theme, and the umbrella represents the conditions that need to be checked back to the original text. There are no causal arrows in the three side-by-side comparisons; the graph is not a measurement result.
Research on Elaraby and Litman This method of "reverse investigation" is used in the long text. They first pick out the important content of the argument from the original text, break a paragraph into small facts that can be checked separately, and then go to see if the abstract is written. The example of the paper's 1st example is very intuitive: the abstract said that a search warrant was issued, and it was also mentioned that the robbery was suspected, but the reader could not see what to search at that time. This does not overturn the first two things; as long as the question is to be searched, the gap will appear. The study separates the missing and contradiction, and even when the coverage score is calculated, neither can get the "correctly covered" score.
What counts is not based on the reader's dissatisfaction afterwards. The legal documents used in the research have disputes, reasons and conclusions; scientific materials are forty computer graphics papers, and the researchers pay attention to the arguments that are marked as highly relevant for the abstract. They use the model to dismantle the facts and auxiliary verification, so the method itself may also be dismantled incorrectly. It measures how much the information selected in the original text is preserved, and does not check whether the original text is correct or not for the reader, and has not measured a Chinese library activity. The borrowing here is the direction of the verification, not a paper set for the manual.
The most interesting comparison: who will ask the question?
If the teacher reads the three-line summary first and then asks the child "what time to start and what to bring", the child may think that the card is enough. Another person who has read the manual will ask the question: "The rain has already started, where should I go?" The card has not been changed a word, but it is very different to use. Where is the difference? The previous questions can only be obtained from the information that has been left; the latter method gives the missing content a chance to show up.
Reader experiments of PU, Gao and Wan Really done two sets of questions. They draw 100 sets of English news and corresponding artificial reference abstracts. One set of questions is written according to the artificial abstract, and the other set is written according to the original news of the news, and each set of four questions in each set. Twenty English-speaking college students are divided into two groups to do questions from different sources; the same person will not read a certain original text before reading its abstract, and rely on the memory just now to make up for the answer. In front of the reader, it may be the original text, manual reference abstract, or it may be a summary generated by the system. What they have to hand in is their own answer, and the researchers also note how long it took to complete the task.
First, just look at the manual reference abstract in the paper table 1. If the question is based on it, the ratio of the reader's answer to the standard answer is 54%; the original text is used to answer this set of questions, which is about 32%. Change to the question based on the original text, and the direction is reversed: about 27% of the artificial abstract is about 27%, and the reading of the original text is about 51%. The score here is People's answer. Picking the two lines of artificial abstract to read is because even if there is no typo in the model, the judgment of "is the summary is not enough" will change. The two sets of questions are answered by different groups, and the questions themselves are also different.
Table 2 also gives another practical choice: when doing the questions from the original text, the average completion time of the merged summaries is about 83 seconds, and the original text is about 212 seconds. This is the time to read the material and complete the answer, not the speed at which the model generates a summary. Exact matching is a strict standard, and the meaning is close but the wording may not be scored. The paper also calculates the degree of overlap of the words; if the reader only remembers which model ranks first, he will miss the most useful part of this experiment: whether the person who asks the question has the opportunity to ask things other than the abstract is more important than choosing an evaluation score first.
There are also reasons to defend short abstracts. The same study also has the task of selecting categories for news and judging whether the two reports are similar. To judge whether to put the article in the same column, you do not necessarily need to know each name and location. When the topic only requires the recognition of the theme, the reader can save time with the abstract. It can't be said that Changwen is always a burden, and it is useless to announce that these three lines of introduction are useless just because the manual did not answer where to go in the rainy day. The teacher who distributes the guide card in the classroom, and the students who are ready to answer the questions, can use the same piece of paper, but there are different reasons to rate it.
Less a few words, the conclusion may be too wide
When reading the paper, this gap is more difficult to see than the "several collection". To give a self-made example: "Students who sign up and complete the exercise this time will answer more questions correctly in the quiz that day." It seems that the words are more neat, but at least three accounts have not been explained: what about the people who have not finished participating, how the test will be done every other week, and whether the difference in grades can be returned to the practice head. Even if "help" is changed to "possible help", the reader still does not know which batch of people was originally measured and which day. The tone became soft and did not return the sample and time.
Peters and Chin-Yee It is this word change that is studied. They compare the short essays written by the original scientific text and the model, and distinguish whether the conclusion is limited to the situation under the study, or whether it is extended into a general judgment of a class of objects. A common extension in Table 4 is that the original text describes how many or which objects are in line, but the summary directly says how such objects are. It doesn't need to create a new number out of thin air, and it may even become more common sense by deleting the numbers. What we are talking about here is how to write the conclusion, and it cannot be used to estimate how many readers will make a wrong decision.
This research is dedicated to referring to the professional editorial abstracts of the same hundred medical articles. Everyone is compressing the space. If the generalization is just "writing short, this can only be done", and the professional abstract should also show a similar tendency. In this group of materials, the generalization difference of professional abstracts is not statistically significant; after the full text abstract of the tested model is combined, the advantage ratio of generalization conclusion is 4.85 (Table 1 and Figure 2). The comparison of this number is the ratio between the occurrence and the absence of such conclusions, and cannot be translated as "the probability of writing a wrong is nearly five times that of a person". Editing and models are also not randomly assigned to the exact same workflow. There is a reason to track down how to choose during compression, but there is no reason to assert that a certain brand will inevitably delete a certain type of words today.
The three papers don’t establish that a parent will take a wrong turn because a model omitted a condition. One asks whether selected facts make it into a summary, another whether particular readers can answer questions using a given text, and the third whether the scope of a claim changes. None randomly assigned parents to read the handbook or the card and counted them at the museum entrance when it rained. Anyone summarizing a real event should allow for one more possibility: the handbook itself may support both readings. Ask the organizer to clarify. Telling the model to “be more accurate” won’t produce fresh evidence.
Leave a summary that can't come out
Here’s something you can try now, though I haven’t tested it for this article. Pick a notice you actually need and decide what you’ll do with it. If you’re choosing an event, ask, “Does the topic interest me?” If you’re about to leave home, ask, “What would make me go somewhere else?” Read only the summary and write down your answer. Leave a blank if the answer is missing, even if you think you can guess. Then open the original, copy just the sentence that settles the question, and see what you would change. A blank you finally fill with “go inside the museum if it rains” deserves a circle more than five meeting times you already had right.
If there are two people, you can let one person only read the original text and ask the other person to answer the question only. The person who asks the question should first leave an answer with the source for himself, and he can't pick the cold knowledge that the original text and the cold knowledge that are not mentioned to make the other party difficult. The answerer can write "I don't know"; then put the two pieces of paper together, distinguish "the abstract is not written", "written but I didn't find it" and "the original text did not explain". Only by allowing "don't know" to be a legitimate answer can you see how much you are sure from the text, and how many are quietly added by yourself.
You can also make a small revision to the abstract with the same paper. Suppose there are eight other lines of recognition of the tree in the original text, and the card of the parent group can only be placed in three lines, what will you delete to make a location for the rainy day? If I send it before going out, I will delete the introduction to the color of the blades and keep the two weather gathering points; Even if the original text is not changed, both versions can be faithful. The trouble is that he forgot to tell him what it was used for when he used the observation notes as a guide for going out, and when he handed it to others.
Revisions of the same size can also be made when reading research. Instead of adding a sentence "there are still limitations" at the end of the note, it is better to stick the words "the day's quiz" close to the result. After two weeks, even if you only copy the sentence in the note, the time and conditions will follow. As for the untested every other week, it is better to leave it as a real question, not to let it become a conclusion that seems to have an answer in the next excerpt.
If there is only one habit today, leave this one: before the abstract, try to say a situation that will make this sentence not valid, and then read the original text to see how the author handles it. If you can't do it, you might as well write the retelling a little narrower: "The manual says there will be activities on Saturday, and I haven't checked the location of the rainy day."
References
- Mohamed Elaraby、Diane Litman,2026, ARC: Argument Representation and Coverage Analysis for Zero-Shot Long Document Summarization with Instruction Following LLMs, the official long article of EACL, see Figures 1, Sections 3 to 4 and limitations.
- Xiao Pu、Mingqi Gao、Xiaojun Wan,2024, Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks, LREC-COLING formal paper, see Section 3 and Tables 1 to 3.
- Uwe Peters、Benjamin Chin-Yee,2025, Generalization bias in large language model summarization of scientific research, Royal Society Open Science official journal papers, see Table 1, Table 4, Figure 2 and discussion.
Apps by the author
I also build small tools for everyday life. Here are two of my apps:
- Daily Expense Journal: Track your income and spending, plan a budget and set savings goals. View on the App Store.
- One Small Action: Pick a small action that fits the time and energy you have today, then record how it felt and what you noticed. View on the App Store.
Find more articles and projects on my website: zffjzz.top. I’d love to hear how the apps work for you and what I could improve.