However, it remains Misinformation Big problem on the network Internet.
Debunking lies online is difficult and time-consuming. Misinformation is so widespread that fact-checkers are often overwhelmed by it. Humans Despite its dissipation, automated fact-checking technology still needs significant improvement.
Lies are also spreading on Social Media It has been known for years that information can spread faster than correct information. But despite these challenges, natural language processing technology offers hope for reducing the impact of misinformation.
Network is full Internet There are many types of misinformation, but one common method is to manipulate the results of scientific studies. This is a particularly effective method because it relies on specialized, reliable research to support a false claim that is partially true.
In fact, misrepresenting scientific studies is a common tactic used during the COVID-19 pandemic to spread lies about the origins of the virus, alternative treatments, and the effectiveness of vaccines. But it can be used in many other contexts.
“Scientific misinformation is a particularly dangerous type of misinformation because it is difficult for non-experts to detect,” says Irina Gurevich, an assistant professor in the Natural Language Processing Department at MBZUAI. “It is difficult for the general public to recognize fallacies based on distorted science.”
Gourevitch and colleagues at MBZUAI and other institutions have conducted a study that represents a step toward combating misinformation that misinterprets scientific research evidence. The results of the study were presented recently at the 62nd Annual Meeting of the Association for Computational Linguistics, one of the largest annual meetings for researchers in natural language processing.
First dataset of its kind
Gorevic and colleagues’ study is based on a dataset called MISSCI, which consists of real-world examples of misinformation that the researchers collected from a fact-checking website. To create this dataset, the researchers commissioned data scientists to collect fact-checking articles written for a website called HealthFeedback, which partners with scientists to review health and medical claims.
The researchers provided the fact-checkers with a total of 208 links to scientific publications that had been misrepresented in the fact-check articles. The fact-checkers manually reconstructed the misleading arguments from the fact-check articles using a method outlined by Gourevitch and her team. The fact-checkers then classified the different types of errors in reasoning, known as fallacies, into nine different categories, including “fallacies of exclusion” and “false equivalence.”
This dataset served as the basis for evaluating large language models, and is the first of its kind to use real-world examples of rigged scientific studies.
Performance Evaluation
The researchers then focused on evaluating the reasoning capabilities of two large language models, GPT-4 developed by OpenAI and LLaMA 2 developed by Meta, with the goal of using large language models to generate false premises that lead to incorrect claims and classify these premises into categories.
The models were given a claim, a false premise, and a scientific publication context, and then asked to classify the false premises used into one of nine categories. The models were also asked to generate the false premises. The researchers tested the models in several different configurations, such as providing the models with a definition of a fallacy, the logic of the fallacy, and the logic of the fallacy plus an example of the fallacy.
GPT-4 performed better than LLaMA 2 at predicting fallacies overall, identifying almost all fallacy classes. When given a fallacy logic with an example of it, GPT-4 was able to predict the correct fallacy class based on correct fallacy premises 77 percent of the time, compared to LLaMA 2 66 percent of the time.
When human raters examined the output of the two models, they found that even if the model was able to generate a plausible fallacy, it might misclassify it.
Other ways to check
The current approach to fact-checking through natural language processing relies on what are known as knowledge bases, repositories of valid information. In this approach, the system compares claims to information in the knowledge base and predicts whether those claims are true. But this approach, which focuses on refuting claims with counter-evidence, fails when there is no counter-evidence, which is the case for most false claims in reality, Gourevitch explains.
In this study, rather than refuting claims, Gourevitch and colleagues’ method refutes the inference between the scientific source and the claims and uses large language models to identify the fallacies required to create false claims based on the source.
Big challenge
Fact-checking is difficult for humans and machines alike. “It’s hard for people to spot fallacies, which makes it hard to run experiments,” says Gourevitch. “We’re not seeing large language models yet that can accurately handle these complex logical questions.”
In the future, Gourevitch and her colleagues plan to analyze other models according to the same framework and develop methods that can handle scientific evidence presented in more than one document.
But the very nature of the task—determining whether a claim is true based on evidence—is often contested, and is a fundamental process in the production of knowledge. “The question of whether evidence supports a claim is at the heart of any empirical science and one of the reasons scientists have developed methods like peer review,” Gourevitch explains. “It’s a very difficult question to answer.”
