Knowledge_Interplay

Abstract

Large language models (LLMs) encode vast amounts of knowledge during pre-training (parametric knowledge, or PK) and can further be enhanced by incorporating contextual knowledge (CK). Can LLMs effectively integrate their internal PK with external CK to solve complex problems?

In this paper, we investigate the dynamic interaction between PK and CK, categorizing their relationships into four types: Supportive, Complementary, Conflicting, and Irrelevant. To support this investigation, we introduce EchoQA, a benchmark spanning scientific, factual, and commonsense knowledge.

Our results show that LLMs tend to suppress their PK when contextual information is available, even when it is complementary or irrelevant. While tailored instructions can encourage LLMs to rely more on their PK, they still struggle to fully leverage it. These findings reveal a key vulnerability in LLMs, raising concerns about their reliability in knowledge-intensive tasks.

Key Findings

Most LLMs struggled to follow instructions to truthfully output their internal knowledge, regardless of the model, knowledge type or reasoning type (Complementary, Conflicting or Irrelevant). LLMs disregard their own knowledge, e.g., more than 60% of cases in scientific knowledge for all tested models, relying solely on the context for reasoning. This highlights the vulnerability of LLMs in leveraging PK.

LLMs are more likely to recall their knowledge for some knowledge and reasoning types, e.g., the commonsense knowledge. We find evidence indicating that the reason behind is likely the imbalance of knowledge in training corpus.

Explicit instruction can help LLMs remember more PK, but still way off from fully leveraging PK. This implies more sophisticated prompt or framework design has the potential to solve this problem to a larger extent.

Detailed Results

For Complementary Reasoning

Although provided with complementary context, LLMs leverage of their own knowledge remains inhibited.

For Conflicting Reasoning

LLMs rarely trust themselves when faced with conflicting context.

For Irrelevant Reasoning

Although grasping the key to the knowledge, LLMs still seek answers in the irrelevant context.

BibTeX

@misc{cheng2024understandinginterplayparametriccontextual,
      title={Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models}, 
      author={Sitao Cheng and Liangming Pan and Xunjian Yin and Xinyi Wang and William Yang Wang},
      year={2024},
      eprint={2410.08414},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.08414}, 
}

Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

We extract Parametric Knowledge (PK) for commonly-used LLMs, and construct Contextual Knowledge (CK) and questions to test LLMs ability to output their PK when CK is given.