Nie jesteś zalogowany | zaloguj się

Wydział Matematyki, Informatyki i Mechaniki Uniwersytetu Warszawskiego

  • Skala szarości
  • Wysoki kontrast
  • Negatyw
  • Podkreślenie linków
  • Reset

Aktualności — Wydarzenia

Sem. "Uczenie maszynowe"

 

Retrieval Augmented Language Models


Prelegent: Konrad Staniszewski

2023-06-01 12:15

Large language models store their knowledge in parameters and require costly fine-tuning to update. An interesting alternative is to provide new knowledge in the model's context. However, typical models have relatively short context lengths.
In this presentation, I will discuss one of the potential solutions to this problem - retrieval augmented transformer models. Those models utilize a large external database to store information about already processed parts of the text and retrieve most matching entries to improve the performance during the inference.