In this paper, we describe the MiBi team’s participation in the BioASQ 2024 Task 12b on biomedical semantic question answering. Our RAG-based systems (retrieval-augmented generation) use GPT-3.5, GPT-4, or Mixtral to generate an answer from some retrieved context. For the retrieval, we use PubMed’s search API or a local BM25 index of PubMed abstracts and potentially re-rank the initially retrieved abstracts/snippets with different neural bi-encoder and cross-encoder re-rankers. We test five different retrieval-augmented generation schemes with different orders of generation and retrieval stages.

The evaluation results for our submitted systems—although partially inconsistent over different test batches—show three general trends. First, combining BM25-based lexical retrieval with neural re-rankers seems to be a good retrieval setup. Second, answers generated from retrieved snippets as context seem more accurate than answers generated from complete abstracts. Third, GPT-4 generates more accurate answers than GPT-3.5, but Mixtral is on par with GPT-4 when employing a generation-then-retrieve-then-generation scheme.