Paulo Cacella’s Post

Happy to announce the publication of my paper titled "Generative vs Intent-based Chatbot for Judicial Advice" in the conference proceedings of IEEE Xplore. The work was presented at the 2nd IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI-2024). The paper presents a generative chatbot and an Intent-based chatbot aimed at providing judicial advice to Indians and explore and compare the two approaches on the basis of various factors such as nature of responses, response quality, handling changing scenarios, training and data requirements and user experience in the context of offering judicial advice specific to Indian laws. Thanking my guide, Medha Wyawahare and my teammate Sarang Zanwar for the shared efforts in making this possible. Link to access the paper: https://lnkd.in/dFJ4-6cb

Just came across this unique approach to evaluating LLM output. Instead of relying on a single large model for evaluating the quality of free-form output from an LLM, this new study proposes using a diverse panel of smaller models. This not only cuts costs, but also significantly reduces bias, providing a more accurate assessment of your LLM's performance. What are some other novel approaches that you have used to evaluate the quality of your LLM output? #GenAI #Innovation HOPPR Check out the full study here: https://lnkd.in/gKxCGACC

Model evaluation is critical for any regulated medical application using LLMs or LVMs. But I think diversity of data used to train and validate the model may be more important Robert.

Fine tuning an AI model can significantly enhance accuracy for specific information subsets. This short course explores PEFT, LoRa, and MoME, demonstrating how individuals can fine-tune models like Meta's Llama at a reasonable cost. #AI #MachineLearning #DataScience

We recently reviewed the OECD - OCDE guidelines and the OECD.AI to implement international standards in our country. Additionally, we're actively working on crafting a proposal in the Senate's AI Technical Committee to steer AI towards fostering economic development through technology. We've thoroughly examined and compared perspectives on technology and its application in smart cities. If you're interested in diving deeper into the session and all its details, check out the YouTube link below: https://lnkd.in/eVVfEwG9 #AI #OECD #TechnologyStandards #EconomicDevelopment #SmartCities #Innovation

Retrieval Augmented Generation (RAG) is a straightforward concept that any company can implement with little effort. You can think of RAG as an NN-based document summarization, like merging the results of a simple elastic search/apache solr query with a text summary. The original concept involved retraining the Llm to have a similar latent space, however in applied concepts the retraining has been dropped and replaced with a vector database and prompt engineering that can be paired with any Llm that exists. Based on Meta's LLAMA3, LlamaIndex, Qdrant, Ollama, Angular, and FastAPI, I built the RAG demo about funding lines for universities of applied sciences in hesse. The demo is available at (not well optimized for mobile use): https://lnkd.in/eBYQwKh4 This demo is a weekend project when the little ones were asleep, so it took me less than 3 days. Imagine your company creating a bot like this with more manpower and effort. Best of all, it's completely free and costs you nothing. Your data stays in your company without any security or GDPR violations. P.S.: On my own behalf: I also work as a speaker and consultant on AI topics. Through my research in context-aware fault diagnosis in industrial production lines, I know the state of the art in AI and have implemented various AI use cases myself using different neural networks under industry constraints. If you need some advice, just get in touch with me. I'm looking forward to talking with you about self-built solutions without buying expensive cloud services.

The first (open) use of #ChatGPT by a Dutch court, more precisely by the Rechtbank Gelderland (ECLI:NL:RBGEL:2024:3636), gained a lot of attention in the Netherlands. But what exactly was the case about? The case revolves around a dispute between two homeowners. One of them had added another floor to his house. Due to this his neighbor expected a lower return from the use of his solar panels and therefore claimed damages. The Rechtbank Gelderland ruled on this case and referred to ChatGPT when calculating the damages in order to determine the average lifespan of solar panels. The court made the following statements in this regard: "The court, also with the help of ChatGPT, estimated the average lifespan of solar modules from 2009 at 25 to 30 years; this lifespan is therefore set at 27.5 years here." Secondly, the court uses ChatGPT to determine the current average price per kilowatt hour for electricity. It states the following: "Assuming, again partly on the basis of ChatGPT, a current average kWh price of €0.30 (€0.29 to €0.34, depending on the type of contract), the loss of revenue at a more or less constant kWh price amounts to around €2,250." To start with the positive: The court is making the use of ChatGPT public and thus transparent, as if it deliberately wanted to initiate a discussion. This is to be welcomed, as a certain number of non-transparent uses of ChatGPT by judges are to be feared and an open discussion in this regard is necessary. In this case, the Rechtbank Gelderland uses a generative AI as a source of information. In doing so, the court seems to make the erroneous assumption that ChatGPT has genuine, trustworthy knowledge. However, it is a Large Language Model (LLM) that only predicts the next word in a sentence. Furthermore, although the court attempts to be transparent about the use of ChatGPT, it fails to provide further important information. For example, it is not clear exactly which prompts were given to ChatGPT. However, this is important for the evaluation and traceability of the response. It also remains unclear which version of ChatGPT the court actually used. Finally, it also remains unclear what exactly the court means when it says that the results were also obtained "with the help of ChatGPT" or "partly on the basis of ChatGPT". Have other trustworthy sources been used? So the court wants to make the use of ChatGPT transparent, but then unfortunately fails in the details, so that it is ultimately a non-transparent approach. The ruling impressively shows how urgently we need to train judges in dealing with generative AI. A court ruling must be based on facts and law and not on "information" whose sources are unclear, as in the case of ChatGPT. https://lnkd.in/e6dupJqK

Excited to dive into the course on "Improving Accuracy of LLM Applications"! 🚀 There are various methods to fine-tune LLMs that can be just as efficient in cost and time as Retrieval-Augmented Generation (RAG). Key strategies include removing hallucinations by leveraging the latest practices like PERF, Memory tuning, ensuring your model is tailored to your specific domain data. For more of the scenarios, Small language models with instruct versions are good enough. By running set of iterations, you can reach the accuracy threshold set for your application, all while optimizing performance and memory. #LLM #MachineLearning #AI #MemoryTuning #HallucinationRemoval #FineTuning #PerformanceOptimization #SLM

Excited to share that TATuP's special issue on AI for decision support is out now! In my piece, I discuss the use of artificial intelligence for the administration of justice, the problem of bias in adjudication and the possibility of using AI as a debiasing tool.

Memory tuning - cure AI of hallucination. Great course by DeepLearning.AI and Lamini #AI #LLM #GenAI