BigML, a pioneer in machine learning, recently held an exclusive event at the Valencian Graduate School and Research Network of Artificial Intelligence (ValgrAI) in Valencia, Spain. The event featured a keynote presentation by Dr. Tom Dietterich, Chief Scientist at BigML and Emeritus Professor at Oregon State University. In this presentation, Dr. Dietterich delved into the shortcomings of Large Language Models (LLMs) and proposed a modular architecture to address their limitations.
What’s Wrong with LLMs and What We Should be Building Instead
Earlier this year, BigML’s Chief Scientist and Oregon State University Emeritus Professor Tom Dietterich gave a keynote presentation titled “What’s wrong with LLMs and what we should be building instead” at the ValgrAI event in Valencia, Spain.
Large Language Models (LLMs) provide a pre-trained foundation for training many interesting AI systems. Among LLMs’ achievements we can count the ability to carry out conversations and answer questions covering a wide range of human knowledge, which Professor Dietterich stresses as our first case of creating a broadly-knowledgeable AI system. Other notable capabilities include summarization and revision of documents, writing code from English descriptions, and in context learning based on a small number of training samples.
However, LLMs have many shortcomings. They are expensive to train and to update, their non-linguistic knowledge is poor, they make false and self-contradictory statements, and these statements can be socially and ethically inappropriate. Professor Dietterich starts his keynote with an overview of well-documented LLM deficiencies and the current efforts to address them within the existing framework.
In the second part of his eye-opening presentation, Dr. Dietterich proposes a more modular architecture that decomposes the functions of existing LLMs and adds several additional components that can potentially address all of the shortcomings of LLMs. Dr. Dietterich’s modular architecture could be built through a combination of state-of-the-art machine learning and software engineering best practices.
Please follow along this noteworthy keynote on YouTube for the specifics of the proposed solution architecture and more.
Conclusion
The presentation by Dr. Tom Dietterich offers valuable insights into the limitations of LLMs and potential solutions to address these limitations. With a growing focus on AI and machine learning, it is essential to understand the potential applications and challenges of these technologies. We believe that a more modular approach to AI systems can lead to more reliable, scalable, and efficient solutions.
Frequently Asked Questions
Question 1: What is a Large Language Model (LLM)?
A Large Language Model (LLM) is a type of artificial intelligence system that is trained on large amounts of text data to generate human-like language output. LLMs are designed to understand and generate text in various forms, including language, stories, and even code.
Question 2: What are the shortcomings of LLMs?
LLMs have several shortcomings, including high costs to train and update, limited non-linguistic knowledge, false and self-contradictory statements, and socially and ethically inappropriate content. Professor Dietterich highlights these deficiencies in his keynote presentation.
Question 3: What is Dr. Dietterich’s proposed solution?
Dr. Dietterich proposes a more modular architecture for LLMs that decomposes the functions of existing systems and adds additional components to address the limitations of LLMs. This approach combines state-of-the-art machine learning and software engineering best practices.
Question 4: Why is a more modular approach necessary?
A more modular approach to AI systems is necessary because it can lead to more reliable, scalable, and efficient solutions. By decomposing complex functions and adding additional components, a modular architecture can address the shortcomings of existing LLMs and provide more comprehensive solutions.
Question 5: Where can I learn more about Dr. Dietterich’s keynote presentation?
Dr. Dietterich’s keynote presentation is available on YouTube, where you can learn more about the proposed solution architecture and limitations of LLMs. Additionally, the presentation slides are available for download, providing a detailed overview of the discussion.
