Charting the path forward: On chatbots, CliffsNotes, and calculators [Students, School, & Chatbots]

Chatbots are becoming increasingly prevalent in our daily lives, handling routine tasks and providing instant information at our fingertips. However, while they excel at generating plausible responses, chatbots often struggle with complex concepts and providing creative solutions.  In this blog post, we will showcase a recent development that aims to address these limitations, exploring the implications for students and how to best prepare them for this evolving landscape.

As Language Learning Models, by design, chatbots focus on generating the next word in a sentence without full comprehension of the underlying context. Recently, University of Rochester researchers tackled these limitations and developed a tool based on GPT-4 that synthesizes chemical compounds - such as novel drugs - on its own. The new model, coined ChemCrow, augments GPT-4 with an expert-curated toolkit, providing it a dependable "language" and powerful chemistry-specific calculators. Furthermore, ChemCrow incorporates a chain-of-thought methodology. When given a task, rather than swiftly generating the next plausible word, ChemCrow considers its current phase in the solution and the objective before choosing which tool to use next. This step-by-step approach permits the injection of "thought" into GPT-4 and the successful automation of a wide range of chemical processes. Let’s take a look at the figure below to see how experts scored the performance of ChemCrow compared to tool-less GPT-4.

Expert scores assigned to each task. Tasks are split in three categories and sorted by order of difficulty within each category.

Tool-less GPT-4 outperforms ChemCrow on simple tasks:

In all three categories, tool-less GPT-4 outperforms ChemCrow on simpler tasks. This is hardly surprising. Chatbots are designed to handle straightforward tasks efficiently, and as the technology continues to advance, routine and mundane tasks will increasingly be delegated to chatbots, allowing human professionals to focus on more complex and specialized aspects of their work.

The demand for human experts will be on the rise:

This shift will increase the demand for professionals to examine the correctness and integrity of the data generated by chatbots. Moreover, there will be a greater need for experts who can offer unique insights and innovative solutions that surpass the capabilities of artificial intelligence.

In today’s digital age, where students may be tempted to rely on chatbots for instant answers, it becomes even more critical for us as educators and parents to emphasize the proper use of these tools. Chatbots can be likened to CliffsNotes, in that they provide a useful resource for quick overviews or assessing a book's relevance, but they lack the depth and complexity required to cultivate true expertise or produce exceptional essays and research papers. And just as calculators are effective for solving complex problems only when we have a thorough understanding of the underlying concepts, so are chatbots.

Developing critical thinking skills and proposing creative solutions is a journey that requires dedication, rigor, and a deep understanding of the subject matter. There are, no shortcuts. In the words of the great Basketball legend Kobe Bryant, “You can’t shortchange yourself. You’re not cheating anybody but yourself.”