As the world explores the new possibilities and uses of artificial intelligence (AI) tools like ChatGPT, researchers at the University of Colorado Department of Biomedical Informatics (DBMI) are integrating similar models into academic authoring.
Recently featured in Nature for their work with generative AI, Casey Greene, PhD, professor in the CU School of Medicine and chair of the DBMI, and Milton Pividori, PhD, developed a new workflow in Manubot, a collaboration platform developed by Greene and others for scientific writing, that uses AI to review and improve research manuscripts.
Manubot was developed to support large-scale collaborative authoring software like GitHub, and this new workflow within the platform allows researchers to spend more time on the creative aspects of research and less time editing manuscripts.
“This workflow automatically revises the text of a manuscript using large language models based on artificial intelligence,” explains Pividori, a postdoctoral fellow who will join the DBMI as an assistant professor of biomedical informatics in May 2023. “I think it will be a huge help for the community to use these tools, incorporate them into our lives, but in a transparent way – perfectly documenting how we use it.”
Prioritizing transparency and human ideation
Manubot connects researchers working on manuscripts and tracks all changes within any document ahead of publication. Although some generative AI-based tools, which focus on generating content from existing data, can pose challenges distinguishing whether humans or machines wrote information, Greene and Pividori designed the new workflow to enable transparency.
“The tool that we developed clearly tracks what text came from the AI and what came from a human,” Pividori says. “If someone wants to see how the manuscript evolved, from the beginning to its readiest state, you can see the evolution exactly.”
Greene and Pividori used the tool to write their paper about its development – applying the tool to existing manuscripts led to it making corrections to formulas and text to improve clarity.
Featured in their preprint, "A publishing infrastructure for AI-assisted academic authoring," the Manubot workflow recommends changes to language in the Discussion section.
“I have not seen a more responsible implementation than this one,” Greene says. “It provides edit-level accountability regarding the text's origin – human or machine. I don’t have concerns about using this tool, though I do have concerns about some approaches that aren’t as transparent.”
Evaluating the performance of a revision software is subjective, Pividori says the tool is best at revising text-based areas, such as introductions, discussions, and sentence structure. However, it isn’t meant to replace the humans who work on this type of research.
“The tool is quite skilled at revising and improving scholarly text, but the truly intelligent component here is the human author behind it,” he says. “At the end, humans need to review the workflow’s suggested changes.”
“I don't think this will replace people,” he continues. “These tools will help us find what's truly human in what we do. For instance, when I do research, I have a question and an intuition on how to proceed it, and then I design an experiment to answer the question. These tools can come into play for the more mechanical parts of the process, such as writing code, but the idea came from a human, not the tool. I think a tool that helps researchers to communicate better is always welcomed, and it will help us to focus more on the things that only humans can do.”
The future of Manubot and ChatGPT
The new workflow is now available to all Manubot users. The current tool uses models based on GPT-3, or Generative Pre-Trained Transformer 3, but does not specifically integrate ChatGPT because it is not yet available for automated use.
“Once ChatGPT becomes available in a programmatic way, or when we have an application programming interface (API), it would be very helpful to incorporate into our tool because it keeps context,” Pividori explains. “That would mean you could feed it the whole manuscript, rather than individual pieces, because ChatGPT can draw connections across the text, such as identifying the definition of an acronym. We implement context in the current tool through custom prompts that layer in important information. When models that incorporate context more directly become available, we will implement them.”
Although the tool is widely available, Greene says it will be up to individual researchers at the CU Anschutz Medical Campus and beyond about whether they want to use it.
“I think ChatGPT demonstrates the potential of these types of systems,” Greene says. “Responsibly incorporating these tools into our workflows can transform the efficiency with which we communicate.”
April 8, 2024