It’s a Little HAIzy: Understanding the Current Use of Artificial Intelligence in Scientific Research

In February of this year, an article was published in Frontiers in Cell Development and Biology that made waves [1]. The peer-reviewed scientific journal quickly retracted the article after major backlash, but not before a slew of readers were startled by the figures within, which were credited to a popular generative AI tool called Midjourney [2]. One AI-generated image featured an anatomically incorrect rat, with nonsensical labels such as “iollotte sserotgomar cell,” which is not a kind of cell at all. While the rat is the figure that circulated social media, all figures in the paper offered false depictions and garbled text. Such blatant oversight of obvious inaccuracies shocked those within and without the field and placed those involved under sudden scrutiny. According to journalists from Motherboard, one of the reviewers pointed fingers at the journal that allowed publication to proceed; the journal pointed fingers at the article’s authors, who failed to address several reviewer comments, and the authors and editor did not want to comment [3].

Diving into the uncharted waters of artificial intelligence makes it difficult to determine where to place the blame, and the fiasco forces us to acknowledge the use of AI in research and consider how it is utilized. Should there be regulations? Many would say yes but are unsure what those regulations would be. So when should the use of AI be noted in an article? Here, the use was clearly cited and images credited to a generative AI tool, but perhaps that is not enough. Furthermore, if AI is used to summarize or assemble data, should it be counted as an author? If not, what counts as plagiarism? There are a multitude of such questions, and in the coming years, we will have to develop answers or else watch AI grow unchecked.

In order to get into the weeds here, let’s first define what the tools are and how they work. Artificial intelligence, or AI, is an attempt to simulate human intelligence with computer systems. The idea is that these systems can learn from new inputs, reason by using what they learn to draw conclusions and make decisions, and self-correct so that they become more and more accurate over time [4]. AI can be trained through machine learning, in which training data is given to the system so that it can learn to recognize patterns within a dataset and begin predicting what should come next and which data points might be outliers. The more training sets provided, the greater the confidence with which the system can make predictions, and the more competent it can become at detecting patterns [5].

Deep learning increases system complexity and functionality through the use of neural networks. Neural networks are an attempt to algorithmically mimic how the human brain processes information, at least as far as we currently understand a brain to function. The network consists of layers of “neurons,” which receive a feature of a dataset as input. Each applies some sort of mathematical transformation to the input, and then decides via an activation function whether the neuron should release an output. The next layer of neurons receives these outputs as inputs and performs their own transformations to give to the next layer, and so on and so forth. The final layer produces a final output, such as deciding how the original data should be classified as a whole (i.e., what kind of cell it must be, based on analysis of all its features) [6, 7, 8]. With extensive training, the system can learn how to adjust the weights given to certain neurons and their mathematical transformations, and produce more accurate predictions.

Once the system has been given training sets to analyze and can “think” on its own, it becomes an extremely powerful tool that we cannot always understand. There can be so many layers to a neural network that occasionally a system will reach a conclusion, but we can’t follow the thought process it took to get there [6, 7, 8]. It can truly operate like another (extremely computationally powerful) person in the room, though its conclusions should ideally be validated, as they are still predictive at the end of the day.

Deep-learning models form the basis of generative AI. The area refers to AI models that go a few steps further from analyzing existing data, and can generate new, original content. This can be in the form of text, images, audio or even videos [9]. Language Learning Models (LLM), such as ChatGPT or Gemini, are the models that can serve almost as a conversational alternative to Google, explaining how to use software programs, how to code, or helping brainstorm ideas for a paper or a project. They can be extremely helpful in scientific research, speeding up progress through the writing or coding process by providing answers that might take a while to find or understand otherwise. However, it is very important to understand that these models aren’t always correct, and in summarizing information from an article or the World Wide Web, GenAI systems sometimes miss major points and can create content that inaccurately or incompletely represents real concepts. The Frontiers article mentioned previously is a potent testament to such shortcomings.

While LLM models are perhaps the best known, they are not the only AI tools widely used in the research space. There are programs that are extremely helpful in analyzing images and performing bioinformatic analyses, and in macromolecular modeling. The hugely popular AlphaFold is a great example, which can predict protein structures based on inputted amino acid sequences [10].

With a large repertoire of applications in mind, AI emerges as a powerful tool generally accepted as the way of the future. That being said, its rise to greatness is certainly not without resistance, especially as its use continues to expand so quickly and often ahead of regulatory measures. Those who don’t adapt and learn how to work alongside AI may find themselves “left behind” in some sense as AI becomes increasingly commonplace, but then again, an excess of caution may not only be smart but essential.

Any facts taken from AI-based technologies should be double-checked, especially if they are advising next steps in an experiment or suggesting what should go into a publication. If not, misinformation can spread like wildfire. Biases are very important to consider in this light. AI programs are trained on datasets provided to them, and so any biases present in a training set will inform all interpretations from the program. Reproducibility is another important feature to check for, especially when using AI in a data processing pipeline. There are many considerations to keep in mind when using AI tools in research, but how can we ensure considerations are enforced? Many scientific journals have now delineated policies on the use of AI, but the rules are not completely uniform and are often stated with the stipulation that in a rapidly evolving field, the rules may change.

Below are links to, and the summarized details of, various journals’ current AI policies. While there is variation among them, the general consensus is that authorship is attributable to humans and not to any AI system. Cell Press may have put it most clearly, pointing out that “authorship implies responsibilities and tasks that can only be attributed to and performed by humans.” Along those lines, authors are always responsible for the factual accuracy of an article, and all content in a manuscript should reflect the authors’ own ideas. Reviewers and editors are expected to use AI sparingly or not at all in their work, to protect confidentiality and ensure up-to-date, expert opinions.

Where the lines get a little more blurry is with AI-generated images, as the complicated landscape of copyright laws and research integrity come into play. Some journals, such as those under Cell Press and Springer Nature, omit the use of AI-generated images entirely, with few exceptions. Others allow images but make clear that authors are fully responsible for all content within them and must ensure accuracy and originality before publication. In short: If you feel you must use AI, then record it as you would any other tool, but at the end of the day, you are the author.

Springer Nature[11]:

• LLMs do not satisfy authorship criteria, and any use of LLMs within the manuscript must be documented.
• AI can be used for copy-editing of human-generated text without citation, but AI-generated writing cannot be used.
• Original, human-derived work must be represented by the paper, and the authors must take accountability for the manuscript.
• GenAI image creation is complicated, especially with copyright and research integrity issues. Springer Nature does not allow any generated images at all, so they don’t have to deal with this, except for companies they are in contractual relationships with (other exceptions, too, on a case-by-case basis).
• Peer reviewers should not upload manuscripts to GenAI tools as that is sensitive information.
• Ideally, peer reviewers will do all their own work as their “expertise is invaluable and irreplaceable.” Generative AI tools are not considered experts in the field and can make errors. If AI is used in review, it must be stated.

Cell Press [12]:

• “Authors must not list or cite AI and AI-assisted technologies as an author or co-author on the manuscript since authorship implies responsibilities and tasks that can only be attributed to and performed by humans.”
• AI can be used for copy-editing, but should not generate textual content that forms large parts of the manuscript.
• “Authors are ultimately responsible and accountable for the contents of the work.”
• Use of generative AI and AI-assisted technologies should be mentioned under their own declarative statement.
• Reviewers and editors cannot use GenAI.
• No AI-generated images allowed, with few exceptions.

Science [13]:

• AI-assisted technologies do not count as authors or co-authors.
• No cited sources should have AI tools listed as authors.
• Any use of AI in the study, or in writing or presenting a manuscript, must be acknowledged and detailed.
• Authors are responsible for factual accuracy and lack of plagiarism.
• If AI tools are used, authors must try to guard against any bias and appropriately cite all sources.
• “Editors may decline to move forward with manuscripts if AI is used inappropriately.”
• Reviewers cannot use AI technology to write their reviews because they contain confidential information.
• No AI-generated images unless editors allow (exceptions mostly to do with papers that are about AI).
• “The Science journals recognize that this area is rapidly developing, and our position on AI-generated multimedia may change with the evolution of copyright law and industry standards on ethical use.”

Frontiers[14]:

• “Generative AI technologies cannot be held accountable for all aspects of a manuscript and consequently do not meet the criteria required for authorship.”
• Authors are responsible for the factual accuracy of any content created by GenAI, and for any plagiarism.
• Any visual or textual content created or edited by GenAI must be acknowledged and explained.

PLOS [15]:

• AI tools should not be used as reviewers or editors — any use of AI by either should be disclosed to manuscript authors.
• Manuscripts should not be uploaded to any online service, including GenAI tools, as this would breach confidentiality.
• Articles should reflect authors’ own work and ideas.
• Any use of AI in the study or manuscript should be reported with an explanation not only of how the tool was used but how its outputs were validated.
• If LLM is used to generate text, authors are responsible for making sure it is accurate and properly cited with no plagiarism.

PNAS [16]:

• AI tools cannot be authors.
• Any use of AI in the manuscript must be acknowledged, including when used for copy-editing.
• Authors are responsible for any outputs created with help of GenAI and should be checked for factual accuracy.
• No generated images unless article is about the use of AI.
• AI tools cannot be used to create cover art.

AI has immense potential, and is already revolutionizing fields from cybersecurity and finance all the way to health care and scientific research. The common use of generative AI is so new, so public, and so powerful, and needs to be wielded responsibly. Directives should be well-established to enforce proper use and to make sure we publish accurately and informatively. But as the landscape rapidly evolves, regulations and copyright laws are evolving, too. The safest way to go seems to be emphasizing interdisciplinary collaboration, to make sure conclusions make sense and that results are reproducible from a variety of angles. Maybe that’s not such a bad thing either, strengthening relations between people, labs, universities and disciplines. It will be exciting to see where AI goes, and it is important to try and understand both its strengths and limitations as AI tools become increasingly commonplace.

References:

1. Guo, X., et al (2024). RETRACTED: Cellular functions of spermatogonial stem cells in relation to JAK/STAT signaling pathway. Front. Cell Dev. Biol., 11. https://doi.org/10.3389/fcell.2023.1339390
2. https://www.midjourney.com/home
3. Pearson, J. (2024, February 15). Scientific Journal Publishes AI-Generated Rat with Gigantic Penis In Worrying Incident. VICE. https://www.vice.com/en/article/scientific-journal-frontiers-publishes-ai-generated-rat-with-gigantic-penis-in-worrying-incident/
4. What is (AI) Artificial Intelligence? (2024, May 7). University of Illinois Chicago. https://meng.uic.edu/news-stories/ai-artificial-intelligence-what-is-the-definition-of-ai-and-how-does-ai-work/#:~:text=Artificial%20Intelligence%20(AI)%20works%20by,%2C%20perception%2C%20and%20language%20understanding.
5. Brown, S. (2021, April 21). Machine learning, explained. MIT Management Sloan School. https://mitsloan.mit.edu/ideas-made-to-matter/machine-learning-explained
6. How Do Neural Networks Work? Your 2024 Guide (2024, April 11). Coursera. https://www.coursera.org/articles/how-do-neural-networks-work
7. IBM Technology. (2024, August 5). AI, Machine Learning, Deep Learning and Generative AI Explained. Youtube. https://www.youtube.com/watch?v=qYNweeDHiyU
8. 3Blue1Brown. (2017, October 5). But what is a neural network? | Chapter 1, Deep learning. Youtube. https://www.youtube.com/watch?v=aircAruvnKk&t=3s
9. Martineau, K. (2023, April 20). What is generative AI? IBM. https://research.ibm.com/blog/what-is-generative-AI
10. https://alphafold.ebi.ac.uk/
11. https://www.nature.com/nature-portfolio/editorial-policies/ai
12. https://www.cell.com/cell/authors
13. https://www.science.org/content/page/science-journals-editorial-policies
14. https://www.frontiersin.org/journals/artificial-intelligence/for-authors/author-guidelines
15. https://journals.plos.org/plosone/s/ethical-publishing-practice
16. https://www.pnas.org/author-center/editorial-and-journal-policies

---

References

• Disability Care and Artificial Intelligence
• From Research to Clinic: Regulatory Frameworks for AI in Medicine
• Unveiling the Dangers of AI Through Dune’s Insights
• Artificial Intelligence in Medicine—Science Fiction Now Science Fact