A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks

A preliminary study compared the performance of ChatGPT-3.5, a large language model (LLM), and humans on Random Number Generation Tasks (RNGTs) to assess if ChatGPT-3.5 exhibits human-like cognitive biases when generating random number sequences. The study found that ChatGPT-3.5 more effectively avoided repetitive and sequential patterns compared to humans, with notably lower repeat frequencies and adjacent number frequencies. The research suggests that further investigation into different models, parameters, and prompting methodologies could help LLMs more closely mimic human random generation behaviors, expanding their applications in cognitive and behavioral science research. The study investigates LLMs' ability to mimic human cognitive processes, specifically in generating random sequences. Unlike traditional computational methods, LLMs process and generate sequences in a way that could potentially mirror human randomness, marked by inherent biases and imperfections. The research aims to compare LLMs' performance against human performance in random number generation tasks (RNGTs) to understand how generative AI might replicate and diverge from human cognitive biases. The study uses contemporary LLMs, focusing on OpenAI's ChatGPT model, to replicate existing human studies' conditions. The researchers crafted a specific user prompt to ensure the model's outputs reflect its standard operational parameters. The results show that ChatGPT demonstrates a higher level of apparent randomness than human outputs but still falls short of achieving perfect randomness. The study's findings contribute to a better understanding of LLMs' capabilities and limitations in generating random sequences. In conclusion, the study highlights the potential of large language models in generating random sequences that mimic human behavior, albeit with some limitations. The findings suggest that further research is needed to optimize LLMs' performance in this area, potentially expanding their applications in cognitive and behavioral science research. The study also underscores the importance of understanding the unique characteristics of LLM-generated sequences compared to human-generated ones, which could inform the design of new experimental methodologies for AI behavioral research.