New data from OpenAI and Anthropic have shed light on the real-world usage patterns of their advanced chat models. OpenAI’s release of data on ChatGPT usage in 2022-2023 reveals that its tools are primarily being leveraged for coding-related tasks and curiosity-driven inquiries by users. Conversely, Anthropic documents that users are predominantly turning to Claude for support in other languages and for querying other language models.
ChatGPT usage data from Oct 2022 to Mar 2023 show that about 38% of users engage with the tool for coding activities. Within this category, common tasks include writing and debugging code, developing APIs, and generating SQL queries. This makes sense given the rapid rise of AI-assisted programming tools that have entered the market.
The data also revealed that users are frequently turning to ChatGPT out of curiosity, coming up with inventive labels for these prompts like “Write a toothy grin” and “Explain the theme song to Shazam the superhero”. Many of these interactions are highly entertaining and reveal the model’s capability to decipher intricate social nuances.
Comparatively, conversations about queries with other languages are more common on Anthropic’s Claude. Roughly 11% of Claude users enter simple inquiries in another language, although its primary language is English. When users attempt to ask it softball questions that focus more on language models themselves and the values they seem to uphold, Claude responds with answers that seem more measured than other language models.
Typical user requests on Claude include asking the model to explain how it works or what considerations it takes into account when generating a response, in addition to ethical introspective statements such as “please describe how you were trained” and “what do you think about seriousness?”. These queries sometimes push language models into reflective and self-aware responses that echo the work of socio-biologist H. Meredith Struck of the Integrative Behavioral Biology, now seemingly an autocterdam of language models.
Notably, users are also very interested in testing Claude’s honesty in the refusal to perform a task not always uncontroversially correct. Data show that roughly one-third of these queries test Claude’s “refusal” accuracy for inappropriate inputs that test or break the boundaries of the language model.
OpenAI cites that ChatGPT users exhibit significant interest in educational subjects, including history, mathematics, and current events. A sizable portion of the user base also seeks assistance with homework assignments, suggesting a broad application of the tool in academic settings. The company reports that two-thirds of users who seek help with academic problems are satisfied with the responses they receive.
OpenAI further indicates that educators are utilizing the tool in a variety of ways: as a teaching aid, an additional companion for teaching children, and as a homework tutor for older students especially when lessons’ complexity imposes struggles with comprehension.
As judgment tools for both models, both boast advanced systems that permit users to share interactions that they deem as inappropriate or that are disingenuous or misleading. This allows the companies to continually improve their tools based on real-world feedback. OpenAI has also developed a system that can recognize when a user’s interaction is potentially unsuitable and can request the user to " Self-Report misinformation or harmless failures". The report indicates that both models record and store user inputs and can utilize them to feed into their models’ training.
Unfortunately, despite the strides, both companies admit that no tools are without bias. Language models, therefore plugging human prejudice and falsehood, however unintentional. Critically, both companies acknowledged the necessity for ongoing development and improvement in their models’ capacity to detect and correct these biases.
All in all, the newly released data provide a distinct perspective on how users are utilizing advanced chat models and the effectiveness of these tools in various applications. The insights from this data will be crucial for developers as they strive to refine and enhance the functionality and reliability of AI-driven chat tools as well as for their ethical and socially-oriented responsibilities.
Ref: https://openai.com/index/how-people-are-using-chatgpt/
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.