• The Horizon
  • Posts
  • What I learned after surveying 150 ChatGPT users

What I learned after surveying 150 ChatGPT users

A deep dive on the productivity gains of using Generative AI

It's been more than a year since ChatGPT was launched in late 2022. ChatGPT was the fastest growing application in history to reach 100 million monthly users. And it attracted the attention of everyone around the world. It promised bold claims on boosting the productivity of knowledge workers, even claiming that a person's billion-dollar business is close to reach.

Google trends of term “ChatGPT“ from September 2022 to March 2024

The new wave of Generative AI promises huge productivity gains for knowledge workers. While I found ChatGPT useful for many tasks such as writing boilerplate code, I was curious to know what difference was ChatGPT making in other people’s lives. Hence, I decided to do a study on the usefulness of ChatGPT in people’s lives.

Data

I conducted a quantitative survey of around 150 users in my network and on Reddit. Most of the users were developers across different parts of the world.

These were the insights I found through the survey:

Tools

Unsurprisingly, 90% of participants used ChatGPT. While most participants use only one AI tool (ChatGPT), some prefer to use more than one AI tool for their workflow.

Percentage of participants and number of AI tools they used

Use Case

Programming was the most popular use case among the participants. This is due to the fact that most of the participants who took the survey were developers. Developers use ChatGPT for multiple purposes:

Writing boiler plate code

Developers like to ask ChatGPT to create a basic template of what they want to make. A functional boilerplate code can be easily edited and customized. This saves hours of research and time in learning documentation of different tools and writing the entire code from scratch.

This can come in handy when the developers are trying to build something they don’t use regularly and have different APIs that they don’t usually memorize (e.g., Regex).

Finding Bugs in Code

Developers also use ChatGPT to find bugs in their code that they could not figure out. By sharing the code snippet, ChatGPT can quickly figure out potential bugs and give suggestions for fixing those bugs.

Write Test Cases

Most developers find writing test cases to be a mundane and tedious task. Hence, they prefer using ChatGPT to automate writing test cases.

Code Documentation

Generating code documentation was another famous text generation use case where LLM can generate code documentation for the function for which they need to write documentation.

Writing Code in Language They are not Familiar with

When developers are not proficient in a new programming language, they usually use ChatGPT to assist them in writing code in that language.

Frequency of Usage

Most participants started using Generative AI tools in the first few months when ChatGPT was launched (November 2022 - January 2023). Since then, the adoption rate of these tools has reduced by 50% and has been constant for the last six months.

45% of participants use generative AI tools every day, and 11% of users use them every hour. More than 90% of users use AI tools monthly, which makes these tools one of the best-in-class SaaS products.

Increase in usage frequency since participants started using Generative AI tools

Over time, most participants have increased their frequency of usage of Generative AI tools. 18% of participants have increased their usage frequency of Generative AI tools from a weekly to a daily basis. 20% of the participants who previously used these models daily continue to use them daily.

Time Saved

Across all participants, the time saved every time they used the Generative AI tool was an average of 1 hour and 20 minutes. This is quite significant considering the increase in speed and output, which allows users to complete valuable programming tasks.

How users feel about Generative AI tools?

When asked participants to rate Generative AI tools on their usefulness in their lives, they rated it 6.0/10 on average. During the survey, we came across participants who found that Generative AI tools saved them days' worth of work, while few claimed that it increased the time they had to spend completing their work. A deeper study would be required to understand the exact use cases where the participants found these tools helpful and when they did not. 

What’s next?

It has only been a year since ChatGPT’s launch, and we are seeing significant improvement in user productivity in text-based tasks such as programming, writing, and content creation. Video models are becoming increasingly capable of real-world video generation and promise improvement in video creation workflow productivity. We are still in the early stages of 3D model generation. However, once they have improved, we should witness productivity improvement in game development, animation, and VFX.

I am amazed at the positive response and interest this survey has received. I plan to do a deeper study on developer use cases of Generative AI. Please comment on this article if you want me to explore any specific area. 

You can access the survey questionnaire used in this survey here. I also encourage you to participate in the survey; I will do another analysis when I have more participants. 

Please subscribe if you found this helpful. I publish an in-depth article every week on AI and Technology.