The hype behind AI technology, particularly the speed of tech suppliers’ advances, has deferred the demand side’s duty to gauge the precise extent of the technology’s contributions. Users are continuously amazed by AI’s actual capability for word string concatenation. However, we would argue that such amusement vanishes as the queried topic hits closer to home, which in turn draws a line over AI’s capability, thereby limiting its contribution. Although there could be a billions of AI use cases, analyzing how much people trust the source when it comes to health issues allows us to clearly see the limitations of technology adoption. This blog post compares responses from the Census Bureau Pulse survey about health information from different sources to approximate an analysis.

Prompting large language models delivers answers just as the internet has done for decades. The way the technology organizes those answers is truly revolutionary. However, such information has always required vetting; if nothing is at stake, AI can get away with useless generalizations. Let us take the case of health information by source to understand the nuances of AI responses. Assuming a health-related query encompasses the well-being of a real-world person, the expectation is that the answer must be fully vetted and understood by the user before implementation. This is where we see how people consume AI text outputs and realize the extent of AI’s usefulness. In other words, if nothing is at stake, AI outputs look stunning. Otherwise, AI outputs may look unreliable.

Obviously, there are so many use cases for leveraging AI (Automation, Vision, Grammar and Spelling); however, we believe that at the core of all of them is the issue of trust. “To what extent do I trust AI outputs?” is the question any use case analyst asks. Precisely, the health information use case data both amplifies user concerns and enables resonance from any other corner of analysis. The Census Bureau asked in its Pulse Survey how much respondents trust information about health from four different sources (Government, AI, Doctor, Scientist). The survey asked the following questions:

• In general, how much would you trust information about health or medical topics from government health agencies?

• In general, how much would you trust information about health or medical topics from scientists?

• In general, how much would you trust information about health or medical topics from a doctor?

• In general, how much would you trust information about health or medical topics from AI tools such as ChatGPT, Google Gemini, or Claude?

Chart 1

The answers reveal how much respondents trust AI when something serious is at stake (Chart 1 above). For tools such as ChatGPT, Google Gemini, or Claude, the aggregated top responses “A great deal” and “Quite a lot” registered 17%, while for doctors, the aggregated top responses equaled 75.8%. In fact, government agencies get a better rank on trust with 32.4%.

Generalizing data points is always a double-edged sword. Claiming that the low level of trust in AI outputs on medical and health topics is applicable to every other type of outcome is naive. However, analyzing these data gets us closer to understanding the fundamentals of AI implementation, and thereby the extent of its adoption.

At Econometricus we deliver quality mixed-methods research that ranks high on both academic and social media impact. We build cogent data-rooted rationales that machines (AI) can not match. Sources wise, we have more than fifteen years of experience leveraging most public sources datasets.

Feel free to get in touch to discuss your research needs.