The individual models also differed in how they used interactive tools to filter and sort web pages.
- Gemini worked with filters in bulk and typically combined multiple types of filters in different tests.
- GPT-4o rarely used filtering and only interacted with it in a few cases.
- Claude filtered more often than GPT-4o, but not as precisely as Gemini.
Consistency of AI agents
The testing also examined how often agents selected the same hotel or had the same behavior when repeatedly entering the same prompt.
The Claude model presented moderately consistent results in selecting the same hotel based on the same challenge, but compared to GPT-4o, it selected from a wider group of hotels.
The Gemini model was the least consistent, providing a wide range of hotel choices and having less predictable results when repeated for the same challenges.
Unambiguity of AI agents
One of the outcomes of the testing was also the clarity, or specificity, of the AI agents. The researchers assessed how often the agent chose a specific hotel and “stuck” to it, rather than offering multiple options or vague suggestions.
Specificity shows how decisive an phone number list agent is when completing a booking. A higher score indicates that they were more likely to commit to one choice, while a lower score indicates that they tended to present multiple options or respond less clearly.
Gemini received the lowest score of
60% because it often offered multiple hotels or an indefinite selection instead of prioritizing one solution.
- Claude reached 74%, most often choosing just one hotel, but with greater variability.
- GPT-4o achieved the highest score of 95% by almost always providing just one clear recommendation for retargeting ads in hubspot hotel selection.
These findings suggest that advertising strategies may need to evolve toward structured, keyword-rich message formats that are more optimized for how AI agents perceive and evaluate information, rather than focusing on eye-catching visual design or emotional appeal.
What can we take from all this?
The study examined how AI agents for three selected major language models interacted with online advertisements for hotel mobile number list bookings on the website. Each model was given the same prompts and performed the same booking actions.
Banner ads showed more clicks than sponsored or native ad formats, but the most important aspect of an ad’s effectiveness was the content of relevant keywords in the visible text.
