For a long time, “social listening” at my desk meant screenshotting a glowing LinkedIn testimonial and dropping it in a Slack channel. That was the whole practice. It felt like research, but it really wasn’t.
The actual research — and the stuff I think is actually worth building a brief around — was sitting a few scrolls further down, in the replies nobody on my team had opened. Someone saying “I really wanted to love this but...” There were whole conversations happening in public that none of us were reading, because we were too busy looking at the post that had the most likes.
That’s the habit I want to talk you out of this week. Not because the viral post is worthless, it’s just not what it looks like. And it’s not just me the viral post fools — it’s whatever you hand it to next, including AI.
Why the ‘viral’ post kept fooling me
94% of marketers say they plan to use AI for content this year, but only 19% of content teams have any real way of checking whether that content works. I don’t think that’s because people are lazy. I think it’s because measuring the right thing is genuinely harder than it sounds, and pulling a viral post feels like measurement even when it isn’t.
For a while I made the same mistake with listening that a lot of teams make with competitive research — I confused visibility with intelligence. I’d see a competitor spending on ads and assume that meant they were winning. I’d see a glowing review and assume that meant customers were happy. Loud is a filter for what gets attention in three seconds of scrolling. It’s not a filter for what’s actually going on.
What changed wasn’t better software. It was slowing down and actually reading it before I ever asked AI to help make sense of any of it — which sounds almost too simple to be a “framework,” but here’s how I’m doing it:
The 5-step workflow I’m trying out
1. Pick one lane. I used to start these projects with “let’s see what people are saying about us,” and every single time it produced a report nobody read, including me. What works instead is picking something specific enough that you already know what you’ll do with the answer. Not “our brand” — “the 48 hours before someone cancels a trial.” Not “our category” — “why people switch away from [specific competitor].” If you’re not sure how to narrow it, ask yourself what decision this is actually for. A headline you want to test, an objection your sales team keeps hitting, a churn email that isn’t working.
2. Pull a real spread of 200 to 400 posts and threads. Not just the top performers. I want the five-star post sitting right next to the “I wanted to love this but...” post, because that contrast is the whole point. If every post you’ve pulled is a hit, you’ve built a highlight reel, not a research base. I try to grab this from more than one place too — Reddit, review sites, the replies under a competitor’s own posts — because people are honest in different ways in different rooms. This is the diet I’m building before any of it goes near AI — and a diet of only hits produces a predictably shallow answer.
3. Actually read the comments. I’m looking for a handful of things specifically: the complaint that keeps coming back, in the customer’s own words, not my cleaned-up paraphrase of it. The thing people clearly want that nobody’s offering yet. The objection that shows up before anyone’s even seriously considering buying. And the exact reason someone says they left a competitor. That’s the moment they were ready to switch, laid out in their own language. I don’t think I’ve ever found a more useful sentence in a comments section than someone explaining, unprompted, why they quit using the other guy.
4. Turn the patterns into something you can use. Raw comments sitting in a doc aren’t a deliverable — I have to actually do something with them. Ad angles, content ideas, landing page lines, answers for the sales team when a prospect pushes back. And the list I build first, every time, is the “stuff we need to stop saying” list. It’s usually the most useful thing to come out of this whole process, and also the most uncomfortable, because it means admitting a line we’ve defended in every deck for two years just isn’t landing the way we assumed.
5. Don’t trust one loud thread. AI is good at spotting a pattern. It’s not good at telling you whether that pattern is actually real or just one person having a bad day in public. So before anything from this exercise goes into a brief, I check it against a second source — search the same complaint on a different platform, or just ask someone on the sales or support team if it matches what they’re hearing on calls. If nobody on the front line recognises it, I hold onto it as a maybe, not a headline.
A quick example, so this isn’t just theory
Say the lane is “why trial users cancel before day 7” for a project management tool. I’d pull a couple hundred posts and threads mentioning it and a couple of close competitors, then actually read what’s underneath, not just the star ratings.
Individually, none of it looks like much. A one-line complaint here. A defensive reply there. One long thread where someone compares three tools side by side. But read together, a shape starts to show up: several people saying nobody tells them what to actually do with the tool on day one, a handful of people who say they stuck with a clunkier competitor because of one specific integration, and — this is the uncomfortable one — the homepage says the tool is “built for teams,” but almost everyone in the cancellation-adjacent comments is describing using it solo, and finding the team features got in their way.
A prompt to try this week
The prompt below is only as good as what you fed it in step 2. Once you’ve got your 200–400 posts and comment threads pulled into one doc, try this:
“Analyze this dataset of [your platform] content and comments about [topic]. Identify recurring pain points, desired outcomes, purchase triggers, hesitation reasons, brands mentioned most, claims people distrust, and best-performing hooks. Turn findings into 10 ad angles, 10 content ideas, 5 positioning recommendations, 5 landing page headlines, and 5 product improvement opportunities.”
Give it the real threads, not a summary you’ve already written yourself — it needs to find the pattern across the raw volume, and it can’t do that if you’ve quietly pre-filtered the data through your own assumptions first.
Once you’ve got that first pass back, I like following up with: “Which of these only showed up in one or two threads?” That question does most of the validating for you, and it usually cuts the list down to what’s actually worth building on.
The AI;DR
Elsewhere in the AIverse
Anthropic just put its entire teaching library in one place. The Claude Academy bundles 355 tutorials, prompting tips, and use-case walkthroughs covering Claude, Cowork, Code, Tag, and the API — basically everything you'd have had to hunt for individually before now. Same week, Anthropic also shipped Concise Mode for Claude Code, which trims its default responses down to short and to the point. Also heads up: Anthropic is reportedly planning to file IPO paperwork as early as this month.
ChatGPT might be breaking up with Reddit. Per Promptwatch, Reddit's share of ChatGPT citations dropped 86% since mid-July — from nearly 4% down to roughly 0.5% by last week. OpenAI hasn't explained the shift, and it's a genuinely abrupt reversal given Reddit has historically been one of the most-cited sources across AI chatbots.
Nous Research gave desktop AI agents a way to talk to each other. Bot Mode in the Hermes Desktop app lets you give each agent its own skillset, model, and memory, then have them communicate directly and share context across threads instead of you relaying everything by hand. Works on macOS, Windows, and Linux.