To summarise

Using AI to summarise social media for you.

Published 31st December, 2023

I like how Reddit allows anyone to share their experience anonymously, on "r/Ask" posts there's usually a good diversity of responses. The comments usually cover common themes and sometimes there is deeper exploration of ideas in threads.

I wonder if it would be helpful to have a summary of the post conversation at the top of the page. Ideally this would

  • filter out low effort comments
  • remove repeated ideas
  • surface novel ideas or perspectives that are relevant to me

AI large language models (ChatGPT etc) can summarise content well in a generic manner. With a bit of careful prompt engineering, they understand the basics of what matters to a user and respond relating to that.

Novelty is more difficult. A mind can easily read thousands of posts and note which are new, yet typical AI chatbots are heavily limited by the size of their conversation history. As of 2023, the more we tell them to remember, the worse they get at recalling a given fact.

So, let's help our chatbot remember by giving it database. It can store all the content it reads and ask a helper service to notify it when any similarity with older ideas happens. This database could also allow the AI to compare the summarised ideas of one Reddit thread to other online discussions or content the user has read before, preventing meta-repetition ("I've read this summary before...").

However, even a chatbot with a database will need careful prompting and instruction to understand what we'd consider meaningful and helpful. I'd imagine it'd be easier to respond to the bots' first draft of a summary and ask for specifics, turning a passive reading experience into a back and forth conversation.

This isn't really the experience I'd want from a quick summary though. To get a perfect summary first time, you'd have to spend a lot of time with the bot in advance so it understands your values, and you'd have to have viewed plenty of articles before, so it has a good understanding of what it thinks you know. At a guess, this should be doable in the next 5-10 years of AI advances.

The downsides

Filtering content based on novelty is likely a bad idea, as oftentimes I find myself re-reading an idea that is familiar to me, but then understanding it in a different way as my perspective has changed over time.

It'd likely be much more valuable to engage with the content yourself, as you'll understand it deeper by thinking it through manually. The summary provided by AI will only be as intelligent as the AI model itself, and current AI intelligence is still significantly sub-human and hallucinogenic.

A lot of our thought and resolution of concepts occurs slowly in the background. If we read summary after summary, how much will stay with us? I imagine there is a scientific study out there which investigates this, but I guess this issue will largely be resolved by people pausing frequently when reading summaries and letting their mind roam a bit.