Ever tried Twitter Live Search and wondered how it works? On the surface, you search for a #hashtag or a keyword and Twitter shows you a continuous stream of results, including new tweets that appear after you searched!

Image: Live twitter search for #GameOfThrones

We search for #GameOfThrones in the Live Mode. Notice the popup showing 10 new results in the time it took to take this screenshot.

How is Twitter able to return a dynamic result feed for any searched keyword? One way to implement a Twitter Search style live feed is to return the original search results from a database (SQL, ElasticSearch, Mongo etc.) and then have a separate feed component using a realtime channel (think socket.io).

Image: Architecture for a naive realtime feed

We can’t know for sure how Twitter internally implements the realtime feed, but the approach described above, while easy to get started, requires scaling two different components and suffers from data consistency issues while offering an okay-ish O(MxN) time complexity (where M=data insertions, N=open keyword searches).

In this post, we describe a scalable approach to building realtime feeds using Meetup’s RSVPs as an example with appbase.io, a hosted data streams API. On the backend, we use a Node.JS worker to fetch Meetup’s RSVP API and insert them into an appbase.io app. On the frontend, we use jQuery (an alternate post with React.JS instead coming soon) to create the feed UI and query appbase.io in realtime with different keyword filters.