Get Even More Visitors To Your Blog, Upgrade To A Business Listing >>

Running query on database after a document/row is of certain age

Running query on database after a document/row is of certain age

Problem

What is the best practice for Running a database-query after any document in a Collection become of certain age?

Let's say this is a node.js web-system with mongoDB, with a collection of posts. After a new post is inserted, it should be updated with some data after 60 minutes.

Would a cron-job that checks all posts with (age 10.000 active users?

Problem courtesy of: JohnC

Solution

Some ideas:

  • Create a second collection as a queue with a "time to update" field which would contain the time at which the source record needs to be updated. Index it, and scan through looking for values older than "now".
  • Include the field mentioned above in the original document and index it the same way
  • You could just clear the value when done or reset it to the next 60 minutes depending on behavior (rather than inserting/deleting/inserting documents into the collection).
  • By keeping the update-collection distinct, you have a better chance of always keeping the entire working set of queued updates in memory (compared to storing the update info in your posts).
  • I'd kick off the update not as a web request to the same instance of Node but instead as a separate process so as to not block user-requests.

As to how you schedule it -- that's up to you and your architecture and what's best for your system. There's no right "best" answer, especially if you have multiple web servers or a sharded data system.

You might use a capped collection, although you'd run the risk of potentially losing records needing to be updated (although you'd gain performance)

Solution courtesy of: WiredPrairie

Discussion

View additional discussion.



This post first appeared on Node.js Recipes, please read the originial post: here

Share the post

Running query on database after a document/row is of certain age

×

Subscribe to Node.js Recipes

Get updates delivered right to your inbox!

Thank you for your subscription

×