I'm building a real-time boat tracker.
The idea:
- A table stores the location updates of each individual boat.
- Another table stores an aggregated JSON document with the positions of all boats. This JSON is sent to all clients with the real-time API.
If there are, for example, 100 boats updating every second, I would end up with 100 updates per second to the aggregated JSON, which is too frequent and creates high load for both clients and server. I want to limit the frequency of updates to the aggregated document (e.g., update it at most once per second), while still keeping all individual boat positions up-to-date. What would be the best approach to implement this?
I think putting items in a queue and processing them at the same interval would probably work here?
I think I would have a boat location queue table that maybe has something like boat id, lat/lng, and status. Then fetch all that information to aggregate it and bulk update the queue status for them? that way you aren't aggregating a lot per second but once per second for every boat that is active?
The problem is who to schedule for 1s loop aggregation
Can you schedule a function that runs on a 1 second cron?
No 😔
why not? too many runs?
No, the minimum time is 1m
Appwrite doesn't allow you to set something less than that due to chron syntax
Well, maybe I can add some kind of cooldown in the function 🤔
So you need something that gets the data and holds it until there's X amount of data or Y time has elapsed.
Once the time has elapsed, update the aggregate document.
You could do this with a function, but I'm slightly worried about conflicting writes on the aggregate document
Yes, that's the issue, how to hold it for X time
couldn't you have a separate table storing everything with a date/time, and then then either have a cron job running every few seconds checking if there are X amount of data or check the "oldest" data and see if Y time elapsed and if either happens, you just use that data however you need it like write it into another table
essentially using this table as a temporary storage
Recommended threads
- Websocket error Realtime
Hi peeps, I am trying to create realtime updates and I am having websocket error Code is below this msg
- Realtime not processing messages
When subscribing to realtime events of database, the client's websocket receives the messages in the websocket feed, but doesn't call the specific message handl...
- Appwrite console is too heavy
The Appwrite console is too heavy And all of my services broken Any support , please