Real-Time Kanban board on Vue.JS
I had a very interesting discussion recently. It was about describing the structure of some real-time application that has to support multiple users working with it at the same time. I don’t remember the exact task, but the core is:
Describe the structure of kanban board application with the following conditions:
- It’s the collection of cards, combined in several columns. Every card is just a title & text;
- Cards can be dragged vertically (to sort them) and horizontally (between columns);
- Multiple simultaneous users supported;
- The App is real-time;
- All changes appear immediately and saved asynchronously. In case of an error, the system returns to the previous state;
- There’s some simultaneous edit prevention logic, like the individual field or the entire card locking while the other user is working on it.
At first, we should think about how to handle the undo. We start editing a card, send the request, and then an error happens. We have to check out to the previous valid state, which means we have to store this valid state separately from the data, entered (but not saved yet) by a user. The other way is to update the data, but keep track of changes in some undo queue.
The next question is data consistency. Obviously sockets are in use, but what if some socket message was lost? We have to think about how to make the system steady and not dependent on a couple of missed socket messages.
Then, what if two users try to update the same card simultaneously? The first request will update the task, but the next one should return an error because it was designated to update the previous version of a card. That leads to the requirement of some versioning.
Cards must save their order. That means there’s some card order property. For example, we have a hundred cards, we drag the 87-rd and drop it between the first and the second. We want to apply it somehow without updating the order of every card between 2 and 87.
The last is the locking logic. We send some locking request when a user starts editing, but what if he leaves the computer for a long time, and the other user wants to edit the same card as well?
So a lot of questions here. And the answer will be the system we’ll build. We’ll use vue/vuex/vuetify at the front end and node/mysql2 at the back end. And we’ll try to make the code as short and simple as possible because… It’s just one article, not a book.
The Result
Here’s the repo: https://github.com/Kasheftin/real-time-kanban-board
Here’s the demo: https://kanban.rag.lt
The code was written as a proof of concept. I don’t recommend to use it as it is in a real application, especially the back end — there’s a lack of validation and error handling (and no ORM at all, just one file and a bunch of raw SQL queries). The node error handling can be improved using my other article: https://codeburst.io/node-express-async-code-and-error-handling-121b1f0e44ba.
The Real-Time Store
The store will contain an array of tasks. It has to be loaded from the API initially, and then we’ll use a socket for real-time updates. But what if the socket will fail to work? We have to be able to request the updated tasks by ourselves. Every task in our database will have created_at and updated_at fields (we’ll make them int = unix timestamp for simplicity). Let’s create /tasks endpoint that will accept dt timestamp parameter and return all the tasks that have updated_at > dt. In this case, if we have any tasks in the front end store, we’ll make a request with dt = max(tasks.updated_at) and receive only these tasks, which were changed since our last request.
Let’s move to the front end store. The state contains an array of tasks. Suppose we have to add or replace some tasks there. We will replace only if the new task updated_at is greater than the updated_at for the current task:
It’s time to write the load action. Initially, it should request all the tasks. Then we want to get updates only. We’ll calculate the max(tasks.updated_at) and request tasks, updated after that time:
loadTasks action can be called anytime. It’s not a bad idea to put it into the timeout and run every minute. Let’s connect WebSocket to add real-time updates. Since we have already written the code that loads tasks through ajax, we’ll not use socket as a data provider. It will be used just for notification. Any task update will broadcast a message with the current timestamp. That’s the route for updating the task (and the similar is for creating the new one):
Finally, let’s move to the primary front end component. There we’ll request loadTasks every minute or by the socket notification:
Delete Task Workflow
I’ve already finished the demo when realized one issue related to the described real-time store. If one user deletes the task, it’s not counted in maxDt calculations, there’s no signal for the other user to remove this deleted task from the store. Obviously we could send a special socket message this case, but what if this message is missed? Any regular request to /tasks?dt=.. does not contain any information about deleted tasks. It has to be fixed.
At first, we’ll follow the soft delete way (it’s called paranoid in sequelize). Instead of deleting the record from the database, we set the special property deleted_at to the current timestamp. Then, we’ll save the timestamp of the initial request tasks?dt=0 to some initDt store variable. And all the following tasks requests will contain it as /tasks?dt=..&since=initDt. They will receive not only tasks, updated since dt, but the deletedTasks since initDt as well. That’s the updated back end for getting tasks:
And the delete action:
Finally, that’s the updated front end store (I added force parameter for the case if we need to refresh all the data):
Sorting
Every task has some unique sort parameter. At first, let’s consider the linked list structure when sort is actually the id of the next task. Then moving the 87-rd task between first and second will require updating the sort for 2 items only (the first will link to 87-rd, the 87–rd will link to the second). But that’s the only advantage here. Linked list is not database-friendly. If we need any type of pagination in the future (to get only the first 100 tasks) there’s no easy way to do that.
We can consider the sort to be initially equal to the task.id. Then it will be unique, and it will be easy to swap any two items. But injecting the 87-rd item between first and second will require swapping of all the tasks between them.
Float could be a good option. Injecting a task between first and second can set sort to 1.5. But we have to think about precision. Moving the next task between first and (new) second will set sort to 1.25, the next inject — to 1.125, etc. The precision of any float is limited in a database. Injecting hoards of tasks here will stop change sort value at some point.
We’ll follow float-like way but make this precision play more obvious. By default, the sort of every task will be equal to id x 1000 . Placing a task after some other task (id1, sort1) will take the next item, (id2, sort2) and try to set sort to (sort1 + sort2) / 2. In the worst case, we have 9 possibilities to do that before the new sort becomes equal to sort1. This will trigger the recalculation of all the sorts back to (index+1) x 1000 once again:
Undoing
Suppose we dragged the card to the other column, but it was not allowed (the API returned an error), and we have to restore the previous state. At first, consider the undo-way. The drag updates the $store.tasks directly and the last user action recorded somehow. Then roll back on error. I believe there’re some systems working that way, but:
- Some extra code for rolling back is required, this code will run in rare cases (ideally never), that’s why there’s a higher probability that it contains bugs by itself.
- The dependent fields are harder to roll back. An address form usually contains several fields (street, country, city, zip), the street field autocomplete (e.g. using google places API) can change the country and city fields. The undo operation for this form requires storing values of all the dependent fields.
That’s why we’ll follow the copy-store-way. We’ll keep the $store.tasks always valid and synchronized with the server. The user will see and will work with the clone, stored directly in the component’s data. And let’s remember we’re writing the kanban board, let it have 4 columns:
If you know any simpler approach, let me know. Here we just copy tasks to the local columns variable and change it according to the user actions. And then send the request. Ideally, the new data will correspond with the picture the user sees on the screen, and he will not notice when the temporary local data will be replaced with the new valid data from the store. In case of error, restoring the previous valid state is done in one row of code.
I used JSON.stringify computed to watch for $store.tasks updates. I don’t like deep watchers and I wrote about this technique here, https://codeburst.io/caution-using-watchers-for-objects-in-vue-ecafb0af6493.
Edit task popup uses the same approach. We copy the task data to some new local variable and work with it using v-model without affecting any data on the other layers.
Versioning
If two users update the same task simultaneously, then the last who sends the request should receive an error, because he is trying to update outdated data. The simplest way is to use updated_at as a task version. We’ll send updated_at of a currently edited task and compare it to the updated_at value in the database:
Locking
The versioned workflow works, but it’s not the best solution. One user starts updating the task description, the other one moves it to the other column, and the first one is unable to save his work. Let’s try to implement the other, locking workflow. When anybody starts editing the task, we lock it. Nobody else can do anything with it (move, delete or edit). What if the user started editing the task, but forgot about that? Let’s add a small user interaction. Anybody can request permission for editing a locked task. If no response from the owner for the next 5 seconds, we switch the ownership.
On the screen below the left user starts editing Task #4. It becomes locked for the right user. The last requests access, but it’s denied. After that the left user saves changes, and the card becomes unlocked. Now the right user starts editing the task and leaves the system. The left user requests access and receives it after 5s timeout.
The workflow is rather complex. Luckily we have the real-time store, we can add any property to the task at the back end, update the updated_at and it will be automatically sent to the client. We need user sessions (in this demo it’s just a randomly generated string sent in the initial /tasks?dt=0 request), and locked_by, locked_switch_requested_by properties. For security reasons, we don’t send them directly. We send a bunch of boolean values instead:
The rest is simple but rather bulk. When the card is opened for editing, we send /tasks/:id/lock request and send /tasks/:id/unlock when the popup is closed. We need two more popups — one for making the request for ownership, and the other one — for granting or denying the request. Then we need /send_unlock_request for substituting the locked_switch_requested_by value. We need /cancel_unlock_request as well. If we made /send_unlock_request but did not receive an answer, we have to send /try_unlock request after 5s timeout. And we need /allow_unlock and /deny_unlock endpoint for the owner as well.
The last chapter (7 more endpoints and 2 more popups) has doubled the code. Any ideas about how to make it smaller?
Here’s the repo: https://github.com/Kasheftin/real-time-kanban-board
Here’s the demo: https://kanban.rag.lt
