My team and I advocate the use of RESTful Service Oriented Architectures (SOA) to our organization where we work with many large data sets used in web-based applications. We strongly believe that the application should live in the client and that the server should be used as a services layer for providing and persisting data. Allowing the client to pull the data it needs can be useful for creating seamless user experiences that include features like sorting, filtering and searching without multiple trips to the server.
Recently, a question was asked about the right time to introduce server-side pagination to the json responses. There are many variables that can impact this decision. Before immediately jumping to pagination, step back and ask yourself a few of these questions.
How will your users interact with the data?
It is important for a developer to know their audience and the audience's workflow. Understand how users need to work with the data. If you are displaying unnecessary data or doing unnecessary work, this will ultimately slow down the experience. Spending time understanding user processes and finding technical ways to solve problems is just as much the developer's job as it is the Business Architect's and ux Designer's jobs.
Does the current workflow need a change?
The current process might simply require a facelift. Just because a process has been followed for years, doesn't mean that it isn't flawed. It might make sense to change or improve a current process in your application to balance the user's needs and the performance of the application. The platform you are using may also warrant a workflow change. What used to work on paper may work different on a computer or a tablet. Put together a strong argument and push the issue to the project owners. The outcome may be surprising.
Is the user experience overwhelming?
Returning thousands of records to a user is simply overwhelming and not practical. On top of the performance issues that can occur, no one is capable of sifting through and working with thousands of records in any meaningful way. Find ways to refine the data before it is even displayed to the user. If the workflow is for a scheduler, initially provide data that is relevant to that day. Or if it is a list of customers, show only the active customers on the system. Then give the user options for filtering the data.
How can the user experience be improved?
This question is really about pulling out the toolbox and finding out what tools should be used to create a better experience for the user. Its a pretty hefty toolbox of possibilities. Don't let it go to waste! Figure out the best ways to keep the user informed, engaged and waiting less time for the information they require.
Are there code issues?
Let's face it… Sometimes our sound logic isn't so perfect. Validate that there aren't logic issues that could be slowing down the experience. Be sure that you aren't looping your JSON object and updating the html DOM in the loop. Save and batch your operations that will generate document reflow until after you have prepared your data. If you find it difficult to refactor your own code, pull in another developer to work through it with you.
Do you need to rework your resource?
A chain of resource requests can be cumbersome and slow as the user is forced to wait for set of data to be retrieved before making another request. Does it make sense for the resources to be separated or should the data be combined to a single resource? If it does make sense for each of these nouns to have their own resource, create a special action on the resource to return all of the data with one response. This can speed up the experience tremendously with very little effort and without breaking general RESTful paradigms.
Can some of the data be prefetched?
It may make sense to grab a full set of data before it is needed and work with it out of memory. Instead of making many small requests for subsets of data, make one big call and return an entire set of data. This removes some of the waiting game and reduces the number of requests on a server and can dramatically speed up the display of data. This method is great for objects that are cacheable.
Is your data cacheable?
Be sure to properly handle caching of your objects. If your large object is fairly static, don't push the user to download it every time. Allowing the client to cache the object will speed somethings up dramatically. Maybe even consider some client-side storage if your requirements and client support lists allow for this.
In the end…
There are numerous ways to improve the user experience that doesn't include server-side paging of data. A good UX design will take this into consideration and most times a better solution will present itself. Just remember to always keep your user informed if there is any waiting and provide them with the best possible workflow experience based on their needs and the performance of the application. If the decision is made to use pagination in your RESTful service, be sure to follow RESTful rules and be sure to think about the UX tools like prefetching.
Article doesn’t really talk about how to do pagination. Waste of time.
That’s because this article isn’t about “How” it is about “When” it is appropriate to use pagination in your REST resources.