Skip to main content Home Creators kunanonj ai-skills-hub cursor-plugin-convex-rule-use-pagination-for-large-datasets
cursor-plugin-convex-rule-use-pagination-for-large-datasets Use cursor-based pagination for large datasets instead of .collect(). Prevents performance issues and provides smooth infinite scroll.
Jump to install Skills Marketplace Discover and explore AI skills built by the community.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Copy promptShow prompt details A direct command skips the review prompt. Inspect the source before running it.
npx skills add https://github.com/KunanonJ/ai-skills-hub --skill cursor-plugin-convex-rule-use-pagination-for-large-datasetsThe command stays on one line. Scroll horizontally to inspect it before copying.
Prefer a local copy? Download the files currently available to SkillsMP.
Download Zip Downloading... More from this repository
cursor-plugin-cf-agents-sdk Build AI agents on Cloudflare Workers using the Agents SDK. Load when creating stateful agents, durable workflows, real-time WebSocket apps, scheduled tasks, MCP servers, chat applications, voice agents, or browser automation. Covers Agent class, state management, callable RPC, Workflows, durable execution, queues, retries, observability, and React hooks. Biases towards retrieval from Cloudflare docs over pre-trained knowledge.
cursor-plugin-cf-cloudflare Comprehensive Cloudflare platform skill covering Workers, Pages, storage (KV, D1, R2), AI (Workers AI, Vectorize, Agents SDK), feature flags (Flagship), networking (Tunnel, Spectrum), security (WAF, DDoS), and infrastructure-as-code (Terraform, Pulumi). Use for any Cloudflare development task. Biases towards retrieval from Cloudflare docs over pre-trained knowledge.
Related occupations SOC
Based on SOC occupation classification
name cursor-plugin-convex-rule-use-pagination-for-large-datasets description Use cursor-based pagination for large datasets instead of .collect(). Prevents performance issues and provides smooth infinite scroll. metadata {"version":"0.1.0"}
Use Pagination for Large Datasets
Never use .collect() on large or unbounded queries. Use Convex's cursor-based pagination instead.
The Problem with .collect()
❌ Bad: Loading Everything
export const getAllTasks = query ({
handler : async (ctx) => {
return await ctx.db .query ("tasks" ).collect ();
},
});
Problems:
Slow for large datasets (100s-1000s of items)
Wastes bandwidth
Poor user experience (long loading)
Can hit memory limits
Doesn't scale
✅ Good: Pagination
export const getTasks = query ({
args : {
: paginationOptsValidator,
},
: (ctx, args) => {
ctx.
. ( )
. ( )
. (args. );
},
});
paginationOpts
handler
async
return
await
db
query
"tasks"
order
"desc"
paginate
paginationOpts
Fast (only loads what's needed)
Scales to millions of items
Smooth infinite scroll
Reactive updates
Better UX
When to Paginate
✅ Always Paginate:
User-generated content (posts, comments, messages)
Activity feeds and timelines
Search results
Notification lists
Any list that could grow unbounded
Lists with > 100 items
⚠️ Maybe Don't Paginate:
Small, bounded lists (user's 3 favorite colors)
Configuration options (< 20 items)
Dropdown menus with limited options
Tags or categories (if truly limited)
Rule of thumb: If it could grow to 100+ items, paginate!
Basic Pagination
Backend: Query with Pagination import { query } from "./_generated/server" ;
import { paginationOptsValidator } from "convex/server" ;
export const listTasks = query ({
args : {
paginationOpts : paginationOptsValidator,
},
handler : async (ctx, args) => {
const user = await getCurrentUser (ctx);
return await ctx.db
.query ("tasks" )
.withIndex ("by_user" , q => q.eq ("userId" , user._id ))
.order ("desc" )
.paginate (args.paginationOpts );
},
});
{
page : Doc <"tasks" >[],
continueCursor : string ,
isDone : boolean ,
}
Frontend: usePaginatedQuery import { usePaginatedQuery } from "convex/react" ;
import { api } from "../convex/_generated/api" ;
function TaskList ( ) {
const { results, status, loadMore } = usePaginatedQuery (
api.tasks .listTasks ,
{},
{ initialNumItems : 20 }
);
return (
<div >
{results?.map(task => (
<TaskItem key ={task._id} task ={task} />
))}
{status === "CanLoadMore" && (
<button onClick ={() => loadMore(20)}>Load More</button >
)}
{status === "LoadingMore" && <div > Loading...</div > }
</div >
);
}
Infinite Scroll Pattern import { useEffect, useRef } from "react" ;
function InfiniteTaskList ( ) {
const { results, status, loadMore } = usePaginatedQuery (
api.tasks .listTasks ,
{},
{ initialNumItems : 20 }
);
const observerRef = useRef<IntersectionObserver >();
const loadMoreRef = useRef<HTMLDivElement >(null );
useEffect (() => {
if (observerRef.current ) observerRef.current .disconnect ();
observerRef.current = new IntersectionObserver ((entries ) => {
if (entries[0 ].isIntersecting && status === "CanLoadMore" ) {
loadMore (20 );
}
});
if (loadMoreRef.current ) {
observerRef.current .observe (loadMoreRef.current );
}
return () => observerRef.current ?.disconnect ();
}, [status, loadMore]);
return (
<div >
{results?.map(task => (
<TaskItem key ={task._id} task ={task} />
))}
{status === "CanLoadMore" && (
<div ref ={loadMoreRef} className ="h-20 flex items-center justify-center" >
Loading more...
</div >
)}
</div >
);
}
Pagination with Filters export const listTasks = query ({
args : {
status : v.optional (v.union (
v.literal ("todo" ),
v.literal ("done" )
)),
paginationOpts : paginationOptsValidator,
},
handler : async (ctx, args) => {
const user = await getCurrentUser (ctx);
let query = ctx.db
.query ("tasks" )
.withIndex ("by_user" , q => q.eq ("userId" , user._id ));
const results = await query.order ("desc" ).paginate (args.paginationOpts );
if (args.status ) {
return {
...results,
page : results.page .filter (task => task.status === args.status ),
};
}
return results;
},
});
Note: For better performance, use compound indexes:
tasks : defineTable ({
userId : v.id ("users" ),
status : v.string (),
}).index ("by_user_and_status" , ["userId" , "status" ])
export const listTasks = query ({
args : {
status : v.union (v.literal ("todo" ), v.literal ("done" )),
paginationOpts : paginationOptsValidator,
},
handler : async (ctx, args) => {
const user = await getCurrentUser (ctx);
return await ctx.db
.query ("tasks" )
.withIndex ("by_user_and_status" , q =>
q.eq ("userId" , user._id ).eq ("status" , args.status )
)
.order ("desc" )
.paginate (args.paginationOpts );
},
});
Reactive Pagination Convex pagination is fully reactive ! When data changes, pages update automatically.
The Challenge (Other Systems) With offset-based pagination, insertions/deletions cause:
Duplicate items across pages
Missing items
Inconsistent pagination
Convex Solution Convex uses cursor-based pagination with automatic tracking:
const { results } = usePaginatedQuery (api.tasks .listTasks , {}, { initialNumItems : 20 });
Cursors track positions, not offsets
Pages automatically adjust to data changes
Seamless reactive updates
Advanced: Custom Pagination For complex cases, use getPage from convex-helpers:
import { getPage } from "convex-helpers/server/pagination" ;
export const customPagination = query ({
args : {
category : v.string (),
paginationOpts : paginationOptsValidator,
},
handler : async (ctx, args) => {
const user = await getCurrentUser (ctx);
const results = await ctx.db
.query ("posts" )
.withIndex ("by_user" , q => q.eq ("userId" , user._id ))
.filter (q => q.eq (q.field ("category" ), args.category ))
.collect ();
return getPage (results, args.paginationOpts );
},
});
Search with Pagination export const searchTasks = query ({
args : {
searchTerm : v.string (),
paginationOpts : paginationOptsValidator,
},
handler : async (ctx, args) => {
const user = await getCurrentUser (ctx);
const allTasks = await ctx.db
.query ("tasks" )
.withIndex ("by_user" , q => q.eq ("userId" , user._id ))
.collect ();
const filtered = allTasks.filter (task =>
task.title .toLowerCase ().includes (args.searchTerm .toLowerCase ())
);
return getPage (filtered, args.paginationOpts );
},
});
Better: Use Convex's built-in text search when available.
Pagination Options interface PaginationOptions {
numItems : number ;
cursor : string | null ;
id ?: string ;
}
const page1 = await query.paginate ({ numItems : 20 , cursor : null });
const page2 = await query.paginate ({ numItems : 20 , cursor : page1.continueCursor });
Common Patterns
Pattern 1: Load More Button const { results, status, loadMore } = usePaginatedQuery (
api.tasks .list ,
{},
{ initialNumItems : 20 }
);
<button
onClick ={() => loadMore(20)}
disabled={status !== "CanLoadMore"}
>
{status === "LoadingMore" ? "Loading..." : "Load More"}
</button >
Pattern 2: Show/Hide Older Items const [showAll, setShowAll] = useState (false );
const { results } = usePaginatedQuery (
api.tasks .list ,
{},
{ initialNumItems : showAll ? 100 : 10 }
);
<button onClick ={() => setShowAll(!showAll)}>
{showAll ? "Show Less" : "Show All"}
</button >
Pattern 3: Paginated Tabs const [activeTab, setActiveTab] = useState<"todo" | "done" >("todo" );
const { results } = usePaginatedQuery (
api.tasks .listByStatus ,
{ status : activeTab },
{ initialNumItems : 20 }
);
Performance Tips
Use Indexes
.withIndex ("by_user" , q => q.eq ("userId" , userId))
.paginate (opts)
.filter (q => q.eq (q.field ("userId" ), userId))
.paginate (opts)
Reasonable Page Sizes
{ initialNumItems : 20 }
{ initialNumItems : 5 }
{ initialNumItems : 500 }
Order Matters
.query ("tasks" )
.withIndex ("by_created" )
.order ("desc" )
.paginate (opts)
Debugging Pagination
Check Total Items (Dev Only) export const debugCount = query ({
handler : async (ctx) => {
const count = (await ctx.db .query ("tasks" ).collect ()).length ;
console .log (`Total tasks: ${count} ` );
return count;
},
});
Log Pagination State const { results, status } = usePaginatedQuery (api.tasks .list , {}, { initialNumItems : 20 });
console .log ({
itemsLoaded : results?.length ,
status,
canLoadMore : status === "CanLoadMore"
});
Checklist
Learn More