createInfiniteResource

A SolidJS primitive for managing paginated data fetching with built-in memory management and intersection observer support.

Features

• Creates refetchOnView directive for simple use-cases with reactive conditions
• Handles pagination and memory management with maxPages option
• Exposes loading states and error handling per page
• Preserves raw page data while providing merged access
• Zero-config array flattening with customizable merge strategies
• Built-in deduplication with uniqueBy for cursor-based pagination
• Bi-directional pagination support (forward/backward)
• AbortSignal integration to prevent race conditions
• Refetch current page without duplicating data

Instalation

npm install @doeixd/create-infinite-resource-solid

Why is this useful?

Managing infinite scroll in SolidJS typically involves coordinating several primitives (resources, signals, effects) while handling pagination state, memory cleanup, and intersection observers. This primitive handles these concerns while remaining flexible enough for cursor-based pagination, complex data structures, and memory constraints.

Core Concepts

This primitive wraps createResource with some key differences:

1. The fetcher receives a context object for pagination control:

type FetcherContext<P> = {
  setPageKey: Setter<P>;   // Set next page/cursor
  hasReachedEnd: Accessor<boolean>; // Check if at end
  setHasReachedEnd: Setter<boolean>; // Mark as complete
  signal: AbortSignal;     // For cancelling stale requests
  direction: 'forward' | 'backward'; // Pagination direction
}

// Usage
const resource = createInfiniteResource(
  async (page, { setPageKey, setHasReachedEnd, signal }) => {
    const data = await fetch(`/api?page=${page}`, { signal });
    const json = await data.json();

    // Either set next page
    if (json.nextPage) {
      setPageKey(json.nextPage);
    } else {
      // Or mark as complete
      setHasReachedEnd(true);
    }

    return json.items;
  }
);

2. Pages are accumulated rather than replaced:

// Default behavior: Flattens arrays
const { data } = createInfiniteResource<string[]>();
data(); // ["item1", "item2", "item3"] (from all pages)

// Custom merging: Preserve page structure
const { data } = createInfiniteResource<Response>({
  mergeData: (prev, next) => [...prev, next]
});
data(); // [page1, page2, page3]

Important Details

1. Memory Management

   createInfiniteResource(fetcher, {
     maxPages: 5 // Only keep last 5 pages
   });

   When maxPages is hit, oldest pages are removed. This affects what's returned from data() but doesn't refetch dropped pages on scroll up.

2. Loading States

   const { pageData, data } = createInfiniteResource();
   pageData.loading; // Current page loading
   data(); // All accumulated data (even during loads)

   Unlike regular resources, you get both the current page's loading state and accumulated data.

3. Intersection Observer

   // Basic
   <div use:refetchOnView={[true, getNextPage]}>
      
   // With conditions
   <div use:refetchOnView={[
     () => !isError() && !hasReachedEnd(),
     getNextPage
   ]}>

   The directive automatically cleans up observers and respects loading states.

Common Patterns

1. Cursor-based Pagination

type Response = { items: Item[], nextCursor: string | null }

createInfiniteResource<Response, string>(
  async (cursor, { setNextPageKey, setHasReachedEnd }) => {
    const data = await fetch(`/api?cursor=${cursor}`);
    
    if (data.nextCursor) {
      setNextPageKey(data.nextCursor);
    } else {
      setHasReachedEnd(true);
    }
    return data;
  },
  {
    initialPageKey: 'initial',
    mergeData: (prev, next) => [...prev, next] // Keep cursor info
  }
);

2. Error Handling with Retries

createInfiniteResource(fetcher, {
  onError: (error) => {
    if (error.status === 429) { // Rate limit
      setTimeout(getNextPage, 1000);
    }
  }
});

3. Virtual Lists

// Keep limited window of data in memory
createInfiniteResource(fetcher, {
  maxPages: 3,
  mergeData: (prev, next) => {
    const window = [...prev, next].slice(-3);
    virtualizer.setItemCount(totalCount);
    return window;
  }
});

New Features

1. Deduplication with uniqueBy

Perfect for cursor-based pagination where items can appear across page boundaries:

type Post = { id: string; title: string; timestamp: number };

const { data } = createInfiniteResource<Post[], string>(
  async (cursor, { setPageKey, signal }) => {
    const response = await fetch(`/api/posts?cursor=${cursor}`, { signal });
    const json = await response.json();

    if (json.nextCursor) setPageKey(json.nextCursor);
    return json.posts;
  },
  {
    initialPageKey: 'initial',
    uniqueBy: (post) => post.id // Automatically deduplicates by ID
  }
);

// data() now contains unique posts only, even if they appear in multiple pages

2. Bi-directional Pagination

Load data in both directions - perfect for chat applications:

const { getNextPage, getPreviousPage, data } = createInfiniteResource(
  async (messageId, { setPageKey, direction }) => {
    const endpoint = direction === 'backward'
      ? `/api/messages/before/${messageId}`
      : `/api/messages/after/${messageId}`;

    const response = await fetch(endpoint);
    const messages = await response.json();

    if (messages.length > 0) {
      const nextId = direction === 'backward'
        ? messages[0].id
        : messages[messages.length - 1].id;
      setPageKey(nextId);
    }

    return messages;
  },
  { initialPageKey: 'latest' }
);

// Load newer messages (default forward direction)
getNextPage();

// Load older messages
getPreviousPage(); // or getNextPage('backward')

3. AbortSignal for Race Conditions

Prevent stale data when users rapidly trigger pagination:

const { getNextPage } = createInfiniteResource(
  async (page, { setPageKey, signal }) => {
    try {
      // Pass signal to fetch - it will auto-cancel on new requests
      const response = await fetch(`/api?page=${page}`, { signal });
      const data = await response.json();

      setPageKey(page + 1);
      return data;
    } catch (err) {
      if (err.name === 'AbortError') {
        // Request was cancelled - this is expected
        return [];
      }
      throw err;
    }
  },
  { initialPageKey: 1 }
);

// Rapid calls won't cause race conditions
getNextPage();
getNextPage(); // First request auto-cancelled
getNextPage(); // Second request auto-cancelled
// Only the last request completes

4. Refetch Current Page

Refresh data without duplicating:

const { refetchCurrentPage, getNextPage } = createInfiniteResource(
  fetcher,
  { initialPageKey: 1 }
);

// User adds new item, refresh current page
onItemAdded(() => {
  refetchCurrentPage(); // Replaces last page data, doesn't append
});

// Load next page as usual
getNextPage(); // Appends as expected

5. Reactive Directive Conditions

The refetchOnView directive now reacts to condition changes:

const [isPaused, setIsPaused] = createSignal(false);

// Observer dynamically connects/disconnects based on condition
<div use:refetchOnView={[
  () => !isPaused() && !hasError(), // Reactive!
  getNextPage
]}>
  {isPaused() ? 'Paused' : 'Loading...'}
</div>

// Pause infinite scroll
setIsPaused(true); // Observer disconnects automatically

// Resume
setIsPaused(false); // Observer reconnects

Gotchas

1. maxPages drops old data but doesn't refetch - consider UX implications
2. Default array flattening assumes uniform page data
3. Page keys must be managed manually through setPageKey
4. The directive assumes element visibility means "load more"
5. uniqueBy only works with default array flattening, not with custom mergeData
6. AbortSignal cancels the fetch request but doesn't prevent the fetcher function from running

Type Details

createInfiniteResource<
  T, // Response type (e.g., Product[])
  P = number | string // Page key type
>

// For complex data:
createInfiniteResource<Response, Cursor>
// Response = { items: Product[], cursor: string }
// Cursor = string

Custom Data Structures

For non-array responses, each page's data is preserved:

type ThreadPage = {
  messages: Message[];
  participants: User[];
  cursor: string;
};

const { data } = createInfiniteResource<ThreadPage, string>(
  async (cursor) => {
    const response = await fetch(`/api/thread?cursor=${cursor}`);
    return response.json();
  },
  {
    initialPageKey: 'initial',
    // Each page is preserved as an array element
    mergeData: (prevPages, newPage) => [...prevPages, newPage]
  }
);

// Access individual pages
data().map(page => ({
  messages: page.messages,
  participants: page.participants
}));

API Reference

Function Signature

function createInfiniteResource<T, P = number | string>(
  fetcher: (
    pageKey: P, 
    context: FetcherContext<P>
  ) => Promise<T>,
  options?: InfiniteResourceOptions<T, P>
): InfiniteResourceReturn<T, P>

Types

Options

type InfiniteResourceOptions<T, P> = {
  // Initial page key passed to fetcher
  initialPageKey: P;

  // Maximum number of pages to keep in memory
  maxPages?: number;

  // Custom function to merge pages
  mergeData?: (prevPages: T[], newPage: T) => T[];

  // Extract unique key from items for deduplication (NEW)
  // Only applies when T is an array type
  uniqueBy?: T extends readonly (infer Item)[]
    ? (item: Item) => string | number
    : never;

  // Called when fetcher throws
  onError?: (error: Error) => void;

  // All createResource options
  initialValue?: T;
  name?: string;
  deferStream?: boolean;
  storage?: () => Signal<T | undefined>;
} & ResourceOptions<T>

Fetcher Context

type FetcherContext<P> = {
  // Set the next page key
  setPageKey: Setter<P>;

  // Check if at end
  hasReachedEnd: Accessor<boolean>;

  // Mark as complete
  setHasReachedEnd: Setter<boolean>;

  // AbortSignal for cancelling requests (NEW)
  signal: AbortSignal;

  // Current pagination direction (NEW)
  direction: 'forward' | 'backward';
}

Return Value

type InfiniteResourceReturn<T, P> = {
  // Merged data from all pages
  // If T is an array type, flattens by default
  data: Accessor<T extends Array<infer U> ? U[] : T[]>;

  // Raw page responses
  allData: Accessor<T[]>;

  // Current page resource
  pageData: Resource<T>;

  // Trigger next page load (NEW: now accepts direction)
  getNextPage: (direction?: 'forward' | 'backward') => void;

  // Convenience for backward pagination (NEW)
  getPreviousPage: () => void;

  // Refetch current page without duplicating (NEW)
  refetchCurrentPage: () => void;

  // Get/set page key
  pageKey: Accessor<P>;
  setPageKey: Setter<P>;

  // End of data tracking
  hasReachedEnd: Accessor<boolean>;
  setHasReachedEnd: Setter<boolean>;

  // Intersection observer directive (NEW: now reactive)
  refetchOnView: Directive<RefetchDirectiveArgs>;

  // Underlying resource
  resource: ResourceReturn<T>;
}

// Directive arguments
type RefetchDirectiveArgs = [
  boolean | (() => boolean), // Condition
  () => void                 // Callback
] | (() => [
  boolean | (() => boolean),
  () => void
])

Methods

getNextPage

Triggers the next page load using the current page key.

const { getNextPage } = createInfiniteResource(fetcher);
getNextPage(); // Loads next page if !hasReachedEnd

refetchOnView

Directive for viewport-based loading.

// Attach to element
<div use:refetchOnView={[condition, callback]} />

// With reactive condition
<div use:refetchOnView={[
  () => canLoadMore(),
  () => loadMore()
]} />

Properties

data

Returns merged data from all pages. By default, flattens arrays:

// With array responses
type T = Product[]
const { data } = createInfiniteResource<T>();
data(); // Product[] (flattened from all pages)

// With custom merging
const { data } = createInfiniteResource<T>({
  mergeData: (prev, next) => [...prev, next]
});
data(); // Product[][] (array of pages)

pageData

Resource for current page with loading states:

const { pageData } = createInfiniteResource();
pageData.loading;  // Current page loading
pageData.error;    // Current page error
pageData();        // Current page data

hasReachedEnd

Tracks if all data has been loaded:

const { hasReachedEnd } = createInfiniteResource();
hasReachedEnd(); // boolean

// Common pattern
<Show 
  when={!hasReachedEnd()} 
  fallback="No more items"
>
  <LoadMoreButton />
</Show>

Resource Access

The underlying resource is exposed for advanced cases:

const { resource } = createInfiniteResource(fetcher);
const [data, { refetch }] = resource;

// Manual refetch with context
refetch({
  setNextPageKey,
  hasReachedEnd,
  setHasReachedEnd
});