Efficiently Displaying 10,000+ Items in Android with Paging 3: A Practical Guide

Problem: Long Lists Crash or Lag

When an app shows thousands of items—say a news feed or a product catalog—loading them all at once quickly exhausts memory. A naïve RecyclerView that requests the entire dataset from a server or database will stall the UI, cause OutOfMemory errors, and frustrate users.

Thesis: Paging 3 Automates Chunking, Caching, and Diffing

AndroidX Paging 3 solves this by fetching data in small pages, keeping only a sliding window in memory, and automatically calculating diffs when the underlying data changes. The result is smooth scrolling, low memory usage, and built‑in retry logic for network hiccups.

1. Understanding Paging 3’s Core Concepts

Paging 3 revolves around three key abstractions:

• PagingSource: Implements load(params) to fetch a page of data given a key.
• PagingData: A Flow that emits pages as they arrive.
• PagingDataAdapter: A RecyclerView.Adapter that consumes PagingData and handles diffing.

Because it uses Kotlin coroutines and Flow, the library is reactive: UI updates automatically when new pages load.

2. Setting Up a PagingSource

Suppose you have a REST endpoint /items?page=1&size=50. Create a Retrofit service:

interface ApiService {
    @GET(\"items\")
    suspend fun getItems(@Query(\"page\") page: Int,
                        @Query(\"size\") size: Int): List<Item>
}

Then implement PagingSource:

class ItemPagingSource(private val api: ApiService) : PagingSource<Int, Item>() {
    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Item> {
        return try {
            val page = params.key ?: 1
            val data = api.getItems(page, params.loadSize)
            LoadResult.Page(
                data = data,
                prevKey = if (page == 1) null else page - 1,
                nextKey = if (data.isEmpty()) null else page + 1
            )
        } catch (e: IOException) {
            LoadResult.Error(e)
        }
    }
}

Key points:

• Use a stable key (page number). If the back‑end uses offsets, pass params.loadSize instead.
• Return prevKey and nextKey to let Paging know how to navigate.

3. Wiring PagingData to the UI

In a ViewModel expose a Flow of PagingData:

class ItemViewModel(private val api: ApiService) : ViewModel() {
    val items = Pager(
        config = PagingConfig(pageSize = 50, enablePlaceholders = false)
    ) {
        ItemPagingSource(api)
    }.flow.cachedIn(viewModelScope)
}

In the Fragment or Activity:

class ItemFragment : Fragment() {
    private val viewModel: ItemViewModel by viewModels()
    private val adapter = ItemAdapter()

    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        recyclerView.adapter = adapter
        lifecycleScope.launchWhenStarted {
            viewModel.items.collectLatest { pagingData ->
                adapter.submitData(pagingData)
            }
        }
    }
}

Define ItemAdapter extending PagingDataAdapter and supply a DiffUtil.ItemCallback:

class ItemAdapter : PagingDataAdapter<Item, ItemViewHolder>(DIFF_CALLBACK) {
    override fun onBindViewHolder(holder: ItemViewHolder, position: Int) {
        holder.bind(getItem(position))
    }
    companion object {
        private val DIFF_CALLBACK = object : DiffUtil.ItemCallback<Item>() {
            override fun areItemsTheSame(old: Item, new: Item) = old.id == new.id
            override fun areContentsTheSame(old: Item, new: Item) = old == new
        }
    }
}

4. Load States & Retry Logic

Paging 3 exposes a loadStateFlow on the adapter. Hook it to show a loading spinner or retry button:

adapter.loadStateFlow
    .onEach { loadState ->
        val isLoading = loadState.refresh is LoadState.Loading
        loadingView.isVisible = isLoading
        retryButton.isVisible = loadState.refresh is LoadState.Error
    }
    .launchIn(lifecycleScope)

retryButton.setOnClickListener { adapter.retry() }

This eliminates manual error handling; the library automatically retries when the user taps retry.

Trade‑offs & Limitations

• Overkill for Small Lists: If your dataset is < 100 items, the extra boilerplate isn’t worth the benefit.
• Kotlin Dependency: Paging 3 is Kotlin‑first. Java projects need to interop with Kotlin or migrate.
• Stable API Required: Changing paging keys or item ordering on the server can break pagination. Ensure the API contract is stable.
• Memory Footprint: While Paging keeps a small window in memory, the PagingDataAdapter still holds a DiffUtil cache. For extremely large data sets, monitor memory with Android Profiler.

Actionable Take‑aways

1. Start with a PagingSource that uses a stable key—page number or offset.
2. Expose PagingData from the ViewModel using Pager and cachedIn.
3. Use PagingDataAdapter with a proper DiffUtil.ItemCallback.
4. Subscribe to loadStateFlow to show progress and retry UI.
5. Profile memory: run the app with thousands of items, scroll, and confirm no OOM.
6. If your list is small or you’re in a Java‑only codebase, consider using a simple RecyclerView with manual pagination instead.

Paging 3 is a powerful tool for large, scroll‑heavy lists. With the right setup, you can deliver a smooth, responsive experience without blowing up device memory.