Preventing Memory Exhaustion: Implementing Pagination and Sorting in JHipster

The Danger of the Full List

When building a new entity in JHipster, the default generated code handles a few dozen records perfectly. However, as your production database grows to thousands or millions of rows, a simple findAll() call becomes a liability. Loading an entire Hibernate entity collection into the JVM can lead to OutOfMemoryError crashes or severe garbage collection pauses that freeze your application.

The solution is to move the data slicing logic from the application layer to the database layer using Spring Data JPA's Pageable and Sort interfaces. This ensures the database only returns the specific subset of records needed for the current view.

How JHipster Handles Data Slicing

JHipster integrates PagingAndSortingRepository (via JpaRepository) to handle the heavy lifting. Instead of returning a List<T>, the backend controllers are designed to return a Page<T> object. This object contains not only the slice of data but also metadata—such as total elements and total pages—which the frontend needs to render pagination controls.

The process follows a specific flow: the frontend sends query parameters (page, size, and sort), the Spring Boot controller maps these to a Pageable object, and Hibernate translates this into a SQL LIMIT and OFFSET clause.

Worked Example: Customizing a Paginated Endpoint

Suppose you have a Product entity and you want to ensure that the API supports dynamic sorting by price or name while maintaining pagination.

1. Repository Layer

Ensure your repository extends JpaRepository. This is the JHipster default, providing the necessary pagination methods.

// ProductRepository.java
public interface ProductRepository extends JpaRepository<Product, Long> {
    // No extra code needed for basic pagination
}

2. Service and Controller Layer

In your REST controller, accept a Pageable parameter. Spring Boot automatically resolves the ?page=X&size=Y&sort=Z query parameters into this object.

// ProductResource.java
@GetMapping("/products")
public ResponseEntity<List<ProductDTO>> getAllProducts(Pageable pageable) {
    Page<ProductDTO> page = productService.findAll(pageable);
    return ResponseEntity.ok().body(page.getContent());
}

3. Verification

Run this command from your terminal (replace localhost:8080 and the endpoint as needed) to verify the database is slicing the data:

# Request page 0, 20 items per page, sorted by price descending
curl "http://localhost:8080/api/products?page=0&size=20&sort=price,desc"

Expected Result: The JSON response should contain a content array with exactly 20 items and a totalElements field indicating the full count of the table.

Performance Trade-offs and Limitations

While pagination solves memory issues, it introduces two specific database risks:

• The Offset Overhead: In relational databases, OFFSET 10000 LIMIT 20 requires the database to scan through the first 10,000 rows before discarding them. This is known as "deep pagination." As page numbers increase, response times will degrade.
• The Sorting Trap: Sorting on a column that is not indexed (e.g., a description text field) forces the database to perform a full table scan and a manual sort in temporary disk space, which can spike CPU and I/O usage.

Practical Safeguards

To prevent Denial of Service (DoS) attacks where a user requests ?size=1000000, you should cap the maximum page size in your application.yml or via a custom interceptor. A common limit is 100 items per page.

To verify your implementation is performant, enable SQL logging in your development environment:

# application-dev.yml
spring:
  jpa:
    show-sql: true
    properties:
      hibernate:
        format_sql: true

Check the logs for LIMIT and OFFSET keywords. If you see a query without a LIMIT clause when calling a paginated endpoint, your Pageable object is not being passed correctly to the repository.