Reducing Network Latency in Microsoft 365 Integrations with Graph API Batching

The Problem: HTTP Overhead and Request Latency

When building integrations for Microsoft 365, developers often need to fetch multiple related resources—such as a user's profile, their manager, and their recent emails—simultaneously. Making these as individual HTTP requests introduces significant network overhead due to repeated TCP handshakes and TLS negotiations. In high-latency environments, this "chattiness" can make an application feel sluggish, even if the backend processing time is minimal.

The Takeaway: Use the /$batch endpoint to combine up to 20 individual Microsoft Graph requests into a single HTTP POST. This reduces the number of round-trips between your application and the Microsoft 365 servers, significantly lowering the total time to retrieve a set of related data.

How JSON Batching Works

Microsoft Graph batching allows you to wrap multiple requests into a single JSON payload. You send this payload to the /v1.0/$batch (or /beta/$batch) endpoint. The server processes these requests and returns a single response containing an array of results, each mapped to a unique ID you provided in the request.

It is important to note that batching is a transport optimization, not a performance bypass. The server still executes the underlying operations, and the total cost of the requests still counts toward your tenant's throttling limits.

Implementation Example: Fetching Multiple User Profiles

To implement batching, you must construct a JSON object containing a requests array. Each item in the array requires a unique id, an HTTP method, and a url (relative to the Graph root).

POST https://graph.microsoft.com/v1.0/$batch
Content-Type: application/json
Authorization: Bearer {token}

{
  "requests": [
    {
      "id": "1",
      "method": "GET",
      "url": "/users/[contact removed]"
    },
    {
      "id": "2",
      "method": "GET",
      "url": "/users/[contact removed]"
    },
    {
      "id": "3",
      "method": "GET",
      "url": "/users/[contact removed]"
    }
  ]
}

Expected Result: The server returns an HTTP 200 OK. The body contains a responses array. Even if one request fails (e.g., [contact removed] does not exist), the overall batch response is 200 OK, but the individual response object for ID "3" will contain a 404 Not Found status.

Operational Constraints and Limitations

While batching is powerful, it has strict limits that can cause 400 Bad Request errors if ignored:

• Maximum Batch Size: You cannot exceed 20 requests per batch. If you have 100 requests, you must split them into five separate batch calls.
• No Atomicity: Batching is not a database transaction. If the third request in a batch of ten fails, the first two remain committed and the subsequent seven continue to process.
• Throttling: Batching does not hide your activity from the throttling engine. If your batch contains 20 heavy queries, you may still receive a 429 Too Many Requests response for the entire batch or specific items within it.
• Sequentiality: While Graph generally processes requests in order, you should not rely on strict sequential execution for complex dependencies unless the API specifically supports dependent requests (which is limited in the standard /$batch endpoint).

Common Implementation Mistakes

Using Absolute URLs: A common error is providing the full URL (https://graph.microsoft.com/v1.0/users/...) inside the batch request. The url field must be a relative path starting from the root (e.g., /users/...).

Ignoring Individual Response Statuses: Developers often check only the top-level HTTP status of the /$batch call. Because the batch endpoint returns 200 OK as long as the JSON was well-formed, you must iterate through the responses array and check the status code of each individual item to ensure the data was actually retrieved.

Verification and Testing

To verify your batching implementation, use the Graph Explorer with the following test case:

1. Create a batch of 5 requests: 3 valid user GETs and 2 intentionally malformed URLs.
2. Execute the POST to /v1.0/$batch.
3. Confirm that the top-level response is 200 OK.
4. Verify that the responses array contains exactly 5 objects, with 3 successful responses and 2 error responses (e.g., 400 or 404).