API Integration Patterns in n8n

Posted on: March 3, 2024 | 9 min read

At the heart of most n8n workflows is the HTTP Request node, the gateway to interacting with thousands of APIs across the web. While making a simple GET request is straightforward, building robust, production-ready integrations requires a more structured approach. Different APIs have different rules, authentication methods, and ways of handling data.

This guide explores common patterns and strategies for integrating with various APIs using n8n, covering everything from authentication and pagination to robust error handling and retry mechanisms.

1. Authentication: The Keys to the Kingdom

Before you can request data, you need to prove you have permission. n8n's built-in credential management is the most secure way to handle this.

  • Header Auth (API Keys): This is the most common method. A service provides you with a secret key (e.g., `sk_live_...`). In the HTTP Request node, go to Authentication > Header Auth, enter `Authorization` as the Name, and `Bearer {{ $credentials.myApi.apiKey }}` as the Value.
  • OAuth2: For services like Google, Salesforce, or HubSpot, OAuth2 is the standard. n8n has dedicated credential types for many popular OAuth2 APIs. When you create a new credential, n8n will guide you through the authorization flow, securely storing the access and refresh tokens. The corresponding application nodes (e.g., HubSpot node) will then use these credentials automatically.
  • Other Methods: For less common methods like HMAC or custom signature-based authentication, you may need to use a "Code" node before your HTTP Request node to generate the required headers or parameters.

2. Handling Pagination: Getting All the Data

Most APIs don't return all their data in a single request. Instead, they "paginate" the results, giving you one "page" of data at a time and a way to ask for the next one. There are two primary patterns for this:

Offset/Page-Based Pagination:

The API response includes the total number of items and the current page. You loop until you've retrieved all pages.

  • Setup: Use a "Code" node to initialize a counter (e.g., `page = 1`).
  • Loop: Use a "Loop Over Items" node. Inside the loop, make your HTTP Request with the current `page` as a query parameter (e.g., `?page={{ page }}`).
  • Increment: After the request, use another "Code" node to increment the page counter.
  • Condition: The loop continues as long as the number of items received is greater than zero or until the current page exceeds the total number of pages.

Cursor-Based Pagination:

The API response includes a `next_page_cursor` or `next_page_url`. You use this value in your next request to get the subsequent set of results.

  • Setup: Your initial HTTP Request gets the first page.
  • Loop: Use a "Loop Over Items" node that is set to continue as long as the `next_page_cursor` from the previous step exists.
  • Next Request: Inside the loop, the HTTP Request node's URL or parameter will reference the cursor from the previous node's output.

3. Rate Limiting and Retries: Playing Nice with the API

Most APIs enforce rate limits—a maximum number of requests you can make in a given time period. If you exceed this, you'll get a `429 Too Many Requests` error. A robust workflow anticipates this.

  • Built-in Retries: In the HTTP Request node settings, you can enable "Retry on Fail." This is a simple way to handle transient network errors or brief rate limit issues.
  • Manual Backoff: For more control, you can build a custom retry loop.
    1. Set "Continue on Fail" in your HTTP Request node.
    2. Connect the failure output to a "Wait" node.
    3. After the wait, loop back to the HTTP Request node. Use a counter in a "Code" node to limit the number of retries to prevent an infinite loop.
  • Respect Headers: Sophisticated APIs often return headers like `X-RateLimit-Remaining` (how many requests you have left) and `X-RateLimit-Reset` (when the limit window resets). You can use "IF" nodes to check these values and proactively pause your workflow with a "Wait" node if you're close to the limit.

4. Data Transformation: Shaping the Output

The data an API returns is rarely in the exact format you need. The "Set" and "Code" nodes are essential for cleaning and structuring it.

  • Use the "Set" Node: For simple tasks like renaming keys, selecting a few fields, or setting static values, the "Set" node is perfect. Its visual interface makes it easy to understand what's happening to the data.
  • Use the "Code" Node: For more complex transformations—like restructuring nested objects, performing calculations, or formatting dates—the "Code" node provides the full power of JavaScript. A common pattern is to loop through an array of items and build a new, cleanly formatted array to pass to the next step.

Conclusion

Mastering these API integration patterns will elevate your n8n workflows from simple scripts to reliable, production-grade automations. By handling authentication securely, managing pagination correctly, respecting rate limits, and transforming data effectively, you can build integrations that are both powerful and resilient.