API throttle rate limits for low‑traffic NetBox instances
0 reputation · 24 Jun 2023, 23:04 UTC
0 reputation · 24 Jun 2023, 23:04 UTC
Goal: Reduce resource consumption and hosting costs for a NetBox deployment that serves minimal API traffic by lowering the API request throttle rate.
Constraint: The API_THROTTLE_RATE setting applies globally to all API endpoints, including the browsable UI and external automation scripts, and only takes effect after a service restart; it does not influence background workers such as RQ or scheduled reports.
Uncertainty: Determining a safe minimum throttle value that avoids frequent HTTP 429 responses for legitimate automation and UI interactions while still delivering measurable savings is not clearly defined in the documentation, especially for variable request patterns across different environments.
What is the lowest API_THROTTLE_RATE that typical automation scripts can sustain without encountering throttling errors?
How can administrators safely test the impact of a reduced throttle setting on UI responsiveness before applying it to production?
Does NetBox provide any mechanism to apply different throttle limits to specific API views or clients?
For a typical low‑traffic NetBox instance that sees under 50 requests per hour per user, a value of 200 requests per hour (200/hr) is a safe starting point. This rate keeps the API_THROTTLE_RATE far above the bursty traffic that automation scripts or the browsable UI may generate, while still reducing the per‑second load compared to the default “no throttling” state.
If your environment is even quieter (e.g., a single‑user lab installation), you can drop the limit to 100/hr or 50/hr without seeing 429 responses. The key is to monitor the Throttle warnings in the NetBox log after each change.
# Make a copy of the settings file
cp /opt/netbox/netbox/netbox/settings.py /opt/netbox/netbox/netbox/settings.py.bak
settings.py:
REST_FRAMEWORK = {
'DEFAULT_THROTTLE_RATES': {
'user': '200/hour',
'anon': '10/hour',
},
'DEFAULT_THROTTLE_CLASSES': [
'rest_framework.throttling.UserRateThrottle',
'rest_framework.throttling.AnonRateThrottle',
],
}
sudo systemctl restart netbox
sudo journalctl -u netbox -f | grep -i throttle
for i in {1..50}; do
curl -s -o /dev/null http://localhost/api/dcim/regions/
sleep 36
done
Throttle warnings. If none appear, the UI remains responsive and the throttle rate is safe.Out of the box, NetBox applies the global API_THROTTLE_RATE to all endpoints. However, because NetBox uses Django REST Framework (DRF), you can:
X-Client-Id) and set a rate accordingly.throttle_classes attribute in the view definition. This requires editing NetBox’s source or creating a plugin that overrides the viewset.limit_req_zone or mod_reqtimeout. The proxy must forward the X-Forwarded-For header so that DRF can distinguish users.Note that background workers (RQ, scheduled reports) bypass DRF throttling entirely, so any limits you set will not affect them.
To fine‑tune the throttle rate precisely, please provide the approximate number of API requests your automation scripts make per hour, and the typical frequency of UI page loads. This will help determine whether 200/hr is sufficient or if a lower value is safe.
Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 25 Jun 2023, 05:10 UTC
Spin up a duplicate NetBox instance (e.g., using Docker or a VM) and apply the new API_THROTTLE_RATE. While logged in, open the Network tab in your browser’s devtools, enable Throttle to Fast 3G, and repeatedly refresh the dashboard. If page loads remain <200 ms and no 429 entries appear in the NetBox log, the setting is likely safe for production.
NetBox itself does not expose per‑view limits in settings.py, but DRF allows you to override the throttle_classes attribute on any view or viewset:
from rest_framework.throttling import UserRateThrottle
class DeviceList(generics.ListAPIView):
queryset = Device.objects.all()
serializer_class = DeviceSerializer
throttle_classes = [UserRateThrottle]
throttle_rate = '50/hour' # custom rate for this endpoint
For external clients, you can create a custom AnonRateThrottle subclass that checks the request’s User-Agent or API key and applies a distinct limit.
RQ jobs and scheduled reports bypass DRF throttling entirely, so lowering API_THROTTLE_RATE will not interfere with them.