Provisioning Grafana Dashboards and Data Sources with YAML for Repeatable Deployments

The problem: UI setup does not survive environment rebuilds

Recreating Grafana dashboards by clicking through the web interface works once. It fails when you need the same dashboards in staging, production, and disaster-recovery instances, or when a new team member must rebuild a monitoring stack from scratch. Manual setup is not version-controlled, hard to review, and easy to drift out of sync.

Grafana provisioning solves this by letting you define data sources, dashboards, and alert rules as YAML configuration files on disk. Grafana reads these files at startup and creates the resources automatically. The useful takeaway: treat Grafana configuration the same way you treat application code—store it in Git, review changes through pull requests, and deploy it with your infrastructure.

Prerequisites

• A running Grafana instance with administrator access to the web UI.
• Write access to the Grafana server filesystem (or the container volume if you run Grafana in Docker or Kubernetes).
• A source control repository to hold provisioning files and dashboard JSON models.
• Basic familiarity with YAML syntax and JSON.

Provisioning directory locations and supported fields vary across Grafana versions and installation methods. Before you begin, confirm the exact paths for your release. On Linux package installations, provisioning files commonly live under /etc/grafana/provisioning. In Docker images, they are often mounted at /etc/grafana/provisioning as well, but the exact path depends on your image tag and configuration. Always check your current grafana.ini or environment variables for the [paths] section.

Step 1: Confirm the provisioning paths

Grafana looks for provisioning files in directories defined in its main configuration file. Open your grafana.ini (or the mounted configuration) and locate the [paths] block. The relevant settings typically include:

[paths]
provisioning = /etc/grafana/provisioning

If you run Grafana in a container, you can inspect the effective configuration by exec'ing into the container and reading /etc/grafana/grafana.ini, or by checking the environment variables passed to the container. If the directory does not exist, create it and ensure the Grafana process user can read it.

Command (Linux package):

sudo mkdir -p /etc/grafana/provisioning/dashboards
sudo mkdir -p /etc/grafana/provisioning/datasources
sudo mkdir -p /etc/grafana/provisioning/alerting
sudo chown -R grafana:grafana /etc/grafana/provisioning

Run these commands on the Grafana host with sudo privileges. Replace grafana:grafana with the user and group your Grafana process actually runs as. If permissions are wrong, Grafana may silently skip the files.

Step 2: Provision a data source

A data source is a configured connection to a backend system such as Prometheus, Loki, InfluxDB, or a SQL database. Create a YAML file inside the datasources directory. The following example defines a Prometheus data source and reads its password from an environment variable rather than storing a plaintext secret in version control.

# /etc/grafana/provisioning/datasources/prometheus.yaml
apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true
    jsonData:
      httpMethod: POST
    secureJsonData:
      password: ${PROMETHEUS_PASSWORD}

Grafana expands environment variables in provisioning files. Set PROMETHEUS_PASSWORD in the Grafana service environment before restarting. If your Prometheus instance requires no authentication, omit the secureJsonData block entirely.

Risk: Committing credentials to version control is a common mistake. Use environment variable expansion or an external secrets manager, and add a pre-commit check to prevent accidental secret leaks.

Step 3: Provision dashboards with a provider

Grafana does not scan arbitrary folders for dashboards. You must create a dashboard provider YAML file that points to a directory containing dashboard JSON models. The provider tells Grafana which folder to scan and whether to allow edits from the UI.

# /etc/grafana/provisioning/dashboards/provider.yaml
apiVersion: 1

providers:
  - name: 'infrastructure-dashboards'
    orgId: 1
    folder: 'Infrastructure'
    type: file
    disableDeletion: false
    editable: false
    options:
      path: /etc/grafana/provisioning/dashboards/json
      foldersFromFilesStructure: true

Place your exported dashboard JSON files in the directory referenced by path. The editable: false setting prevents accidental UI changes, but remember that provisioned dashboards are overwritten on every restart regardless of this flag. Export any UI changes back to JSON and commit them.

Version caution: Dashboard JSON models must match the schema expected by your target Grafana version. Dashboards exported from a newer Grafana may fail to render or may require migration after an upgrade. Test dashboard JSON in a staging instance before promoting it to production.

Step 4: Restart Grafana

Provisioning happens at startup. After adding or changing files, restart the Grafana service.

Linux package (systemd):

sudo systemctl restart grafana-server

Docker:

docker restart <container_name_or_id>

Run the restart command on the Grafana host or container orchestrator with sufficient privileges. Grafana reads the provisioning directory, validates the YAML, and creates or updates the resources. If a file contains invalid YAML or references a missing path, Grafana logs the error and skips that specific resource while continuing to run.

Expected checks

After the restart, verify the result before moving to production.

1. UI check: Log in to Grafana. Navigate to Configuration > Data sources. Confirm your provisioned data source appears without a red error indicator. Open the dashboard list and confirm the provisioned dashboards render correctly.
2. Log check: Inspect the Grafana server log for provisioning failures. On Linux package installations, the log is commonly at /var/log/grafana/grafana.log. Search for lines containing provisioning or error. A clean startup with no provisioning errors is the expected result.
3. Deliberate failure test (staging only): Introduce a syntax error into a copy of a provisioning file in a staging instance, restart Grafana, and confirm that the error appears in the log while the rest of Grafana remains available. This validates that you can detect failures in production.

Command to inspect logs (Linux package):

sudo grep -i "provisioning\|error" /var/log/grafana/grafana.log

Adjust the log path if your installation uses a different location. You need read access to the log file.

Recovery options

Provisioning changes state on disk and in the Grafana database. If something goes wrong, use these recovery paths.

• Invalid YAML or missing path: Grafana skips the resource and continues running. Correct the syntax or path in version control, redeploy the file, and restart Grafana. No database cleanup is usually required for skipped files.
• Accidental overwrite of a UI-edited dashboard: Provisioned dashboards are replaced on restart. If you made important changes in the UI, export the dashboard JSON before the next restart and commit it to the provisioning repository. Without an export, the UI changes are lost.
• Permission failures: If resources silently fail to load, verify that the Grafana process user can read the provisioning files and dashboard directories. Use ls -l and namei -l to trace ownership along the path.
• Full rollback: Because provisioning files live in version control, roll back by reverting the commit, redeploying the previous files, and restarting Grafana. If you must return to a purely manual state, remove the provisioning files and restart; Grafana will stop managing those resources, but existing database entries may remain until manually deleted.

Limitations to plan for

Provisioning is powerful but not universal. Do not use it for ad-hoc, one-off dashboards that should never be replicated across environments. Reserve it for resources that must be identical everywhere.

Provisioned resources are declarative: the file is the source of truth. Manual UI edits to provisioned dashboards or data sources are lost on restart unless exported back to files. This is by design, but it requires discipline from the team.

Finally, the provisioning schema and directory layout have evolved across Grafana major versions. A configuration that works on one version may use deprecated fields on another. Pin your Grafana version in infrastructure code, and validate provisioning changes in a staging environment that matches production before deploying.