Indexing a Field in CouchDB with a MapReduce View

Useful answer

CouchDB’s built‑in MapReduce views let you define a JavaScript map function that emits key/value pairs from documents. Once the view is built, you can look up documents by the emitted key without scanning the whole database. This is the standard way to create secondary indexes in CouchDB.

How MapReduce views work

A design document (_design/<name>) contains one or more views. Each view has a map function that CouchDB runs against every document in the database. The function may call emit(key, value) zero or more times. The emitted pairs are stored in a B‑tree index that is updated incrementally as documents change. Queries against the view read this index, which is far faster than a full database scan.

Worked example: indexing the email field

Suppose you have a database users with documents that include a type field set to 'user' and an email field. You want to find a user by email address.

1. Create the design document

Run the following curl command on a machine with network access to the CouchDB instance. You need permission to write to the _design endpoint (typically admin or a user with the _designer role).

curl -X PUT http://localhost:5984/users/_design/by_email \
  -H "Content-Type: application/json" \
  -d '{
    "views": {
      "by_email": {
        "map": "function(doc) {\n          if (doc.type === \"user\" && doc.email) {\n            emit(doc.email, null);\n          }\n        }"
      }
    }
  }'

This creates (or updates) the design document _design/by_email with a single view named by_email. The map function emits the email as the key and null as the value because we only need the key for lookups.

2. Indexing behavior

After the design document is stored, CouchDB starts building the view index. You can monitor progress with:

curl http://localhost:5984/users/_active_tasks

Look for a task with "type":"indexer" and "design_doc":"_design/by_email". When the task disappears or shows "progress":100, the index is ready.

3. Query the view

To retrieve the user document for [contact removed]:

curl "http://localhost:5984/users/_design/by_email/_view/by_email?key=\"[contact removed]\"&include_docs=true"

The response will be a JSON object with a rows array. Each row contains the id, key, value (null), and, because we used include_docs=true, the full document under doc.

Limits and performance considerations

• Initial build time: On a large database the first index creation can take minutes or hours because CouchDB must process every document. Subsequent updates are incremental and much faster.
• Disk usage: The view index stores one entry per emitted key. If you emit large values (e.g., whole documents) the index size grows quickly, increasing both storage and query latency.
• Update frequency: Each write to a document triggers an incremental view update for all views that include that document. Many views or frequent writes can increase write latency.
• Map function purity: The map function must be pure and side‑effect free. Side effects (e.g., database writes) can lead to inconsistent index states during concurrent updates.

As a rule of thumb, keep emitted values small (often null or a short scalar) and limit the number of views per design document to what your query patterns truly require.

Common mistakes and how to avoid them

1. Forgetting to guard against missing fields

If a document lacks the email field, accessing doc.email returns undefined. Emitting undefined as a key is allowed but usually not what you want. Wrap the emit in a condition:

if (doc.type === \"user\" && doc.email) { emit(doc.email, null); }

2. Emitting non‑JSON‑serializable values

The map function can only emit values that JSON can represent (strings, numbers, booleans, null, objects, arrays). Emitting a function, Date object, or circular structure will cause the view to fail compilation, and you will see an error in _active_tasks.

3. Updating the map function without changing the design document revision

CouchDB uses the design document’s _rev to detect changes. If you edit the map function but forget to save the design document (i.e., you send a PUT with the same content), the view will not rebuild. Always ensure the PUT request includes the updated function; the response will contain a new _rev.

4. Using complex keys that hurt performance

Keys are used for sorting and range queries. Very long strings or large arrays increase the size of the B‑tree nodes, slowing both index building and lookups. Keep keys as short as reasonably possible.

Verification steps

1. Create a test database (e.g., curl -X PUT http://localhost:5984/testdb).
2. Insert a few sample documents with the field you plan to index:

curl -X POST http://localhost:5984/testdb \
  -H "Content-Type: application/json" \
  -d '{\"type\":\"user\",\"email\":\"[contact removed]\",\"name\":\"Alice\"}'

3. Add the design document as shown in the worked example.
4. Query the view with a known key and verify:
• HTTP status code is 200.
• The JSON payload contains "total_rows":1 (or the expected count).
• The rows array holds an object with "key":"[contact removed]" and a doc matching the inserted document.
5. Check _active_tasks to confirm the indexer task has completed without errors.

If any of these checks fail, examine the CouchDB log (couch.log) for view‑related error messages.

Summary

MapReduce views are the primary mechanism for creating efficient secondary indexes in CouchDB. By defining a pure map function that emits the field you want to query, you gain fast lookups without full‑database scans. Keep the emitted values small, guard against missing fields, and monitor index build progress to ensure healthy performance.