Query Builder limitations with SQLite in-memory integration testing
0 reputation · 07 Jan 2021, 05:00 UTC
0 reputation · 07 Jan 2021, 05:00 UTC
CodeIgniter 4 enables database-agnostic query construction through its Query Builder, allowing developers to run integration tests using an SQLite in-memory database via the DatabaseTrait. This approach facilitates testing CRUD logic without requiring production credentials or a live database server.
The challenge arises when the application relies on database-specific SQL features that are not universally supported across engines. While the Query Builder abstracts standard operations seamlessly, advanced features like JSONB operations or specific spatial functions may fail or behave differently when switching from a MySQL or PostgreSQL production environment to SQLite.
How does the Query Builder handle non-standard SQL syntax when the test environment is toggled to SQLite in-memory? Are there specific patterns within the framework to ensure compatibility for JSON-specific queries without rewriting the core query logic?
34275 reputation · 07 Jan 2021, 07:12 UTC
The Query Builder does not automatically translate database-specific JSON or spatial syntax when you switch the connection to SQLite in-memory. Core CRUD methods (select(), where(), orderBy()) generate portable SQL, but any call that maps to MySQL JSON_EXTRACT, PostgreSQL jsonb operators, or spatial functions will emit the original dialect's syntax. SQLite will reject or silently ignore that SQL unless the JSON1 extension is loaded and you execute the query through $db->query() with SQLite-compatible functions like json_extract().
select(), where(), join(), groupBy(), limit() produce standard SQL that runs on SQLite without changes.->select('json_extract(data, "$.key")') or raw expressions using ->where('JSON_EXTRACT(col, "$.path")', 'value') are passed through to the driver. On MySQL/PostgreSQL they work; on SQLite they fail unless JSON1 is present and the function name matches SQLite's json_extract.LIKE or string comparison may return rows but produce incorrect results (e.g., matching partial strings instead of structured paths).Wrap JSON access behind a driver-aware helper so the same repository method works in production and test environments.
// app/Helpers/JsonQueryHelper.php
if (! function_exists('json_extract_expr')) {
function json_extract_expr(\CodeIgniter\Database\BaseConnection $db, string $column, string $path): string
{
$driver = $db->DBDriver;
if ($driver === 'MySQLi') {
return "JSON_EXTRACT({$column}, '{$path}')";
}
if ($driver === 'Postgre') {
return "{$column}->>'{$path}'";
}
// SQLite with JSON1
return "json_extract({$column}, '{$path}')";
}
}
// Usage in a model or repository
$expr = json_extract_expr($this->db, 'data', '$.user.id');
$builder->select($expr . ' as user_id')->where($expr, $userId);
For tests, ensure the SQLite connection loads JSON1. In phpunit.xml or your bootstrap:
<php>
<env name="DB_DATABASE" value=":memory:" />
<env name="DB_DBDriver" value="SQLite3" />
</php>
Then in your test base, after $this->db = \Config\Database::connect():
$this->db->query('PRAGMA compile_options;'); // verify JSON1 appears
// or explicitly load if built as extension
// $this->db->query('SELECT load_extension("json1");');
JSON_EXTRACT or ->> appears unless you injected it via the helper.$db->query('SELECT json_extract(col, "$.key") FROM test_table'). If it returns a result, JSON1 is active.SELECT sqlite_version(); and look for ENABLE_JSON1 in PRAGMA compile_options;.What is the exact SQLite version and compile-time configuration in your CI/test runner? Run php -r "$db=new SQLite3(':memory:'); print_r($db->querySingle('PRAGMA compile_options;'));"; and share the output. If ENABLE_JSON1 is missing, the helper must fall back to LIKE pattern matching or you must compile SQLite with JSON1 enabled.
Use comments to ask for clarification. Post a solution as an answer.
5,610 reputation · 07 Jan 2021, 09:31 UTC
A critical detail regarding the use of SQLite for integration testing is that the JSON1 extension is not enabled by default in many standard PHP/PDO SQLite builds. Even if you use the correct json_extract() syntax, the query will fail if the underlying SQLite binary lacks this extension.
To verify if your test environment supports these operations, you can run a diagnostic query before your test suite executes:
$db->query("SELECT json_extract('{\"a\":1}', '$.a');");
If this throws an error, the Query Builder cannot resolve the issue because the limitation is at the database engine level, not the framework level. In such cases, you may need to ensure your testing environment uses a SQLite build with JSON1 enabled or avoid JSON-specific logic in the DatabaseTrait tests by mocking the repository layer for those specific complex queries.