Compression property fails when moving Phoenix 4.x from HBase 1.x to 2.x
26.5K reputation · 17 Dec 2024, 15:12 UTC
Compression Property Compatibility
The goal is to maintain the same column‑family compression settings when a Phoenix 4.x deployment is shifted from a local HBase 1.x cluster to a production HBase 2.x cluster.
In the local environment, the CREATE TABLE … WITH COLUMN FAMILIES … COMPRESSION=<type> clause is accepted and the table functions normally. In the production environment, the same clause results in table creation errors or runtime failures, with Phoenix logs indicating unsupported or invalid compression properties. The Phoenix 4.x documentation explicitly limits compression support to HBase 1.x, and no migration path is provided for HBase 2.x.
Unresolved decisions remain regarding whether future Phoenix releases will reintroduce compression support for HBase 2.x or prescribe an alternative approach.
Unresolved Questions
- Will Phoenix 5.x re‑enable compression support for HBase 2.x?
- Is there an officially recommended workflow for applying compression to Phoenix tables on HBase 2.x?
- What is the impact on performance and compatibility when setting compression at the HBase shell level after Phoenix table creation?
1 answer
1 question comment
Use comments to ask for clarification. Post a solution as an answer.
26,525 reputation · 17 Dec 2024, 17:18 UTC
Compression can be applied post‑creation
When Phoenix 4.x runs against HBase 2.x, the DDL clause COMPRESSION=... may be rejected before the property reaches HBase. However, once the table exists (even if created without compression), you can still set or change the column‑family compression using the HBase shell:
alter 'MY_TABLE', {NAME => 'cf1', COMPRESSION => 'SNAPPY'}This bypasses the Phoenix coprocessor limitation because the change is made directly in HBase metadata. After altering, new writes will use the selected codec, while existing store files retain their original encoding until a major compaction rewrites them. Verify the change with:
hbase> describe 'MY_TABLE'
and check the output for COMPRESSION => 'SNAPPY' (or LZ4/ZSTD as appropriate). This approach lets you keep Phoenix 4.x operational on HBase 2.x while still achieving the desired compression settings.