Configuration and connection lifecycle

Documentation index · Troubleshooting

Minimal configuration

import os

from hbasedriver.hbase_constants import HConstants

config = {
    HConstants.ZOOKEEPER_QUORUM: os.getenv("HBASE_ZK", "127.0.0.1:2181"),
    HConstants.CONNECTION_POOL_SIZE: 10,
    HConstants.CONNECTION_IDLE_TIMEOUT: 300,
}

The application reads HBASE_ZK and passes it into Client(config); it is not an implicit driver setting. Use explicit host:port addresses with no whitespace around commas.

Settings consumed by the current implementation

Key	Default	Meaning
hbase.zookeeper.quorum	Required	Comma-separated ZooKeeper contact addresses
hbase.connection.pool.size	10	RegionServer pool size setting for each Table
hbase.connection.idle.timeout	300 seconds	RegionServer pool idle timeout setting
hbase.meta.connection.pool.size	5	Cluster metadata pool size setting
hbase.meta.connection.idle.timeout	300 seconds	Cluster metadata pool idle timeout setting

Pool size is not a promise of that many concurrent requests or a strict application-wide socket cap. The code contains pool and shared-cache limitations; do not infer production thread safety from the presence of locks. Benchmark your actual workload.

Settings not supported end-to-end

The presence of a name in HConstants is not proof that Client honors it. In particular:

• The high-level startup lookup uses /hbase/master and /hbase/meta-region-server. A custom zookeeper.znode.parent is not consistently applied.
• Do not rely on hbase.zookeeper.property.clientPort to alter every connection; put ports in quorum addresses.
• Java timeout, retry, authentication, TLS, and Kerberos settings are not interchangeable with this Python client's configuration.
• A quorum string is accepted, but the current startup loop does not guarantee failover when its first contact times out.

Keep credentials out of source files, diagnostic output, README examples, and Git. This client uses simple RPC authentication; adding a password to this dictionary does not enable secure-cluster support.

Docker addressing

Within hbase_dev, use hbase-zk:2181, not localhost:2181. The latter means the development container itself. The cluster advertises hbase-master and hbase-rs1/2/3; those names are resolvable on the Compose network. Host port mappings alone do not rewrite advertised names or RegionServer ports.

Run from this repository's root:

docker compose ps
docker compose exec -T dev python -c "from hbasedriver.client.client import Client; print('import OK')"

The import check does not connect to HBase. For an end-to-end check, copy and run the complete example.

Lifecycle and ownership

Reuse a Client for a bounded unit of work. Keep only one active Table handle at a time per Client unless your workload has independently validated shared-cache behavior. Table closure also closes the Client's attached cluster connection, so do not close one handle while another is still using it.

from hbasedriver.client.client import Client
from hbasedriver.operations.get import Get

with Client(config) as client:
    with client.get_table(b"default", b"mytable") as table:
        row = table.get(Get(b"row1"))

Close scanners promptly, including after an early break. Context-manager exits invoke cleanup methods, but are not a guarantee that all underlying sockets are currently released; the implementation has cleanup gaps. For long-running services, monitor file descriptors and connection counts rather than relying on garbage collection.

For buffered writes, explicitly owning the Table is clearer than hiding it inside the factory:

from hbasedriver.client.buffered_mutator import BufferedMutator, BufferedMutatorParams
from hbasedriver.operations.put import Put

params = BufferedMutatorParams()
params.write_buffer_size = 2 * 1024 * 1024
params.write_buffer_periodic_flush = 0  # Disable periodic flushing; milliseconds otherwise.
with Client(config) as client:
    with client.get_table(b"default", b"mytable") as table:
        with BufferedMutator(table, params) as mutator:
            mutator.mutate(Put(b"row1").add_column(b"cf", b"q", b"value"))

BufferedMutatorParams() takes no constructor arguments. A mutator flushes on normal close, but is not a transaction or durable queue. Do not acknowledge ingestion solely because mutate() returned; establish a successful flush boundary and retain source records for reconciliation.