Scanning, filters, and advanced writes

Documentation home · Runnable recipes · API reference

Examples on this page assume an open table with a cf family. They are contextual snippets, not a complete script. Use the cookbook for independent examples with setup, data, and assertions.

Design the scan boundary first

Row keys are ordered as bytes. The constructor Scan(start_row=b"", end_row=b"") includes both endpoints by default; empty bounds are unbounded. Make a half-open interval explicit:

from hbasedriver.operations.scan import Scan

scan = (
    Scan(start_row=b"user:001")
    .with_end_row(b"user:100", inclusive=False)
    .add_family(b"cf")
)
with table.scan(scan) as scanner:
    first_page = scanner.next_batch(20)
for row in first_page:
    print(row.rowkey, row.get(b"cf", b"name"))

next_batch(20) obtains up to 20 rows from that scanner. Exiting the context closes it; the example does not continue the entire scan.

For long reads, consume bounded chunks:

from hbasedriver.operations.scan import Scan

with table.scan(Scan().add_family(b"cf")) as scanner:
    while True:
        rows = scanner.next_batch(100)
        if not rows:
            break
        for row in rows:
            print(row.rowkey)  # Replace with your bounded processing step.

Avoid list(scanner) on an unbounded table. Iteration currently requests one row through next_batch(1); explicit chunks let you control application memory and request size. Neither Scan.set_limit() nor PageFilter should be treated as a reliable global cap in this driver.

Resume after the last key

table.scan_page(scan, page_size) opens/closes a scanner for one page and returns (rows, last_returned_key). Continue with:

from hbasedriver.operations.scan import Scan

# resume_key was returned by a previous nonempty page.
scan = (
    Scan()
    .with_start_row(resume_key, inclusive=False)
    .with_end_row(b"user:100", inclusive=False)
    .add_family(b"cf")
)
rows, next_key = table.scan_page(scan, page_size=20)

Do not pass include_start_row=False to the Scan constructor: that keyword is not accepted. Reapply the original filter and projection on every page. A nonempty page always has a resume key, even if no more rows remain; stop on empty rows. The pagination recipe demonstrates the complete loop.

Pagination across changing data is not snapshot-consistent. Re-creating a scanner per page trades scanner lifetime for additional setup overhead. Reverse scans, wide/partial rows, and split/move recovery need separate integration tests for your topology.

Choose the correct filter

Goal	Filter	Important distinction
Row-key prefix	PrefixFilter(b"user:")	Also use start/end keys to constrain the search
Select a qualifier	QualifierFilter(CompareOperator.EQUAL, BinaryComparator(b"name"))	Filters cells, not whole-row business predicates
Qualifier prefix	ColumnPrefixFilter(b"metric:")	Matches qualifier names
Row must have a matching column	SingleColumnValueFilter(...)	Set missing-column behavior explicitly
Combine conditions	FilterList(operator=..., filters=[...])	Nested enum: FilterList.Operator
Return row keys with reduced cells	FirstKeyOnlyFilter() / KeyOnlyFilter()	Inspect transformed result shape before decoding values

Current Scan.add_column() has a protobuf serialization bug for bytes qualifiers. To select a single qualifier in a family, use this alternative:

from hbasedriver.filter import BinaryComparator, CompareOperator, QualifierFilter
from hbasedriver.operations.scan import Scan

scan = (
    Scan(start_row=b"user:")
    .with_end_row(b"user;", inclusive=False)
    .add_family(b"cf")
    .set_filter(QualifierFilter(CompareOperator.EQUAL, BinaryComparator(b"name")))
)
with table.scan(scan) as scanner:
    selected = scanner.next_batch(20)

For whole-row predicates, use SingleColumnValueFilter and include the tested family in the Scan. Its filter_if_missing and latest_version_only options are constructor arguments, not setter methods. The filter recipe shows a working AND combination.

Filter comparisons are byte comparisons unless you deliberately choose a different comparator/encoding. Text b"100" does not sort like integer 100 relative to b"20". See data model.

Time ranges, versions, and scan settings

Do not use Scan.set_time_range() to enforce a query window: its fields are not serialized. Get's finite time range has a separate protobuf-assignment failure. Also, Row does not preserve multiple versions or cell timestamps.

For time-oriented queries, plan an application row-key scheme and test its byte ordering; do not claim a client-side timestamp filter over Row when timestamps are unavailable.

Java-style setters such as setCaching, setBatch, setMaxResultSize, setReversed, and setAllowPartialResults exist. They do not imply full Java Client behavior. In particular:

• caching is not the number of rows in the final result;
• batch refers to cell/row-fragment handling, not BatchPut;
• partial and reverse scans need workload-specific validation;
• a method that accepts a setting may not carry it through every execution path.

Pick the correct write abstraction

Requirement	API	Boundary
Write one row	Put	Bytes supplied by the application
Submit several rows conveniently	BatchPut / table.batch()	Sequential calls; partial success possible
Defer writes in memory	BufferedMutator	Not durable, no combined RPC guarantee
Atomically increase one counter	Increment	Signed 8-byte numeric storage, retries can double-count
Compare then write	CheckAndPut	Same-row atomic condition, not a full lock service
Append bytes	Append	Replays duplicate the suffix
Put/Delete together on one row	RowMutations	Not a cross-row transaction

BufferedMutatorParams() takes no keyword arguments. Assign write_buffer_size (bytes), write_buffer_periodic_flush (milliseconds), or max_key_value_size after construction. Use BufferedMutator(table, params) with a caller-owned Table and explicit flush() at an application checkpoint.

A successful flush means that invocation returned without error; it does not make the client buffer a recovery log. A failed flush can remove unsent work from the buffer. Preserve input/checkpoints and reconcile server state before retrying. See configuration, limitations, and the buffering recipe.

Lifecycle and concurrency

Use nested Client → Table → Scanner/Mutator contexts, closing inner resources first. This is the intended usage pattern, not a guarantee that every low-level socket is reclaimed. Table handles share some Client internals, and closing a Table can affect shared state.

Do not assume thread-safe sharing, full automatic reconnection, or correct cache isolation for arbitrary multi-table scans. Use conservative bounded sessions, test your concurrency pattern, and monitor sockets and memory. See configuration for details.