starling.search.search_engine.SearchEngine

class SearchEngine[source]

Bases: object

FAISS-backed similarity search with rich post-filtering.

The SearchEngine wraps a trained faiss.Index and optional SequenceStore to provide high-level search primitives used across STARLING. In addition to ANN lookups, it supports length gating, identity thresholds, exact-match suppression, and optional reranking with the encoder.

Parameters:

  • index (faiss.Index) – Trained FAISS index (e.g., IVFPQ, HNSW) matching the embedding metric.

  • metric ({'cosine', 'l2'}, default 'cosine') – Similarity/distance metric used during index construction.

  • seq_store (SequenceStore, optional) – Sequence/metadata store generated alongside the index. Required for filters that need sequence lengths or raw sequences, and for reranking.

  • verbose (bool, default True) – When True the engine logs key operations (loading, filters, rerank).

Variables:

  • index (faiss.Index) – Underlying FAISS index used for coarse ANN search.

  • metric (str) – Metric string supplied during construction ("cosine" or "l2").

  • seq_store (SequenceStore or None) – Attached sequence store used for metadata-aware filtering.

  • total (int) – Number of vectors stored in the index.

  • dim (int) – Embedding dimensionality of the index.

Examples

>>> from starling.search import SearchEngine
>>> engine = SearchEngine.load("/data/index.faiss")
>>> hits = engine.search(queries, k=50, nprobe=128, rerank=True)
>>> len(hits[0])
50

Methods

__init__

load

Load a serialized FAISS index (and optional sequence store).

search

Approximate nearest neighbor search with optional filtering and reranking.
__init__(index: faiss.Index, metric: str = 'cosine', seq_store: SequenceStore | None = None, verbose: bool = True)[source]
classmethod load(index_path: str, metric: str = 'cosine', verbose: bool = True) → SearchEngine[source]

Load a serialized FAISS index (and optional sequence store).

This attempts to read index_path (a .faiss file) and, if present, a sibling .seqs.sqlite file with the same base name. The sequence store is optional; absence only disables filters / reranking that need it.

Parameters:

  • index_path (str) – Path to the serialized FAISS index (typically ends with .faiss).

  • metric ({'cosine', 'l2'}, default 'cosine') – Metric that will be used for queries. Must match how embeddings were prepared.

  • verbose (bool, default True) – Whether to emit log messages during loading.

Returns:

A ready-to-query search engine instance.

Return type:

SearchEngine

search(queries: Tensor, k: int = 10, nprobe: int | None = None, return_similarity: bool = False, query_sequences: List[str] | None = None, exclude_exact: bool = False, sequence_identity_max: float | None = None, identity_denominator: str = 'query', min_l2_distance: float | None = None, max_cosine_similarity: float | None = None, overfetch: int | None = None, rerank: bool = False, rerank_device: str | None = None, rerank_batch_size: int = 64, rerank_ionic_strength: int | None = None, length_min: int | None = None, length_max: int | None = None) → List[List[Tuple[float, int, str | None, int | None]]][source]

Approximate nearest neighbor search with optional filtering and reranking.

Parameters:

  • queries (torch.Tensor, shape (Q, D)) – Query embeddings. For cosine metric they should be L2-normalized externally (the method re-normalizes defensively before search).

  • k (int, default 10) – Number of final results per query.

  • nprobe (int, optional) – IVF probe count override; higher => better recall, slower.

  • return_similarity (bool, default False) – If True (cosine), return similarity in [0,1]; otherwise distance.

  • query_sequences (list of str, optional) – Raw sequences aligned with queries (enables exact / identity filters & rerank).

  • exclude_exact (bool, default False) – Remove candidates whose sequence matches the query exactly.

  • sequence_identity_max (float, optional) – Maximum allowed identity (0-1). Requires query_sequences and seq store.

  • identity_denominator ({'query','target','max','min','avg'}, default 'query') – Mode for identity normalization (forwarded to identity function).

  • min_l2_distance (float, optional) – Minimum allowed L2 distance (metric=’l2’).

  • max_cosine_similarity (float, optional) – Maximum allowed cosine similarity (metric=’cosine’).

  • overfetch (int, optional) – Multiply k by this before filtering. Auto: 5 if filters active else 1.

  • rerank (bool, default False) – If True, re-embed surviving candidates (exact scoring) and reorder.

  • rerank_device (str, optional) – Device for reranking encoder; auto-select if None.

  • rerank_batch_size (int, default 64) – Batch size for rerank embedding pass.

  • rerank_ionic_strength (int, optional) – Ionic strength forwarded to encoder (if None uses model default).

  • length_min (int, optional) – Minimum acceptable candidate length.

  • length_max (int, optional) – Maximum acceptable candidate length.

Returns:

Per-query results: [(score, gid, header, length), ...] up to k.

Return type:

list of list of tuple

Raises:

RuntimeError – If filters requiring a sequence store are requested but none is attached.