Using BetterContext Copy

Ingestion Pipelines

Stream codebases, documentation, wikis, and event logs into your memory graph with automated AST extraction and causal dependency linking.

BetterContext provides multiple ingestion pathways tailored for initial repository bootstrapping, continuous git branch synchronization, and automated enterprise webhooks.

FRAME 01 · MULTI-PATHWAY INGRESS CHANNELS 01 · Ingress Pathways CLI Batch Git Scanner Webhook Event Streams REST Programmatic SDK GitHub & Linear Merges SOURCES Streaming Ingest Channels • Git push & PR review webhook hooks • Multi-threaded parallel file tree indexer • Instant <14ms/file Parsing Latency STREAM FRAME 02 · LEXICAL PARSING & DATA SANITIZATION CORE 02 · Tree-sitter AST 120+ Language Grammars Class & Symbol Trees Deterministic Call Graph Cross-File Import DAG PARSER 03 · Secret Sanitizer Entropy & Regex Scrub API Keys & JWT Redaction Synthetic Placeholders Zero Data Leakage FILTER INDEX FRAME 03 · DETERMINISTIC GRAPH STORE & TOMBSTONING 04 · Graph & Store Deterministic Nodes Submodular Context Pack Dual-Clock Tombstoning Historical Bisect Safe INDEXED Dual-Clock Tombstone State • Git deleted files marked soft-deprecated • Fully preserved for bisect & post-mortems • Prevents Broken Graph Invariants Dual-Clock Tombstoning: Soft-Deprecate on Git Delete

Ingestion Pathways

CLI Ingestion

The fastest way to index a project:

# Ingest the entire current directory on the 'main' branch
bettercontext ingest . \
  --branch main \
  --include "*.py,*.ts,*.md" \
  --exclude "tests/*,dist/*" \
  --threads 8

Programmatic Ingestion via Python SDK

Ingest custom documents with role-based ACLs and metadata:

from bettercontext import BetterContextClient

client = BetterContextClient()

# Upsert a sensitive architectural decision record
doc = client.memory.upsert(
    title="ADR-042: Database Encryption at Rest",
    content="All customer PII and authentication tokens must be stored in Postgres columns encrypted with pgcrypto.",
    format="markdown",
    tags=["security", "compliance", "database"],
    acls={
        "read": ["engineer", "security-lead", "admin"],
        "write": ["security-lead"]
    }
)

print(f"Created Node ID: {doc.node_id}")
print(f"Entities Extracted: {doc.extracted_entities}")

Handling Deletions & Tombstoning

When code or documentation is deleted in git, BetterContext uses Dual-Clock Tombstoning. Rather than instantly obliterating the node and breaking downstream historical queries, the node is marked as deprecated for current queries while remaining queryable during historical git-bisect and retrospective sessions.

Next Steps

Was this page helpful?

Powered By BetterContext