merkle-inclusion-proof-verification-protocol

Tx-Rails Merkle Inclusion-Proof Verification Protocol

A chain-of-custody procedure for cryptographic transaction records

Document status: first published edition. Effective date: 2026-07-31. Companion to docs/ultimate-trust-model.html, which this document was the last identified gap in.

0. Purpose and how to use this document

This protocol lets an independent party — a court-appointed expert, opposing counsel’s expert, a regulator, an internal or external auditor — verify that a specific transaction record was genuinely part of the Tx-Rails Byzantine-fault-tolerant ledger at a specific time, without trusting ITOTCCA’s software, ITOTCCA’s database, or ITOTCCA’s cooperation. Every step below can be carried out independently, using only the disputed record, one JSON artifact, and public information (a running CometBFT node or archive, and a public distributed ledger).

This is the cryptographic equivalent of a chain-of-custody procedure for physical evidence: a fixed sequence of checks that, followed exactly, either confirms or contradicts a claimed fact, and produces a written record of which check (if any) failed. It is written to be followed by someone with ordinary competence in applied cryptography and command-line tooling, not exclusive Tx-Rails expertise. Where a step requires a value or a citation an expert witness’s report should state independently rather than take on this document’s word, that is flagged explicitly.

What this protocol is not. It does not establish that a record’s business content is true, complete, or was reported honestly by the party who submitted it — see §7 (Scope and Residual Trust) and docs/ultimate-trust-model.html §“What this means for storage.” It establishes something narrower and unconditionally checkable: that a specific hash, claimed to represent a specific transaction, was included in the ledger structure that was anchored to a public distributed ledger at a specific time, and that the hash chain connecting the two has not been altered since. What that hash represents, and whether it was computed correctly at the point of submission, are separate questions this protocol does not answer and does not claim to.

1. Definitions

  • Record. The disputed transaction — typically produced by the party under investigation via ordinary disclosure/discovery, not by ITOTCCA. This protocol does not specify how a Verifying Party obtains the Record; it begins once the Record and a matching Proof Artifact are in hand.
  • Proof Artifact. The single JSON file this protocol verifies against — produced by the merkle-audit tool’s --level=tx mode (§3), schema documented in full at §5.
  • Producing Party. Whoever runs merkle-audit to generate the Proof Artifact — ordinarily ITOTCCA or the party operating the relevant Tx-Rails infrastructure, but the tool is open source (AGPLv3) and nothing about it requires that party’s continued cooperation once a Proof Artifact exists; anyone with the required data access (§2) can run it, including a court-appointed expert directly.
  • Verifying Party. Whoever independently checks the Proof Artifact against the Record and the public anchor — the role this protocol is written for.
  • Anchor / Anchored Root. The Merkle root value committed to a public distributed ledger (Hedera by default, or Ethereum/an EVM-compatible chain — see §6). This is the one value in the whole chain the Producing Party cannot alter without also rewriting that public ledger’s own consensus history.
  • Slab, Batch, Block. The three intermediate groupings the proof chain climbs through before reaching the Anchor — defined precisely in §3’s diagram. A Slab is 24 Batches; a Batch is one CometBFT block’s transaction set within the aggregation pipeline.

2. Prerequisites

Before starting, the Verifying Party needs:

  1. The Record itself, or at minimum the raw bytes/content that were submitted as the transaction (obtained via ordinary disclosure, not from this protocol).
  2. A Proof Artifact for that Record (§3, §5) — either produced on request by the Producing Party, or generated independently by the Verifying Party directly (the tool and its data-access requirements are documented, not proprietary; see §3.1).
  3. A working installation of CometBFT’s public Go module (github.com/cometbft/cometbft), specifically its crypto/merkle package — or a from-spec reimplementation (§4) — to independently recompute the two inclusion-proof steps this protocol’s tooling itself uses. This is public, open-source software, not anything ITOTCCA publishes or controls.
  4. Network access to whichever public ledger the Record’s Anchor was submitted to (Hedera Mirror Node’s public REST API, or an Ethereum-compatible JSON-RPC endpoint / block explorer — §6). No ITOTCCA-operated infrastructure is required for this step.
  5. A SHA-256 implementation (standard in essentially every programming language and OS toolchain) to independently verify the Record’s own content hash if the dispute concerns whether a given hash genuinely represents the Record’s content, rather than only whether that hash was included in the ledger.

Nothing above requires access to ITOTCCA’s production database, ITOTCCA’s private infrastructure, or a live Tx-Rails validator node under ITOTCCA’s control, except as noted in §3.1 for Proof Artifact generation (as distinct from verification, which this protocol is primarily about).

3. The proof chain, and where the Proof Artifact comes from

A Record’s inclusion is established through four chained Merkle inclusion proofs, each independently checkable, connecting the transaction to the publicly-anchored root:

tx (leaf)               --[tx_to_data_hash_proof]-->    block.Header.DataHash
DataHash (as one leaf)   --[data_hash_to_block_proof]--> block_hash
block_hash (leaf)        --[block_to_batch_proof]-->     batch.final_merkle_root
batch.final_merkle_root  --[batch_to_anchor_proof]-->    anchored (compounded) root

Each arrow is a Merkle inclusion proof in the RFC 6962 sense (§4): given a leaf, a list of sibling hashes (“aunts”), and a claimed root, anyone can recompute the path and confirm or refute inclusion — no access to the full leaf set required, only the sibling hashes on the path from that one leaf to the root. Chained end to end, the whole thing is roughly 15 sibling hashes and a few hundred bytes, regardless of how many transactions the ledger actually contains at that point.

3.1 Generating the Proof Artifact

The Proof Artifact is produced by the open-source merkle-audit tool (aggd/cmd/merkle-audit), --level=tx mode:

merkle-audit --db-url $AGGREGATOR_DB_URL --level=tx \
             --trace-db-url $TRACE_DB_URL \
             --comet-rpc-addr http://<a-live-cometbft-node>:26657 \
             --network-id <source-network-id> \
             --tx-hash <hex-encoded content hash of the disputed Record>

The --tx-hash form resolves the Record purely from its own content — CometBFT’s own types.Tx.Hash(), the same value CometBFT’s own transaction index keys on — rather than from an internal ITOTCCA database record asserting where it is. (An alternative --tx-id/--federation-id form exists, keyed on Tx-Rails’ own internal transaction identifier, for cases where that identifier rather than content-hash lookup is the operative one; the two are mutually exclusive and produce the identical Proof Artifact shape either way.)

Generation itself requires access to the relevant Postgres databases and a live (or archived — §3.2) CometBFT node, which is why it will ordinarily be the Producing Party who runs this step and hands the Verifying Party the resulting artifact. Nothing about verification (§3.3 onward) depends on trusting that hand-off: the artifact is a self-contained, portable file, and every value in it is independently recomputable by the Verifying Party from public primitives, which is the entire point of this protocol.

3.2 If the live node no longer holds the block

CometBFT nodes may prune old block data. A companion tool, block-archiver (aggd/cmd/block-archiver), independently archives blocks (and a hash → location index) as they’re produced, before any pruning occurs. merkle-audit --level=tx falls back to an archive directory (--archive-dir) automatically whenever the live node reports a height has been pruned. From the Verifying Party’s side this is invisible — the artifact and its verification procedure are identical whichever source the block bytes came from, since both round-trip through CometBFT’s own canonical encoding. It is noted here because a Verifying Party asking “how do I know this block data hasn’t simply been lost or is unavailable to check independently” should know an independent-of-the-validator archival path exists and can be asked for.

3.3 Verification, step by step

Given the Record and its Proof Artifact, the Verifying Party performs the following checks in order. Each is a single deterministic computation — not an iterative or probabilistic one: at each step, either the hashes chain correctly, or they don’t.

  1. Confirm the Record’s own hash. Independently compute the content hash of the Record using CometBFT’s transaction-hash convention (SHA-256 over the raw transaction bytes, per types.Tx.Hash()) and confirm it matches the leaf value the Proof Artifact’s first tier (tx_to_data_hash_proof) is built on. If the Record was supplied by content hash (§3.1’s --tx-hash path), this step is definitional and should already match by construction; perform it anyway as an independent check, not a formality — it is the one step that ties the abstract proof back to the actual disputed document.
  2. Verify Tier 1: transaction → block DataHash. Using the tx_to_data_hash_proof field (a MerkleStepProof: total, index, leaf_hash_hex, aunts_hex, root_hash_hex — §5), recompute the inclusion path with CometBFT’s own merkle.Proof.Verify (or a from-spec reimplementation — §4) and confirm the computed root equals root_hash_hex, which should equal the block’s Header.DataHash.
  3. Verify Tier 2: DataHash → block hash. Using data_hash_to_block_proof, the same verification, confirming the chain reaches the block’s own hash (block_hash_hex).
  4. Verify Tier 3: block hash → batch root. Using block_to_batch_proof, confirming the block’s hash is included under batch.final_merkle_root — the value the aggregation pipeline actually anchors (see §3.4’s architecture note on why this, and not a per-transaction root, is the anchored unit).
  5. Verify Tier 4: batch root → anchored root. Using batch_to_anchor_proof, confirming the batch’s root is included under the compounded root that was actually submitted to the public ledger — anchored_root_hex in the artifact.
  6. Verify the anchored root against the public ledger itself. This is the step that removes ITOTCCA (and this protocol’s own tooling) from the trust chain entirely — see §6 for the exact procedure on Hedera and Ethereum respectively. Confirm the value actually recorded on the public ledger, fetched independently, equals anchored_root_hex.
  7. Record the result. If every tier verified and the public-ledger value matches, the Record is confirmed included in the ledger structure that was anchored at the stated time. If any tier fails to verify, or the public-ledger value disagrees, that is the finding — document precisely which comparison failed and what the two disagreeing values were (§8 on interpreting a negative result).

3.4 Architecture note: why the chain has four tiers, not one

A natural question: why not a single Merkle tree straight from transactions to the anchor? Because that is not how the underlying pipeline is actually built, and describing it as if it were would misstate the evidence. A “batch” in the aggregation pipeline is a Slab of 24 consecutive CometBFT block hashes, Merkleized into final_merkle_root — individual transaction hashes are never themselves folded into that root. Proving one transaction’s inclusion therefore has to reach through CometBFT’s own internal transaction-tree and header-hash construction (Tiers 1–2) before it even reaches the level the aggregation pipeline anchors (Tiers 3–4). This is a more faithful description of what was actually computed and submitted than a simplified one would be, which matters directly for court admissibility: an expert witness’s report should describe the actual construction, not an idealised one that would be easier to explain but doesn’t match what happened.

4. Cryptographic primitives this protocol relies on

  • SHA-256 (FIPS 180-4) — the hash function underlying every leaf, node, and root value in this chain. Collision resistance is approximately 2128; the specific evidentiary framing an expert witness should use for this is discussed in docs/ultimate-trust-model.html §“Statistical framing via expert testimony.”
  • Domain-separated Merkle construction, RFC 6962-style. CometBFT’s merkle.HashFromByteSlices/merkle.Proof.Verify (the functions Tiers 3–4 are built on, and the pattern Tiers 1–2’s from-spec reimplementation follows) use a leaf/internal-node hash-prefix distinction, the same approach Certificate Transparency (RFC 6962) uses. This closes a known forgery class in undifferentiated Merkle trees, where an attacker can sometimes pass off an internal node’s hash as if it were a valid leaf. This is publicly documented, citable behaviour of a public, open-source library — not a Tx-Rails-specific claim requiring Tx-Rails’ own credibility.
  • CometBFT’s Header.Hash() construction (Tier 2 only). CometBFT’s block-header hash is a 14-leaf Merkle tree over the header’s own fields (including DataHash), using an internal encoding function (cdcEncode) that CometBFT does not export as public API. merkle-audit‘s txproof.go carries a from-spec reimplementation of this encoding, and — critically for a court-facing claim — self-checks that reimplementation against the real, running Header.Hash() on every invocation, not only in the tool’s own test suite. A Verifying Party who wants to confirm this independently, rather than take the tool’s self-check on faith, should compare a sample header’s computed hash against the same header’s hash as reported by the live (or archived) CometBFT node directly — they must be identical, since both are computing the same public, specified value.

5. Proof Artifact schema reference

The Proof Artifact is a single JSON file (InclusionProofArtifact). Field-by-field, for --level=tx (the full four-tier form this protocol targets; --level=batch produces the same shape with only Tiers 3–4 populated, for a block-level rather than transaction-level claim):

Field Meaning
schema_version Artifact format version. This protocol targets version "1".
issued_at When the Producing Party generated this artifact — not itself part of the chain of proof, informational only.
source_network_id Which network/federation the block belongs to.
block_height The CometBFT block height the transaction was committed in.
tx_id, federation_id Present if the artifact was generated via --tx-id/--federation-id rather than --tx-hash.
tx_hash_hex The transaction’s own content hash, if generated via --tx-hash.
block_index The transaction’s literal position within the block’s transaction list.
tx_to_data_hash_proof Tier 1: a MerkleStepProof (see below) from the transaction to block.Header.DataHash.
data_hash_to_block_proof Tier 2: a MerkleStepProof from DataHash to the block’s own hash.
block_hash_hex The block hash both Tier 2 and Tier 3 pivot on.
batch_id, batch_start_height, batch_end_height Which Batch (aggregation-pipeline slab unit) the block falls in, and its height range.
block_to_batch_proof Tier 3: a MerkleStepProof from the block hash to batch.final_merkle_root.
anchor_id Which anchoring event (row in the aggregator’s anchor outbox) covers this batch.
hedera_tx_id The public transaction identifier the compounded root was submitted under — the handle for §6’s independent public-ledger check. (Field name is a holdover from the Hedera-only original build; it holds the equivalent transaction identifier for whichever chain was actually used — see §6.2 for Ethereum.)
anchor_timestamp When the anchoring submission was made.
batch_to_anchor_proof Tier 4: a MerkleStepProof from batch.final_merkle_root to the compounded root.
anchored_root_hex The value that should equal both the root of batch_to_anchor_proof and what’s independently observable on the public ledger (§6). This is the single value the entire chain is ultimately about.
verified Whether the Producing Party’s own tooling considered the chain internally consistent at generation time. A Verifying Party should not rely on this field — it is exactly what this protocol exists to let a third party check independently, not take on trust.
note If verified is false, a human-readable statement of which two values disagreed. Present for completeness; again, independently confirm rather than rely on the tool’s own characterisation.

MerkleStepProof (the shape of each of the four *_proof fields above):

Field Meaning
total Total number of leaves in this tier’s tree.
index This leaf’s position (0-indexed) among them.
leaf_hash_hex The leaf value itself, hex-encoded.
aunts_hex The sibling hashes on the path from leaf to root — typically log₂(total) of them.
root_hash_hex The root this tier’s proof claims to reach — should equal the leaf value of the next tier up the chain.

6. Checking the anchored root against the public ledger

This is the step that makes the whole chain independently verifiable rather than merely internally consistent: confirming anchored_root_hex against a value neither party in a dispute controls. The procedure differs by which chain the deployment anchors to (ANCHOR_CHAIN at submission time; both are built, tested, and in current use — see docs/ultimate-trust-model.html §“anchor-network lever” for the tradeoffs between them).

6.1 Hedera (default)

The compounded root is submitted as the full message body of a Hedera Consensus Service (HCS) topic message — not merely referenced in the transaction memo (a separate, shorter field used for an unrelated internal replay-check, not the primary evidentiary path). The message body is the JSON-serialised anchor payload, containing compounded_root_hex among other fields identical in name and meaning to this protocol’s own artifact fields.

  1. Using hedera_tx_id from the artifact (or the topic ID and sequence number, if that’s what the Producing Party’s records identify the submission by), query Hedera’s public Mirror Node REST API for that topic message. Hedera operates multiple independent, publicly-documented Mirror Node deployments; a Verifying Party should use the current, officially-documented endpoint for the relevant network (mainnet or testnet) rather than take any single URL on trust from this document — Hedera’s own developer documentation is the authority on the current API surface.
  2. The Mirror Node returns the message body, base64-encoded. Decode it.
  3. Parse the decoded bytes as JSON and read its compounded_root_hex field.
  4. Confirm it is byte-identical to the Proof Artifact’s own anchored_root_hex.

The Mirror Node is Hedera’s independent, publicly-operated read side — separate infrastructure from the consensus nodes that accepted the original submission. A party who had somehow compromised both an anchor-submission credential and ITOTCCA’s own database would still need to separately compromise the Mirror Node (or Hedera’s underlying consensus itself) to make this specific check pass on a forged value, which is the residual-trust boundary this protocol is honest about in §7.

6.2 Ethereum (or an EVM-compatible chain)

The compounded root is submitted as the plain calldata of a zero-value transaction — no smart contract involved, so there is nothing to trust beyond the public chain itself recording the transaction.

  1. Using the transaction hash recorded in the artifact’s hedera_tx_id field (name is a legacy holdover — see §5’s note; it holds the Ethereum transaction hash when that chain was used), query any Ethereum JSON-RPC endpoint’s eth_getTransactionByHash, or any public block explorer for the relevant network, for that transaction.
  2. Read the transaction’s input (calldata) field.
  3. Decode it as UTF-8 JSON and read its compounded_root_hex field, identically to §6.1’s step 3.
  4. Confirm it is byte-identical to the Proof Artifact’s own anchored_root_hex.

Because the submission code talks to any EVM-compatible network by RPC endpoint and chain ID alone, this identical procedure applies whether the specific deployment anchors to Ethereum mainnet, a testnet, or an EVM sidechain such as Elastos — only which network’s public RPC/explorer to query changes, per docs/ultimate-trust-model.html‘s discussion of that tradeoff.

7. Scope and residual trust

Consistent with the standard this whole architecture is held to elsewhere (docs/ultimate-trust-model.html; federation-trust-model.html‘s own treatment of Regime C’s assumption (a)), this protocol states its residual trust explicitly rather than implying a completed proof of certainty:

  • The public ledger’s own consensus safety is not, and cannot be, established by this protocol. Confirming a value is recorded on Hedera or Ethereum relies on that network’s own consensus never having forked, reordered, or been equivocated to different observers at that point in its history — a property of the public ledger, not of anything Tx-Rails controls or this protocol can independently verify. This is precisely the same category of residual trust federation-trust-model.html names for Regime C’s assumption (a); §6.2’s choice-of-anchor discussion in docs/ultimate-trust-model.html is the relevant framing for why a given deployment picked the chain it did.
  • This protocol confirms inclusion, not content-level correctness. A verified chain confirms that a hash claimed to represent the Record was genuinely part of the anchored structure. It does not confirm that hash was computed correctly from genuine underlying business data at the point of original submission — that is a question about the Producing Party’s own systems, upstream of anything this protocol observes. See docs/ultimate-trust-model.html‘s discussion of hash-only integration mode for the corresponding disclosure-and-discovery framing.
  • CometBFT’s own implementation is trusted to match its own public specification, mitigated (not eliminated) by merkle-audit‘s self-check of its from-spec header-hash reimplementation against the real, running function on every invocation (§4) — a stronger form of confirmation than an unchecked reimplementation would offer, but not a formal proof that CometBFT’s own code is bug-free.
  • This is a first published edition. It has not yet been used in an actual court proceeding or reviewed by retained legal counsel for a specific jurisdiction’s evidentiary rules; a party relying on it in an active matter should have it reviewed for that purpose before filing, the same way any expert protocol would be.

8. Interpreting a negative result

A verification failure at any tier, or a mismatch at §6’s public-ledger check, is itself a meaningful finding, not merely an inconclusive one — see docs/ultimate-trust-model.html‘s discussion of why the mismatch case is often more legally interesting than the match case. Forging a passing chain at any tier requires either a SHA-256 collision or rewriting the relevant public ledger’s own consensus history; a Verifying Party documenting a failure should record, precisely:

  • Which numbered step (§3.3) first failed to verify.
  • The two specific values that disagreed (both hex strings, quoted in full, not summarised).
  • Whether the failure was internal (one tier’s computed root doesn’t match the next tier’s claimed leaf — a problem within the artifact or the archived data itself) or external (§6’s public-ledger value doesn’t match anchored_root_hex — a problem with what was actually anchored, or a forged artifact).

9. Worked example

A minimal worked example, illustrating the shape of a Proof Artifact and the verification outcome, based on a real Proof Artifact structure (values illustrative — see aggd/cmd/merkle-audit/README.md for the full field set and a batch-level example):

{
  "schema_version": "1",
  "issued_at": "2026-07-28T12:00:00Z",
  "source_network_id": "audit-nat-a",
  "block_height": 10057,
  "tx_hash_hex": "0x8f2a...",
  "block_index": 3,
  "tx_to_data_hash_proof": {
    "total": 1024, "index": 3, "leaf_hash_hex": "0x8f2a...",
    "aunts_hex": ["0x...", "... (10 entries)"], "root_hash_hex": "0xa1b2..."
  },
  "data_hash_to_block_proof": {
    "total": 14, "index": 6, "leaf_hash_hex": "0xa1b2...",
    "aunts_hex": ["0x...", "... (4 entries)"], "root_hash_hex": "0xc3d4..."
  },
  "block_hash_hex": "0xc3d4...",
  "batch_id": 42, "batch_start_height": 10033, "batch_end_height": 10056,
  "block_to_batch_proof": {
    "total": 24, "index": 24, "leaf_hash_hex": "0xc3d4...",
    "aunts_hex": ["0x...", "... (5 entries)"], "root_hash_hex": "0xe5f6..."
  },
  "anchor_id": 3,
  "hedera_tx_id": "0.0.1234@1753600000.000000000",
  "anchor_timestamp": "2026-07-28T11:58:00Z",
  "batch_to_anchor_proof": {
    "total": 24, "index": 17, "leaf_hash_hex": "0xe5f6...",
    "aunts_hex": ["0x...", "... (5 entries)"], "root_hash_hex": "0x4f3a..."
  },
  "anchored_root_hex": "0x4f3a...",
  "verified": true
}

Reading this chain: the transaction’s own hash (tx_hash_hex) is Tier 1’s leaf; Tier 1’s root_hash_hex equals Tier 2’s leaf (both 0xa1b2..., the block’s DataHash); Tier 2’s root equals the artifact’s block_hash_hex and Tier 3’s leaf (both 0xc3d4...); Tier 3’s root equals Tier 4’s leaf (both 0xe5f6..., batch.final_merkle_root); and Tier 4’s root equals anchored_root_hex (0x4f3a...) — the value §6 checks against the public ledger directly. Each equality in that sentence is exactly one of §3.3’s numbered checks.

10. References

  • docs/ultimate-trust-model.html — the discussion this protocol was written to complete; the fuller regulatory/evidentiary framing, the anchor-choice tradeoffs, and the record of how the underlying tooling (including the archival and content-hash-lookup pieces §3.1–3.2 depend on) was verified against a real, independently-running chain rather than only synthetic test data.
  • federation-trust-model.html — the regime model (A/B/C) §7 refers to, including Regime C’s own stated assumption (a) and its treatment as permanent residual trust rather than a to-do.
  • aggd/cmd/merkle-audit/README.md — the tool’s own usage documentation, worked examples for all three verification levels, and the architecture note on why the proof chain has the shape it does.
  • FIPS 180-4 — the SHA-256 specification.
  • RFC 6962 (Certificate Transparency) — the domain-separated Merkle tree construction this architecture’s hashing follows the same approach as.
  • CometBFT’s public documentation and source (github.com/cometbft/cometbft), specifically the crypto/merkle package and types.Header.Hash()/types.Tx.Hash().
  • Hedera’s public developer documentation (Mirror Node REST API, Hedera Consensus Service) and Ethereum’s JSON-RPC specification — both are the current, authoritative sources for §6’s exact endpoints; this document deliberately does not freeze a specific URL that could drift out of date.

Revision history

Date Change
2026-07-31 First published edition, written once the underlying tooling (--level=tx, block-archiver, --tx-hash content-based lookup) was built and verified against a real chain — the gap docs/ultimate-trust-model.html identified as “paperwork” rather than a missing capability.

Tx-Rails — Built by IT/OT Chain & Cloud Australia (ITOTCCA). Sydney · London.