atlas-ledger

Summary

atlas-ledger is the central persistence and execution crate of the Rust workspace. It owns the durable proposal log, proposal and transaction indexes, in-memory ledger state, state transition execution, shard and AEC persistence, KYC persistence, and the public ledger gRPC service.

This crate is not just “the database.” In the current design it is both:

• the authoritative ledger state and execution engine
• the persistence layer used by the rest of the runtime

That makes it one of the highest-leverage crates in the workspace.

Why This Crate Exists

• persist proposals and transaction metadata durably
• apply proposal contents into ledger state
• expose read APIs for balances, assets, aliases, KYC, and proposals
• maintain secondary persistent views such as account shards, AEC storage, and KYC storage
• provide the concrete LedgerPort implementation consumed by consensus-facing code

Current Role In The Workspace

atlas-ledger currently owns six major concerns:

1. Ledger, the main facade used by application and consensus code
2. State, the in-memory state machine for accounts, assets, delegations, aliases, KYC, and institutions
3. append/query/checkpoint/prune lifecycle around persisted proposals
4. state execution through BlockExecutor, validation handlers, and interceptors
5. durable persistence through binlog, redb index, shard files, AEC storage, and KYC storage
6. public ledger gRPC service definitions and implementation

There is also a separate storage::Storage type that is not the same thing as Ledger. That type acts as a higher-level consensus/audit store for proposals, votes, and results, and can optionally hold an Arc<Ledger>.

Public Surface

Top-Level Modules

• core
• interface
• telemetry

Main Public Reexports

• Ledger
• TxStatusEvent
• state
• storage
• AccountNature
• AccountSection
• AccountState
• AccountingClassification
• AssetDefinition
• AssetType
• ATLAS_FULL_ID
• ATLAS_SYMBOL
• AccountingMap
• core::runtime::{binlog, index}

Important Types

• core::ledger::manager::Ledger
• core::ledger::state::State
• core::ledger::manager::{AppendProposalOutcome, LedgerCheckpointSnapshot, PruneOutcome, RollbackOutcome}
• core::ledger::executor::BlockExecutor
• core::runtime::binlog::Binlog
• core::runtime::index::Index
• core::ledger::storage::Storage
• core::ledger::storage::kyc::{KycStorage, PersistedKycRecord}

Important Traits Implemented

• atlas_common::ports::LedgerPort for Ledger

Key Modules

• `lib.rs`: public facade and LedgerPort implementation
• `core/ledger/manager/mod.rs`: Ledger struct and core owned resources
• `core/ledger/manager/append.rs`: proposal append flow, dedup, binlog write, indexing, and execution handoff
• `core/ledger/manager/queries.rs`: read/query facade, reconstruction flow, and helper factories
• `core/ledger/state/mod.rs`: in-memory state model and entry application rules
• `core/ledger/executor/mod.rs`: execution orchestration over state, index, shards, AEC, and KYC persistence
• `core/ledger/transaction_engine/validation.rs`: stateless and stateful validation of signatures, assets, and nonce
• `core/ledger/transaction_engine/interceptors/mod.rs`: post-parse transaction hooks that mutate domain state beyond raw balance transfers
• `core/ledger/genesis.rs`: genesis bootstrap for allocations, institutions, and issuance pool setup
• `core/runtime/binlog.rs`: durable append-only proposal log
• `core/runtime/index.rs`: redb-backed proposal, tx hash, signature, and status indexes
• `core/ledger/storage/mod.rs`: higher-level proposal/vote/result storage used by consensus flows
• `core/ledger/storage/kyc.rs`: persisted KYC record storage
• `core/ledger/reader.rs`: read-only balance, asset, alias, KYC, and proposal queries
• `interface/api/service.rs`: gRPC LedgerService implementation
• `proto/ledger.proto`: public gRPC contract

Inputs And Outputs

Inputs

• Proposal values from consensus/runtime
• signed transactions inside proposal payloads
• genesis bootstrap state
• ledger queries from node, admin, and gRPC clients
• persistence backend configuration such as data directory paths

Outputs

• persisted proposals in binlog
• indexed proposal locations, tx hashes, signatures, and status
• updated in-memory state
• account shard files and AEC views
• persisted KYC registry snapshots
• gRPC responses for transaction submission and reads
• transaction status broadcast events

Internal Dependencies

Workspace Dependencies

• atlas-common
• atlas-bank
• atlas-kyc
• atlas-mempool

atlas-ledger is where those lower-level and domain-level crates are assembled into executable ledger behavior.

External Dependencies That Shape The Design

• redb for durable indexes and KYC persistence
• tokio for async file and runtime work
• tonic and prost for public ledger gRPC
• axum and related tower crates for interface support

Used By

These packages depend directly on atlas-ledger today:

• atlas-node-admin
• atlas-consensus
• atlas-node

Observed usage in the workspace includes:

• `cluster/proposals.rs`: consensus commits accepted proposals through ledger.append_proposal(...)
• `env/runtime.rs`: consensus runtime uses atlas_ledger::storage::Storage
• `main.rs`: node startup applies genesis using ledger.apply_genesis_state(...)
• `config.rs`: node config builds Ledger::new(...)
• `rpc/server.rs`: node serves the ledger gRPC service via LedgerServiceImpl
• `handlers/accounts.rs`: admin flows use ledger state and bank services together
• `auth.rs`: admin auth checks ledger balances of system KYC assets

Execution Model

The main append path in `append.rs` is:

1. parse proposal content into signed transactions
2. derive transaction hashes and detect duplicates
3. append the proposal to the binlog
4. execute the proposal against in-memory state through BlockExecutor
5. index proposal location and transaction metadata in redb

Inside execution, the crate uses:

• validation handlers for signatures, sender and fee-payer envelopes, assets, and nonce
• atlas-bank::AccountingEngine for transfer entry generation
• interceptor actions for KYC, alias, institution, staking, mint, and system parameter flows
• async background persistence workers for shards, AEC, and KYC records

State And Persistence Model

The crate maintains multiple views of the ledger at once:

• State: authoritative in-memory view for runtime decisions
• Binlog: append-only proposal history
• Index: redb lookup tables for proposals, tx hashes, signatures, and statuses
• ShardStorage: account-focused persistence
• AecStorage: account event chain persistence
• KycStorage: persisted KYC registry records

There is also a reconstruction flow in `queries.rs` that rebuilds persistent views from the binlog when needed.

Testing

The crate has a broad test surface:

• unit tests across Binlog, Index, Reader, validation, interceptors, storage, AEC, and state modules
• `test_transaction_pipeline.rs`: verifies append, idempotency, indexing, state mutation, and tx status updates
• `test_institutional_genesis.rs`: verifies genesis registration of institutions and issuance application

This is one of the clearer signals that atlas-ledger is treated as a core integration point, not just a utility crate.

Risks Or Design Tension

Very Broad Responsibility

atlas-ledger currently combines:

• state machine logic
• persistence backends
• query facade
• execution orchestration
• gRPC service
• telemetry
• recovery and reconstruction

That breadth is practical, but it makes the crate cognitively heavy.

Two Storage Abstractions

There is both:

• Ledger, the main runtime ledger facade
• storage::Storage, a separate proposal/vote/result store used by consensus code

Both are valid, but the naming overlap is easy to misread.

Parallel KYC Systems

The main runtime flow appears to use:

• core/ledger/state/kyc.rs
• core/ledger/storage/kyc.rs

But the crate also contains a separate `core/nft_kyc/` subsystem. Based on the current code search, that module looks present and tested, but not obviously wired into the main append/query path. It may be legacy, experimental, or parallel design work.

Mixed Scope Inside The Crate

This Rust-only pass is focused on the Rust crate, but atlas-ledger also contains non-Rust assets such as the explorer frontend under `explorer/`. That reinforces that this package boundary is wider than the ledger engine alone.

Placeholder Or Thin Areas

Some internal modules are clearly less mature than the core append/query/state path:

• `core/service/mod.rs` is currently a placeholder
• some interface modules are thin or mostly organizational

Open Questions

• should storage::Storage eventually be renamed to reduce confusion with the main Ledger
• should public ledger gRPC live here long term, or move upward into the application layer
• should core/nft_kyc stay in this crate if the main KYC runtime has moved elsewhere
• should some of the recovery, persistence, or interface responsibilities split into smaller crates over time

Relevant Files

• `Cargo.toml`
• `lib.rs`
• `manager/mod.rs`
• `append.rs`
• `queries.rs`
• `state/mod.rs`
• `executor/mod.rs`
• `validation.rs`
• `interceptors/mod.rs`
• `genesis.rs`
• `binlog.rs`
• `index.rs`
• `storage/mod.rs`
• `storage/kyc.rs`
• `reader.rs`
• `service.rs`
• `ledger.proto`
• `test_transaction_pipeline.rs`
• `test_institutional_genesis.rs`