Function Pools¶

1. Purpose & Motivation¶

1.1 What Problem Does Function Pools Solve?¶

Viper is a metadata-driven C++ runtime that enables strongly-typed interoperability between C++ and Python. At its core, Viper provides a type system (Type, Value) and persistence (Database, CommitSystem). However, business logic must be exposed as callable operations for both Python clients and remote services to invoke.

Function Pools solves the fundamental problem: "How do we expose pure C++ business logic as remotely callable operations while maintaining type safety, avoiding Viper pollution of business code, and enabling seamless regeneration?"

The Challenge¶

Consider a typical scenario: You've built a complex C++ calculation engine for financial modeling. You need to:

Expose operations to Python (e.g., calculate_risk(portfolio, market_data) -> risk_score)
Maintain type safety (ensure arguments match expected types at runtime)
Keep business logic pure (no Viper types polluting your C++ algorithms)
Support regeneration (DSM schema changes shouldn't break user code)
Enable remote invocation (Services layer can call functions over RPC)
Integrate with Commit System (some functions need mutation tracking)

Without Function Pools, developers would manually write: - Wrapper classes implementing Viper::Function interface - Value encode/decode logic (C++ types ↔ Viper Values) - Function registry and lookup infrastructure - Python bindings for each function - Validation logic for argument types and counts

This is tedious, error-prone, and creates coupling between business logic and Viper infrastructure.

The Solution¶

Function Pools introduces the Bridges Pattern: a clean separation between stable user code (business logic in pure C++) and regenerable wrappers (Kibo-generated infrastructure). The workflow is:

DSM Definition → Kibo Code Generation → User Bridges (C++) → Generated Wrappers → Python/Services

Key insight: User writes pure C++ functions in FunctionPoolBridges namespace (never regenerated). Kibo generates wrapper classes in FunctionPools namespace (regenerated anytime) that handle Value encoding, validation, and registry management. Business logic remains stable across regenerations.

1.2 Why is Function Pools Needed?¶

Function Pools is the abstraction layer that decouples business logic from Viper infrastructure, while Services is the orchestration layer that manages remote invocation and distributed execution. They solve orthogonal problems:

Concern	Function Pools	Services
Problem	Expose C++ as callable operations	Orchestrate remote function execution
Coupling	Bridges Pattern (stable ↔ generated)	RPC protocol + function pool discovery
Type Safety	FunctionPrototype validation	Serialization/deserialization
Scope	Single-process callable registry	Multi-process distributed services
Reusability	Used by ALL Viper projects	Optional remote infrastructure

Dependency relationship: Services → Function Pools (unidirectional). Services uses Function Pools but Function Pools is independently useful (GE project uses it without Services for Python bindings).

Why Not Merge With Services?¶

The coupling is unidirectional, not bidirectional like Type ↔ Value (which justify merging into "Type and Value System"). Function Pools can exist without Services (local Python bindings), but Services cannot exist without Function Pools (needs callable operations). Merging would conflate local abstraction with distributed orchestration, harming modularity.

1.3 Primary Use Cases¶

Use Case 1: Python Business Logic Bindings¶

Scenario: GE project exposes geometric calculations to Python.

DSM Definition (Pool_Tools.dsm):

function_pool Tools {dc9740c9-9d1d-4c1e-9caa-4c8843b91e82} {
    int64 add(int64 a, int64 b);
    bool isEven(int64 a);
}

User Bridge (GE_FunctionPoolBridges.cpp - STABLE):

namespace GE::FunctionPoolBridges::Tools {
    std::int64_t add(std::int64_t a, std::int64_t b) {
        return a + b;  // Pure C++, no Viper types
    }
}

Generated Wrapper (GE_FunctionPools.cpp - REGENERABLE):

class F_add final : public Viper::Function {
    std::shared_ptr<Value> checkedCall(...) const override {
        auto a = ValueDecoder::decode_int64(ValueInt64::cast(args.at(0)));
        auto b = ValueDecoder::decode_int64(ValueInt64::cast(args.at(1)));
        return ValueEncoder::encode(FunctionPoolBridges::Tools::add(a, b));
    }
};

Python Usage:

import ge
pool = ge.function_pools.tools()
result = pool.add(10, 32)  # Calls C++ through Viper

Use Case 2: Commit-Aware Mutations¶

Scenario: Service project tracks mutations for collaborative editing.

DSM Definition:

commit_function_pool Commit {550e8400-e29b-41d4-a716-446655440000} {
    void applyDelta(CommitDelta delta);
}

User Bridge (with CommitMutating context):

namespace Service::FunctionPoolBridges::Commit {
    void apply_delta(Viper::CommitMutating const & commitMutating,
                     std::shared_ptr<Service::CommitDelta> const & delta) {
        commitMutating.applyDelta(delta);  // Mutation tracked
    }
}

Use Case 3: Polymorphic Functions¶

Scenario: Comparison function works with any Viper type.

DSM Definition:

function_pool Utils {uuid} {
    bool isGreater(any a, any b);
}

Implementation:

bool is_greater(Viper::Any const & a, Viper::Any const & b) {
    return Viper::isGreater(a.value()->compare(b.value()));
}

Why This Works: TypeCode::Any bypasses prototype validation, allowing runtime polymorphism.

1.4 Architecture Position¶

Function Pools resides in Functional Layer 1, depending on:

Foundation Layer 0: Type and Value System (for Value encoding/decoding, type validation)
Functional Layer 1: Commit System (for commit-aware function pools)

Function Pools is consumed by:

Functional Layer 1: Services (for remote function invocation)
All Viper Projects: GE, Service, and any project needing Python bindings

Architectural role: Function Pools is the semantic boundary between pure C++ business logic and Viper's metadata-driven infrastructure. It enables type-safe exposure without pollution of user code.

2. Domain Overview¶

2.1 Scope and Boundaries¶

In Scope: - Function pool abstractions (FunctionPool, CommitFunctionPool) - Callable function abstractions (Function, CommitGettingFunction, CommitMutatingFunction) - Function signatures with type validation (FunctionPrototype) - Bridges Pattern (separation of stable user code from generated wrappers) - Kibo code generation (templates for C++/Python) - Value encode/decode infrastructure (C++ types ↔ Viper Values) - Function registry and O(1) lookup - Python bindings for function invocation

Out of Scope: - Remote invocation and RPC (Services domain) - Distributed execution and load balancing (Services domain) - Business logic implementation (user responsibility) - DSM parsing and validation (DSM domain, though tests overlap)

Boundary with Services: Function Pools provides the callable abstraction (what can be called, how to validate, how to invoke locally). Services provides the remote orchestration (where to call, how to serialize, how to distribute). Services depends on Function Pools but not vice versa.

2.2 Key Concepts¶

2.2.1 Function Pool¶

Definition: A registry container that manages a collection of callable functions, identified by UUID and name.

Purpose: - Provides O(1) lookup of functions by name - Groups related operations (e.g., "Tools" pool for utilities, "Commit" pool for mutations) - Serves as the discovery mechanism for Python/Services

Key characteristics: - Immutable after construction (functions added during initialization) - UUID-identified for global uniqueness - Thread-safe for concurrent reads (const methods)

C++ API (Viper_FunctionPool.hpp:15):

class FunctionPool final {
public:
    UUId const uuid;
    std::string const name;

    void add(std::shared_ptr<Function> const & function);
    std::shared_ptr<Function> check(std::string const & funcName) const;
    std::vector<std::shared_ptr<Function>> functions() const;
};

2.2.2 Commit Function Pool¶

Definition: A commit-aware variant of FunctionPool where functions receive a CommitGetting or CommitMutating context.

Purpose: - Enable functions to read/modify commit state - Enforce read vs. write intent (CommitGetting for queries, CommitMutating for mutations) - Support collaborative editing with mutation tracking

Key difference from FunctionPool: Functions receive explicit context parameter, enabling integration with Commit System's event sourcing.

C++ API (Viper_CommitFunctionPool.hpp:12):

class CommitFunctionPool final {
public:
    UUId const uuid;
    std::string const name;

    void add(std::shared_ptr<CommitFunction> const & function);
    std::shared_ptr<CommitFunction> check(std::string const & funcName) const;
};

2.2.3 Function¶

Definition: An abstract callable that validates arguments and returns a typed result.

Purpose: - Enforce type safety via FunctionPrototype validation - Implement Template Method pattern (call → validate → checkedCall) - Decouple validation (generated) from implementation (user)

Critical design: call() method is sealed (enforces validation), checkedCall() is virtual (user implements). This ensures validation cannot be bypassed.

C++ API (Viper_Function.hpp:18):

class Function {
public:
    std::shared_ptr<FunctionPrototype> const prototype;
    std::string const documentation;

    std::shared_ptr<Value> call(std::vector<std::shared_ptr<Value>> const & args) const {
        prototype->checkArguments(args);  // ALWAYS validates
        return prototype->checkResult(checkedCall(args));
    }

protected:
    virtual std::shared_ptr<Value> checkedCall(...) const = 0;  // User implements
};

2.2.4 Function Prototype¶

Definition: A function signature specifying name, parameters (name + type), and return type.

Purpose: - Validate argument count and types at runtime - Validate return type matches declaration - Support introspection (list parameters, inspect types) - Enable polymorphism via TypeCode::Any special case

Validation logic (Viper_FunctionPrototype.cpp:67):

void checkArguments(std::vector<std::shared_ptr<Value>> const & arguments) const {
    if (arguments.size() != parameters.size()) {
        throw std::runtime_error("Expected " + std::to_string(parameters.size()) +
                                 " arguments, got " + std::to_string(arguments.size()));
    }
    for (size_t i = 0; i < parameters.size(); ++i) {
        auto const & param = parameters[i];
        if (param.type->typeCode() != TypeCode::Any &&  // Special case: 'any' bypasses check
            !param.type->equals(arguments[i]->type())) {
            throw std::runtime_error("Type mismatch for parameter '" + param.name + "'");
        }
    }
}

TypeCode::Any semantics: When a parameter type is any, validation is skipped, allowing polymorphic functions that accept any Viper value.

2.2.5 Bridges Pattern ⭐¶

Definition: A code organization pattern that separates stable user code (business logic) from regenerable wrappers (infrastructure).

Purpose: - Keep business logic pure C++ (no Viper types pollution) - Enable safe regeneration (Kibo can regenerate wrappers without touching user code) - Maintain semantic stability across DSM schema changes

Workflow:

DSM Definition: User declares function signature in .dsm file dsm function_pool Tools {uuid} { int64 add(int64 a, int64 b); }
Kibo Generation: Generates two files:
ProjectName_FunctionPoolBridges.hpp (declarations, user implements in .cpp)
ProjectName_FunctionPools.{hpp,cpp} (wrappers, fully regenerable)
User Implementation (Bridges.cpp - NEVER regenerated): cpp namespace Project::FunctionPoolBridges::Tools { std::int64_t add(std::int64_t a, std::int64_t b) { return a + b; // Pure C++ } }
Generated Wrapper (FunctionPools.cpp - regenerated anytime): cpp class F_add : public Viper::Function { std::shared_ptr<Value> checkedCall(...) const override { auto a = ValueDecoder::decode_int64(ValueInt64::cast(args[0])); auto b = ValueDecoder::decode_int64(ValueInt64::cast(args[1])); return ValueEncoder::encode(FunctionPoolBridges::Tools::add(a, b)); } };

Key insight: Bridge functions have pure C++ signatures (stable API), wrapper classes have Viper signatures (regenerable infrastructure). The bridge is the semantic stability boundary.

2.3 High-Level Architecture¶

┌─────────────────────────────────────────────────────────────────┐
│                         Python Layer                            │
│  import project                                                 │
│  pool = project.function_pools.tools()                          │
│  result = pool.add(10, 32)  # Typed call                       │
└────────────────────────┬────────────────────────────────────────┘
                         │ P_Viper binding (tp_call)
┌────────────────────────▼────────────────────────────────────────┐
│                  Generated Wrappers (REGENERABLE)               │
│  class F_add : public Viper::Function {                         │
│      Value checkedCall(args) {                                  │
│          a = decode_int64(args[0]);  // Value → C++            │
│          b = decode_int64(args[1]);                             │
│          result = Bridge::add(a, b);  ◄─────────┐              │
│          return encode(result);  // C++ → Value │              │
│      }                                           │              │
│  }                                               │              │
└──────────────────────────────────────────────────┼──────────────┘
                                                   │
┌──────────────────────────────────────────────────▼──────────────┐
│              User Bridges (STABLE - never regenerated)          │
│  namespace Project::FunctionPoolBridges::Tools {                │
│      int64_t add(int64_t a, int64_t b) {                        │
│          return a + b;  // Pure C++ business logic              │
│      }                                                           │
│  }                                                               │
└──────────────────────────────────────────────────────────────────┘

Data flow: 1. Python calls pool.add(10, 32) → P_Viper binding 2. Binding calls Function::call([Value(10), Value(32)]) → Validates via FunctionPrototype 3. Calls F_add::checkedCall() → Generated wrapper 4. Wrapper decodes Values → 10 (int64), 32 (int64) 5. Wrapper calls FunctionPoolBridges::Tools::add(10, 32) → User bridge (pure C++) 6. Bridge returns 42 (int64) 7. Wrapper encodes → Value(42) 8. Binding converts → Python int object

CRITICAL: User code at step 5 is pure C++, no Viper pollution. Regeneration replaces steps 1-4 and 6-8, but never touches step 5.

2.4 Two Pool Types: FunctionPool vs CommitFunctionPool¶

Why Two Separate Types?¶

Design rationale: Type safety and compile-time enforcement of mutation intent.

Aspect	FunctionPool	CommitFunctionPool
Function Signature	`Value func(Values args)`	`Value func(CommitContext, Values args)`
Use Case	Stateless operations	Commit-aware operations
Mutation Tracking	No	Yes (via CommitMutating)
Read-Only Enforcement	N/A	CommitGetting vs CommitMutating
Examples	Math, string utils, conversions	Commit deltas, node mutations

Example: Stateless Function¶

DSM:

function_pool Tools {uuid} {
    string format(string template, any value);
}

Bridge:

std::string format(std::string const & tmpl, Viper::Any const & value) {
    return std::regex_replace(tmpl, std::regex("\\{\\}"), value.value()->toString());
}

No commit context needed - pure transformation.

Example: Commit-Aware Function¶

DSM:

commit_function_pool Commit {uuid} {
    void mutating applyDelta(CommitDelta delta);
    CommitNode getting getNode(UUId id);
}

Bridges:

void apply_delta(Viper::CommitMutating const & cm, std::shared_ptr<CommitDelta> delta) {
    cm.applyDelta(delta);  // Mutation tracked in commit history
}

std::shared_ptr<CommitNode> get_node(Viper::CommitGetting const & cg, Viper::UUId const & id) {
    return cg.getNode(id);  // Read-only access
}

Key difference: mutating keyword → CommitMutating context, no keyword → CommitGetting context. Compile-time enforcement of read vs. write intent.

2.5 Value Encode/Decode Infrastructure¶

Function wrappers bridge between C++ types (user code) and Viper Values (infrastructure). The encode/decode layer is auto-generated by Kibo.

Decoding (Value → C++)¶

Primitive types:

// args[i] is std::shared_ptr<Value>
auto a = ValueDecoder::decode_int64(ValueInt64::cast(args[0]));  // Value → int64_t
auto b = ValueDecoder::decode_bool(ValueBool::cast(args[1]));    // Value → bool
auto c = ValueDecoder::decode_string(ValueString::cast(args[2])); // Value → string

DSM types:

auto vec = Demo_Vector3::cast(args[0]);  // Value → Demo_Vector3 (concept)

Special case: any:

auto any = Viper::Any::cast(args[0]);  // Value → Any wrapper
auto underlying = any.value();  // Access std::shared_ptr<Value>

Encoding (C++ → Value)¶

Primitive types:

return ValueEncoder::encode(42);           // int64_t → ValueInt64
return ValueEncoder::encode(true);         // bool → ValueBool
return ValueEncoder::encode("hello");      // string → ValueString

DSM types:

return vector3.vpr_value;  // Demo_Vector3 already wraps Value

Pattern: Generated code handles all encoding/decoding complexity. User bridge functions have pure C++ signatures.

3. Functional Decomposition¶

This section breaks down Function Pools into 7 sub-domains, with special emphasis on the Bridges Pattern (3.4), which is the defining characteristic of this domain.

3.1 Function Pool Registry¶

Purpose: Container abstraction for managing collections of callable functions with O(1) lookup.

Key Files: - src/Viper/Viper_FunctionPool.{hpp,cpp} (stateless pools) - src/Viper/Viper_CommitFunctionPool.{hpp,cpp} (commit-aware pools)

Core Responsibilities: 1. Registration: Add functions during pool construction 2. Discovery: Lookup functions by name (throws if not found) 3. Introspection: List all functions in pool 4. Identity: UUID-based global identification

Design Pattern: Registry Pattern - Centralized lookup with unordered_map<string, Function> for O(1) access.

Implementation Details (Viper_FunctionPool.cpp:34-46):

void FunctionPool::add(std::shared_ptr<Function> const & function) {
    auto const & name = function->prototype->name;
    if (_functionByName.contains(name)) {
        throw std::runtime_error("Function '" + name + "' already exists");
    }
    _functionByName[name] = function;
}

std::shared_ptr<Function> FunctionPool::check(std::string const & funcName) const {
    auto it = _functionByName.find(funcName);
    if (it == _functionByName.end()) {
        throw std::runtime_error("Function '" + funcName + "' not found in pool '" + name + "'");
    }
    return it->second;
}

Thread Safety: Read operations (check(), functions()) are thread-safe (const methods, immutable after construction). Write operations (add()) are not synchronized (intended for initialization phase only).

WHY Registry Pattern? - Performance: O(1) lookup vs. linear search - Safety: Duplicate detection prevents shadowing - Discoverability: Services can enumerate available operations - Simplicity: Single responsibility (registration + lookup), no business logic

3.2 Function Abstractions¶

Purpose: Abstract callable interface with validation enforcement and Template Method pattern.

Key Files: - src/Viper/Viper_Function.{hpp,cpp} (base abstraction) - src/Viper/Viper_FunctionPrototype.{hpp,cpp} (signature validation) - src/Viper/Viper_CommitFunction.{hpp,cpp} (commit-aware base) - src/Viper/Viper_CommitGettingFunction.{hpp,cpp} (read-only operations) - src/Viper/Viper_CommitMutatingFunction.{hpp,cpp} (mutating operations)

Core Responsibilities: 1. Validation: Enforce argument count, types, return type via FunctionPrototype 2. Abstraction: Decouple validation from implementation (Template Method) 3. Type Safety: Prevent invalid calls at runtime 4. Documentation: Associate docstrings with functions

Design Pattern: Template Method Pattern - Sealed call() method delegates to virtual checkedCall() after validation.

Critical Implementation (Viper_Function.cpp:19-26):

std::shared_ptr<Value> Function::call(std::vector<std::shared_ptr<Value>> const & args) const {
    prototype->checkArguments(args);  // Step 1: ALWAYS validate

    auto const result = checkedCall(args);  // Step 2: Delegate to user implementation

    return prototype->checkResult(result);  // Step 3: Validate return type
}

WHY Template Method? - Safety: Validation cannot be bypassed (sealed method) - Separation: Infrastructure (validation) separated from business logic (checkedCall) - Consistency: All functions validated uniformly - Testability: Prototype validation is unit-testable independently

Polymorphism via TypeCode::Any: When a parameter is declared any in DSM, FunctionPrototype skips type checking for that parameter:

if (param.type->typeCode() != TypeCode::Any &&
    !param.type->equals(arguments[i]->type())) {
    throw std::runtime_error("Type mismatch");
}

This enables generic functions like isGreater(any a, any b) that work with any Viper type.

3.3 Commit Function Integration¶

Purpose: Extend function abstractions with commit context for mutation tracking and read/write enforcement.

Key Files: - src/Viper/Viper_CommitGettingFunction.{hpp,cpp} (read-only functions) - src/Viper/Viper_CommitMutatingFunction.{hpp,cpp} (mutating functions)

Core Responsibilities: 1. Context Injection: Provide CommitGetting or CommitMutating context to functions 2. Intent Enforcement: Compile-time separation of read vs. write operations 3. Integration: Bridge Function Pools with Commit System's event sourcing

Design Pattern: Strategy Pattern - Context injection enables different behaviors (read vs. write) without changing function interface.

Type Hierarchy:

Function (stateless)
    ↓
CommitFunction (commit-aware base)
    ↓
    ├─ CommitGettingFunction (read-only)
    └─ CommitMutatingFunction (read-write)

CommitGetting vs CommitMutating:

Aspect	CommitGettingFunction	CommitMutatingFunction
Intent	Read-only queries	State mutations
Context Type	`CommitGetting const &`	`CommitMutating const &`
DSM Keyword	(none)	`mutating`
Operations	getNode, query state	applyDelta, modify state
Example	`CommitNode getNode(UUId id)`	`void applyDelta(CommitDelta d)`

Implementation Example (Viper_CommitMutatingFunction.cpp:20-28):

std::shared_ptr<Value> CommitMutatingFunction::call(
    std::shared_ptr<CommitMutating> const & commitMutating,
    std::vector<std::shared_ptr<Value>> const & args
) const {
    prototype->checkArguments(args);  // Validate as usual

    // Pass CommitMutating context to user implementation
    auto const result = checkedCall(*commitMutating, args);

    return prototype->checkResult(result);
}

WHY Strategy Pattern? - Type Safety: Compiler prevents calling mutations with CommitGetting context - Clarity: Function signature explicitly declares intent (read vs. write) - Integration: CommitMutating tracks mutations for event sourcing - Flexibility: Same Function abstraction works for stateless and commit-aware operations

3.4 Bridges Pattern ⭐ (CRITICAL)¶

Purpose: Separate stable user code (business logic) from regenerable wrappers (infrastructure) to enable safe code generation across DSM schema changes.

This is the defining characteristic of Function Pools and cannot be understood from Viper tests alone. Must analyze GE project for real examples.

Key Files: - Templates: templates/cpp/FunctionPool/FunctionPoolBridges.{hpp,cpp}.stg (bridge declarations) - Templates: templates/cpp/FunctionPool/FunctionPools.{hpp,cpp}.stg (wrapper implementations) - GE Example: GE/src/GE/GE_FunctionPoolBridges.{hpp,cpp} (STABLE user code) - GE Example: GE/src/GE/GE_FunctionPools.{hpp,cpp} (REGENERABLE wrappers)

The Problem Bridges Pattern Solves¶

Without Bridges, regenerating code after DSM changes would overwrite user implementations. Developers would need to: 1. Manually backup business logic before regeneration 2. Copy logic back into regenerated files 3. Risk merge conflicts and lost changes

This is error-prone and breaks the DSM-driven workflow.

The Solution: Two Namespaces, Clear Contract¶

Bridges Pattern establishes:

DSM Definition → Kibo Generation → Two Namespaces:
    1. FunctionPoolBridges::<Pool> (STABLE - user implements)
    2. FunctionPools::<Pool> (REGENERABLE - Kibo generates)

Contract: - User never edits FunctionPools namespace (will be overwritten) - Kibo never regenerates FunctionPoolBridges.cpp (only generates .hpp declarations once) - Wrappers call bridges: FunctionPoolBridges::<pool>::<function>(args...)

Complete Workflow Example (GE Project)¶

Step 1: DSM Definition (GE/definitions/Ge/Pool_Tools.dsm):

"""This pool provides access to the various utility functions."""
function_pool Tools {dc9740c9-9d1d-4c1e-9caa-4c8843b91e82} {
    """Return a + b."""
    int64 add(int64 a, int64 b);

    """Return true if a is even."""
    bool isEven(int64 a);

    """Return true if a > b."""
    bool isGreater(any a, any b);
}

Step 2: Kibo Generation (run once or when signatures change):

cd ge && python3 generate.py --cpp --package

Generates: 1. GE/GE_FunctionPoolBridges.hpp (declarations - generated ONCE, user safe to regenerate) 2. GE/GE_FunctionPools.{hpp,cpp} (wrappers - regenerated EVERY TIME)

Step 3: User Implements Bridges (GE/src/GE/GE_FunctionPoolBridges.cpp - STABLE):

// Copyright (c) Digital Substrate 2025, All rights reserved.
// THIS FILE IS NEVER REGENERATED - USER CODE

namespace GE::FunctionPoolBridges::Tools {

    // Pure C++ implementation - no Viper types
    std::int64_t add(std::int64_t a, std::int64_t b) {
        return a + b;  // Business logic
    }

    bool is_even(std::int64_t a) {
        return a % 2 == 0;
    }

    // Polymorphic function - uses Viper::Any
    bool is_greater(Viper::Any const & a, Viper::Any const & b) {
        return Viper::isGreater(a.value()->compare(b.value()));
    }
}

Step 4: Generated Wrappers (GE/src/GE/GE_FunctionPools.cpp - REGENERABLE):

// Copyright (c) Digital Substrate 2025, All rights reserved.
// Generated from Ge.dsmb by kibo-1.2.0.jar
// THIS FILE IS REGENERATED - DO NOT EDIT

#include "GE_FunctionPools.hpp"
#include "GE_FunctionPoolBridges.hpp"  // Calls into user code

namespace GE::FunctionPools {

std::shared_ptr<Viper::FunctionPool> tools() {
    auto pool = std::make_shared<Viper::FunctionPool>(
        Viper::UUId::make("dc9740c9-9d1d-4c1e-9caa-4c8843b91e82"),
        "Tools"
    );

    // Add each function wrapper
    pool->add(F_add::make());
    pool->add(F_is_even::make());
    pool->add(F_is_greater::make());

    return pool;
}

namespace Tools {

    // Wrapper for 'add' function
    class F_add final : public Viper::Function {
    public:
        static std::shared_ptr<F_add> make() {
            std::vector<Viper::FunctionPrototype::Parameter> parameters;
            parameters.push_back({"a", Viper::ValueType::type_int64()});
            parameters.push_back({"b", Viper::ValueType::type_int64()});

            return std::make_shared<F_add>(
                Viper::FunctionPrototype::make("add", std::move(parameters),
                                              Viper::ValueType::type_int64()),
                "Return a + b."
            );
        }

        F_add(std::shared_ptr<Viper::FunctionPrototype> prototype,
              std::string const & documentation)
            : Function(std::move(prototype), documentation) {}

    protected:
        // Template Method implementation - called after validation
        std::shared_ptr<Viper::Value> checkedCall(
            std::vector<std::shared_ptr<Viper::Value>> const & args
        ) const override {
            // DECODE: Viper Values → C++ types
            auto const a = Viper::ValueDecoder::decode_int64(
                Viper::ValueInt64::cast(args.at(0))
            );
            auto const b = Viper::ValueDecoder::decode_int64(
                Viper::ValueInt64::cast(args.at(1))
            );

            // CALL BRIDGE: Pure C++ function (STABLE user code)
            auto const result = FunctionPoolBridges::Tools::add(a, b);

            // ENCODE: C++ type → Viper Value
            return Viper::ValueEncoder::encode(result);
        }
    };

    // Similar wrappers for is_even and is_greater...
}

} // namespace GE::FunctionPools

Step 5: Python Binding (auto-generated by Kibo):

# ge/python/ge/function_pools.py
# Generated from Ge.dsmb by kibo-1.2.0.jar

import dsviper

class Tools:
    """This pool provides access to the various utility functions."""

    def __init__(self, pool: dsviper.FunctionPool):
        self._pool = pool
        self._funcs = pool.funcs

    def add(self, a: int, b: int) -> int:
        """Return a + b."""
        result = self._funcs["add"](a, b)
        return result

    def is_even(self, a: int) -> bool:
        """Return true if a is even."""
        result = self._funcs["is_even"](a)
        return result

def tools() -> Tools:
    """Get the Tools function pool."""
    import ge_cpp  # C++ extension module
    return Tools(ge_cpp.function_pools.tools())

Why Bridges Pattern Works¶

Separation of Concerns: | Concern | Bridges (user) | Wrappers (generated) | |---------|---------------|----------------------| | Signature | Pure C++ types | Viper Values | | Logic | Business algorithms | Value encode/decode | | Stability | STABLE (never regenerated) | REGENERABLE (anytime) | | Dependencies | Standard C++, domain types | Viper infrastructure | | Testing | Unit tests on pure C++ | Integration tests |

Regeneration Safety: 1. User adds new function to DSM → Kibo regenerates wrappers 2. Existing bridge implementations untouched 3. New bridge declarations added to .hpp (user implements in .cpp) 4. No merge conflicts, no lost code

Semantic Boundary: The bridge function signature is the contract. As long as the signature remains compatible (same C++ types), regeneration is safe. If signature changes (e.g., int64 → double), compiler catches incompatibility.

WHY Bridges Pattern? - Stability: User code immune to regeneration - Purity: Business logic stays pure C++ (no Viper pollution) - Productivity: Developers focus on logic, not infrastructure - Safety: Compiler enforces contract between stable and regenerable code - Scalability: Pattern works for 1 function or 1000 functions

Historical Note: This pattern is unique to Function Pools. Other Viper domains (Data, Commit, Database) generate complete implementations because they have no user logic. Function Pools is the only domain where user implements behavior, hence the need for Bridges.

3.5 Code Generation (Kibo Templates)¶

Purpose: Generate C++ wrapper classes, Python bindings, and bridge declarations from DSM definitions.

Key Files (Templates): - templates/cpp/FunctionPool/FunctionPools.{hpp,cpp}.stg (C++ wrappers) - templates/cpp/FunctionPool/FunctionPoolBridges.{hpp,cpp}.stg (C++ bridge declarations) - templates/cpp/FunctionPool/CommitFunctionPools.{hpp,cpp}.stg (Commit-aware wrappers) - templates/cpp/FunctionPool/CommitFunctionPoolBridges.{hpp,cpp}.stg (Commit bridge declarations) - templates/python/function_pools.py.stg (Python wrapper classes)

Core Responsibilities: 1. Parse DSM: Extract function_pool and commit_function_pool definitions 2. Generate Wrappers: Create F_<function> classes implementing Function interface 3. Generate Bridges: Create bridge declarations in FunctionPoolBridges namespace 4. Generate Python: Create typed Python wrapper classes 5. Type Mapping: Map DSM types to C++ types (int64 → std::int64_t, string → std::string, etc.)

Design Pattern: Factory Pattern - Each pool has a factory function (e.g., tools()) that constructs and populates the FunctionPool.

Template Workflow:

DSM (.dsm) → DSM Compiler → Binary (.dsmb) → Kibo → StringTemplate → Generated Code

Key Template Variables (from DSM): - pool.name: Pool name (e.g., "Tools") - pool.uuid: UUID for global identification - pool.documentation: Docstring for pool - pool.functions: List of DSMFunction objects - func.name: Function name (e.g., "add") - func.parameters: List of (name, type) pairs - func.returnType: Return type (DSMTypeReference) - func.documentation: Docstring for function

Example Template Fragment (FunctionPools.cpp.stg):

class F_<func.name> final : public Viper::Function {
public:
    static std::shared_ptr<F_<func.name>> make() {
        std::vector<Viper::FunctionPrototype::Parameter> parameters;
        <func.parameters:{param|
        parameters.push_back({"<param.name>", <param.type.viperType>()});
        }>
        return std::make_shared<F_<func.name>>(
            Viper::FunctionPrototype::make("<func.name>", std::move(parameters),
                                          <func.returnType.viperType>()),
            "<func.documentation>"
        );
    }

protected:
    std::shared_ptr<Viper::Value> checkedCall(...) const override {
        <func.parameters:{param|
        auto const <param.name> = ValueDecoder::decode_<param.type.decoderName>(
            <param.type.castType>::cast(args.at(<i0>))
        );
        }>

        auto const result = FunctionPoolBridges::<pool.name>::<func.cppName>(<func.parameters:{param|<param.name>}; separator=", ">);

        return ValueEncoder::encode(result);
    }
};

WHY Factory Pattern? - Initialization: Pool construction + function registration in one call - Encapsulation: Internal implementation hidden (F_add class is private to .cpp file) - Discoverability: Single entry point per pool (e.g., GE::FunctionPools::tools())

3.6 Value Encode/Decode Infrastructure¶

Purpose: Bridge between C++ types (user code) and Viper Values (infrastructure) in generated wrappers.

Key Files: - templates/cpp/FunctionPool/FunctionPools.cpp.stg (contains encode/decode logic) - Viper ValueEncoder/ValueDecoder utilities (part of Value System)

Core Responsibilities: 1. Decode: Convert Viper Values to C++ types before calling bridge 2. Encode: Convert C++ types to Viper Values after bridge returns 3. Type Safety: Use typed casts (ValueInt64::cast) to enforce types 4. Error Handling: Throw exceptions on type mismatches

Decode Pattern (Value → C++):

// Primitive types
auto const i = ValueDecoder::decode_int64(ValueInt64::cast(args[0]));
auto const b = ValueDecoder::decode_bool(ValueBool::cast(args[1]));
auto const s = ValueDecoder::decode_string(ValueString::cast(args[2]));

// DSM types (concepts/clubs)
auto const vec = Demo_Vector3::cast(args[0]);  // Already wraps Value

// Special: 'any' type
auto const a = Viper::Any::cast(args[0]);  // Wrapper, get Value via a.value()

Encode Pattern (C++ → Value):

// Primitive types
return ValueEncoder::encode(42);          // int64_t → ValueInt64
return ValueEncoder::encode(true);        // bool → ValueBool
return ValueEncoder::encode("hello");     // string → ValueString

// DSM types
return vec.vpr_value;  // Demo_Vector3 contains std::shared_ptr<Value> member

// void returns
return ValueVoid::make();  // Special case for void functions

Type Mapping Table:

DSM Type	C++ Type	Cast Function	Decode Function	Encode Function
`int64`	`std::int64_t`	`ValueInt64::cast`	`decode_int64`	`encode`
`double`	`double`	`ValueDouble::cast`	`decode_double`	`encode`
`bool`	`bool`	`ValueBool::cast`	`decode_bool`	`encode`
`string`	`std::string`	`ValueString::cast`	`decode_string`	`encode`
`uuid`	`Viper::UUId`	`ValueUUId::cast`	`decode_uuid`	`encode`
`blob`	`std::shared_ptr<Viper::Blob>`	`ValueBlob::cast`	(direct)	`blob.vpr_value`
`<Concept>`	`std::shared_ptr<Concept>`	`Concept::cast`	(direct)	`.vpr_value`
`any`	`Viper::Any`	`Any::cast`	(direct)	`.value()`

WHY Encode/Decode Layer? - Purity: Bridge functions have pure C++ signatures (no Value types) - Type Safety: Casts throw exceptions if types don't match - Consistency: Uniform pattern across all generated wrappers - Maintainability: Kibo handles complexity, user never touches encode/decode

3.7 Python Bindings¶

Purpose: Expose function pools to Python with typed interfaces and docstrings.

Key Files: - Templates: templates/python/function_pools.py.stg (Python wrapper generation) - P_Viper: src/P_Viper/P_Viper_Function.cpp (makes Function tp_callable) - P_Viper: src/P_Viper/P_Viper_FunctionPool.cpp (FunctionPool bindings)

Core Responsibilities: 1. Wrapper Classes: Generate Python classes wrapping FunctionPool 2. Typed Methods: Generate methods with type hints and docstrings 3. Callable Protocol: Make Function objects callable from Python (via tp_call) 4. Error Handling: Convert C++ exceptions to Python exceptions

Generated Python Structure:

class Tools:
    """This pool provides access to the various utility functions."""

    def __init__(self, pool: dsviper.FunctionPool):
        self._pool = pool
        self._funcs = pool.funcs  # Dict[str, Function]

    def add(self, a: int, b: int) -> int:
        """Return a + b."""
        result = self._funcs["add"](a, b)  # Calls C++ via tp_call
        return result

P_Viper tp_call Implementation (P_Viper_Function.cpp:156-168):

static PyObject * tp_call(P_Viper_Function * self, PyObject * args, PyObject *) {
    P_TRY {
        // Decode Python args to Viper Values
        P_ViperDecoderContext ctx{"call." + (*self->v)->prototype->name};
        auto const arguments = P_ViperWrapper::checkValue(ctx, args, 0,
                                                          (*self->v)->prototype);

        // Call C++ Function
        auto const result = (*self->v)->call(arguments);

        // Encode Viper Value back to Python
        return P_ViperWrapper::wrapValueOrEncode(result);
    } P_CATCH_P  // Converts C++ exceptions to Python
}

Complete Data Flow (Python → C++ → Bridge):

Python: pool.add(10, 32)
    ↓ tp_call
P_Viper: Decode args → [Value(10), Value(32)]
    ↓ Function::call
Viper: Validate via FunctionPrototype
    ↓ F_add::checkedCall
Generated Wrapper: Decode Values → (10, 32)
    ↓ FunctionPoolBridges::Tools::add(10, 32)
User Bridge: return 10 + 32  // Pure C++
    ↓ return
Generated Wrapper: Encode → Value(42)
    ↓ return
P_Viper: Encode Value → PyLong(42)
    ↓ return
Python: result = 42

WHY Python Bindings Generation? - Type Safety: Python has type hints matching DSM declarations - Documentation: Docstrings from DSM appear in Python help() - Discoverability: IDE autocomplete shows available functions - Ergonomics: pool.add(a, b) is more Pythonic than pool.call("add", [a, b])

4. Developer Usage Patterns (Golden Scenarios)¶

This section demonstrates real-world Function Pools usage through 7 golden scenarios, drawn from test_dsm_functions.py (DSM parsing/introspection) and GE/Service projects (runtime execution).

Scenario 1: Introspecting Function Pool Metadata¶

Source: python/tests/unit/test_dsm_functions.py:61-82 Purpose: Query DSM function pool properties (name, UUID, documentation, functions)

When to use: Service discovery, debugging, documentation generation

Python Code:

import dsviper

# Load DSM definitions
builder = dsviper.DSMBuilder.assemble("test.dsm")
report, dsm_defs, _ = builder.parse()
assert not report.has_error()

# Get function pool
pools = list(dsm_defs.function_pools())
pool = next(p for p in pools if p.name() == "UtilityFunctions")

# Introspect metadata
print(f"Pool Name: {pool.name()}")
print(f"UUID: {pool.uuid()}")
print(f"Documentation: {pool.documentation()}")

# List functions
functions = list(pool.functions())
print(f"Function Count: {len(functions)}")
for func in functions:
    print(f"  - {func.name()}")

Output:

Pool Name: UtilityFunctions
UUID: 12345678-1234-5678-1234-567812345678
Documentation: Pool for utility functions.
Function Count: 5
  - add
  - multiply
  - formatString
  - randomString
  - isGreater

Key Insights: - DSMDefinitions.function_pools() returns iterator of DSMFunctionPool - Each pool has name(), uuid(), documentation(), functions() - Useful for code generators, documentation tools, service registries

Scenario 2: Introspecting Function Prototypes¶

Source: python/tests/unit/test_dsm_functions.py:198-220 Purpose: Inspect function signatures (parameters, return types)

When to use: Validation logic, API documentation, client generation

Python Code:

# Get function from pool
funcs = list(pool.functions())
add_func = next(f for f in funcs if f.name() == "add")

# Get prototype
prototype = add_func.prototype()
print(f"Function: {prototype.name()}")

# Inspect parameters
params = list(prototype.parameters())
print(f"Parameters ({len(params)}):")
for param_name, param_type in params:
    type_ref = param_type  # DSMTypeReference
    print(f"  {param_name}: {type_ref.name()}")

# Inspect return type
return_type = prototype.return_type()
print(f"Returns: {return_type.name()}")

Output:

Function: add
Parameters (2):
  a: int64
  b: int64
Returns: int64

Key Insights: - DSMFunction.prototype() returns DSMFunctionPrototype - prototype.parameters() returns [(name: str, type: DSMTypeReference)] - DSMTypeReference.name() gives type name (e.g., "int64", "string", "Vector3") - Useful for generating client stubs, OpenAPI specs, RPC schemas

Scenario 3: Commit Function Pool Introspection¶

Source: python/tests/unit/test_dsm_functions.py:276-298 Purpose: Distinguish commit-aware function pools and mutation intent

When to use: Commit system integration, permission checks

Python Code:

# Get commit function pool
commit_pools = list(dsm_defs.commit_function_pools())
pool = next(p for p in commit_pools if p.name() == "CommitOperations")

# Iterate functions
for func in pool.functions():
    print(f"{func.name()}: ", end="")

    # Check if mutating
    if func.is_mutating():
        print("MUTATING (writes)")
    else:
        print("GETTING (read-only)")

Output:

getNode: GETTING (read-only)
listNodes: GETTING (read-only)
applyDelta: MUTATING (writes)
deleteNode: MUTATING (writes)

Key Insights: - DSMCommitFunctionPool separate from DSMFunctionPool - DSMCommitFunction.is_mutating() checks mutating keyword in DSM - mutating → CommitMutatingFunction (can write) - no keyword → CommitGettingFunction (read-only) - Runtime enforces via context type (CommitMutating vs CommitGetting)

Scenario 4: Complete Function Pool Workflow (GE Project)¶

Source: GE project (/Volumes/DigitalSubstrate/com.digitalsubstrate.ge/) Purpose: End-to-end workflow from DSM → C++ → Python

When to use: Adding new business logic to Viper project

Step 1: Define DSM (definitions/Ge/Pool_Tools.dsm):

"""This pool provides access to the various utility functions."""
function_pool Tools {dc9740c9-9d1d-4c1e-9caa-4c8843b91e82} {
    """Return a + b."""
    int64 add(int64 a, int64 b);

    """Return a random string of given size."""
    string randomString(int64 size);
}

Step 2: Generate Code:

cd /Volumes/DigitalSubstrate/com.digitalsubstrate.ge
cd definitions && python3 generate.py --cpp --package && cd ..

Generates: - ge/GE/GE_FunctionPoolBridges.hpp (bridge declarations) - ge/GE/GE_FunctionPools.{hpp,cpp} (wrappers) - ge/python/ge/function_pools.py (Python bindings)

Step 3: Implement Bridges (src/GE/GE_FunctionPoolBridges.cpp):

#include "GE_FunctionPoolBridges.hpp"
#include <random>

namespace GE::FunctionPoolBridges::Tools {

    std::int64_t add(std::int64_t a, std::int64_t b) {
        return a + b;  // Pure C++ business logic
    }

    std::string random_string(std::int64_t size) {
        static const char chars[] = "abcdefghijklmnopqrstuvwxyz0123456789";
        std::random_device rd;
        std::mt19937 gen(rd());
        std::uniform_int_distribution<> dis(0, sizeof(chars) - 2);

        std::string result(size, '\0');
        for (std::int64_t i = 0; i < size; ++i) {
            result[i] = chars[dis(gen)];
        }
        return result;
    }

} // namespace GE::FunctionPoolBridges::Tools

Step 4: Build C++ Extension:

cd ge_wheel
python3 build.py -id  # -i=install, -d=dev mode
cd ..

Step 5: Use from Python:

import ge

# Get function pool
pool = ge.function_pools.tools()

# Call functions
result = pool.add(10, 32)
print(f"10 + 32 = {result}")  # Output: 42

random_str = pool.random_string(16)
print(f"Random: {random_str}")  # Output: Random: k3p9zlq2m8f7x1w4

Key Insights: - User only implements Step 3 (pure C++ business logic) - Steps 1, 2, 4 are mechanical (DSM, generation, build) - Step 5 is seamless (Python calls C++ transparently) - Bridge code (Step 3) never regenerated - safe from DSM changes

Scenario 5: Polymorphic Functions with `any` Type¶

Source: python/tests/unit/test_dsm_functions.py:341-363 + GE project Purpose: Implement functions that work with any Viper type

When to use: Generic utilities (comparison, serialization, formatting)

DSM Definition:

function_pool Generic {uuid} {
    """Return true if a > b (works for any comparable type)."""
    bool isGreater(any a, any b);

    """Convert any value to JSON string."""
    string toJson(any value);
}

Bridge Implementation:

#include "GE_FunctionPoolBridges.hpp"
#include "Viper_Any.hpp"

namespace GE::FunctionPoolBridges::Generic {

    bool is_greater(Viper::Any const & a, Viper::Any const & b) {
        // Access underlying Value
        auto val_a = a.value();
        auto val_b = b.value();

        // Use Value::compare() for polymorphic comparison
        auto cmp = val_a->compare(val_b);
        return Viper::isGreater(cmp);
    }

    std::string to_json(Viper::Any const & value) {
        // Use Value::toJSON() polymorphic method
        return value.value()->toJSON();
    }

} // namespace GE::FunctionPoolBridges::Generic

Python Usage:

import ge

pool = ge.function_pools.generic()

# Works with integers
assert pool.is_greater(42, 10) == True

# Works with strings
assert pool.is_greater("zebra", "apple") == True

# Works with DSM concepts
vec1 = ge.Vector3(1.0, 2.0, 3.0)
vec2 = ge.Vector3(0.5, 1.0, 1.5)
# Comparison uses Vector3::compare() implementation

# Serialize to JSON
json_str = pool.to_json(vec1)
print(json_str)  # Output: {"x": 1.0, "y": 2.0, "z": 3.0}

Key Insights: - any type bypasses FunctionPrototype validation (TypeCode::Any special case) - Bridge receives Viper::Any wrapper (not raw Value pointer) - any.value() returns std::shared_ptr<Value> for polymorphic operations - Enables generic algorithms without template code in DSM

Scenario 6: Commit-Aware Function Implementation¶

Source: Service project + Exp project commit pool examples Purpose: Implement functions that read/modify commit state

When to use: Collaborative editing, event sourcing, CRDT operations

DSM Definition (definitions/Service/Pool_Commit.dsm):

commit_function_pool Commit {550e8400-e29b-41d4-a716-446655440000} {
    """Get commit node by UUID (read-only)."""
    CommitNode getNode(UUId id);

    """List all commit nodes (read-only)."""
    CommitNode[] listNodes();

    """Apply delta to commit state (mutating)."""
    void mutating applyDelta(CommitDelta delta);
}

Bridge Implementation:

#include "Service_CommitFunctionPoolBridges.hpp"

namespace Service::FunctionPoolBridges::Commit {

    // GETTING function - read-only context
    std::shared_ptr<Service::CommitNode> get_node(
        Viper::CommitGetting const & cg,
        Viper::UUId const & id
    ) {
        // Access commit state (read-only)
        auto node = cg.getNode(id);
        if (!node) {
            throw std::runtime_error("Node not found: " + id.toString());
        }
        return Service::CommitNode::cast(node);
    }

    // GETTING function - read-only context
    std::vector<std::shared_ptr<Service::CommitNode>> list_nodes(
        Viper::CommitGetting const & cg
    ) {
        auto all_nodes = cg.allNodes();
        std::vector<std::shared_ptr<Service::CommitNode>> result;
        for (auto const & node : all_nodes) {
            result.push_back(Service::CommitNode::cast(node));
        }
        return result;
    }

    // MUTATING function - read-write context
    void apply_delta(
        Viper::CommitMutating const & cm,
        std::shared_ptr<Service::CommitDelta> const & delta
    ) {
        // Mutation tracked in commit history
        cm.applyDelta(delta);

        // Additional business logic...
        auto affected_nodes = delta->affectedNodes();
        for (auto const & node_id : affected_nodes) {
            auto node = cm.getNode(node_id);
            // ... process node mutation ...
        }
    }

} // namespace Service::FunctionPoolBridges::Commit

Python Usage:

import service
from dsviper import UUId, CommitDatabase

# Create commit database
db = CommitDatabase.make(":memory:", service.definitions())

# Get commit function pool
pool = service.function_pools.commit()

# Perform mutations in transaction
with db.transaction() as txn:
    commit_mutating = txn.commit_mutating()

    # Call mutating function
    delta = service.CommitDelta()
    delta.add_node(...)
    pool.apply_delta(commit_mutating, delta)  # Tracked in commit history

# Perform queries (read-only)
with db.transaction() as txn:
    commit_getting = txn.commit_getting()

    # Call getting functions
    node = pool.get_node(commit_getting, node_id)
    all_nodes = pool.list_nodes(commit_getting)

Key Insights: - GETTING functions: CommitGetting const & context (read-only) - MUTATING functions: CommitMutating const & context (can write) - DSM mutating keyword → CommitMutatingFunction C++ class - Python must pass context explicitly (via transaction) - Mutations automatically tracked by Commit System event sourcing

Scenario 7: Testing Function Pools (DSM Validation)¶

Source: python/tests/unit/test_dsm_functions.py (381 lines, 30+ tests) Purpose: Validate DSM function pool definitions

When to use: CI/CD, DSM schema validation, regression testing

Test Structure:

import unittest
from dsviper import DSMBuilder

class TestDSMFunctions(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        """Load DSM definitions once for all tests."""
        builder = DSMBuilder.assemble("test_comprehensive.dsm")
        report, cls.dsm_defs, _ = builder.parse()
        if report.has_error():
            for error in report.errors():
                print(error)
            raise RuntimeError("DSM parsing failed")

    def _get_function_pool(self, name: str):
        """Helper to get function pool by name."""
        pools = list(self.dsm_defs.function_pools())
        return next(p for p in pools if p.name() == name)

    def _get_function(self, pool, name: str):
        """Helper to get function by name."""
        funcs = list(pool.functions())
        return next(f for f in funcs if f.name() == name)

    def test_function_pool_basic_properties(self):
        """Validate pool name, UUID, documentation."""
        pool = self._get_function_pool("UtilityFunctions")

        self.assertEqual(pool.name(), "UtilityFunctions")
        self.assertIsNotNone(pool.uuid())
        self.assertTrue(len(pool.documentation()) > 0)

    def test_function_prototype_parameters(self):
        """Validate function parameters (count, names, types)."""
        pool = self._get_function_pool("UtilityFunctions")
        func = self._get_function(pool, "add")

        prototype = func.prototype()
        params = list(prototype.parameters())

        self.assertEqual(len(params), 2)
        param_a_name, param_a_type = params[0]
        param_b_name, param_b_type = params[1]

        self.assertEqual(param_a_name, "a")
        self.assertEqual(param_a_type.name(), "int64")
        self.assertEqual(param_b_name, "b")
        self.assertEqual(param_b_type.name(), "int64")

    def test_function_return_type(self):
        """Validate function return type."""
        pool = self._get_function_pool("UtilityFunctions")
        func = self._get_function(pool, "add")

        prototype = func.prototype()
        return_type = prototype.return_type()

        self.assertEqual(return_type.name(), "int64")

    def test_commit_function_pool_mutating(self):
        """Validate commit function mutation intent."""
        commit_pools = list(self.dsm_defs.commit_function_pools())
        pool = next(p for p in commit_pools if p.name() == "CommitOps")

        func_getting = self._get_function(pool, "getNode")
        self.assertFalse(func_getting.is_mutating())

        func_mutating = self._get_function(pool, "applyDelta")
        self.assertTrue(func_mutating.is_mutating())

    def test_polymorphic_any_parameter(self):
        """Validate 'any' type parameters."""
        pool = self._get_function_pool("Generic")
        func = self._get_function(pool, "isGreater")

        prototype = func.prototype()
        params = list(prototype.parameters())

        # Both parameters should be 'any' type
        param_a_name, param_a_type = params[0]
        param_b_name, param_b_type = params[1]

        self.assertEqual(param_a_type.name(), "any")
        self.assertEqual(param_b_type.name(), "any")

if __name__ == '__main__':
    unittest.main()

Running Tests:

cd python/tests/unit
python3 -m unittest test_dsm_functions.py -v

Output:

test_function_pool_basic_properties ... ok
test_function_prototype_parameters ... ok
test_function_return_type ... ok
test_commit_function_pool_mutating ... ok
test_polymorphic_any_parameter ... ok

Ran 30 tests in 0.123s
OK

Key Insights: - Tests focus on DSM parsing/introspection, not runtime execution - Use DSMBuilder.assemble() to load DSM definitions - Validate pool metadata (name, UUID, docs) - Validate function prototypes (parameters, return types) - Validate commit function mutation intent - No runtime tests in Viper repo - see GE/Service for integration tests

5. Technical Constraints & Design Patterns¶

5.1 Performance Characteristics¶

Function Pool Registry: - Lookup: O(1) via std::unordered_map<string, Function> - Insertion: O(1) during pool construction (no dynamic growth) - Memory: Linear in number of functions (small overhead per function)

Function Invocation: - Validation: O(n) where n = parameter count (checks each parameter type) - Encode/Decode: O(1) per primitive, O(depth) for nested DSM types - Python Bridge: 3 copies (Python → Value → C++ and back) - Optimization: ValueWrappedPyObject for zero-copy when possible

Bottlenecks: - Type validation on every call (cannot be disabled, by design) - Value encoding for complex DSM types (nested concepts/arrays) - Python C/API overhead (tp_call, argument parsing)

Optimization opportunities: - Cache FunctionPrototype validation (currently per-call) - Batch function calls to amortize Python/C++ crossing overhead - Use ValueWrappedPyObject for zero-copy when Python object wraps Value

5.2 Thread Safety¶

Thread-Safe Operations: - FunctionPool::check() - const method, immutable after construction - FunctionPool::functions() - returns copy of function list - Function::call() - const method, no shared mutable state - FunctionPrototype validation - stateless, operates on arguments

Not Thread-Safe: - FunctionPool::add() - intended for initialization only, no synchronization - Bridge implementations - user responsibility to ensure thread safety

Concurrency Model: - Immutable pools: Once constructed, function pools are immutable (read-only) - Stateless functions: Functions should be stateless (no mutable members) - User bridges: If bridge logic requires state, user must provide synchronization

Best Practice:

// ❌ BAD: Mutable state in bridge (not thread-safe)
namespace Bridges::Counter {
    static int64_t counter = 0;  // Shared mutable state!

    int64_t increment() {
        return ++counter;  // Race condition
    }
}

// ✅ GOOD: Stateless bridge or externalized state
namespace Bridges::Counter {
    int64_t increment(std::int64_t current) {
        return current + 1;  // Pure function
    }
}

// ✅ ACCEPTABLE: Bridge with synchronized state
namespace Bridges::Counter {
    static std::atomic<int64_t> counter{0};  // Atomic for thread safety

    int64_t increment() {
        return counter.fetch_add(1, std::memory_order_relaxed);
    }
}

5.3 Error Handling¶

Validation Errors (FunctionPrototype): - Wrong argument count: std::runtime_error("Expected N arguments, got M") - Type mismatch: std::runtime_error("Type mismatch for parameter 'x'") - Return type mismatch: std::runtime_error("Return type mismatch")

Registry Errors (FunctionPool): - Function not found: std::runtime_error("Function 'x' not found in pool 'y'") - Duplicate function: std::runtime_error("Function 'x' already exists")

Bridge Implementation Errors: - User exceptions: Propagate through wrapper → P_Viper → Python - Uncaught exceptions: Terminate process (C++ semantics)

Python Error Mapping:

// P_Viper exception handling (P_CATCH_P macro)
P_TRY {
    auto result = (*self->v)->call(arguments);
    return P_ViperWrapper::wrapValueOrEncode(result);
} P_CATCH_P  // Converts C++ exceptions to Python exceptions

Python receives:

try:
    result = pool.add(10, "invalid")  # Type mismatch
except RuntimeError as e:
    print(e)  # "Type mismatch for parameter 'b': expected int64, got string"

5.4 Design Patterns Summary¶

Function Pools leverages 6 key design patterns:

1. Registry Pattern (FunctionPool) - Purpose: Centralized function lookup with O(1) access - Implementation: std::unordered_map<string, Function> - Benefit: Discoverability for Services, Python bindings

2. Template Method Pattern (Function) - Purpose: Enforce validation before user implementation - Implementation: Sealed call() method, virtual checkedCall() - Benefit: Validation cannot be bypassed, safety guaranteed

3. Strategy Pattern (CommitFunction) - Purpose: Context injection (CommitGetting vs CommitMutating) - Implementation: Different context types passed to bridge - Benefit: Compile-time enforcement of read vs. write intent

4. Factory Pattern (FunctionPools generation) - Purpose: Encapsulate pool construction and function registration - Implementation: Generated tools() factory functions - Benefit: Single entry point, internal classes hidden

5. Adapter Pattern (Function wrappers) - Purpose: Bridge Viper Values ↔ C++ types - Implementation: Generated checkedCall() with encode/decode - Benefit: User code stays pure C++, no Viper pollution

6. Bridges Pattern (FunctionPoolBridges) ⭐ - Purpose: Separate stable user code from regenerable infrastructure - Implementation: Two namespaces (Bridges stable, Pools regenerable) - Benefit: Safe regeneration, semantic stability

Pattern Relationships:

Registry (discovery) → Factory (construction) → Adapter (wrappers)
                                                     ↓
                                         Template Method (validation)
                                                     ↓
                                              Bridges (user code)
                                                     ↑
                                         Strategy (commit context)

5.5 Limitations and Known Issues¶

1. No Function Overloading - Limitation: DSM does not support function overloading (same name, different signatures) - Reason: Python bindings require unique names per function - Workaround: Use descriptive names (e.g., addInt, addDouble, addVector)

2. No Variadic Functions - Limitation: Functions must have fixed number of parameters - Reason: FunctionPrototype validates exact parameter count - Workaround: Use arrays (int64[] values) for variable-length inputs

3. No Default Parameters - Limitation: DSM does not support default/optional parameters - Reason: Prototype validation expects exact argument count - Workaround: Create multiple functions (e.g., formatSimple, formatAdvanced)

4. No Generic Type Parameters - Limitation: DSM does not support generic/template types (e.g., List<T>) - Reason: Viper types are runtime-resolved, not compile-time templates - Workaround: Use any type for polymorphism (runtime type checking)

5. Bridges Pattern Requires Discipline - Limitation: Kibo generates bridge .hpp declarations, but user must implement .cpp - Risk: Forgetting to implement bridge causes linker errors (not caught until build) - Mitigation: Build system should fail fast with clear error message

6. Limited Test Coverage in Viper Repo - Limitation: Only DSM parsing tests exist, no runtime execution tests - Reason: Function Pools requires user implementations (bridges) - Impact: Cannot validate runtime behavior without real projects (GE, Service) - Mitigation: GE/Service projects serve as integration test suites

5.6 Code Generation Constraints¶

DSM Requirements: - UUID uniqueness: Each function pool must have globally unique UUID - Valid identifiers: Function names must be valid C++ and Python identifiers - Type compatibility: Parameters/return types must exist in DSM definitions

Kibo Template Constraints: - No conditional generation: Cannot conditionally include functions based on config - Fixed namespace structure: Project::FunctionPools::<Pool> and Project::FunctionPoolBridges::<Pool> - Name mangling: DSM names converted to C++ (camelCase → snake_case)

Python Binding Constraints: - Type hint limitations: Python type hints approximate C++ types (e.g., int for int64) - No operator overloading: Functions must be explicit methods (cannot use +, -, etc.) - GIL contention: Python Global Interpreter Lock serializes C++ function calls

6. Cross-References & Dependencies¶

6.1 Dependencies (What Function Pools Requires)¶

Foundation Layer 0: Type and Value System - Purpose: Type validation, value encoding/decoding, polymorphism - Files: - Viper_Type.{hpp,cpp} - Type hierarchy and TypeCode enumeration - Viper_Value.{hpp,cpp} - Abstract value interface - Viper_ValueInt64.{hpp,cpp}, Viper_ValueString.{hpp,cpp}, etc. - Concrete value types - Viper_Any.{hpp,cpp} - Wrapper for any type parameters - Integration: - FunctionPrototype uses Type for parameter/return validation - Function wrappers use Value for argument passing - any parameters use Viper::Any wrapper - Documentation: doc/domains/Type_And_Value_System.md

Functional Layer 1: Commit System (optional) - Purpose: Commit-aware function pools with mutation tracking - Files: - Viper_CommitGetting.{hpp,cpp} - Read-only commit context - Viper_CommitMutating.{hpp,cpp} - Read-write commit context - Viper_CommitFunction.{hpp,cpp} - Base for commit-aware functions - Integration: - CommitFunctionPool requires CommitFunction hierarchy - CommitGettingFunction/CommitMutatingFunction inject context - Bridge functions receive context parameter - Documentation: doc/domains/Commit_System.md

Functional Layer 1: DSM (for code generation) - Purpose: Parse DSM definitions to extract function pool metadata - Files: - DSMP_FunctionPool.{hpp,cpp} - DSM AST for function pools - DSMFunctionPool.{hpp,cpp} - Runtime representation - DSMBuilder.{hpp,cpp} - DSM parser/validator - Integration: - Kibo uses DSMFunctionPool to generate wrappers - Tests use DSMBuilder to validate DSM definitions - Documentation: doc/domains/Dsm.md

6.2 Dependents (What Depends on Function Pools)¶

Functional Layer 1: Services - Purpose: Remote function invocation over RPC - Integration: - Services discovers FunctionPools via registry - RPC layer serializes function calls (name + arguments) - Remote Services invoke FunctionPool::check() to resolve function - Coupling: Unidirectional (Services → Function Pools) - Documentation: (Pending - Services not yet documented)

All Viper Projects (GE, Service, etc.) - Purpose: Expose C++ business logic to Python - Integration: - Projects define function_pool in DSM - Kibo generates wrappers and bridges - Python imports generated function_pools module - Examples: - GE/definitions/Ge/Pool_*.dsm (7 pools in GE project) - Service/definitions/Service/Pool_*.dsm (1 pool in Service project)

Getting Started Guides: - doc/Getting_Started.md - DSM language tutorial (includes function_pool syntax) - doc/Getting_Started_With_Templated_Feature.md - Code generation workflow

Core Reference: - doc/DSM.md - DSM language specification (Section: function_pool and commit_function_pool) - doc/Kibo.md - Code generator manual (Section: FunctionPool templates) - doc/Kibo_Template_Model.md - Template model and StringTemplate usage

Domain Documentation: - doc/domains/Type_And_Value_System.md - Type validation, Value encoding - doc/domains/Commit_System.md - Commit context injection, mutation tracking - doc/domains/Dsm.md - DSM parsing, semantic validation

Internals (for Viper developers): - doc/Internal_Viper.md - Function abstraction architecture - doc/Internal_P_Viper.md - Python binding implementation (tp_call, argument decoding) - doc/Internal_Kibo.md - StringTemplate rendering for FunctionPool generation

6.4 Real-World Examples¶

GE Project (/Volumes/DigitalSubstrate/com.digitalsubstrate.ge/): - Purpose: Geometric engine with 7 function pools - Pools: - Tools - Utility functions (add, random_string, etc.) - Mesh - Mesh operations (merge, subdivide, etc.) - Geometry - Geometric calculations (distance, intersect, etc.) - Color - Color conversions (rgb_to_hsv, etc.) - Random - Random generation (uniform, normal, etc.) - String - String utilities (format, split, etc.) - Math - Mathematical functions (sin, cos, sqrt, etc.) - Files: - definitions/Ge/Pool_*.dsm - DSM definitions - src/GE/GE_FunctionPoolBridges.cpp - User implementations (STABLE) - ge/GE/GE_FunctionPools.cpp - Generated wrappers (REGENERABLE)

Service Project (/Volumes/DigitalSubstrate/com.digitalsubstrate.service/): - Purpose: Remote service infrastructure with 1 commit function pool - Pools: - Commit - Commit operations (getNode, applyDelta, etc.) - Files: - definitions/Service/Pool_Commit.dsm - DSM definition - src/Service/Service_CommitFunctionPoolBridges.cpp - User implementations - service/Service/Service_CommitFunctionPools.cpp - Generated wrappers

Exp Project (/Volumes/DigitalSubstrate/com.digitalsubstrate.viper/exp/): - Purpose: Experimental features for Viper testing - Pools: - CommitFunctionPools::commit() - Commit testing utilities - Files: - Exp/Exp_CommitFunctionPools_Commit.hpp - Factory declaration - exp/Exp.dsmb - Binary DSM definitions

6.5 Migration Guide: Adding New Function Pool¶

Scenario: You want to add new business logic to an existing Viper project.

Step 1: Define DSM (definitions/<Project>/Pool_<Name>.dsm):

"""Documentation for your pool."""
function_pool MyPool {<uuid>} {
    """Documentation for function."""
    <return_type> myFunction(<param_type> paramName);
}

Step 2: Generate Code:

cd definitions && python3 generate.py --cpp --package && cd ..

Generates (if first time): - <project>/<Project>/<Project>_FunctionPoolBridges.hpp - Bridge declarations - Creates empty file: src/<Project>/<Project>_FunctionPoolBridges.cpp (if doesn't exist)

Always regenerates: - <project>/<Project>/<Project>_FunctionPools.{hpp,cpp} - Wrappers - <project>/python/<project>/function_pools.py - Python bindings

Step 3: Implement Bridge (src/<Project>/<Project>_FunctionPoolBridges.cpp):

#include "<Project>_FunctionPoolBridges.hpp"

namespace <Project>::FunctionPoolBridges::MyPool {

    <return_cpp_type> my_function(<param_cpp_type> param_name) {
        // Pure C++ implementation
        return ...;
    }

} // namespace

Step 4: Build and Install:

# C++ project build
mkdir build && cd build && cmake .. && cmake --build . && cd ..

# Python wheel build
cd <project>_wheel && python3 build.py -id && cd ..

Step 5: Use from Python:

import <project>

pool = <project>.function_pools.my_pool()
result = pool.my_function(arg)

Important Notes: - UUID generation: Use python3 -c "import uuid; print(uuid.uuid4())" for globally unique UUIDs - Bridges stability: Never edit <Project>_FunctionPools.cpp (will be overwritten) - Type mapping: Ensure DSM types exist in definitions (concepts, primitives) - Testing: Add integration tests in project's Python test suite

Document Change History¶

v1.0 (2025-11-14) - Initial documentation - Comprehensive coverage of Function Pools domain (6 documented, 8 pending = 43% coverage) - 7 sub-domains documented with emphasis on Bridges Pattern - 7 golden scenarios from test_dsm_functions.py and GE/Service projects - Real-world examples from GE project (critical for understanding Bridges Pattern) - 6 design patterns analyzed (Template Method, Factory, Registry, Adapter, Bridges, Strategy)

Future Improvements: - Add performance benchmarks for function invocation overhead - Document RPC integration once Services domain is documented - Add migration examples for Python → C++ function translation - Expand test coverage recommendations (integration tests)

END OF DOCUMENT

Function Pools¶

1. Purpose & Motivation¶

1.1 What Problem Does Function Pools Solve?¶

The Challenge¶

The Solution¶

1.2 Why is Function Pools Needed?¶

Why Not Merge With Services?¶

1.3 Primary Use Cases¶

Use Case 1: Python Business Logic Bindings¶

Use Case 2: Commit-Aware Mutations¶

Use Case 3: Polymorphic Functions¶

1.4 Architecture Position¶

2. Domain Overview¶

2.1 Scope and Boundaries¶

2.2 Key Concepts¶

2.2.1 Function Pool¶

2.2.2 Commit Function Pool¶

2.2.3 Function¶

2.2.4 Function Prototype¶

2.2.5 Bridges Pattern ⭐¶

2.3 High-Level Architecture¶

2.4 Two Pool Types: FunctionPool vs CommitFunctionPool¶

Why Two Separate Types?¶

Example: Stateless Function¶

Example: Commit-Aware Function¶

2.5 Value Encode/Decode Infrastructure¶

Decoding (Value → C++)¶

Encoding (C++ → Value)¶

3. Functional Decomposition¶

3.1 Function Pool Registry¶

3.2 Function Abstractions¶

3.3 Commit Function Integration¶

3.4 Bridges Pattern ⭐ (CRITICAL)¶

The Problem Bridges Pattern Solves¶

The Solution: Two Namespaces, Clear Contract¶

Complete Workflow Example (GE Project)¶

Why Bridges Pattern Works¶

3.5 Code Generation (Kibo Templates)¶

3.6 Value Encode/Decode Infrastructure¶

3.7 Python Bindings¶

4. Developer Usage Patterns (Golden Scenarios)¶

Scenario 1: Introspecting Function Pool Metadata¶

Scenario 2: Introspecting Function Prototypes¶

Scenario 3: Commit Function Pool Introspection¶

Scenario 4: Complete Function Pool Workflow (GE Project)¶

Scenario 5: Polymorphic Functions with any Type¶

Scenario 6: Commit-Aware Function Implementation¶

Scenario 7: Testing Function Pools (DSM Validation)¶

5. Technical Constraints & Design Patterns¶

5.1 Performance Characteristics¶

5.2 Thread Safety¶

5.3 Error Handling¶

5.4 Design Patterns Summary¶

5.5 Limitations and Known Issues¶

5.6 Code Generation Constraints¶

6. Cross-References & Dependencies¶

6.1 Dependencies (What Function Pools Requires)¶

6.2 Dependents (What Depends on Function Pools)¶

6.3 Related Documentation¶

6.4 Real-World Examples¶

6.5 Migration Guide: Adding New Function Pool¶

Document Change History¶

Scenario 5: Polymorphic Functions with `any` Type¶