Federated Learning & Distributed Compute

Froglet turns horizontal and vertical federated learning (FL) and distributed map-reduce into cryptographically auditable, privacy-governed, Byzantine-resilient, hardware-accelerated, and per-round payable compute transactions. Instead of trusting unverified JSON/gRPC weight updates, opening inbound firewall ports at hospitals or banks, or embedding datasets in guest code, each silo runs a Froglet provider node that mounts its resident data via capability policy ([sqlite.handles.cohort] for local SQLite EHR/AML views, [http] for local HL7 FHIR R4 servers, and [gpu] for compute.gpu.tensor host GPU tensor acceleration), executes deterministic WebAssembly (froglet.wasm.host_json.v1), and emits a hash-linked 5-artifact (free) or 6-artifact (Lightning) signed chain for every round.

The reference implementation in examples/fl_distributed_demo.py, examples/wasm-services/fl-compute/, python/froglet_fl.py, src/wasm_gpu.rs, src/gpu.rs, python/tests/test_froglet_fl.py, and examples/test_fl_distributed_demo.py is verified at TRL 7–8 (Enterprise Multi-Consortium System Demonstrated in Live Multi-Node Cluster) across disposable froglet-node processes (73 verified deals, 365 verified signed artifacts), supporting Flower (flwr), NVIDIA FLARE (nvflare), Cryptographic Secure Aggregation (SecAgg), Rényi DP (RDP) Privacy Ledgers, Byzantine Defense + Shapley Value Settlement, MLP & LoRA adapters, 3-Layer Deterministic GPU Acceleration (NVIDIA CUDA & Apple Metal MPS), Storage-Agnostic Multi-Domain Schemas (OMOP CDM v5.4, HL7 FHIR R4, Financial AML), Cross-Silo Vertical FL (PSI + Split Learning), and zero-inbound-port WebSocket Relay tunneling.

---

Benefits of Using Froglet with Enterprise Federated Learning

Benefit	What Froglet Adds	Why It Matters for Enterprise FL
Signed Round Receipts & Cohort Merkle Roots	BIP-340 Schnorr receipt + binary SHA-256 cohort_commitment_hash per round	Eliminates unverified “trust me, I trained” gradient updates and pins the exact cohort
In-Wasm Cryptographic SecAgg	Pairwise secp256k1 ECDH + HKDF-SHA256 additive masking in	Plaintext weights are omitted from Wasm output; coordinator only sees the unmasked sum
Silo-Enforced RDP Privacy Ledger	Rényi DP () accountant + hash-chained privacy_ledger_entry	Silos enforce cumulative budgets inside Wasm and reject over-budget deals
Byzantine Defense & Shapley Settlement	CoordinateMedian, TrimmedMean, MultiKrum, L2 clipping + exact Shapley ()	Rejects poisoned updates and gates Lightning satoshi payouts on positive marginal validation gain
Multi-Architecture & Vertical FL	logistic_regression, mlp_2layer (ReLU MLP), lora_adapter (), psi_intersect & vertical_fl_step	Supports both non-linear/adapter fine-tuning and cross-industry hospital+bank vertical split learning
3-Layer Deterministic GPU Acceleration	compute.gpu.tensor Wasm host bridge + PyTorch cuda/mps GpuTensorAccelerator + GpuDevicePool	Hardware-accelerates MLP/LoRA/SplitNN/Multi-Krum on NVIDIA CUDA and Apple Metal MPS with bit-exact fixed-point output_hash parity
Storage-Agnostic Domain Adapters	Unified sqlite (db.query), http (http.fetch), and inline adapters	Out-of-the-box support for clinical_cohort, omop_cdm_v5_4, fhir_r4_bundle, and financial_aml
Zero Inbound Ports & Configurable Quotas	Outbound WebSocket Relay (docs/RELAY.md v1) + configurable FROGLET_WASM_* ceilings	Hospitals and banks participate with 0 inbound firewall ports and tunable memory/fuel caps

---

Provable TRL 6 + TRL 7–8 Enterprise Verification Pillars

examples/fl_distributed_demo.py verifies all 8 pillars (TRL 6 Pillars 1–4 + Enterprise TRL 7–8 Pillars 5–8) end-to-end across live multi-process froglet-node instances:

Pillar	Architectural Mechanism	Live Verification Result
1. Live Host SQLite Capability (TRL 6)	fl_compute.wasm (176,842 B, SHA-256 fc9e4eb7...) contains 0 embedded patient records and imports only froglet_host::call_json. Each silo provisions an isolated SQLite database (silo_1_ehr.db, silo_2_ehr.db, silo_3_ehr.db) mounted via FROGLET_WASM_POLICY_PATH.	Cross-silo SQLite isolation verified: querying silo_2 inside silo_1’s Wasm sandbox returns 0 rows and fails closed.
2. Per-Sample L2 Clipping & DP-SGD (TRL 6)	Per-record gradients are clipped in L2 norm (max_grad_norm_micro = 120,000) and perturbed with deterministic SplitMix64 Irwin-Hall Gaussian noise (dp_noise_multiplier_micro = 150,000, dp_seed) inside fl_compute.wasm.	600 per-sample gradients clipped across 3 silos (200 per silo); DP-SGD global model reaches 100.0% (60/60) unseen holdout accuracy (loss = 0.038944).
3. 360-Record Non-IID Benchmark + Holdout (TRL 6)	360 clinical records across 3 hospital SQLite databases (100 train + 20 holdout per silo): Silo 1 (83.3% negative), Silo 2 (83.3% positive), and Silo 3 (50/50 balanced).	5-round FedAvg improves train accuracy 49.0% → 100.0% (loss 0.250000 → 0.040046) and unseen holdout accuracy 55.0% → 100.0% (60/60, loss 0.040038), outperforming isolated skewed silos (80.0% and 78.3%).
4. Zero-Inbound-Port Outbound Relay (TRL 6)	A hospital silo with 0 inbound ports dials outbound via WebSocket (FROGLET_RELAY_URL), completes BIP-340 Schnorr challenge authentication (froglet-relay-auth/v1), and activates frame.v1 forwarding.	/v1/provider/transports/relay/activate transitions to status = "up", and /v1/feed serves the silo’s granted offer over the outbound WebSocket tunnel (HTTP 200).
5. Cryptographic Secure Aggregation (SecAgg, TRL 7–8)	Pairwise secp256k1 ECDH + HKDF-SHA256 seeds expand via counter-mode HMAC-SHA256 inside fl_compute.wasm ( for , for ). Raw updated_weights_micro are omitted from Wasm output; secagg_unmask_peer recovers from dropped silos.	Sum of 3 masked silo vectors cancels pairwise masks in to recover exact FedAvg weights [-55114, 407724, 550719, 533112]; simulated silo_3 dropout recovery via secagg_unmask_peer recovers exact 2-silo FedAvg [-53937, 431511, 580609, 560784].
6. Silo-Enforced RDP Privacy Ledger & Merkle Commitment (TRL 7–8)	fl_compute.wasm computes the binary SHA-256 Merkle root (cohort_commitment_hash) of queried records, tracks cumulative Rényi DP across , and emits a hash-chained privacy_ledger_entry (prev_ledger_hash -> ledger_hash).	2-round hash chain verified (head_ledger_hash = sha256:bc0935b4..., ); a 3rd round exceeding max_epsilon_micro = 8,500,000 is rejected inside Wasm (status = "rejected_privacy_policy", reason "cumulative_epsilon_exceeds_max_budget").
7. Byzantine-Resilient Aggregation & Shapley Settlement (TRL 7–8)	ByzantineAggregator implements L2 update clipping, CoordinateMedian, TrimmedMean, and MultiKrum. ShapleyContributionEvaluator computes exact coalition Shapley values () and LOO marginal gains to gate Lightning settlement.	Injecting a sign-flipped poisoning update (silo_byzantine) degrades naive FedAvg holdout accuracy to 45.0% (loss = 0.520189), whereas MultiKrum rejects silo_byzantine and preserves 100.0% holdout accuracy (loss = 0.112896). Shapley attribution assigns , quarantines silo_byzantine (0 sats), and allocates 230 / 324 / 444 sats to honest silo_1..3.
8. Multi-Architecture (MLP/LoRA), GPU Tensor Acceleration & Vertical FL (TRL 7–8)	fl_enterprise_train supports mlp_2layer (16 params) and lora_adapter (16 rank-2 params) with compute.gpu.tensor host GPU tensor ops across sqlite, http, and inline data sources (clinical_cohort, omop_cdm_v5_4, fhir_r4_bundle, financial_aml), plus psi_intersect and vertical_fl_step (forward/backward + GpuTensorAccelerator).	mlp_2layer reduces loss 0.242004 → 0.147758; lora_adapter reduces loss 0.267609 → 0.229018 (100/100 accuracy). Host compute.gpu.tensor (6 ops, 4,272 elements) and PyTorch cuda/mps (max_drift_micro = 3) achieve bit-exact JCS output_hash parity with CPU. Cross-silo PSI aligns 30 shared entities and Vertical Split Learning reduces joint loss 0.159303 → 0.117471.

---

3-Layer Deterministic GPU Acceleration (CUDA & Metal MPS)

Froglet accelerates federated learning and self-hosted compute across three complementary layers while preserving bit-exact RFC 8785 JCS output_hash and BIP-340 Schnorr receipt signatures:

1. Wasm Host GPU Tensor Bridge (compute.gpu.tensor, src/wasm_gpu.rs):
   • Exposes gpu.tensor over froglet.wasm.host_json.v1 when granted by the node’s Wasm policy ([gpu] enabled = true, max_elements = 1048576, max_dim = 4096).
   • Supports status, matmul, mlp_forward_backward, lora_forward_backward, and batched_l2_norm, quantizing all inputs and outputs to fixed-point micro-units at the host boundary so CPU, Apple Metal (mps), and NVIDIA CUDA (cuda) produce identical canonical receipt hashes.
2. PyTorch Coordinator & Vertical-FL GPU Accelerator (GpuTensorAccelerator, python/froglet_fl.py):
   • Auto-detects NVIDIA CUDA (torch.cuda), Apple Silicon Metal (torch.backends.mps), or deterministic CPU fallback for vertical SplitNN cut-layer forward/backward passes, rank- LoRA adapters, 2-layer ReLU MLPs, and vectorized MultiKrum pairwise Euclidean distance matrices.
   • Supports quantize_boundary=True (default, matching integer fixed-point micro-units bit-for-bit) and use_gpu_float32=True (executing raw torch.float32 tensor kernels on cuda:0 or mps with max_abs_drift_micro <= 3 — relative precision).
3. Exclusive GPU Device Pool & Fail-Closed Quarantine (GpuDevicePool, src/gpu.rs):
   • Hardware discovery via nvidia-smi (Linux/CUDA) and system_profiler SPDisplaysDataType (macOS Apple Silicon Metal).
   • Enforces exclusive single-GPU slot reservation (--gpus device=<idx>, NVIDIA_VISIBLE_DEVICES), post-job process cleanup verification, and automatic fail-closed device quarantine on timeout or dirty exit (compute.gpu, verified via scripts/gpu_smoke.sh).

Live Hardware Verification Matrix (Apple M4 Metal MPS vs. GCP NVIDIA Tesla T4 CUDA)

Metric / Capability	Apple Silicon (Apple M4, macOS)	GCP Cloud GPU (NVIDIA Tesla T4, Linux)
Hardware & Driver	Apple M4 (10-core GPU, Metal 3)	NVIDIA Tesla T4 (15,360 MiB GDDR6, Driver 580.178.04, CUDA 12.9)
Wasm compute.gpu.tensor Telemetry	backend: "mps", hardware_accelerated: true, 6 tensor ops, 4,272 elements	backend: "cuda", hardware_accelerated: true, 6 tensor ops, 4,272 elements
PyTorch GpuTensorAccelerator	torch 2.11.0, device: "mps", gpu_float32_max_drift_micro: 3	torch 2.9.1+cu129, device: "cuda:0", gpu_float32_max_drift_micro: 3
Container compute.gpu Smoke (scripts/gpu_smoke.sh)	Host GpuDevicePool + Metal probe verified (gpu::tests 4/4 pass)	Live Docker --gpus device=0 deal 2c521736fd23389d800227a41cb0e6bc (visible: true, 15,360 MiB)
End-to-End 5-Node FL Suite (examples/fl_distributed_demo.py)	73 deals, 365 signed artifacts, cpu_vs_gpu_bit_exact_parity_verified: true	73 deals, 365 signed artifacts, cpu_vs_gpu_bit_exact_parity_verified: true

---

Turnkey Python SDK (python/froglet_fl.py)

python/froglet_fl.py provides a self-contained SDK for orchestrating enterprise FL consortia:

• SecAggProtocol: Derives pairwise secp256k1 ECDH + HKDF-SHA256 seeds, builds per-silo secagg Wasm configs, aggregates masked updates in , and orchestrates secagg_unmask_peer dropout recovery.
• PrivacyBudgetLedger: Computes integer Rényi DP (), converts to , and verifies hash-chained privacy_ledger_entry audit trails (prev_ledger_hash -> ledger_hash) and cohort_commitment_hash Merkle roots.
• ByzantineAggregator: L2 update norm clipping (clip_update_l2_norm), fedavg, coordinate_median, trimmed_mean, and multi_krum distance-based outlier rejection.
• ShapleyContributionEvaluator: Exact combinatorial coalition Shapley values () and Leave-One-Out (LOO) marginal holdout loss attribution, automated poisoning quarantine (phi_k <= 0), and proportional Lightning satoshi settlement allocation.
• GpuTensorAccelerator: Auto-detects PyTorch cuda (NVIDIA Tesla T4 / A100 / H100) and Apple Silicon mps (Apple M4) with deterministic CPU fallback for mlp_forward_backward, lora_forward_backward, vertical_top_step, and multi_krum_distance_matrix.
• VerticalFLCoordinator: Cross-silo blinded HMAC-SHA256 psi_intersect alignment + 2-party Split Learning coordinator top-model forward/backward orchestration (vertical_fl_step).
• FrogletEnterpriseFLOrchestrator: High-level multi-round orchestrator combining SecAgg, PrivacyBudgetLedger, ByzantineAggregator, ShapleyContributionEvaluator, and GpuTensorAccelerator.

---

What a Signed FL Receipt Proves (and Does Not Prove)

In line with Froglet’s protocol guarantees, keep the verification boundary explicit:

• What the receipt chain proves offline:
  • Identity & Non-Repudiation: Which silo secp256k1 / BIP-340 Schnorr identity offered the service, quoted the terms, accepted the deal, and signed the terminal receipt.
  • Exact Artifact & Cohort Bindings: The SHA-256 workload_hash of fl_compute.wasm and its host capability (db.sqlite.query.read.cohort or net.http.fetch), the canonical input_hash of the round parameters, and the canonical output_hash containing the binary SHA-256 Merkle root (cohort_commitment_hash), secagg_proof_hash, and hash-chained privacy_ledger_entry (ledger_hash).
  • Deterministic Replay: When re-executed against the same cohort shard and fl_compute.wasm module, the Wasm kernel produces a bit-for-bit identical result_hash.
• What the receipt chain does not prove by itself:
  • A signature alone does not prove hardware TEE attestation or that a silo’s underlying database was not synthetically populated on the host. Pairing Froglet receipts with cohort_commitment_hash Merkle roots, unseen holdout-split evaluation (split = "holdout"), MultiKrum outlier filtering, and Shapley value () contribution gating makes every round’s submission attributable, Byzantine-resilient, and economically incentive-compatible.

---

8-Claim Positive + Adversarial Verification Suite

The verification harness in examples/fl_distributed_demo.py and examples/test_fl_distributed_demo.py verifies all 8 claims against both positive executions and adversarial failure injections across live froglet-node processes:

#	Claim	Positive Proof	Adversarial / Negative Proof	Result
1	Cryptographic Audit Trail	Valid 5-artifact chain (Descriptor → Offer → Quote → Deal → Receipt) passes offline froglet_verify.validate_full_chain	Mutating 1 weight (weights_micro[0] += 1) or flipping 1 receipt signature bit is rejected offline	PASS
2	Data-Silo Compensation	Priced silo (30 sats) settles base_fee_msat + success_fee_msat (30,000 msat) with a verified 6-artifact chain (InvoiceBundle included)	Unpaid direct /v1/node/execute/wasm returns HTTP 409; under-priced deal (5 sats < 30 sats) and premature accept before payment are rejected	PASS
3	Zero-Trust Code Execution	fl_compute.wasm imports only froglet_host::call_json (0 embedded records) and runs within 50M fuel / 8 MiB RAM	Infinite-loop Wasm module traps with execution_limit_exceeded (signed failure receipt verified); out-of-bounds integer payload fails closed	PASS
4	Strict Data Locality	0 raw patient records transmitted in Map-Reduce or FL deals; each silo queries its own physical silo_k_ehr.db	Injecting raw records, patient IDs (s1_p001), or PHI_SENTINEL into any compute payload raises an immediate security violation	PASS
5	Bit-Exact Reproducibility	Running fl_local_sgd with DP-SGD (max_grad_norm_micro=250,000, dp_seed=202605) on independent nodes yields identical result_hash (054c70c9...)	Matches the pure-Python fixed-point DP-SGD oracle bit-for-bit (72 clipped gradients, dp_noise_l2_micro = 6431)	PASS
6	Idempotent Retries	Re-submitting an identical idempotency_key returns the cached Deal + Receipt (0 duplicate deals or charges)	Reusing the same idempotency_key with modified round weights is rejected (idempotency key reused with a different requester deal)	PASS
7	Framework-Agnostic Convergence	5-round FedAvg on 300 train records converges loss 0.250000 → 0.040046 (49.0% → 100.0%) and reaches 100.0% (60/60) on unseen holdout	Outperforms isolated models trained on skewed silo_1 (80.0% holdout) or silo_2 (78.3% holdout)	PASS
8	Redacted Cohort Discovery	Curated data.structured.query.v1 snapshots publish 6 non-PHI columns (300 train rows); holdout split and patient_private_note remain unexposed	Querying excluded PHI column patient_private_note is rejected by the provider node	PASS

---

Architecture & Five-Node Topology

+-------------------------------------------------------------------------+
|                  Coordinator / Aggregator (Runtime Node)                |
|  - Turnkey Enterprise SDK (python/froglet_fl.py: SecAgg, RDP, Shapley)  |
|  - GpuTensorAccelerator (PyTorch CUDA / Apple Metal MPS / CPU)          |
|  - Flower (flwr NumPyClient + FedAvg Strategy)                          |
|  - NVIDIA FLARE (nvflare Executor + InTimeAccumulateWeightedAggregator) |
|  - Offline Verifier (python/froglet_verify.py BIP-340 + RFC 8785 JCS)   |
+-------------------+-------------------------+---------------------------+
                    |                         |                         |
   Signed FL Round  |        Signed FL Round  |        Signed FL Round  |
   Deal + Receipt   |        Deal + Receipt   |        Deal + Receipt   |
                    v                         v                         v
        +-----------------------+ +-----------------------+ +-----------------------+
        | Silo 1 (Provider)     | | Silo 2 (Provider)     | | Silo 3 (Provider)     |
        |  - silo_1_ehr.db      | |  - silo_2_ehr.db      | |  - silo_3_ehr.db      |
        |    (EHR + OMOP + AML) | |    (EHR + OMOP + AML) | |    (EHR + OMOP + AML) |
        |  - FHIR R4 HTTP Adapter| |  - fl_compute.wasm    | |  - fl_compute.wasm    |
        |  - compute.gpu.tensor | |  - compute.gpu.tensor | |  - compute.gpu.tensor |
        |  - SecAgg + RDP Ledger| |  - SecAgg + RDP Ledger| |  - SecAgg + RDP Ledger|
        |  - BIP-340 Schnorr Key| |  - BIP-340 Schnorr Key| |  - BIP-340 Schnorr Key|
        +-----------------------+ +-----------------------+ +-----------------------+

Round-by-Round Convergence (360-Record Non-IID Clinical Cohort)

Round	Global Fixed-Point Weights [b, w_age, w_alpha, w_beta]	Global Train Loss (N=300)	Global Train Accuracy (N=300)	Unseen Holdout Accuracy (N=60)
Round 0 (Init)	[0, 0, 0, 0]	0.250000	49.0% (147/300)	55.0% (33/60)
Round 1	[-55114, 407724, 550719, 533112]	0.113340	100.0% (300/300)	—
Round 2	[-106880, 702007, 950608, 921289]	0.076141	100.0% (300/300)	—
Round 3	[-150141, 945589, 1282730, 1243885]	0.058090	100.0% (300/300)	—
Round 4	[-187543, 1158013, 1573006, 1525926]	0.047286	100.0% (300/300)	—
Round 5 (Final)	[-220768, 1348603, 1833861, 1779431]	0.040046	100.0% (300/300)	100.0% (60/60, loss 0.040038)
Round 5 + DP-SGD	[-230154, 1382572, 1879758, 1824309]	—	—	100.0% (60/60, loss 0.038944)

---

Quick Start

1. Run the Complete TRL 6 + TRL 7–8 Enterprise Verification Suite

From a repository checkout with Rust (wasm32-unknown-unknown target) and Python 3.10+:

rustup target add wasm32-unknown-unknown

# Auto-builds froglet-node and fl_compute.wasm, then runs all 4 TRL 6 pillars, all 4 Enterprise TRL 7–8 pillars, and the 8-claim suite:
python3 examples/fl_distributed_demo.py --output /tmp/fl_trl7_report.json

2. Run with Flower (flwr) and NVIDIA FLARE (nvflare)

python3 -m venv .venv-fl
.venv-fl/bin/pip install "flwr>=1.10" "nvflare>=2.5" numpy

.venv-fl/bin/python examples/fl_distributed_demo.py \
  --require-frameworks \
  --no-build \
  --output /tmp/fl_trl7_report.json

3. Run the Automated Unit Tests

python3 -W error -m unittest python.tests.test_froglet_fl examples.test_fl_distributed_demo -v