Network Topology¶
S5 Topology Diagram¶
┌──────────────────────────────────────────────┐
│ S5 (Day-0) | V=45 T=30 │
│ 5 GCP nodes (us-central1-a) │
│ Allocation: [12, 12, 11, 5, 5] │
└──────────────────────────────────────────────┘
┌──────────────────────────────── CORE (full mesh) ───────────────────────────────────────┐
│ │
│ core1 (aws-a-1) core2 (aws-b-1) core3 (aws-c-1) │
│ ┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ 104.198.139.4 │ │ 34.69.0.125 │ │ 34.28.8.159 │ │
│ │ EL: go-centurion │ │ EL: reth │ │ EL: go-centurion │ │
│ │ CL: Lighthouse │ │ CL: Lighthouse │ │ CL: Prysm │ │
│ │ VC: Vouch │ │ VC: Vouch │ │ VC: Vouch │ │
│ │ Vals: 12 │ │ Vals: 12 │ │ Vals: 11 │ │
│ └───┬────────┬────────┘ └───┬────────┬────────┘ └───┬────────┬────────┘ │
│ │ │ │ │ │ │ │
│ │ │ mTLS │ │ mTLS │ │ mTLS │
│ │ │ │ │ │ │ │
│ │ ┌────┴──────┐ │ ┌────┴──────┐ │ ┌────┴──────┐ │
│ │ │ dirk-1 │ │ │ dirk-2 │ │ │ dirk-3 │ │
│ │ │ 10.128. │ │ │ 10.128. │ │ │ 10.128. │ │
│ │ │ 0.12 │ │ │ 0.14 │ │ │ 0.10 │ │
│ │ │ :13141 │ │ │ :13141 │ │ │ :13141 │ │
│ │ └───────────┘ │ └───────────┘ │ └───────────┘ │
│ │ │ │ │
│ ├────────────────────────────┼────────────────────────────┘ │
│ │ │ │
│ ┌───┴────────────────────────────┴──────────────────────────────────────────────┐ │
│ │ P2P full mesh (EL ↔ EL · CL ↔ CL · Engine API) │ │
│ └──────────────────────────┬────────────────────────────────────────────────────┘ │
└─────────────────────────────│───────────────────────────────────────────────────────────┘
│
┌───────────────┴───────────────────────────────┐
│ CORE ↔ EDGE P2P (EL ↔ EL · CL ↔ CL) │
└──────────────────┬───────────────────┬────────┘
│ │
┌────────────────────────────────│───────────────────│───────────────────────────────────────┐
│ │ │ │
│ edge1 (hetz-1) │ │ edge2 (linode-1) │
│ ┌─────────────────────┐ │ │ ┌─────────────────────┐ │
│ │ 104.154.98.59 │◄──────┘ └──────►│ 34.59.194.206 │ │
│ │ EL: go-centurion │ │ EL: nethermind │ │
│ │ CL: Prysm │ │ CL: Lodestar │ │
│ │ VC: Vouch │ │ VC: Vouch │ │
│ │ Vals: 5 │ │ Vals: 5 │ │
│ └───┬────────┬────────┘ └───┬────────┬────────┘ │
│ │ │ │ │ │
│ │ │ mTLS │ │ mTLS │
│ │ │ │ │ │
│ │ ┌────┴──────┐ │ ┌────┴──────┐ │
│ │ │ dirk-4 │ │ │ dirk-5 │ │
│ │ │ 10.128. │ │ │ 10.128. │ │
│ │ │ 0.11 │ │ │ 0.13 │ │
│ │ │ :13141 │ │ │ :13141 │ │
│ │ └───────────┘ │ └───────────┘ │
│ │ │ │
│ ├─────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───┴──────────────────────────────────────────────────────────────────────────────┐ │
│ │ P2P inter-edge (EL ↔ EL · CL ↔ CL) │ │
│ └──────────────────────────────────────────────────────────────────────────────────┘ │
│ EDGE (keep alive, small stake) │
└────────────────────────────────────────────────────────────────────────────────────────────┘
Signing path per node: CL (Beacon API :5052) ──► Vouch ──► mTLS ──► Dirk (:13141)
Dirk fleet: 5x GCP e2-micro, Dirk v1.2.1, internal VPC (10.128.0.0/24)CORE / EDGE Model¶
CORE nodes (aws-a-1, aws-b-1, aws-c-1) form a full P2P mesh. Each CORE node peers with every other CORE node at both EL and CL layers.
EDGE nodes (hetz-1, linode-1) peer with all CORE nodes and with each other. They carry a smaller stake but improve client and provider diversity.
All nodes are currently on GCP us-central1-a. The node names (aws-, hetz-, linode-*) are logical identifiers from the original multi-cloud design.
Network Protocols¶
| Layer | Protocol | Port | Bind | Purpose |
|---|---|---|---|---|
| EL P2P | DevP2P | 30303 (TCP+UDP) | 0.0.0.0 | Block/tx propagation |
| CL P2P | libp2p | 9000 (TCP+UDP) | 0.0.0.0 | Attestations, blocks, sync |
| Engine API | HTTP/JWT | 8551 | 127.0.0.1 | EL <-> CL communication |
| EL RPC | HTTP | 8545 | 0.0.0.0 | JSON-RPC |
| EL WS | WebSocket | 8546 | 0.0.0.0 | WebSocket subscriptions |
| CL HTTP | HTTP | 5052 | 0.0.0.0 | Beacon API |
| CL Metrics | HTTP | 6061 | 0.0.0.0 | Prometheus metrics (CL) |
| VC Metrics | HTTP | 6062 | 0.0.0.0 | Prometheus metrics (Vouch) |
| Dirk | gRPC/mTLS | 13141 | 0.0.0.0 | Remote signing (internal VPC) |
Peering Mechanics¶
Peers are configured by deploy-node.sh --peer, which collects live identities from running CORE nodes and distributes them to all nodes.
Execution Layer (enodes)¶
EL peering uses enodes collected via admin_nodeInfo RPC on port 8545. Enodes are written to:
/etc/centurion/el_bootnodes-- comma-separated, single line (used by geth--bootnodesand reth--bootnodes)/etc/centurion/el_static_peers-- same content (used by nethermind--Network.StaticPeers)
Peers are also added live via admin_addPeer RPC (no EL restart required).
Consensus Layer (ENRs)¶
CL peering uses ENRs collected via /eth/v1/node/identity on port 5052. ENRs are written to:
/etc/centurion/cl_boot_enrs-- comma-separated, single line
CL services must be restarted to pick up new boot ENRs.
Client-Specific Peering Flags¶
| Client | EL Peering Flag | Source File |
|---|---|---|
| geth (go-centurion) | --bootnodes <comma-separated> | /etc/centurion/el_bootnodes |
| reth (rustcen) | --bootnodes <comma-separated> | /etc/centurion/el_bootnodes |
| nethermind | --Network.StaticPeers <comma-separated> | /etc/centurion/el_static_peers |
| Client | CL Peering Flag | Source File |
|---|---|---|
| Lighthouse | --boot-nodes <comma-separated> | /etc/centurion/cl_boot_enrs |
| Prysm | --bootstrap-node <comma-separated> | /etc/centurion/cl_boot_enrs |
| Lodestar | --bootnodes <enr1> --bootnodes <enr2> ... (split) | /etc/centurion/cl_boot_enrs |
Lodestar ENR Format
Lodestar expects each ENR as a separate --bootnodes argument. The startup script splits the comma-separated file into individual args.
How deploy-node.sh --peer Works¶
- SSH to each CORE node, call
admin_nodeInfoRPC to get the live enode - SSH to each CORE node, call
/eth/v1/node/identityto get the live ENR - For each node (CORE + EDGE), write peering files excluding the node's own identity for CORE nodes
- Add EL peers live via
admin_addPeerRPC (no restart needed) - Restart all CL services to pick up the new boot ENRs
Node Inventory¶
| Node | IP | Role | EL | CL | VC | Validators |
|---|---|---|---|---|---|---|
| aws-a-1 | 104.198.139.4 | CORE | go-centurion | Lighthouse | Vouch -> Dirk-1 | 12 |
| aws-b-1 | 34.69.0.125 | CORE | reth (rustcen) | Lighthouse | Vouch -> Dirk-2 | 12 |
| aws-c-1 | 34.28.8.159 | CORE | go-centurion | Prysm | Vouch -> Dirk-3 | 11 |
| hetz-1 | 104.154.98.59 | EDGE | go-centurion | Prysm | Vouch -> Dirk-4 | 5 |
| linode-1 | 34.59.194.206 | EDGE | nethermind | Lodestar | Vouch -> Dirk-5 | 5 |
Dirk Fleet Inventory¶
| Instance | Internal IP | Port | Machine Type | Mapped Validator Node |
|---|---|---|---|---|
| dirk-1 | 10.128.0.12 | 13141 | e2-micro | aws-a-1 (core1) |
| dirk-2 | 10.128.0.14 | 13141 | e2-micro | aws-b-1 (core2) |
| dirk-3 | 10.128.0.10 | 13141 | e2-micro | aws-c-1 (core3) |
| dirk-4 | 10.128.0.11 | 13141 | e2-micro | hetz-1 (edge1) |
| dirk-5 | 10.128.0.13 | 13141 | e2-micro | linode-1 (edge2) |
JWT Authentication (Engine API)¶
The Engine API (port 8551) is authenticated via a shared JWT secret at /etc/centurion/jwt.hex. It only secures local EL<->CL communication (bound to 127.0.0.1). Not used for P2P traffic.
Verifying Connectivity¶
# Check EL peer count (should be 4 for all nodes)
gcloud compute ssh centurion@aws-a-1 --zone us-central1-a --quiet --command \
"curl -s http://127.0.0.1:8545 -X POST \
-H 'Content-Type: application/json' \
-d '{\"jsonrpc\":\"2.0\",\"method\":\"net_peerCount\",\"params\":[],\"id\":1}' \
| python3 -c 'import sys,json; print(int(json.load(sys.stdin)[\"result\"],16),\"EL peers\")'"
# Check CL peer count -- same Beacon API for all CL clients
gcloud compute ssh centurion@aws-a-1 --zone us-central1-a --quiet --command \
"curl -s http://127.0.0.1:5052/eth/v1/node/peer_count \
| python3 -c 'import sys,json; d=json.load(sys.stdin)[\"data\"]; print(d[\"connected\"],\"CL peers\")'"