> For the complete documentation index, see [llms.txt](https://docs.blockcast.network/main/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.blockcast.network/main/getting-started/how-do-i-participate-in-the-network/relay/relay-amt.md). # Juniper MX AMT Hardware Offload Offload AMT multicast relay processing to Juniper MX hardware Every RELAY node runs Linux kernel AMT by default. For operators with Juniper MX routers (MX204, MX240, MX480, MX960, etc.), you can offload AMT relay processing to dedicated hardware for better performance and scale. Hardware offloading is optional. Your RELAY works without it. This is for operators who want to handle thousands of concurrent AMT tunnels at line rate. ## What This Gives You | | Linux Kernel AMT (default) | Juniper MX Hardware | | -------------------- | -------------------------------- | ------------------------------------------------------------- | | **Setup** | Automatic, zero-config | Requires gNMI setup | | **Tunnel scale** | Hundreds | Thousands (line rate) | | **Stats collection** | amt-astats reads kernel counters | amt-astats reads via gNMI | | **CDNi billing** | amt-astats HTTP logs | Per-(S,G) records: gNMI counters + JET identity (gRPC :32767) | | **DRIAD discovery** | TR advertises gateway IP | TR advertises MX anycast IP | ## Prerequisites * Juniper MX router running JunOS 22.2R3 or later (validated on **24.4R2-S3.5**, 2026-09-01) * MX registered as `AMT_RELAY` server in Traffic Ops with profile `AMT_RELAY_MX_PROFILE` * `amt-astats` deployed on your gateway pod (included in the default RELAY runtime) * Magma gateway certificates (auto-provisioned during RELAY registration) ## Setup via Portal During RELAY onboarding (after the Bandwidth Commitment step), the portal asks if you want to add hardware AMT offloading.
Hardware AMT Offload dialog with two options: use default Linux AMT or configure Juniper MX offload

Select "Yes, configure Juniper MX offload" to proceed with hardware setup

Select **Yes, configure Juniper MX offload** to open the configuration screen:
Juniper MX Configuration showing gNMI target, optional password auth, optional IPFIX liveness port, and AMT Relay Public IP

Configure the gNMI target and AMT Relay Public IP (the IPFIX port is optional — liveness only, not billing)

| Field | Description | Example | | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | | **gNMI Target** | Auto-discover from Traffic Ops (recommended) or enter a manual IP:port. **A correct target is not sufficient** — your gateway's egress address must also be permitted by the MX's gNMI source allowlist (see [step 3](#3-permit-the-gateway-source-on-the-re-filter)). IPv4 only; IPv6 targets are denied. | `69.25.95.1:32767` | | **IPFIX Port** (optional) | Liveness/staleness only — not the billing path | `4739` | | **AMT Relay Public IP** | The public IP of your hardware AMT relay. This may differ from the auto-detected gateway IP. Traffic Router uses this IP for DRIAD DNS responses so clients reach the MX directly for AMT tunnel setup. | `69.25.95.1` | **Authentication**: gNMI uses certificate-based TLS by default. Your gateway presents its Magma client certificate and the MX verifies it — that client-certificate check gates the channel. The TLS is mutually authenticated only when the gateway also verifies the MX's server certificate, which needs the CA that signed it (see [gNMI Authentication](#gnmi-authentication)). gRPC authorization then follows a dedicated **read-only** username (configured for you, not entered in the wizard); see [gNMI Authentication](#gnmi-authentication) below for the least-privilege login class and CA-loading details. If your MX additionally requires a metadata password, expand the **Password Auth (optional)** section and enter the JunOS credentials. After completing the MX configuration, the portal proceeds to the **Node Verification** step:
Node Verification checklist showing ports, Traffic Monitor, Traffic Router, Cache, and AMT Relay status

All ports and services must pass verification before the node goes live

This checks that all required ports (53, 80, 443, 2268) are forwarded and all services (Traffic Monitor, Traffic Router, cache, AMT relay) are reachable and healthy. ## gNMI Authentication By default, gNMI uses **certificate-based TLS**. Your gateway's Magma client certificate authenticates the gateway to the MX router, and the MX's client-certificate check gates the channel. **Verify the MX's server certificate too.** The gateway authenticates the MX only if it verifies the MX's gNMI server certificate. Set `gnmiTLSSkipVerify: false` and store the CA that signed that certificate as `ca.crt` in the secret named by `gnmiTLSCASecret` (see the [Helm values](#5-configure-amt-astats) below). The server certificate must carry the gNMI target address as an IP SAN (for example `IP:69.25.95.1`), because the name `amt-astats` checks is taken from the dial target. A self-signed server certificate does not force you to skip verification: it is its own trust anchor, so pin the certificate itself as `ca.crt` ([step 5](#5-configure-amt-astats) shows how). With `gnmiTLSSkipVerify: true` (the chart default), authentication is one-way: the MX authenticates the gateway, but the gateway does not authenticate the MX, so `amt-astats` cannot detect a spoofed or intercepted gNMI endpoint. This channel carries the [billing](#how-it-works) counters, so skip verification only if you accept that exposure. **Use a dedicated read-only gRPC user.** Junos gRPC runs with `skip-authentication`, so authorization follows the **per-RPC metadata `username`**, not the cert CN. Point `amt-astats` at a purpose-built read-only login class rather than a broad/super-user account, so a client authenticating as that user cannot issue config-changing RPCs: ```junos set system login class grpc-billing-ro permissions view set system login class grpc-billing-ro allow-grpc-rpc-regexps "(gnmi\.gNMI/(Subscribe|Capabilities)|jnx\.jet\.management\.Management/OpCommandGet|jnx\.jet\.authentication\.Authentication/Login)" set system login user amt-billing class grpc-billing-ro set system login user amt-billing authentication plain-text-password ``` `permissions view` is what makes this user read-only: config-changing RPCs like `gnmi.gNMI/Set` are denied at commit (verified: the MX returns *permission denied*). The `allow-grpc-rpc-regexps` whitelist adds defense-in-depth, narrowing the callable RPC surface to exactly what `amt-astats` calls: `gnmi.gNMI/Subscribe` in STREAM/SAMPLE mode for the `logical/usage` billing counters (on Junos 22.2R `gnmi.gNMI/Get` returns configuration only and `Subscribe ONCE` returns nothing, so billing never uses `Get`), plus JET `Authentication/Login` and `Management/OpCommandGet` for tunnel identity. If you must reuse an existing user, a `deny-grpc-rpc-regexps "(gnmi\.gNMI/Set|jnx\.jet\.management\.Management/(Config|EphemeralConfig)Set)"` deny-list is the lower-risk alternative. It is **not** equivalent: it blocks only the config-changing RPCs and leaves every other RPC callable, including `gnmi.gNMI/Get`. Both configurations permit the billing path. Apply with `commit confirmed`, verify `/_astats` + `/health`, then `commit` to confirm. The password is only carried by JET `Login`; under `skip-authentication` it isn't verified, but set one for forward-compatibility. **The login class limits the intended client, not a compromised one.** Under `skip-authentication` the username comes from gRPC metadata and the password isn't verified, so any client that completes the TLS handshake can claim a different, more privileged username and is authorized as that user. The access-control boundary is the client-certificate check plus the RE source filter from [step 3](#3-permit-the-gateway-source-on-the-re-filter). ### Loading the CA Certificate on the MX The MX must trust the Magma certifier CA to verify your gateway's client certificate during the gNMI mTLS handshake. This covers only that direction. For the gateway to verify the MX, pin the MX's own server certificate as described in [step 5](#5-configure-amt-astats). 1. Extract the CA cert from your gateway pod: ```bash kubectl exec -n production-blockcastd -- \ cat /var/opt/magma/certs/certifier.pem > /tmp/certifier.pem ``` 2. Copy it to the MX: ```bash scp -O /tmp/certifier.pem @:/tmp/certifier.pem ``` 3. Load it on the MX: ```junos configure set security pki ca-profile magma-certifier ca-identity commit ``` Then from operational mode: ```junos request security pki ca-certificate load ca-profile magma-certifier filename /tmp/certifier.pem ``` 4. **Reboot the MX** (required — JSD does not reliably reload CA certs without a full reboot): ```junos request system reboot ``` ## Manual Setup (CLI) If you prefer to configure the MX manually instead of using the portal wizard: ### 1. Load Magma Certifier CA on the MX The MX needs to trust the Magma certifier CA to verify gateway client certificates during the gNMI mTLS handshake. ```bash # Extract certifier.pem from your gateway pod kubectl exec -n production-blockcastd -- \ cat /var/opt/magma/certs/certifier.pem > /tmp/certifier.pem # Copy to the MX router scp -O /tmp/certifier.pem @:/tmp/certifier.pem ``` On the MX: ```junos configure set security pki ca-profile magma-certifier ca-identity commit ``` Then from operational mode: ```junos request security pki ca-certificate load ca-profile magma-certifier filename /tmp/certifier.pem ``` ### 2. Configure gNMI Extension Service The service presents the local certificate `gnmi-server`. If the MX has none yet, generate it with the MX's gNMI address as an IP SAN, because `amt-astats` checks the name against the dial target: ```junos request security pki generate-key-pair certificate-id gnmi-server size 2048 request security pki local-certificate generate-self-signed certificate-id gnmi-server subject "CN=,O=" ip-address ``` Then configure the service: ```junos configure set system services extension-service request-response grpc ssl port 32767 set system services extension-service request-response grpc ssl local-certificate gnmi-server set system services extension-service request-response grpc ssl mutual-authentication certificate-authority magma-certifier set system services extension-service request-response grpc ssl mutual-authentication client-certificate-request require-certificate-and-verify set system services extension-service request-response grpc skip-authentication commit ``` ### 3. Permit the gateway source on the RE filter **Enabling the service does not make it reachable.** gNMI terminates on the routing engine, so the `protect-re` loopback filter decides who may connect. Without an explicit allow term ahead of the deny, a correctly-configured extension service is unreachable and every symptom points at the wrong thing (see [Known Gotchas](#known-gotchas)). Term order matters — the allow must precede the deny: ```junos set firewall family inet filter protect-re term allow-amt-relay-gnmi from source-address set firewall family inet filter protect-re term allow-amt-relay-gnmi from protocol tcp set firewall family inet filter protect-re term allow-amt-relay-gnmi from destination-port 32767 set firewall family inet filter protect-re term allow-amt-relay-gnmi then count amt-relay-gnmi-allowed set firewall family inet filter protect-re term allow-amt-relay-gnmi then accept set firewall family inet filter protect-re term deny-other-gnmi from protocol tcp set firewall family inet filter protect-re term deny-other-gnmi from destination-port 32767 set firewall family inet filter protect-re term deny-other-gnmi then count amt-relay-gnmi-denied set firewall family inet filter protect-re term deny-other-gnmi then log set firewall family inet filter protect-re term deny-other-gnmi then discard commit confirmed 10 ``` Use `commit confirmed` — a filter that locks you out rolls back on its own if you cannot confirm. After you have verified access from the permitted source, confirm with a plain `commit` before the 10 minutes run out. (`commit check` only validates syntax; it does not confirm a pending `commit confirmed`.) `` is the egress prefix of the gateway running `amt-astats`, not the MX's own address. On the Blockcast production MX204 this is `69.25.95.36/30`. **IPv6 is deny-only.** `protect-re-v6` carries `deny-other-gnmi-v6` with no matching allow term, so gNMI over IPv6 is discarded regardless of source. Use an IPv4 target. Confirm the terms are counting rather than trusting the commit: ```junos show firewall filter protect-re counter amt-relay-gnmi-allowed show firewall filter protect-re counter amt-relay-gnmi-denied ``` A rising `denied` counter is the signature of a gateway whose source address is outside the allowlist. ### 4. Reboot the MX JSD does not reliably reload CA certs without a full reboot. ```junos request system reboot ``` ### 5. Configure amt-astats In your Helm values or docker-compose environment: ```yaml amtRelay: enabled: true relayType: "juniper" gnmiAuthMode: "tls" gnmiUsername: "amt-billing" # dedicated read-only gRPC user (least-privilege login class); see gNMI Authentication gnmiPasswordSecret: "amt-relay-gnmi-billing-password" # the dedicated amt-billing user's password gnmiTLSCASecret: "amt-relay-gnmi-ca" # secret whose ca.crt is the CA that signed the MX gNMI server certificate gnmiTLSSkipVerify: false # verify the MX server certificate; true = one-way auth (see gNMI Authentication) amtRelayPublicIP: "69.25.95.1" # Public IP of hardware AMT relay (may differ from gateway IP) ipfixPort: 4739 # optional; liveness/staleness only — billing uses gNMI counters + JET identity ``` **Pin the MX server certificate before deploying `gnmiTLSSkipVerify: false`.** If a CA signed the MX's `gnmi-server` certificate, put that CA in `ca.crt`. If the certificate is self-signed (as generated in step 2), the certificate itself is the trust anchor. Capture it from a host inside the allowlist and confirm it is the MX's: ```bash set -o pipefail rm -f /tmp/gnmi-server.pem openssl s_client -connect :32767 \ -cert /var/opt/magma/certs/gateway.crt \ -key /var/opt/magma/certs/gateway.key `. If the fingerprint does not match, you captured someone else's certificate: stop. If the IP SAN is missing, regenerate the certificate as in step 2 and capture again. Only when both checks pass, create the secret: ```bash kubectl create secret generic amt-relay-gnmi-ca -n production-blockcastd \ --from-file=ca.crt=/tmp/gnmi-server.pem ``` amt-astats auto-discovers the MX address from Traffic Ops by querying for `AMT_RELAY_MX_PROFILE` servers in the same cachegroup. Set `gnmiTarget` explicitly only if you need to override discovery. **Discovery resolves the address; it does not grant access.** Traffic Ops returning the correct MX says nothing about whether this gateway's source address is admitted by the RE filter in step 3. A successfully discovered target that never connects is an allowlist problem, not a discovery problem. ### 6. Verify > **Run these from a host inside the gNMI allowlist.** From anywhere else they fail at TCP connect, which looks identical to a broken service. Take the control first: > > ```bash > nc -vz 22 # control: SSH is not source-restricted > nc -vz 32767 # subject > ``` > > If 22 succeeds and 32767 does not, the router is healthy and your source address is not allowlisted. Do not reboot the MX and do not touch the CA — fix the filter (step 3). ```bash # TLS handshake test timeout 5 openssl s_client -connect :32767 \ -cert /var/opt/magma/certs/gateway.crt \ -key /var/opt/magma/certs/gateway.key \ -alpn h2 2>&1 | grep "SSL handshake" # Verify MX requests Magma CA for client certs timeout 5 openssl s_client -connect :32767 \ -cert /var/opt/magma/certs/gateway.crt \ -key /var/opt/magma/certs/gateway.key \ 2>&1 | grep "Acceptable client" # Verify the MX server certificate the way amt-astats does with # gnmiTLSSkipVerify: false. Expect "Verify return code: 0 (ok)". timeout 5 openssl s_client -connect :32767 \ -cert /var/opt/magma/certs/gateway.crt \ -key /var/opt/magma/certs/gateway.key \ -CAfile /tmp/gnmi-server.pem -verify_ip -verify_return_error \ &1 | grep "Verify return code" # Check amt-astats is reporting curl -s http://localhost:8080/_astats | grep plugin.amt ``` ## How It Works ``` amt-astats (on gateway pod) ├── gNMI mTLS ──→ MX :32767 (JGrpcServer) │ ├── logical/usage counters → per-tunnel egress bytes/pkts (billable) │ └── JET OpCommandGet (get-amt-tunnel-information) → per-(S,G) + subscriber gateway identity │ └── correlate on ud- unit → per-(S,G) CDNi billing records → CDN Controller │ ├── /_astats ──HTTP──→ Traffic Monitor │ └── AMT relay stats (tunnels, bandwidth, utilization) │ └── IPFIX receiver (optional) ←── MX flow export └── liveness/staleness only — NOT the billing path ``` When the MX is the active AMT relay, Traffic Router advertises the **MX's anycast IP** in DRIAD DNS responses (not the gateway IP). Clients performing RFC 8777 DRIAD discovery reach the MX directly for AMT tunnel setup. **Per-(S,G) billing model.** Billing records are produced by `amt-astats` correlating two structured interfaces on the MX's mTLS gRPC endpoint (`:32767`): **JET** `OpCommandGet` (running `get-amt-tunnel-information`) supplies the per-tunnel subscriber **gateway + (source, group)** identity, and **gNMI** `logical/usage` supplies the per-tunnel egress **bytes/packets**. They join on the `ud-` logical unit to emit one CDNi record per **(subscriber, source, group)** window. IPFIX/jFlow is *not* used for billing — its templates don't carry the AMT outer-tunnel gateway or (S,G), so the receiver is kept only as a liveness/staleness signal. ## Known Gotchas **A filtered port looks exactly like a broken JSD.** If gNMI connect fails, check the source allowlist *before* the two CA/reboot gotchas below. Both of those produce a connection that fails, and so does a `discard` term — but only one of them is fixed by rebooting a production router. `nc -vz 22` as a control and a rising `amt-relay-gnmi-denied` counter distinguish them in seconds. **With `gnmiTLSSkipVerify: false`, a handshake that fails can be a pinning problem.** If the server-certificate check in step 6 does not print `Verify return code: 0 (ok)`, the cause is the pinned certificate or the MX certificate itself, not the filter or the CA reload. Tell them apart with `openssl x509 -in /tmp/gnmi-server.pem -noout -ext subjectAltName`. If `IP Address:` is listed, `ca.crt` is the wrong certificate or not its signer: re-pin as in step 5. If it is missing, re-pinning cannot help, because the SAN is fixed when the certificate is generated. Regenerate it as in step 2, then re-pin. Do not reboot the MX for this. **Ed25519 TLS works on JunOS 22.2+.** Magma Ed25519 gateway certificates pass TLS verification. The MX advertises `Ed25519` in its requested signature algorithms during the TLS CertificateRequest. **JSD requires reboot after CA change.** After replacing the CA cert in a PKI profile, `restart jsd immediately` is not sufficient. The gRPC connection closes with EOF. A full reboot is required. **Only one CA profile for gNMI mutual auth.** The `certificate-authority` configuration accepts a single CA profile. Switching CAs is a replace operation. **`skip-authentication` still needs a gRPC metadata `username`.** Despite its name, JGrpcServer reads `username`/`password` from gRPC metadata and **authorization follows that username**. Set `GNMI_USERNAME` to a **dedicated read-only** JunOS user (see [gNMI Authentication](#gnmi-authentication)); `GNMI_PASSWORD` is only used by JET `Login` and isn't verified under `skip-authentication`, but set it for forward-compatibility. An empty/unknown username is rejected. **SANs may affect identity extraction.** Magma gateway certs have multiple DNS SANs. Since `skip-authentication` + gRPC metadata password handles auth, this doesn't block connectivity, but is relevant if you try cert CN as the sole auth method.