Shielded Vote PIR Server Setup

Fresh PIR Setup

Use Automatic for the supported one-command install. Switch to Manual when you need to download, verify, configure, and start nf-server yourself without running start_pir.sh.

Provision A Dedicated PIR Host

Minimum production requirement: run PIR on its own machine with at least this capacity.

Platform

Linux amd64

CPU

4 vCPU + AVX-512

Memory

32 GB RAM

Storage

65 GB free SSD

More CPU and storage are recommended when available to keep query serving and future snapshot growth comfortable.

The production linux-amd64 binary requires AVX-512. If the host does not expose AVX-512, the service will crash before binding port 3000, usually with status=4/ILL in systemd logs.
Run The PIR Installer

Run this on the PIR host. It installs the verified nf-server release, writes the systemd service, starts nullifier-query-server, and bootstraps the active PIR snapshot from Valar-hosted object storage.
```
curl -fsSL https://vote.fra1.digitaloceanspaces.com/start_pir.sh | sudo bash
```
The installer targets Linux hosts with systemd and must run as root because it writes under /opt and /etc.
Expose HTTPS To The Local Service

nf-server serves plaintext HTTP on local port 3000. Set up your own Caddy, nginx, or load balancer so the public URL uses HTTPS and proxies to 127.0.0.1:3000. Do not expose the raw port directly to clients.
```
pir.example.org {
    reverse_proxy 127.0.0.1:3000

    @debug path /tier1/row/* /tier2/row/*
    handle @debug {
        respond 403
    }
}
```
Replace pir.example.org with your hostname. The public endpoint should look like https://pir.example.org.
Publish The PIR Endpoint

Add the public HTTPS URL to pir_endpoints[] in dynamic-voting-config.json. Add one JSON object with url and label, then open a PR from GitHub's edit flow. Edit voting config.
```
{
  "url": "https://pir.example.org",
  "label": "Example PIR"
}
```

Provision A Dedicated PIR Host

Minimum production requirement: run PIR on its own machine with at least this capacity.

Platform

Linux amd64

CPU

4 vCPU + AVX-512

Memory

32 GB RAM

Storage

65 GB free SSD

More CPU and storage are recommended when available to keep query serving and future snapshot growth comfortable.

The production linux-amd64 binary requires AVX-512. If the host does not expose AVX-512, the service will crash before binding port 3000, usually with status=4/ILL in systemd logs.
Install Host Requirements

Install the shell tools used by the manual download, checksum, and local service checks.
```
sudo apt-get update
sudo apt-get install -y curl ca-certificates jq
command -v systemctl >/dev/null
```

Download And Verify The Release

Pin TAG if you need a specific release. By default this downloads the latest GitHub release, prefers Valar-hosted object storage for the binary and checksum, and falls back to GitHub release assets.

set -euo pipefail
REPO="valargroup/vote-nullifier-pir"
BINARY_BASE_URL="${BINARY_BASE_URL:-https://vote.fra1.digitaloceanspaces.com/binaries/vote-pir}"
TAG="${TAG:-$(curl -fsSL "https://api.github.com/repos/${REPO}/releases/latest" | jq -r '.tag_name')}"
WORKDIR="${WORKDIR:-/tmp/pir-manual-install}"

case "$(uname -m)" in
  x86_64) PLATFORM=linux-amd64 ;;
  aarch64|arm64) PLATFORM=linux-arm64 ;;
  *) echo "Unsupported CPU architecture: $(uname -m)" >&2; exit 1 ;;
esac

rm -rf "$WORKDIR"
mkdir -p "$WORKDIR"

curl -fsSL --retry 3 --retry-delay 2 -o "${WORKDIR}/nf-server-${PLATFORM}" \
  "${BINARY_BASE_URL}/nf-server-${TAG}-${PLATFORM}" \
  || curl -fsSL --retry 3 --retry-delay 2 -o "${WORKDIR}/nf-server-${PLATFORM}" \
    "https://github.com/${REPO}/releases/download/${TAG}/nf-server-${PLATFORM}"

curl -fsSL --retry 3 --retry-delay 2 -o "${WORKDIR}/SHA256SUMS" \
  "${BINARY_BASE_URL}/SHA256SUMS-${TAG}" \
  || curl -fsSL --retry 3 --retry-delay 2 -o "${WORKDIR}/SHA256SUMS" \
    "https://github.com/${REPO}/releases/download/${TAG}/SHA256SUMS"

curl -fsSL --retry 3 --retry-delay 2 -o "${WORKDIR}/nullifier-query-server.service" \
  "https://github.com/${REPO}/releases/download/${TAG}/nullifier-query-server.service"

grep -q " nf-server-${PLATFORM}$" "${WORKDIR}/SHA256SUMS"
grep -q " nullifier-query-server.service$" "${WORKDIR}/SHA256SUMS"
(cd "$WORKDIR" && sha256sum -c SHA256SUMS --ignore-missing)

Install The Binary And Service Files

This installs the verified release under /opt/nf-ingest, exposes nf-server on PATH, and writes the same environment defaults the installer writes.

set -euo pipefail
WORKDIR="${WORKDIR:-/tmp/pir-manual-install}"
case "$(uname -m)" in
  x86_64) PLATFORM=linux-amd64 ;;
  aarch64|arm64) PLATFORM=linux-arm64 ;;
  *) echo "Unsupported CPU architecture: $(uname -m)" >&2; exit 1 ;;
esac

sudo install -d /opt/nf-ingest /opt/nf-ingest/pir-data
sudo install -m 0755 "${WORKDIR}/nf-server-${PLATFORM}" /opt/nf-ingest/nf-server
sudo install -m 0644 "${WORKDIR}/nullifier-query-server.service" /etc/systemd/system/nullifier-query-server.service
sudo ln -sf /opt/nf-ingest/nf-server /usr/local/bin/nf-server

sudo tee /etc/default/nf-server >/dev/null <<'EOF'
SVOTE_PIR_VOTING_CONFIG_URL=https://valargroup.github.io/token-holder-voting-config/voting-config.json
SVOTE_PIR_PRECOMPUTED_BASE_URL=https://vote.fra1.digitaloceanspaces.com
EOF

/opt/nf-ingest/nf-server --version
/opt/nf-ingest/nf-server doctor --pir-data-dir /opt/nf-ingest/pir-data

Optional: write SENTRY_DSN=... to /opt/nf-ingest/.env; the service reads that file when present.

Start And Bootstrap The Service

On first start, nf-server serve resolves the active voting snapshot height from the voting config and downloads the matching PIR tiers from Valar-hosted object storage before serving local port 3000.
```
sudo systemctl daemon-reload
sudo systemctl enable --now nullifier-query-server
sudo systemctl status nullifier-query-server
curl -fsS http://127.0.0.1:3000/ready
```
If no active round exists on a fresh host, set SVOTE_PIR_BOOTSTRAP_SNAPSHOT_HEIGHT in /etc/default/nf-server to a published snapshot height, then restart the service.

Expose HTTPS To The Local Service

Proxy public HTTPS traffic to 127.0.0.1:3000 and block debug row routes. Set PIR_URL to the public URL you will publish for wallets.

PIR_DOMAIN="pir.example.org"
PIR_URL="https://${PIR_DOMAIN}"

sudo apt-get install -y caddy
sudo tee /etc/caddy/Caddyfile >/dev/null <<EOF
${PIR_DOMAIN} {
    reverse_proxy 127.0.0.1:3000

    @debug path /tier1/row/* /tier2/row/*
    handle @debug {
        respond 403
    }
}
EOF
sudo caddy validate --config /etc/caddy/Caddyfile
sudo systemctl restart caddy

If TLS is handled upstream, skip the Caddy block and set PIR_URL directly before publishing.

PIR_URL="https://pir.example.org"

Verify And Publish The PIR Endpoint

Confirm local readiness and public HTTPS access, then add the public URL to pir_endpoints[] in dynamic-voting-config.json.

curl -fsS http://127.0.0.1:3000/ready
curl -fsS http://127.0.0.1:3000/health | jq -r '.status'
curl -fsS http://127.0.0.1:3000/root | jq '{height, pir_depth, num_ranges}'
curl -fsS "${PIR_URL}/root" | jq '{height, pir_depth, num_ranges}'

{
  "url": "https://pir.example.org",
  "label": "Example PIR"
}

Edit voting config.

Rebootstrap PIR Data

Bootstrapped PIR tier data is disposable. If the local PIR data is corrupt or stuck on the wrong height, clear it and restart the service so it downloads the current snapshot again.

sudo systemctl stop nullifier-query-server
sudo rm -rf /opt/nf-ingest/pir-data/*
sudo systemctl start nullifier-query-server

This is for bootstrapped serve-only hosts. If you intentionally run synced mode, back up nullifier artifacts before wiping the data directory.

Node Operations

Follow service logs

sudo journalctl -u nullifier-query-server -f

View local readiness

curl -fsS http://127.0.0.1:3000/ready
curl -fsS http://127.0.0.1:3000/health | jq -r '.status'
curl -fsS http://127.0.0.1:3000/root | jq '{height, pir_depth, num_ranges}'

Restart service

sudo systemctl restart nullifier-query-server

Run host check

nf-server doctor --pir-data-dir /opt/nf-ingest/pir-data

Destructive PIR Teardown Permanently removes local PIR data, configuration, and services.

Destructive PIR Teardown

This permanently removes the local PIR service, configuration, and PIR data under /opt/nf-ingest. Bootstrapped serve-only hosts can rebuild from published snapshots, but synced-mode or local-only nullifier artifacts should be backed up first.

Use this when a test host should stop serving PIR. Remove the public endpoint from pir_endpoints[] before clients continue routing requests to a deleted server.

Run The Teardown Script

Run this on the PIR host. The script requires typing REMOVE PIR before it deletes local state.
```
curl -fsSL https://vote.fra1.digitaloceanspaces.com/remove-pir.sh | sudo bash
```
- Stops and removes nullifier-query-server.
- Deletes /opt/nf-ingest, /etc/default/nf-server, and the nf-server symlink when owned by this setup.
- Backs up and removes generated Caddy config only when it clearly belongs to this PIR server.
Remove Published References

Remove this host's public HTTPS URL from pir_endpoints[]. Otherwise clients may keep routing PIR requests to a server that no longer exists. Edit voting config.

Stop And Remove The Service

set -euo pipefail

sudo systemctl stop nullifier-query-server 2>/dev/null || true
sudo systemctl disable nullifier-query-server 2>/dev/null || true
sudo rm -f /etc/systemd/system/nullifier-query-server.service
sudo rm -f /etc/default/nf-server

if [ -L /usr/local/bin/nf-server ] && [ "$(readlink /usr/local/bin/nf-server)" = "/opt/nf-ingest/nf-server" ]; then
  sudo rm -f /usr/local/bin/nf-server
fi

sudo systemctl daemon-reload
sudo systemctl reset-failed nullifier-query-server 2>/dev/null || true

Delete PIR Data

This removes the default install root. It also handles the bind-mount layout used by infra-provisioned PIR hosts.

if mountpoint -q /opt/nf-ingest 2>/dev/null; then
  sudo find /opt/nf-ingest -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +
  if sudo umount /opt/nf-ingest; then
    sudo rm -rf /opt/nf-ingest
  else
    echo "Deleted /opt/nf-ingest contents, but could not unmount it." >&2
  fi
else
  sudo rm -rf /opt/nf-ingest
fi

if [ -d /mnt/pir-data ]; then
  if mountpoint -q /mnt/pir-data 2>/dev/null; then
    sudo find /mnt/pir-data -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +
    if sudo umount /mnt/pir-data; then
      sudo rm -rf /mnt/pir-data
    else
      echo "Deleted /mnt/pir-data contents, but could not unmount it." >&2
    fi
  else
    sudo rm -rf /mnt/pir-data
  fi
fi

if grep -Eq '[[:space:]](/opt/nf-ingest|/mnt/pir-data)[[:space:]]' /etc/fstab 2>/dev/null; then
  sudo cp /etc/fstab "/etc/fstab.before-pir-remove.$(date -u +%Y%m%dT%H%M%SZ)"
  sudo sed -i -e '\@[[:space:]]/opt/nf-ingest[[:space:]]@d' -e '\@[[:space:]]/mnt/pir-data[[:space:]]@d' /etc/fstab
fi

Remove Dedicated Caddy Config

Only run this if the Caddyfile was created solely for this PIR server. Leave shared Caddy configs in place and remove only the PIR reverse proxy block.

CADDYFILE="/etc/caddy/Caddyfile"
if [ -f "$CADDYFILE" ] && grep -Eq 'reverse_proxy[[:space:]]+(localhost|127[.]0[.]0[.]1):3000' "$CADDYFILE"; then
  sudo cp "$CADDYFILE" "${CADDYFILE}.before-pir-remove.$(date -u +%Y%m%dT%H%M%SZ)"
  sudo rm -f "$CADDYFILE"
  sudo systemctl stop caddy 2>/dev/null || true
  sudo systemctl disable caddy 2>/dev/null || true
fi

PIR Server Setup

Fresh PIR Setup

Provision A Dedicated PIR Host

Run The PIR Installer

Expose HTTPS To The Local Service

Publish The PIR Endpoint

Provision A Dedicated PIR Host

Install Host Requirements

Download And Verify The Release

Install The Binary And Service Files

Start And Bootstrap The Service

Expose HTTPS To The Local Service

Verify And Publish The PIR Endpoint

Rebootstrap PIR Data

Node Operations

Follow service logs

View local readiness

Restart service

Run host check

Destructive PIR Teardown

Run The Teardown Script

Remove Published References

Stop And Remove The Service

Delete PIR Data

Remove Dedicated Caddy Config

References