最后更新于:2026年07月

K8s 代理集群
Kubernetes 自建代理集群

当单台 VPS 无法满足业务需求时——无论是并发量激增、地域覆盖不足,还是节点稳定性堪忧——Kubernetes(K8s)都能帮你将多台服务器统一管理为一个代理集群。通过 Helm Chart 一键部署 xray/sing-box,配合多地域 Pod 调度与健康检查,实现自动扩缩容、故障自愈和流量智能分发。

本文面向已掌握基础代理运维的进阶用户,将带你从零搭建一个生产级的 K8s 代理集群。


🧭 为什么选择 K8s 管理代理?

传统方案的瓶颈

PLAINTEXT
单节点 VPS 方案:

  客户端 → 单台 VPS(xray/sing-box) → 目标网站
     │
     ├── 性能上限 = 单台 VPS 性能
     ├── 单点故障风险
     ├── 无法自动扩容
     └── 地域固定,无法智能调度

多节点手动方案:

  客户端 → 负载均衡器 → VPS A / VPS B / VPS C
     │
     ├── 需要手动管理每台 VPS
     ├── 配置不一致风险
     ├── 故障切换靠人工
     └── 扩容 = 重复手动部署

K8s 代理集群的优势

PLAINTEXT
┌──────────────────────────────────────────────────────────┐
│              K8s 代理集群架构                            │
│                                                          │
│  客户端                                                   │
│     │                                                    │
│     │  Cloudflare Load Balancer / 自建 DNS              │
│     ▼                                                    │
│  ┌─────────────────────────────────────┐                │
│  │         Kubernetes 集群              │                │
│  │                                      │                │
│  │  ┌─────┐  ┌─────┐  ┌─────┐        │                │
│  │  │Pod 1│  │Pod 2│  │Pod 3│        │                │
│  │  │日本 │  │美国 │  │新加坡│        │                │
│  │  │Node │  │Node │  │Node │        │                │
│  │  └──┬──┘  └──┬──┘  └──┬──┘        │                │
│  │     └─────────┴─────────┘          │                │
│  │            Service                  │                │
│  │            (ClusterIP/NodePort)     │                │
│  └─────────────────────────────────────┘                │
│                                                          │
│  优势:                                                   │
│  1. 自动调度:Pod 自动分配到不同地域 Node                 │
│  2. 自动扩容:CPU/带宽高时自动增加 Pod                   │
│  3. 故障自愈:Pod 崩溃后自动重启/重建                   │
│  4. 配置统一:ConfigMap 集中管理,所有 Pod 一致          │
│  5. 滚动更新:零停机更新配置/镜像                        │
│  6. 多地域:不同地域的 Node 组成一个集群                 │
└──────────────────────────────────────────────────────────┘

📦 环境准备

集群规划

节点类型数量配置地域
Master1-32 CPU / 4GB / 50GB日本
Worker-日本1-22 CPU / 2GB / 30GB日本
Worker-美国1-22 CPU / 2GB / 30GB美国
Worker-新加坡1-22 CPU / 2GB / 30GB新加坡

推荐: 3 Worker × 3 地域 = 9 节点,形成高可用集群。

安装 K8s(kubeadm 方式)

1. 所有节点基础配置

BASH
# 关闭 swap
swapoff -a
sed -i '/swap/d' /etc/fstab

# 加载内核模块
cat > /etc/modules-load.d/k8s.conf << 'EOF'
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

# 设置 sysctl
cat > /etc/sysctl.d/k8s.conf << 'EOF'
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sysctl --system

# 安装 containerd
apt update
apt install -y containerd

# 配置 containerd
mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
systemctl restart containerd

# 安装 kubeadm/kubelet/kubectl
apt install -y apt-transport-https ca-certificates curl gnupg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | \
  gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
  https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | \
  tee /etc/apt/sources.list.d/kubernetes.list

apt update
apt install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

2. 初始化 Master 节点

BASH
# 只在 Master 节点执行
kubeadm init \
  --pod-network-cidr=10.244.0.0/16 \
  --service-cidr=10.96.0.0/12 \
  --apiserver-advertise-address=<MASTER_IP>

# 配置 kubectl
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# 保存 join token(Worker 节点需要)
kubeadm token create --print-join-command
# 输出类似:
# kubeadm join 1.2.3.4:6443 --token xxx --discovery-token-ca-cert-hash sha256:xxx

3. Worker 节点加入集群

BASH
# 在每个 Worker 节点执行
kubeadm join <MASTER_IP>:6443 --token <TOKEN> \
  --discovery-token-ca-cert-hash sha256:<HASH>

4. 安装网络插件(Flannel)

BASH
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

# 验证
kubectl get nodes
kubectl get pods -n kube-system

5. 为节点添加地域标签

BASH
# 为日本节点打标签
kubectl label node worker-jp-1 region=jp
kubectl label node worker-jp-2 region=jp

# 为美国节点打标签
kubectl label node worker-us-1 region=us
kubectl label node worker-us-2 region=us

# 为新加坡节点打标签
kubectl label node worker-sg-1 region=sg
kubectl label node worker-sg-2 region=sg

# 验证
kubectl get nodes --show-labels

🚀 部署 xray 代理服务

方案一:原生 K8s 资源(理解原理)

1. 创建 ConfigMap(代理配置)

YAML
# xray-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: xray-config
data:
  config.json: |
    {
      "inbounds": [
        {
          "port": 443,
          "protocol": "vless",
          "settings": {
            "clients": [
              {
                "id": "your-uuid-here",
                "level": 0
              }
            ],
            "decryption": "none"
          },
          "streamSettings": {
            "network": "ws",
            "wsSettings": {
              "path": "/secretpath"
            },
            "security": "tls",
            "tlsSettings": {
              "certificates": [
                {
                  "certificateFile": "/etc/xray/cert.pem",
                  "keyFile": "/etc/xray/key.pem"
                }
              ]
            }
          }
        }
      ],
      "outbounds": [
        {
          "protocol": "freedom"
        }
      ]
    }
BASH
kubectl apply -f xray-config.yaml

2. 创建 Secret(TLS 证书)

BASH
# 创建 TLS Secret
kubectl create secret tls xray-tls \
  --cert=/path/to/fullchain.pem \
  --key=/path/to/privkey.pem

3. 创建 Deployment

YAML
# xray-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: xray-proxy
  labels:
    app: xray-proxy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: xray-proxy
  template:
    metadata:
      labels:
        app: xray-proxy
    spec:
      containers:
      - name: xray
        image: teddysun/xray:latest
        ports:
        - containerPort: 443
        volumeMounts:
        - name: config
          mountPath: /etc/xray/config.json
          subPath: config.json
        - name: tls
          mountPath: /etc/xray/cert.pem
          subPath: tls.crt
        - name: tls
          mountPath: /etc/xray/key.pem
          subPath: tls.key
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        readinessProbe:
          tcpSocket:
            port: 443
          initialDelaySeconds: 5
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 443
          initialDelaySeconds: 15
          periodSeconds: 20
      volumes:
      - name: config
        configMap:
          name: xray-config
      - name: tls
        secret:
          secretName: xray-tls
BASH
kubectl apply -f xray-deployment.yaml

4. 创建 Service

YAML
# xray-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: xray-service
spec:
  selector:
    app: xray-proxy
  ports:
  - protocol: TCP
    port: 443
    targetPort: 443
  type: NodePort
BASH
kubectl apply -f xray-service.yaml

# 查看分配的 NodePort
kubectl get svc xray-service
# 输出类似:443:32123/TCP

5. 配置地域调度(Node Affinity)

YAML
# xray-deployment-affinity.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: xray-jp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: xray-jp
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: region
                operator: In
                values:
                - jp
      containers:
      - name: xray
        image: teddysun/xray:latest
        # ... 其他配置
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: xray-us
spec:
  replicas: 2
  selector:
    matchLabels:
      app: xray-us
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: region
                operator: In
                values:
                - us
      containers:
      - name: xray
        image: teddysun/xray:latest
        # ... 其他配置
BASH
kubectl apply -f xray-deployment-affinity.yaml

📦 方案二:Helm Chart(生产推荐)

创建 Helm Chart

BASH
# 创建 Chart
helm create proxy-cluster

# 目录结构
# proxy-cluster/
# ├── Chart.yaml
# ├── values.yaml
# ├── templates/
# │   ├── deployment.yaml
# │   ├── service.yaml
# │   ├── configmap.yaml
# │   ├── secret.yaml
# │   ├── hpa.yaml
# │   └── ingress.yaml

Chart.yaml

YAML
apiVersion: v2
name: proxy-cluster
description: A Helm chart for deploying xray/sing-box proxy cluster
type: application
version: 1.0.0
appVersion: "1.8.0"

values.yaml

YAML
# 全局配置
replicaCount: 3

image:
  repository: teddysun/xray
  pullPolicy: IfNotPresent
  tag: "latest"

# 地域配置
regions:
  - name: jp
    replicas: 2
    nodeSelector:
      region: jp
  - name: us
    replicas: 2
    nodeSelector:
      region: us
  - name: sg
    replicas: 2
    nodeSelector:
      region: sg

# 代理配置
proxy:
  protocol: vless
  port: 443
  uuid: "your-uuid-here"
  network: ws
  wsPath: "/secretpath"
  tls: true
  domain: "proxy.example.com"

# 资源限制
resources:
  requests:
    memory: "128Mi"
    cpu: "100m"
  limits:
    memory: "512Mi"
    cpu: "500m"

# 自动扩缩容
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 20
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 80

# 健康检查
healthCheck:
  enabled: true
  readinessProbe:
    initialDelaySeconds: 5
    periodSeconds: 10
  livenessProbe:
    initialDelaySeconds: 15
    periodSeconds: 20

# 服务类型
service:
  type: NodePort
  port: 443

templates/deployment.yaml

YAML
{{- range .Values.regions }}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "proxy-cluster.fullname" $ }}-{{ .name }}
  labels:
    {{- include "proxy-cluster.labels" $ | nindent 4 }}
    region: {{ .name }}
spec:
  replicas: {{ .replicas }}
  selector:
    matchLabels:
      {{- include "proxy-cluster.selectorLabels" $ | nindent 6 }}
      region: {{ .name }}
  template:
    metadata:
      labels:
        {{- include "proxy-cluster.selectorLabels" $ | nindent 8 }}
        region: {{ .name }}
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: region
                operator: In
                values:
                - {{ .name }}
      containers:
      - name: xray
        image: "{{ $.Values.image.repository }}:{{ $.Values.image.tag }}"
        imagePullPolicy: {{ $.Values.image.pullPolicy }}
        ports:
        - containerPort: {{ $.Values.proxy.port }}
        volumeMounts:
        - name: config
          mountPath: /etc/xray/config.json
          subPath: config.json
        - name: tls
          mountPath: /etc/xray/cert.pem
          subPath: tls.crt
        - name: tls
          mountPath: /etc/xray/key.pem
          subPath: tls.key
        resources:
          {{- toYaml $.Values.resources | nindent 10 }}
        {{- if $.Values.healthCheck.enabled }}
        readinessProbe:
          tcpSocket:
            port: {{ $.Values.proxy.port }}
          initialDelaySeconds: {{ $.Values.healthCheck.readinessProbe.initialDelaySeconds }}
          periodSeconds: {{ $.Values.healthCheck.readinessProbe.periodSeconds }}
        livenessProbe:
          tcpSocket:
            port: {{ $.Values.proxy.port }}
          initialDelaySeconds: {{ $.Values.healthCheck.livenessProbe.initialDelaySeconds }}
          periodSeconds: {{ $.Values.healthCheck.livenessProbe.periodSeconds }}
        {{- end }}
      volumes:
      - name: config
        configMap:
          name: {{ include "proxy-cluster.fullname" $ }}-config
      - name: tls
        secret:
          secretName: {{ include "proxy-cluster.fullname" $ }}-tls
---
{{- end }}

templates/hpa.yaml(自动扩缩容)

YAML
{{- if .Values.autoscaling.enabled }}
{{- range .Values.regions }}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: {{ include "proxy-cluster.fullname" $ }}-{{ .name }}
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include "proxy-cluster.fullname" $ }}-{{ .name }}
  minReplicas: {{ .replicas }}
  maxReplicas: {{ $.Values.autoscaling.maxReplicas }}
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: {{ $.Values.autoscaling.targetCPUUtilizationPercentage }}
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: {{ $.Values.autoscaling.targetMemoryUtilizationPercentage }}
---
{{- end }}
{{- end }}

安装 Helm Chart

BASH
# 安装
helm install proxy-cluster ./proxy-cluster \
  --set proxy.uuid="your-uuid-here" \
  --set proxy.domain="proxy.example.com"

# 升级
helm upgrade proxy-cluster ./proxy-cluster \
  --set proxy.uuid="new-uuid" \
  --set autoscaling.maxReplicas=30

# 查看状态
helm list
kubectl get pods -l app.kubernetes.io/name=proxy-cluster
kubectl get hpa

🎯 多地域调度策略

方案一:地域专属 Deployment

每个地域独立部署,客户端手动选择:

YAML
# 日本地域
apiVersion: apps/v1
kind: Deployment
metadata:
  name: proxy-jp
spec:
  replicas: 3
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: region
                operator: In
                values: ["jp"]
---
# 美国地域
apiVersion: apps/v1
kind: Deployment
metadata:
  name: proxy-us
spec:
  replicas: 3
  template:
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: region
                operator: In
                values: ["us"]

客户端配置:

YAML
# Clash Meta 配置
proxies:
  - name: "JP-Cluster"
    type: vless
    server: jp-node-ip
    port: 443
    # ...
  - name: "US-Cluster"
    type: vless
    server: us-node-ip
    port: 443
    # ...

proxy-groups:
  - name: "Auto-Select"
    type: url-test
    proxies:
      - JP-Cluster
      - US-Cluster
    url: http://www.gstatic.com/generate_204
    interval: 300

方案二:智能 DNS 调度

使用 GeoDNS 根据用户地理位置返回最近节点:

PLAINTEXT
用户在中国 → DNS 返回日本节点 IP
用户在欧洲 → DNS 返回美国节点 IP
用户在东南亚 → DNS 返回新加坡节点 IP

使用 Cloudflare Load Balancer:

BASH
# 在 Cloudflare Dashboard 中配置
# Traffic → Load Balancing

# 创建 Origin Pool
Pool-JP:
  - jp-node-1.example.com
  - jp-node-2.example.com

Pool-US:
  - us-node-1.example.com
  - us-node-2.example.com

Pool-SG:
  - sg-node-1.example.com
  - sg-node-2.example.com

# 配置 Geo-Routing
规则 1:中国大陆 → Pool-JP
规则 2:北美 → Pool-US
规则 3:亚太(除中国) → Pool-SG
规则 4:默认 → Pool-JP

方案三:Anycast IP + BGP

如果你有自治系统号(ASN),可以广播 Anycast IP:

PLAINTEXT
┌──────────────────────────────────────────────────────────┐
│              Anycast 调度                                │
│                                                          │
│  用户请求 → Anycast IP                                  │
│                │                                         │
│                ├── 中国用户 → 日本节点(最近)           │
│                ├── 美国用户 → 美国节点(最近)           │
│                └── 印度用户 → 新加坡节点(最近)         │
│                                                          │
│  实现:BGP 广播相同 IP 前缀到不同地域                    │
│  要求:需要 ASN 和 BGP 会话                              │
└──────────────────────────────────────────────────────────┘

📊 监控与运维

Prometheus + Grafana 监控

部署 Prometheus:

BASH
# 使用 Helm 安装
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace

配置 xray 指标导出:

YAML
# 在 xray 配置中启用 stats
apiVersion: v1
kind: ConfigMap
metadata:
  name: xray-config
data:
  config.json: |
    {
      "stats": {},
      "api": {
        "tag": "api",
        "services": ["StatsService"]
      },
      "policy": {
        "levels": {
          "0": {
            "statsUserUplink": true,
            "statsUserDownlink": true
          }
        },
        "system": {
          "statsInboundUplink": true,
          "statsInboundDownlink": true
        }
      }
    }

Grafana Dashboard:

YAML
# proxy-dashboard.json(关键指标)
- Pod CPU 使用率
- Pod 内存使用率
- 网络流量(入站/出站)
- Pod 重启次数
- HPA 扩缩容事件
- 地域分布
- 延迟分布

日志收集

BASH
# 使用 EFK / PLG 堆栈
helm install fluent-bit fluent/fluent-bit \
  --namespace logging \
  --create-namespace

# 查看 xray 日志
kubectl logs -l app=xray-proxy --tail=100 -f

# 查看特定 Pod 日志
kubectl logs xray-proxy-jp-7d8f9b2c5-x1a2b

自动故障转移

YAML
# Pod Disruption Budget(保证最少可用 Pod)
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: xray-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: xray-proxy

🔒 安全加固

网络策略

YAML
# 只允许特定端口和来源
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: xray-network-policy
spec:
  podSelector:
    matchLabels:
      app: xray-proxy
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from: []
    ports:
    - protocol: TCP
      port: 443
  egress:
  - to: []
    ports:
    - protocol: TCP
      port: 443
    - protocol: UDP
      port: 53

Secret 管理

BASH
# 使用 Sealed Secrets 加密
kubeseal --format=yaml < secret.yaml > sealed-secret.yaml
kubectl apply -f sealed-secret.yaml

# 或使用 External Secrets Operator
# 从 Vault / AWS Secrets Manager 获取

RBAC 权限控制

YAML
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: proxy-operator
rules:
- apiGroups: [""]
  resources: ["pods", "services", "configmaps"]
  verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "update", "patch"]

🔍 故障排查

问题 1:Pod 无法调度

BASH
# 查看 Pod 状态
kubectl get pods -o wide

# 查看事件
kubectl describe pod xray-proxy-jp-xxx

# 常见原因:
# 1. 节点资源不足
kubectl describe node worker-jp-1

# 2. Node Affinity 不匹配
kubectl get nodes --show-labels

# 3. 镜像拉取失败
kubectl logs xray-proxy-jp-xxx

问题 2:HPA 不生效

BASH
# 检查 HPA 状态
kubectl get hpa
kubectl describe hpa xray-jp

# 检查 metrics-server
kubectl get pods -n kube-system | grep metrics

# 安装 metrics-server
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

问题 3:配置更新不生效

BASH
# 更新 ConfigMap 后需要重启 Pod
kubectl rollout restart deployment xray-proxy

# 或设置自动滚动更新
kubectl set env deployment/xray-proxy UPDATE_TIMESTAMP=$(date +%s)

📋 完整部署检查清单

PLAINTEXT
集群搭建:
□ 准备 3+ 台服务器(Master + Worker)
□ 安装 containerd
□ 安装 kubeadm/kubelet/kubectl
□ 初始化 Master 节点
□ Worker 节点加入集群
□ 安装网络插件(Flannel)
□ 为节点打地域标签

代理部署:
□ 创建 ConfigMap(代理配置)
□ 创建 Secret(TLS 证书)
□ 创建 Deployment(多地域)
□ 创建 Service(NodePort)
□ 配置 HPA(自动扩缩容)
□ 配置 PDB(故障转移)

监控运维:
□ 安装 Prometheus + Grafana
□ 配置 xray 指标导出
□ 创建 Grafana Dashboard
□ 配置日志收集
□ 设置告警规则

安全配置:
□ 配置 NetworkPolicy
□ 使用 Sealed Secrets
□ 配置 RBAC
□ 定期更新镜像

结语

Kubernetes 为代理服务带来了企业级的管理能力——自动调度、自动扩缩容、故障自愈、统一配置。通过多地域节点部署,你可以构建一个高可用、高性能的全球代理网络。

总结要点:

推荐架构:

PLAINTEXT
入门(单地域):
  1 Master + 2 Worker + Helm Chart + HPA

进阶(多地域):
  1 Master + 6 Worker(3 地域 × 2)+ GeoDNS + HPA

生产(高可用):
  3 Master + 9 Worker(3 地域 × 3)+ Anycast + 完整监控

记住:K8s 不是银弹,但对于需要管理多台代理服务器的场景,它是最优雅的选择。

愿你的代理集群永远稳定、高效、弹性!⚓

版权声明

作者: 易邦

链接: https://blog.e8k.net/posts/k8s-proxy-cluster/

许可证: 知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议

本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可。