部署ATS（Apache Traffic Server）和Nginx正向代理服务&性能对比

1. 正向代理的用途

正向代理一般是用于内部网络出去，反向代理一般是用于外部网络进入

?? 某同学喜欢面向搜索引擎编程，想通过 百度搜索引擎查找一些学习资料，但是有些网站直接访问可能不太安全，会暴露自己的IP，同学比较苦恼，想着怎样才能使用百度 搜索自己想要的学习资料，又不会暴露自己的IP在网站上呢？这时我告诉该同学，手上刚好有一台代理服务器，这台代理服务器通过nginx配置了正向代理转发http和https请求，你呢，只需要在自己的Windows本地电脑的网关配置一下这台代理服务器的IP和端口号，就能正常通过代理服务器访问到百度 并搜索相关的学习资料了，还不会暴露自己真实的IP
?? 正向代理用来进行上网等功能。如果把局域网外的Internet想象成一个巨大的资源库，则局域网中的客户端要访问Internet，则需要通过代理服务器来访问，这种代理服务就称为正向代理（也就是大家常说的，通过正向代理进行上网功能）
?? 现在的网站基本上都是https，要解决既能访问http80端口也能访问https443端口的网站，需要配置两个SERVER节点，一个处理HTTP转发，另一个处理HTTPS转发，而客户端都通过HTTP来访问代理，通过访问代理不同的端口，来区分HTTP和HTTPS请求

?

2. ATS(Apache Traffic Server)正向代理服务器部署

?? Apache Traffic Server官方部署手册

[root@vm10-1-248-42 ~]# wget -O /etc/yum.repos.d/aliyun.repo http://mirrors.aliyun.com/repo/Centos-7.repo
[root@vm10-1-248-42 ~]# wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[root@vm10-1-248-42 ~]# rpm -Uvh epel-release-latest-7*.rpm
[root@vm10-1-248-42 ~]# yum install trafficserver -y

?? 配置Configuring A Forward Proxy（正向代理）：

[root@vm10-1-248-42 ~]# cp /etc/trafficserver/records.config /etc/trafficserver/records.config.bak
[root@vm10-1-248-42 ~]# vim /etc/trafficserver/records.config

CONFIG proxy.config.http.server_ports STRING 8088 				// 配置traffic server开放的端口。默认端口号是8080，推荐更换
CONFIG proxy.config.http.insert_request_via_str INT 0  
CONFIG proxy.config.url_remap.remap_required INT 0				// 禁用重映射规则存在的要求，并在traffic server将请求代理到远程主机之前匹配传入请求
CONFIG proxy.config.http.cache.http INT 0						// 关闭代理HTTP请求的缓存
CONFIG proxy.config.reverse_proxy.enabled INT 0					// 关闭反向代理
CONFIG proxy.config.log.max_space_mb_for_logs INT 200000

[root@vm10-1-248-42 ~]# systemctl start trafficserver 
[root@vm10-1-248-42 ~]# systemctl status trafficserver 
● trafficserver.service - Apache Traffic Server is a fast, scalable and extensible caching proxy server.
   Loaded: loaded (/usr/lib/systemd/system/trafficserver.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2024-01-12 13:59:22 CST; 4s ago
     Docs: man:traffic_server(8)
 Main PID: 20309 (traffic_manager)
   CGroup: /system.slice/trafficserver.service
           ├─20309 /usr/bin/traffic_manager
           └─20314 /usr/bin/traffic_server -M --httpport 8088:fd=8

Jan 12 13:59:22 vm10-1-248-42.cloud.local systemd[1]: Started Apache Traffic Server is a fast, scalable and extensible caching proxy server..
Jan 12 13:59:22 vm10-1-248-42.cloud.local traffic_manager[20309]: [E. Mgmt] log ==> [TrafficManager] using root directory '/usr'
Jan 12 13:59:22 vm10-1-248-42.cloud.local traffic_manager[20309]: NOTE: --- Manager Starting ---
Jan 12 13:59:22 vm10-1-248-42.cloud.local traffic_manager[20309]: NOTE: Manager Version: Apache Traffic Server - traffic_manager - 9.2.3 - (build #...:08:30)
Jan 12 13:59:22 vm10-1-248-42.cloud.local traffic_manager[20309]: NOTE: RLIMIT_NOFILE(7):cur(687832),max(687832)
Jan 12 13:59:25 vm10-1-248-42.cloud.local traffic_server[20314]: NOTE: --- traffic_server Starting ---
Jan 12 13:59:25 vm10-1-248-42.cloud.local traffic_server[20314]: NOTE: traffic_server Version: Apache Traffic Server - traffic_server - 9.2.3 - (bu...:08:30)
Jan 12 13:59:25 vm10-1-248-42.cloud.local traffic_server[20314]: NOTE: RLIMIT_NOFILE(7):cur(687832),max(687832)
Hint: Some lines were ellipsized, use -l to show in full.

[root@vm10-1-248-42 ~]# ss -tnlp | grep traffic_manager
LISTEN     0      128          *:8088                     *:*                   users:(("[TS_MAIN]",pid=20314,fd=8),("traffic_manager",pid=20309,fd=8))

[root@vm10-1-248-48 ~]# vim /etc/profile
export http_proxy='10.1.248.42:8088'
export https_proxy='10.1.248.42:8088'

[root@vm10-1-248-48 ~]# source /etc/profile
[root@vm10-1-248-48 ~]# echo $http_proxy
10.1.248.42:8088
[root@vm10-1-248-48 ~]# echo $https_proxy
10.1.248.42:8088

?? Linux客户端代理访问：

[root@vm10-1-248-48 ~]# curl http://www.baidu.com/ -v
* About to connect() to proxy 10.1.248.42 port 8088 (#0)
*   Trying 10.1.248.42...
* Connected to 10.1.248.42 (10.1.248.42) port 8088 (#0)
> GET http://www.baidu.com/ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Content-Length: 2381
< Content-Type: text/html
< Date: Fri, 12 Jan 2024 06:26:24 GMT
< Etag: "588604c8-94d"
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Pragma: no-cache
< Server: ATS/9.2.3
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Age: 0
< Proxy-Connection: keep-alive
< 
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;

[root@vm10-1-248-48 ~]# curl https://www.baidu.com/ -v
* About to connect() to proxy 10.1.248.42 port 8088 (#0)
*   Trying 10.1.248.42...
* Connected to 10.1.248.42 (10.1.248.42) port 8088 (#0)
* Establish HTTP proxy tunnel to www.baidu.com:443
> CONNECT www.baidu.com:443 HTTP/1.1
> Host: www.baidu.com:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Date: Fri, 12 Jan 2024 06:27:02 GMT
< Proxy-Connection: keep-alive
< Server: ATS/9.2.3
< 
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",L=beijing,ST=beijing,C=CN
* 	start date: Jul 06 01:51:06 2023 GMT
* 	expire date: Aug 06 01:51:05 2024 GMT
* 	common name: baidu.com
* 	issuer: CN=GlobalSign RSA OV SSL CA 2018,O=GlobalSign nv-sa,C=BE
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2443
< Content-Type: text/html
< Date: Fri, 12 Jan 2024 06:27:03 GMT
< Etag: "588603eb-98b"
< Last-Modified: Mon, 23 Jan 2017 13:23:55 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< 
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;

3. Nginx正向代理服务器部署

[root@vm10-1-248-197 ~]# /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.22.0
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx --user=nginx --group=nginx --pid-path=/var/run/nginx/nginx.pid --lock-path=/var/lock/nginx.lock --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --with-http_gzip_static_module --with-http_gunzip_module --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --http-client-body-temp-path=/usr/local/nginx/client --http-proxy-temp-path=/usr/local/nginx/proxy --http-fastcgi-temp-path=/usr/local/nginx/fastcgi --http-uwsgi-temp-path=/usr/local/nginx/uwsgi --http-scgi-temp-path=/usr/local/nginx/scgi

?? nginx官方并不支持直接转发https请求,但是阿里对nginx拓展了一个ngx_http_proxy_connect_module模块，并且在github上开源了 https://github.com/chobits/ngx_http_proxy_connect_module。网络上很多没有patch该模块，而是直接用http的配置，只是修改一下端口为443，这样是不可行的，当正向代理https站点时，会报非200错误码。

?? 这里安装nginx通过install进行编译安装，编译后默认安装目录为/usr/local/nginx，后续配置新模块ngx_http_proxy_connect_module还需要重新install编译一次
?? 将新模块ngx_http_proxy_connect_module源码压缩包上传到服务器上，并解压并重命名

[root@vm10-1-248-197 ~]# unzip ngx_http_proxy_connect_module-master.zip
[root@vm10-1-248-197 ~]# mv ngx_http_proxy_connect_module-master ngx_http_proxy_connect_module

[root@vm10-1-248-197 ~]# cd nginx-1.22.0
[root@vm10-1-248-197 nginx-1.22.0]# pwd
/root/nginx-1.22.0
[root@vm10-1-248-197 nginx-1.22.0]# patch -p1 < /root/ngx_http_proxy_connect_module/patch/proxy_connect_rewrite_102101.patch
patching file src/http/ngx_http_core_module.c
patching file src/http/ngx_http_parse.c
patching file src/http/ngx_http_request.c
patching file src/http/ngx_http_request.h
patching file src/http/ngx_http_variables.c

[root@vm10-1-248-197 nginx-1.22.0]# ./configure --add-module=/root/ngx_http_proxy_connect_module 
[root@vm10-1-248-197 nginx-1.22.0]# make && make install 

[root@vm10-1-248-197 ~]# /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.22.0
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
configure arguments: --add-module=/root/ngx_http_proxy_connect_module

?? 修改nginx的配置：
?? 修改nginx的配置分别添加http和https的server，其他配置保持不变。

    server {
        listen       80;
        resolver 114.114.114.114;
        location / {
            proxy_pass http://$host$request_uri;
            proxy_set_header HOST $host;
            proxy_buffers 256 4k;
            proxy_max_temp_file_size 0k;
            proxy_connect_timeout 30;
            proxy_send_timeout 60;
            proxy_read_timeout 60;
            proxy_next_upstream error timeout invalid_header http_502;
        }
   }


    server {
     	 resolver 114.114.114.114;
      	 listen 443;

     	 proxy_connect;
     	 proxy_connect_allow            443 563;
     	 proxy_connect_connect_timeout  10s;
     	 proxy_connect_read_timeout     10s;
     	 proxy_connect_send_timeout     10s;
       	 location / {
            	proxy_pass https://$host$request_uri;
           	    proxy_set_header Host $host;
       }
  }

[root@vm10-1-248-197 conf]# /usr/local/nginx/sbin/nginx -s reload

?? 目前比较主流的DNS：
?? ● （国外）谷歌：8.8.8.8 ：developers.google.com
?? ● （国外）OpenDNS：208.67.222.222 signup.opendns.com
?? ● （国内）114：114.114.114.114 www.114dns.com
?? ● （国内）腾讯：119.29.29.29 www.dnspod.cn
?? ● （国内）阿里：223.5.5.5 alidns.com
?? ● （国内）百度：180.76.76.76 dudns.baidu.com

?? Linux客户端代理访问：

[root@vm10-1-248-48 ~]# curl http://www.baidu.com/ -v -x 10.1.248.197:80
* About to connect() to proxy 10.1.248.197 port 80 (#0)
*   Trying 10.1.248.197...
* Connected to 10.1.248.197 (10.1.248.197) port 80 (#0)
> GET http://www.baidu.com/ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Server: nginx/1.22.0
< Date: Mon, 08 Jan 2024 05:44:21 GMT
< Content-Type: text/html
< Content-Length: 2381
< Connection: keep-alive
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Etag: "588604c8-94d"
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/

[root@vm10-1-248-48 ~]# curl https://www.baidu.com/ -v -x 10.1.248.197:443
* About to connect() to proxy 10.1.248.197 port 443 (#0)
*   Trying 10.1.248.197...
* Connected to 10.1.248.197 (10.1.248.197) port 443 (#0)
* Establish HTTP proxy tunnel to www.baidu.com:443
> CONNECT www.baidu.com:443 HTTP/1.1
> Host: www.baidu.com:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection Established
< Proxy-agent: nginx
< 
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: CN=baidu.com,O="Beijing Baidu Netcom Science Technology Co., Ltd",L=beijing,ST=beijing,C=CN
* 	start date: Jul 06 01:51:06 2023 GMT
* 	expire date: Aug 06 01:51:05 2024 GMT
* 	common name: baidu.com
* 	issuer: CN=GlobalSign RSA OV SSL CA 2018,O=GlobalSign nv-sa,C=BE
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Accept-Ranges: bytes
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Connection: keep-alive
< Content-Length: 2443
< Content-Type: text/html
< Date: Mon, 08 Jan 2024 06:38:08 GMT
< Etag: "588603eb-98b"
< Last-Modified: Mon, 23 Jan 2017 13:23:55 GMT
< Pragma: no-cache
< Server: bfe/1.0.8.18
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/

?? 在nginx正向代理服务器中查看日志：

[root@vm10-1-248-197 ~]# tail -f  /usr/local/nginx/logs/access.log 
// http代理日志
10.1.248.48 - - [08/Jan/2024:14:52:54 +0800] "GET http://www.baidu.com/ HTTP/1.1" 200 2381 "-" "curl/7.29.0"

// https代理日志
10.1.248.48 - - [08/Jan/2024:14:53:49 +0800] "CONNECT www.baidu.com:443 HTTP/1.1" 200 8223 "-" "curl/7.29.0"

?? 设置Linux客户端全局代理：

[root@vm10-1-248-48 ~]# export https_proxy=10.1.248.197:443
[root@vm10-1-248-48 ~]# export http_proxy=10.1.248.197:80
[root@vm10-1-248-48 ~]# echo $http_proxy
10.1.248.197:80
[root@vm10-1-248-48 ~]# echo $https_proxy
10.1.248.197:443

[root@vm10-1-248-48 ~]# vim /etc/profile
export http_proxy='10.1.248.197:80'
export https_proxy='10.1.248.197:443'
[root@vm10-1-248-48 ~]# source /etc/profile

[root@vm10-1-248-48 ~]# curl http://www.baidu.com -v
[root@vm10-1-248-48 ~]# curl https://www.baidu.com -v

4. 性能对比

?? 安装webbench：

[root@vm10-1-248-48 ~]# yum install ctags wget make apr* autoconf automake gcc gcc-c++
[root@vm10-1-248-48 ~]# tar xvf webbench-1.5.tar.gz 
[root@vm10-1-248-48 ~]# mkdir -p /usr/local/man
[root@vm10-1-248-48 ~]# chmod 644 /usr/local/man/
[root@vm10-1-248-48 ~]# cd webbench-1.5
[root@vm10-1-248-48 ~]# make && make install 

webbench -c 并发数 -t 运行测试时间 URL

// -c 500：表示同时产生500个并发连接
// -t 600：表示持续600秒

?? 使用webbench工具对使用traffic server正向代理时进行压测：

[root@vm10-1-248-48 ~]# webbench --proxy 10.1.248.42:8088 -c 2000 -t 600 http://www.baidu.com/
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET http://www.baidu.com/
2000 clients, running 600 sec, via proxy server 10.1.248.42:8088.

Speed=14697 pages/min, -1485793 bytes/sec.
Requests: 146974 susceed, 0 failed.

[root@vm10-1-248-48 ~]# webbench --proxy 10.1.248.42:8088 -c 2000 -t 600 https://www.baidu.com/
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET https://www.baidu.com/
2000 clients, running 600 sec, via proxy server 10.1.248.42:8088.

Speed=1112225 pages/min, 349243 bytes/sec.
Requests: 11122256 susceed, 0 failed.

?? 使用webbench工具对使用nginx正向代理时进行压测：

[root@vm10-1-248-48 ~]# webbench --proxy 10.1.248.197:80 -c 2000 -t 600 http://www.baidu.com/
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET http://www.baidu.com/
2000 clients, running 600 sec, via proxy server 10.1.248.197:80.

Speed=51170 pages/min, 571571 bytes/sec.
Requests: 206650 susceed, 305050 failed.

[root@vm10-1-248-48 ~]# webbench --proxy 10.1.248.197:443 -c 2000 -t 600 https://www.baidu.com/
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET https://www.baidu.com/
2000 clients, running 600 sec, via proxy server 10.1.248.197:443.

Speed=1004612 pages/min, -1482305 bytes/sec.
Requests: 10045971 susceed, 151 failed.

[root@vm10-1-248-48 ~]# webbench --proxy 10.1.248.197:443 -c 2000 -t 600 https://cn.bing.com/
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.

Benchmarking: GET https://cn.bing.com/
2000 clients, running 600 sec, via proxy server 10.1.248.197:443.

Speed=1011487 pages/min, -1443486 bytes/sec.
Requests: 10114676 susceed, 195 failed.