HttpTunnel工作原理及源程序分析
2012年01月13日 点击数: 21780 字体: 大 中 小
这份报告对开源工程HttpTunnel的工作原理及源程序运行流程进行分析，重点分析HttpTunnel对HTTP协议的封装与实现。
1 工作原理概述
HttpTunnel通过HTTP请求方式，提供了一个双向的、虚拟的数据通路，可以通过http proxy来使用（不是必需）。
HttpTunnel编译连接之后得到两个可执行的文件，hts和htc，hts是服务器端程序，用于要连接的外部主机之上，htc是客户端程序，用于本地主机之上。这两个部分连接用于产生一个虚拟的数据通道（tunnel）。
Http Tunnel利用HTPP的POST与GET两个命令建立了两个连接，分别用于客户端向服务器发送和接收数据，而且考虑到了HTTP数据的合法性，会随时检查所接收或发出的数据是否超过content-length规定的长度，如果是则填充完后重新开连接进行处理新的数据，保证了数据始终是合法的HTTP数据，完全等价于客户端在同时用HTTP协议在上传和下载一个大文件，文件数据中不需要任何的HTTP命令或HTML语言标记。
2 HttpTunnel程序流程分析
2.1 服务器端程序流程分析
hts是服务器端,安装在外部网络一侧，也就是没有防火墙的一端。
hts源文件包括hts.c, common.c, tunnel.c, http.c这几个文件和port目录下的库。
流程如下：
main() (hts.c)
parse_arguments (argc, argv, &arg),解析命令行参数;
调用tunnel = tunnel_new_server (arg.port, arg.content_length)创建新的tunnel的服务端;
初始化tunnel, in_fd, out_fd都为-1
在tunnel->server_socket监听server_socket (tunnel->dest.host_port, 1), backlog=1, 注意用了SO_REUSEADDR选项; àcommon.c
通道的一些选项:strict_content_length, keep_alive, max_connection_age, 这些在HTTP头说明中会用到;
写PID文件
进入无限循环
如果定义输入输出的终端设备(arg.device),打开设备fd = open_device (arg.device),common.c;
tunnel_accept接受外部连接 àtunnel.c
accept()连接
解析HTTP协议请求数据: http_parse_request (s, &request), http.c
POST或PUT命令, 将此socket设置为tunnel的in_fd, 用于接收客户端的数据;
GET命令, 将此socket设置为tunnel的out_fd, 用于发送去往客户端的数据;
如果设置了转发端口(arg.forward_port != -1), 调用do_connect连接到这个端口,描述符为fd, 注意这个连接是本机的
自己连自己的内部连接, hts起到代理转发作用, 目标端口也就是真正的被HTTP包裹的协议端口;
进入下一个循环
poll: fd 和tunnel->server_socket, 也就是网络通道数据和内部连接数据对倒;
handle_input()
handle_device_input()处理fd输入信息; common.c
handle_input()
handle_tunnel_input()处理来自通道的信息; common.c
服务器端接收了客户端的两个连接, POST命令对应的连接服务器接收数据, GET命令对应的连接服务器发送数据。
2.2 客户端程序流程分析
htc是客户端,安装在防火墙内部网络一侧，也即客户浏览器一端。
htc源文件包括htc.c, common.c, tunnel.c, http.c, base64.c这几个文件和port目录下的库程序
流程如下：
main.c (htc.c)
parse_arguments (argc, argv, &arg); 解析命令行参数, àhtc.c
如果转发端口(arg.forward_port != -1),在此端口打开socket监听s = server_socket
(arg.forward_port, 0),
backlog=0, 其实不收连接, 注意用了SO_REUSEADDR选项, àcommon.c
进入无限循环for (;;);
定义输入输出的终端设备(arg.device),打开设备fd = open_device (arg.device),common.c;
否则如果定义了转发端口(arg.forward_port != -1),输入输出通过连接的socket来进行fd = wait_for_connection_on_socket (s),就是accept() , à htc.c
打开一个新的通道 tunnel = tunnel_new_client (arg.host_name, arg.host_port,
arg.proxy_name, arg.proxy_port,
arg.content_length); tunnel.c
注意tunnel结构中的in_fd, out_fd都初始化为-1,表示没有连接, 缺省的content_length是100K字节.
设置通道的一些选项:strict_content_length, keep_alive, max_connection_age, user_agent这些在HTTP头说明中会用到, 支持代理;
如果要进行代理认证(arg.proxy_authorization != NULL),将认证参数进行base64编码,作为通道的proxy_authorization参数;
通道连接对方tunnel_connect (tunnel), tunnel.c, 建立http tunnel,主要是调用函数tunnel_write_request (tunnel, TUNNEL_OPEN, auth_data, sizeof auth_data)
如果要写入和已经写入的数据长度超过content_length, 对tuenel进行填充;
对于客户端已经连接好的tunnel,超时时进行断开;
对于断开(或第一次连接)的客户端, 调用tunnel_out_connect (tunnel)发起连接
调用do_connect()连接服务器(第一个连接), socket描述符为tunnel->out_fd.
设置该socket的一些选项;
调用shutdown(out_fd, 0), 不接收数据;
调用http_post()函数向服务器发送HTTP的POST命令; http.c
调用tunnel_write_data (tunnel, &request, sizeof request)函数向服务器写要执行的请求(此时为TUNNEL_OPEN)
继续写data部分,先写长度,然后是数据(此时为dummy的auth_data=42, length=1),(即HTTP的POST命令向服务器写TUNNEL_OPEN命令和一个dummy数据),然后进行数据长度判断是否在此连接中数据写多了.
进入函数tunnel_in_connect (tunnel) , 数据进入的连接
调用do_connect()连接服务器(第2个连接),socket描述符为tunnel->in_fd.
设置该socket的一些选项;
调用http_get()函数向服务器发送HTTP的GET命令; à http.c
调用shutdown(in_fd, 1), 不再写数据;
调用http_parse_response (tunnel->in_fd, &response)解析HTTP服务器返回数据;
处理统计信息
此时tunnel_connect完成
进入下一个循环
poll选择用户的输入设备fd和网络tunnel的in_fd(第2条连接)数据,也就是用户的输入信息和服务器返回的信息进行对倒:
fd输入->tunnel->out_fd输出; tunnel->in_fd输入->fd输出.
handle_input()
handle_device_input()处理用户输入信息, àcommon.c
handle_input()
handle_tunnel_input()处理来自通道的信息, àcommon.c
客户端一共向服务器发起了两个连接, 第一个连接用于发送数据,第2个连接用于接收数据
3 HttpTunnel源码相关分析
3.1 HTTP头的封装与实现
3.1.1 HTTP头结构
在HttpTunnel中，将HTTP头实现为名值对。是一个递归的结构。
typedef struct http_header Http_header;
struct http_header
{
const char *name;
const char *value;
Http_header *next; /* FIXME: this is ugly; need cons cell. */
};
3.1.2 创建HTTP头
static inline Http_header *
http_alloc_header (const char *name, const char *value)
{
Http_header *header;
header = malloc (sizeof (Http_header));
if (header == NULL)
return NULL;
header->name = header->value = NULL;
header->name = strdup (name);
header->value = strdup (value);
if (name == NULL || value == NULL)
{
if (name == NULL)
free ((char *)name);
if (value == NULL)
free ((char *)value);
free (header);
return NULL;
}
return header;
}
3.1.3 添加HTTP头
Http_header *
http_add_header (Http_header **header, const char *name, const char *value)
{
Http_header *new_header;
new_header = http_alloc_header (name, value);
if (new_header == NULL)
return NULL;
new_header->next = NULL;
while (*header)
header = &(*header)->next;
*header = new_header;
return new_header;
}
3.1.4 解析HTTP头
static ssize_t
parse_header (int fd, Http_header **header)
{
unsigned char buf[2];
unsigned char *data;
Http_header *h;
size_t len;
ssize_t n;
*header = NULL;
n = read_all (fd, buf, 2);
if (n <= 0)
return n;
if (buf[0] == ' ' && buf[1] == ' ')
return n;
h = malloc (sizeof (Http_header));
if (h == NULL)
{
log_error ("parse_header: malloc failed");
return -1;
}
*header = h;
h->name = NULL;
h->value = NULL;
n = read_until (fd, ':', &data);
if (n <= 0)
return n;
data = realloc (data, n + 2);
if (data == NULL)
{
log_error ("parse_header: realloc failed");
return -1;
}
memmove (data + 2, data, n);
memcpy (data, buf, 2);
n += 2;
data[n - 1] = 0;
h->name = data;
len = n;
n = read_until (fd, ' ', &data);
if (n <= 0)
return n;
data[n - 1] = 0;
h->value = data;
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
return n;
free (data);
if (n != 1)
{
log_error ("parse_header: invalid line ending");
return -1;
}
len += n;
log_verbose ("parse_header: %s:%s", h->name, h->value);
n = parse_header (fd, &h->next);
if (n <= 0)
return n;
len += n;
return len;
}
3.1.5 写HTTP头
static ssize_t
http_write_header (int fd, Http_header *header)
{
ssize_t n = 0, m;
if (header == NULL)
return write_all (fd, " ", 2);
m = write_all (fd, (void *)header->name, strlen (header->name));
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, ": ", 2);
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, (void *)header->value, strlen (header->value));
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, " ", 2);
if (m == -1)
{
return -1;
}
n += m;
m = http_write_header (fd, header->next);
if (m == -1)
{
return -1;
}
n += m;
return n;
}
3.1.6 查找HTTP头
static Http_header *
http_header_find (Http_header *header, const char *name)
{
if (header == NULL)
return NULL;
if (strcmp (header->name, name) == 0)
return header;
return http_header_find (header->next, name);
}
3.1.7 获取HTTP头
const char *
http_header_get (Http_header *header, const char *name)
{
Http_header *h;
h = http_header_find (header, name);
if (h == NULL)
return NULL;
return h->value;
}
3.1.8 销毁HTTP头
static void
http_destroy_header (Http_header *header)
{
if (header == NULL)
return;
http_destroy_header (header->next);
if (header->name)
free ((char *)header->name);
if (header->value)
free ((char *)header->value);
free (header);
}
3.2 HTTP方法的封装与实现
3.2.1 HTTP方法简介
OPTIONS :
GET :
HEAD :
POST :
PUT :
DELETE:
TRACE :
CONNECT:
3.2.2 HTTP方法枚举
在程序中，定义以下的枚举类型。
typedef enum
{
HTTP_GET,
HTTP_PUT,
HTTP_POST,
HTTP_OPTIONS,
HTTP_HEAD,
HTTP_DELETE,
HTTP_TRACE
} Http_method;
3.2.3 HTTP方法的通用实现
在HttpTunnel里面，首先定义了一个通用的实现HTTP方法的函数，即。
static inline ssize_t
http_method (int fd, Http_destination *dest,
Http_method method, ssize_t length)
{
char str[1024]; /* FIXME: possible buffer overflow */
Http_request *request;
ssize_t n;
if (fd == -1)
{
log_error ("http_method: fd == -1");
return -1;
}
n = 0;
if (dest->proxy_name != NULL)
n = sprintf (str, "http://%s:%d", dest->host_name, dest->host_port);
sprintf (str + n, "/index.html?crap=%ld", time (NULL));
request = http_create_request (method, str, 1, 1);
if (request == NULL)
return -1;
sprintf (str, "%s:%d", dest->host_name, dest->host_port);
http_add_header (&request->header, "Host", str);
if (length >= 0)
{
sprintf (str, "%d", length);
http_add_header (&request->header, "Content-Length", str);
}
http_add_header (&request->header, "Connection", "close");
if (dest->proxy_authorization)
{
http_add_header (&request->header,
"Proxy-Authorization",
dest->proxy_authorization);
}
if (dest->user_agent)
{
http_add_header (&request->header,
"User-Agent",
dest->user_agent);
}
n = http_write_request (fd, request);
http_destroy_request (request);
return n;
}
该函数工作流程分析如下：
3.2.4 GET方法的实现
ssize_t
http_get (int fd, Http_destination *dest)
{
return http_method (fd, dest, HTTP_GET, -1);
}
3.2.5 POST方法的实现
http_post (int fd, Http_destination *dest, size_t length)
{
return http_method (fd, dest, HTTP_POST, (ssize_t)length);
}
3.3 HTTP请求的封装与实现
3.3.1 HTTP请求结构
typedef struct
{
Http_method method;
const char *uri;
int major_version;
int minor_version;
Http_header *header;
} Http_request;
3.3.2 为HTTP请求分配内存空间
static inline Http_request *
http_allocate_request (const char *uri)
{
Http_request *request;
request = malloc (sizeof (Http_request));
if (request == NULL)
return NULL;
request->uri = strdup (uri);
if (request->uri == NULL)
{
free (request);
return NULL;
}
return request;
}
3.3.3 创建HTTP请求
Http_request *
http_create_request (Http_method method,
const char *uri,
int major_version,
int minor_version)
{
Http_request *request;
request = http_allocate_request (uri);
if (request == NULL)
return NULL;
request->method = method;
request->major_version = major_version;
request->minor_version = minor_version;
request->header = NULL;
return request;
}
3.3.4 解析HTTP请求
ssize_t
http_parse_request (int fd, Http_request **request_)
{
Http_request *request;
unsigned char *data;
size_t len;
ssize_t n;
*request_ = NULL;
request = malloc (sizeof (Http_request));
if (request == NULL)
{
log_error ("http_parse_request: out of memory");
return -1;
}
request->method = -1;
request->uri = NULL;
request->major_version = -1;
request->minor_version = -1;
request->header = NULL;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (request);
return n;
}
request->method = http_string_to_method (data, n - 1);
if (request->method == -1)
{
log_error ("http_parse_request: expected an HTTP method");
free (data);
free (request);
return -1;
}
data[n - 1] = 0;
log_verbose ("http_parse_request: method = "%s"", data);
free (data);
len = n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (request);
return n;
}
data[n - 1] = 0;
request->uri = data;
len += n;
log_verbose ("http_parse_request: uri = "%s"", request->uri);
n = read_until (fd, '/', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
else if (n != 5 || memcmp (data, "HTTP", 4) != 0)
{
log_error ("http_parse_request: expected "HTTP"");
free (data);
http_destroy_request (request);
return -1;
}
free (data);
len = n;
n = read_until (fd, '.', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
data[n - 1] = 0;
request->major_version = atoi (data);
log_verbose ("http_parse_request: major version = %d",
request->major_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
data[n - 1] = 0;
request->minor_version = atoi (data);
log_verbose ("http_parse_request: minor version = %d",
request->minor_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
free (data);
if (n != 1)
{
log_error ("http_parse_request: invalid line ending");
http_destroy_request (request);
return -1;
}
len += n;
n = parse_header (fd, &request->header);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
len += n;
*request_ = request;
return len;
}
3.3.5 销毁HTTP请求
void
http_destroy_request (Http_request *request)
{
if (request->uri)
free ((char *)request->uri);
http_destroy_header (request->header);
free (request);
}
3.4 HTTP响应的封装与实现
3.4.1 HTTP响应结构
typedef struct
{
int major_version;
int minor_version;
int status_code;
const char *status_message;
Http_header *header;
} Http_response;
3.4.2 为HTTP响应分配内存空间
static inline Http_response *
http_allocate_response (const char *status_message)
{
Http_response *response;
response = malloc (sizeof (Http_response));
if (response == NULL)
return NULL;
response->status_message = strdup (status_message);
if (response->status_message == NULL)
{
free (response);
return NULL;
}
return response;
}
3.4.3 创建HTTP响应
Http_response *
http_create_response (int major_version,
int minor_version,
int status_code,
const char *status_message)
{
Http_response *response;
response = http_allocate_response (status_message);
if (response == NULL)
return NULL;
response->major_version = major_version;
response->minor_version = minor_version;
response->status_code = status_code;
response->header = NULL;
return response;
}
3.4.4 解析HTTP响应
ssize_t
http_parse_response (int fd, Http_response **response_)
{
Http_response *response;
unsigned char *data;
size_t len;
ssize_t n;
*response_ = NULL;
response = malloc (sizeof (Http_response));
if (response == NULL)
{
log_error ("http_parse_response: out of memory");
return -1;
}
response->major_version = -1;
response->minor_version = -1;
response->status_code = -1;
response->status_message = NULL;
response->header = NULL;
n = read_until (fd, '/', &data);
if (n <= 0)
{
free (response);
return n;
}
else if (n != 5 || memcmp (data, "HTTP", 4) != 0)
{
log_error ("http_parse_response: expected "HTTP"");
free (data);
free (response);
return -1;
}
free (data);
len = n;
n = read_until (fd, '.', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->major_version = atoi (data);
log_verbose ("http_parse_response: major version = %d",
response->major_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->minor_version = atoi (data);
log_verbose ("http_parse_response: minor version = %d",
response->minor_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->status_code = atoi (data);
log_verbose ("http_parse_response: status code = %d",
response->status_code);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->status_message = data;
log_verbose ("http_parse_response: status message = "%s"",
response->status_message);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_response (response);
return n;
}
free (data);
if (n != 1)
{
log_error ("http_parse_request: invalid line ending");
http_destroy_response (response);
return -1;
}
len += n;
n = parse_header (fd, &response->header);
if (n <= 0)
{
http_destroy_response (response);
return n;
}
len += n;
*response_ = response;
return len;
}
3.5 HTTP目标的封装
typedef struct
{
const char *host_name;
int host_port;
const char *proxy_name;
int proxy_port;
const char *proxy_authorization;
const char *user_agent;
} Http_destination;
还有一些其他相关函数可以参考源程序。
