分类目录归档：技术

关于somaxconn参数

6条回复

我们线上服务器net.core.somaxconn都是默认的128,这个参数会影响到所有AF_INET类型socket的listen队列

Man 2 listen可以知道:

int listen(int s, int backlog);

The  backlog  parameter  defines the maximum length the queue of pending connections may grow to.  If a connection request

arrives with the queue full the client may receive an error with an indication of ECONNREFUSED or, if the underlying  pro-

tocol supports retransmission, the request may be ignored so that retries succeed.

BUGS

If the socket is of type AF_INET, and the backlog argument is greater than the constant SOMAXCONN  (128  in  Linux  2.0  &

2.2),  it  is silently truncated to SOMAXCONN.

也就是说,web应用中listen函数的backlog会给我们内核参数的net.core.somaxconn 限制到128,在高突发的请求中可能会导致链接超时或者触发重传

比如nginx 定义NGX_LISTEN_BACKLOG默认到511, 却由于我们参数未曾优化会限制到128,只有128个connections can be queued in kernel listen
queue(by Igor Sysoev).

#define NGX_LISTEN_BACKLOG 511/ls.backlog = NGX_LISTEN_BACKLOG;/ if (listen(s, ls.backlog) == -1) {

相信其他应用比如squid也会有类似问题,突发的大并发connect请求会由于内核listen队列的限制导致链接超时或者重传,从而影响用户体验

以下是实验测试情况,使用2台机器分别以1000个并发,benchmark方式,请求服务器 ( 相当于2000个并发请求同时请求服务器 )

情景1,默认配置, net.core.somaxconn=128,服务器为nginx

测试客户端A:

Transactions:                2072870 hits

Availability:                  99.99 %

Elapsed time:                 179.59 secs

Data transferred:            6096.59 MB

Response time:                  0.08 secs

Transaction rate:           11542.24 trans/sec

Throughput:                    33.95 MB/sec

Concurrency:                  927.34

Successful transactions:     2072871

Failed transactions:             300

Longest transaction:           45.30

Shortest transaction:           0.00

错误率大概是1.5%

测试客户端B:

Transactions:                1859454 hits

Availability:                  99.99 %

Elapsed time:                 179.11 secs

Data transferred:            5468.90 MB

Response time:                  0.09 secs

Transaction rate:           10381.63 trans/sec

Throughput:                    30.53 MB/sec

Concurrency:                  904.45

Successful transactions:     1859454

Failed transactions:             276

Longest transaction:           49.60

Shortest transaction:           0.00

错误率大概也是1.5%

错误提示大都为:

socket: connection timed out

warning: socket: -1803417280 select timed out: Connection timed out

情景2,调整配置, net.core.somaxconn=8192,  nginx显式配置  listen     80 default backlog=8192;

测试客户端A:

** SIEGE 2.69

** Preparing 1000 concurrent users for battle.

The server is now under siege…

Lifting the server siege…      done.

Transactions:                1789818 hits

Availability:                 100.00 %

Elapsed time:                 180.00 secs

Data transferred:            5264.09 MB

Response time:                  0.10 secs

Transaction rate:            9943.43 trans/sec

Throughput:                    29.24 MB/sec

Concurrency:                  997.06

Successful transactions:     1789818

Failed transactions:               0

Longest transaction:            0.87

Shortest transaction:           0.00

错误率是0

测试客户端B:

** SIEGE 2.69

** Preparing 1000 concurrent users for battle.

The server is now under siege…

Lifting the server siege…      done.

Transactions:                1768585 hits

Availability:                 100.00 %

Elapsed time:                 179.31 secs

Data transferred:            5201.65 MB

Response time:                  0.10 secs

Transaction rate:            9863.28 trans/sec

Throughput:                    29.01 MB/sec

Concurrency:                  998.30

Successful transactions:     1768588

Failed transactions:               0

Longest transaction:            3.10

Shortest transaction:           0.03

错误率是0

10个超酷的命令行

4条回复
  1. 以 root 帐户执行上一条命令。
  2. python -m SimpleHTTPServer

    利用 Python 搭建一个简单的 Web 服务器，可通过 http://$HOSTNAME:8000 访问。

  3. :w !sudo tee %

    在 Vim 中无需权限保存编辑的文件。

  4. cd -

    更改到上一次访问的目录。

  5. ^foo^bar

    将上一条命令中的 foo 替换为 bar，并执行。

  6. cp filename{,.bak}

    快速备份或复制文件。

  7. mtr google.com

    traceroute + ping。

  8. !whatever:p

    搜索命令历史，但不执行。

  9. $ssh-copy-id user@host

    将 ssh keys 复制到 user@host 以启用无密码 SSH 登录。

  10. ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/out.mpg

    把 Linux 桌面录制为视频。

nginx core dump trace

发表评论

问题: 如何core dump

答案: 配置方法如下

1. 配置nginx跑在非daemon和非master_process模式，配置样例

daemon off;

master_process off;

2. 配置worker_processes个数，可设置为1

worker_processes 1;

3. 启动nginx

ulimit -c unlimited;

/opt/soft/nginx/sbin/nginx -c /opt/soft/nginx/conf/nginx.conf;

4. 阅读coredump

gdb sbin/nginx  -c core.$pid$

gdb>where

#貌似说最新的coredump办法,不影响在线业务

nginx配置方法
worker_rlimit_core  500M;
working_directory   /path/to/cores/;
要保证nginx进程用户对working_directory可写
原来的配置都不用动，也跑在daemon模式下，有master进程
只需要增加这两行就成了
worker_rlimit_core限制生成core大小的，如果占内存多就搞大
确认需要以下方式:
#新建一个文件夹, 并确认nginx可以读写
$ mkdir /tmp/cores
$ sudo chown root:root /tmp/cores
$ sudo chmod 1777 /tmp/cores
#设置unlimited core file dump
$ ulimit -c unlimited
#设置系统级别的core file
$ echo "/tmp/cores/core.%e.%p" | sudo tee /proc/sys/kernel/core_pattern
#允许suid dumpable
$ sudo sysctl -w fs.suid_dumpable=2
$ sysctl -p
可以发送信号, 让nginx 直接生成coredump, 测试以上配置是否生效
比如kill -11 $work_process_pid

python 的与或非

9条回复

def intersect(a, b):

“”” return the intersection of two lists “””
return list(set(a) & set(b))

def union(a, b):
“”” return the union of two lists “””
return list(set(a) | set(b))

def difference(a, b):
“”” show whats in list b which isn’t in list a “””
return list(set(b).difference(set(a)))
备注:这里应该是list(set(b)-set(a))

这个代码非常方便的使用在比较两份海量url的共同元素上