最开始以为是C# api接口调用问题,后来定位到nginx反代上,抓nginx日志,看日志分析出来应该是读取body的时候耗时,读取body时,有时会创建临时文件,然后读取临时文件时,每次只能读取到一部分,一直在这种情况下耗时几秒钟,最后才读取完成, 问题是有时快有时慢,不清楚为什么,在电脑cpu很低的情况下都会出现
读取临时文件日志在补充问题中
快的时候读取就是一大块一大块读取
8192 - 2737 - 1412 - 2824 - 1219
慢的时候基本每次都是小于1412字节在读取
8192 - 1108 - 1412 - 1412 - 1412 - 1412 - 1412 -24
发现是读取临时文件时造成的
这是快的读取日志
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http body new buf t:1 f:0 000055CDDEB29420, pos 000055CDDEB29420, size: 8192 file: 0, size: 0
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http write client request body, bufs 000055CDDEAF98D8
2023/06/11 13:06:59 [debug] 12894#12894: *63298 write: 6, 000055CDDEB29420, 8192, 15655
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: eof:0, avail:1
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: fd:4 2737 of 8192
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http client request body recv 2737
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http client request body rest 51360
2023/06/11 13:06:59 [debug] 12894#12894: *63298 event timer: 4, old: 7423435396, new: 7423435396
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http read client request body
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: eof:0, avail:1
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: fd:4 1412 of 5455
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http client request body recv 1412
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http client request body rest 51360
2023/06/11 13:06:59 [debug] 12894#12894: *63298 event timer: 4, old: 7423435396, new: 7423435396
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http read client request body
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: eof:0, avail:1
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: fd:4 2824 of 4043
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http client request body recv 2824
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http client request body rest 51360
2023/06/11 13:06:59 [debug] 12894#12894: *63298 event timer: 4, old: 7423435396, new: 7423435396
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http read client request body
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: eof:0, avail:1
2023/06/11 13:06:59 [debug] 12894#12894: *63298 recv: fd:4 1219 of 1219
2023/06/11 13:06:59 [debug] 12894#12894: *63298 http client request body recv 1219
这是慢的读取日志
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body recv 864
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http body new buf t:1 f:0 000055CDDEB29420, pos 000055CDDEB29420, size: 8192 file: 0, size: 0
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http write client request body, bufs 000055CDDEB68758
2023/06/11 13:08:23 [debug] 12894#12894: *63542 write: 6, 000055CDDEB29420, 8192, 50320
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: fd:4 548 of 8192
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body recv 548
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body rest 31709
2023/06/11 13:08:23 [debug] 12894#12894: *63542 event timer: 4, old: 7423519523, new: 7423519739
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: fd:4 1412 of 7644
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body rest 31709
2023/06/11 13:08:23 [debug] 12894#12894: *63542 event timer del: 4: 7423519523
2023/06/11 13:08:23 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423519847
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: fd:4 1412 of 6232
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body rest 31709
2023/06/11 13:08:23 [debug] 12894#12894: *63542 event timer: 4, old: 7423519847, new: 7423519955
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:23 [debug] 12894#12894: *63542 recv: fd:4 1412 of 4820
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:23 [debug] 12894#12894: *63542 http client request body rest 31709
2023/06/11 13:08:23 [debug] 12894#12894: *63542 event timer: 4, old: 7423519847, new: 7423520059
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 3408
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 31709
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer del: 4: 7423519847
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423520167
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 1996
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 31709
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer: 4, old: 7423520167, new: 7423520275
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 584 of 584
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 584
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http body new buf t:1 f:0 000055CDDEB29420, pos 000055CDDEB29420, size: 8192 file: 0, size: 0
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http write client request body, bufs 000055CDDEB68758
2023/06/11 13:08:24 [debug] 12894#12894: *63542 write: 6, 000055CDDEB29420, 8192, 58512
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 828 of 8192
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 828
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 23517
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer: 4, old: 7423520167, new: 7423520383
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 7364
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 23517
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer del: 4: 7423520167
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423520491
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 5952
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 23517
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer: 4, old: 7423520491, new: 7423520595
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 4540
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 23517
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer: 4, old: 7423520491, new: 7423520703
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 3128
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 23517
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer del: 4: 7423520491
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423520811
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 1716
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 23517
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer: 4, old: 7423520811, new: 7423520919
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 304 of 304
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 304
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http body new buf t:1 f:0 000055CDDEB29420, pos 000055CDDEB29420, size: 8192 file: 0, size: 0
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http write client request body, bufs 000055CDDEB68758
2023/06/11 13:08:24 [debug] 12894#12894: *63542 write: 6, 000055CDDEB29420, 8192, 66704
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1108 of 8192
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1108
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 15325
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer: 4, old: 7423520811, new: 7423521027
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:24 [debug] 12894#12894: *63542 recv: fd:4 1412 of 7084
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:24 [debug] 12894#12894: *63542 http client request body rest 15325
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer del: 4: 7423520811
2023/06/11 13:08:24 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423521135
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 5672
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 15325
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer: 4, old: 7423521135, new: 7423521239
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 4260
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 15325
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer: 4, old: 7423521135, new: 7423521347
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 2848
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 15325
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer del: 4: 7423521135
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423521455
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 1436
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 15325
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer: 4, old: 7423521455, new: 7423521563
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 24 of 24
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 24
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http body new buf t:1 f:0 000055CDDEB29420, pos 000055CDDEB29420, size: 8192 file: 0, size: 0
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http write client request body, bufs 000055CDDEB68758
2023/06/11 13:08:25 [debug] 12894#12894: *63542 write: 6, 000055CDDEB29420, 8192, 74896
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1388 of 7133
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1388
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 7133
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer: 4, old: 7423521455, new: 7423521671
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 5745
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 7133
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer del: 4: 7423521455
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423521775
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 4333
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 7133
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer: 4, old: 7423521775, new: 7423521883
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 2921
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 7133
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer: 4, old: 7423521775, new: 7423521991
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 1412 of 1509
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 1412
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body rest 7133
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer del: 4: 7423521775
2023/06/11 13:08:25 [debug] 12894#12894: *63542 event timer add: 4: 60000:7423522099
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http run request: "/api/v1/region/evaluate?"
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http read client request body
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: eof:0, avail:1
2023/06/11 13:08:25 [debug] 12894#12894: *63542 recv: fd:4 97 of 97
2023/06/11 13:08:25 [debug] 12894#12894: *63542 http client request body recv 97
试试修改一下 client_body_buffer_size
,参考 Optimizing nginx client_body_buffer_size for web app accepting file uploads
感谢回复, 我参数到这篇文章, 把反代的buffer搞大,https://www.javatang.com/archives/2020/08/11/22403140.html
不知道能起作用不,我看他还有修改linux内核
综合以上的说明并根据服务器实际的情况,将上述参数的值设置如下:
proxy_buffering on;
proxy_buffer_size 512k;
proxy_buffers 64 512k;
proxy_busy_buffers_size 1m;
经过以上设置之后,error.log中不再出现上述错误,也没有发现其他的错误。虽然客户方感觉有一些改变,但偶尔依然会有卡顿的情况。
修改Linux内核设置
记得在Resin服务器出现大量的ESTABLISHED和TIME_WAIT连接造成响应缓慢这篇文章中,为了减少TIME_WAIT的数量,将net.ipv4.tcp_tw_reuse和net.ipv4.tcp_tw_recycle参数都设置成了1,这两个参数的含义如下:
tcp_tw_recycle:该参数用于回收TIME_WAIT连接,但是存在一个陷阱:当多个客户端通过NAT方式联网并与服务端交互时,服务端看到的是同一个IP,也就是说对服务端而言这些客户端实际上等同于一个,可惜由于这些客户端的时间戳可能存在差异,于是乎从服务端的视角看,便可能出现时间戳错乱的现象,进而直接导致时间戳小的数据包被丢弃。正式因如此,Linux从4.12内核之后移除了这个参数。
tcp_tw_reuse:顾名思义就是复用TIME_WAIT连接。当创建新连接的时候,如果可能的话会考虑复用相应的TIME_WAIT连接。通常认为tcp_tw_reuse比tcp_tw_recycle安全一些,这是因为一来TIME_WAIT创建时间必须超过一秒才可能会被复用;二来只有连接的时间戳是递增的时候才会被复用。
通常情况下,客户方都是使用NAT方式联网访问系统的,如果将这两个参数都打开的话会出现误判的情况,因此需要打开/etc/sysctl.conf文件,将以下三个参数都设置为0:
net.ipv4.tcp_tw_reuse = 0
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_timestamps = 0
保存退出,然后执行sysctl -p命令生效。注意:一定要将上面三个参数的值设置为0,而不是将原来的设置注释掉,那样是无法生效的。
这样设置之后,问题得到完美解决。
我把 client_body_buffer_size 512k;设置成这个大小,目前还没有试出来变慢等下再试试
刚一段时间没操作后, 又出现变慢的情况
还是有问题, 同一个接口, 在同一时间段点的
@油墨路: 如果条件允许,建议把 nginx 换成 YARP 验证一下是 nginx 的问题还是网络问题
另外,ningx 有没有配置 keepalive
?
@dudu: 有一个 keepalive_timeout 65;不知道是不是
@dudu: 我看了日志,这次没存临时文件,不过读取的时候依然很慢
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: fd:4 1412 of 34851
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer: 4, old: 7502958336, new: 7502958444
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: fd:4 1412 of 33439
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer: 4, old: 7502958336, new: 7502958552
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: fd:4 1412 of 32027
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer del: 4: 7502958336
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502958656
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: fd:4 1412 of 30615
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer: 4, old: 7502958656, new: 7502958764
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: fd:4 1412 of 29203
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer: 4, old: 7502958656, new: 7502958872
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: fd:4 1412 of 27791
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer del: 4: 7502958656
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502958976
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:22 [debug] 2432#2432: *35986 recv: fd:4 1412 of 26379
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:22 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:22 [debug] 2432#2432: *35986 event timer: 4, old: 7502958976, new: 7502959084
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 24967
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer: 4, old: 7502958976, new: 7502959192
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 23555
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer del: 4: 7502958976
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502959296
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 22143
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer: 4, old: 7502959296, new: 7502959404
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 20731
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer: 4, old: 7502959296, new: 7502959508
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 19319
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer del: 4: 7502959296
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502959616
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 17907
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer: 4, old: 7502959616, new: 7502959724
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 16495
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer: 4, old: 7502959616, new: 7502959828
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 15083
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer del: 4: 7502959616
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502959936
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 13671
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer: 4, old: 7502959936, new: 7502960044
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:23 [debug] 2432#2432: *35986 recv: fd:4 1412 of 12259
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:23 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:23 [debug] 2432#2432: *35986 event timer: 4, old: 7502959936, new: 7502960148
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 1412 of 10847
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer del: 4: 7502959936
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502960256
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 1412 of 9435
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer: 4, old: 7502960256, new: 7502960364
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 1412 of 8023
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer: 4, old: 7502960256, new: 7502960472
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 1412 of 6611
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer del: 4: 7502960256
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502960576
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 1412 of 5199
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer: 4, old: 7502960576, new: 7502960684
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 1412 of 3787
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer: 4, old: 7502960576, new: 7502960788
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 1412 of 2375
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 1412
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 85683
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer del: 4: 7502960576
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer add: 4: 60000:7502960896
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http run request: "/api/v1/region/evaluate?"
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http read client request body
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: eof:0, avail:1
2023/06/12 11:12:24 [debug] 2432#2432: *35986 recv: fd:4 963 of 963
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body recv 963
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http body new buf t:1 f:0 000055CE7B1B28D0, pos 000055CE7B1B28D0, size: 85683 file: 0, size: 0
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http client request body rest 0
2023/06/12 11:12:24 [debug] 2432#2432: *35986 event timer del: 4: 7502960896
2023/06/12 11:12:24 [debug] 2432#2432: *35986 http init upstream, client timer: 0
2023/06/12 11:12:24 [debug] 2432#2432: *35986 epoll add event: fd:4 op:3 ev:80002005
2023/06/12 11:12:24 [debug] 2432#2432: *35986 posix_memalign: 000055CE7B17ADD0:4096 @16
根据提供的日志和描述,问题似乎出现在读取临时文件时的性能问题。读取临时文件的时间可能取决于文件的大小和存储介质的性能。
以下是一些可能的解决方法和建议:
优化存储介质:如果可能的话,将临时文件存储在更快速的存储介质上,例如固态硬盘(SSD)。这可以显著提高文件读取速度。
调整临时文件大小:根据日志中的观察结果,读取速度受到临时文件大小的影响。尝试调整临时文件的大小,以找到一个在大多数情况下都能获得较好性能的值。
调整Nginx配置:尝试调整Nginx的相关配置参数,以提高性能。例如,增加client_body_buffer_size参数的值,以便更多的请求体能够在内存中进行处理,而不是写入到临时文件中。
使用缓存:如果可能的话,考虑使用缓存机制来减少对C#接口的频繁请求。通过缓存经常请求的数据,可以减轻对C#接口的压力,提高整体性能。
性能分析和优化:使用工具和方法对系统进行性能分析,以确定瓶颈所在。根据分析结果,对性能瓶颈进行优化,可能需要对C#接口和Nginx进行优化。
需要注意的是,由于我无法直接访问您的环境或进行更详细的调查,以上建议仅供参考。您可能需要根据您的具体情况进行进一步的调查和实验。如果问题仍然存在,可能需要寻求更专业的技术支持来解决该问题。
感觉应该是这个连接就是慢连接了,之前有nginx代理时,nginx读取body耗时长, 没有nginx代理就是c#读取body长, 其实还有一层,加了个阿里的负载均衡,我打算把阿里负载均衡取消了再看看
– 油墨路 1年前我感觉是阿里负载均衡的锅,我直接走端口了,没走阿里负载均衡没卡了,等几天再看看,再结帖