summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--.gitignore3
-rw-r--r--DESIGN99
-rw-r--r--Makefile11
-rw-r--r--README12
-rw-r--r--man4/opentracker.conf.411
-rw-r--r--opentracker.c12
-rw-r--r--opentracker.xcodeproj/project.pbxproj320
-rw-r--r--ot_http.c34
-rw-r--r--ot_http.h5
-rw-r--r--ot_socket.c21
-rw-r--r--ot_socket.h1
-rw-r--r--tests/accept_filter_probe.py110
-rw-r--r--tests/integration/test_expiry.py7
-rw-r--r--tests/syscount.c144
-rw-r--r--tests/t_event.c37
-rw-r--r--tests/t_socket.c43
16 files changed, 506 insertions, 364 deletions
diff --git a/.gitignore b/.gitignore
index b5c40f0..3e469ff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,5 @@
1*.o
2*.dSYM 1*.dSYM
2*.o
3opentracker 3opentracker
4opentracker.debug 4opentracker.debug
5opentracker.integration 5opentracker.integration
@@ -7,4 +7,5 @@ opentracker.tsan
7proxy 7proxy
8proxy.debug 8proxy.debug
9tests/runner 9tests/runner
10tests/runner.poll
10tests/runner.sanitize 11tests/runner.sanitize
diff --git a/DESIGN b/DESIGN
new file mode 100644
index 0000000..ff37706
--- /dev/null
+++ b/DESIGN
@@ -0,0 +1,99 @@
1opentracker design notes
2========================
3
4Decisions that cost measurement to reach, and the alternatives that were
5tried and rejected.
6
7What the tracker is optimised for
8---------------------------------
9
10UDP carries most of the traffic. An HTTP request costs about ten times a UDP
11announce, 26 microseconds of tracker CPU against 2.5. Nearly all of that is
12the kernel setting up and tearing down a connection rather than tracker
13code.
14
15A connection carries one request and closes. Keep-alive works and is rarely
16asked for.
17
18Statistics are approximate on purpose. Counters are per thread and summed
19when read, so a /stats read can under-report once. Nothing drifts.
20
21
22Decisions
23---------
24
25One UDP socket per worker, where the kernel spreads datagrams over a group
26bound to one address. Cost per datagram then stays flat as workers are added,
270.71 to 0.72 microseconds from one worker to eight. Sharing one socket grows
28it eightfold over the same range, 0.71 to 6.07.
29
30listen.udp.workers defaults to 1. One worker beats none by 19 to 37 per cent
31everywhere, because a blocking read costs one system call where the event
32loop costs two. Beyond one the gain depends on the kernel balancing, so
33raising it is the operator's decision.
34
35A thread parked in a blocking read is woken by a signal, not by closing or
36shutting down its socket:
37
38 mechanism Linux FreeBSD macOS
39 close() blocked blocked 0 ms
40 shutdown(SHUT_RD) 1 ms 1 ms blocked
41 SO_RCVTIMEO works works works
42 pthread_kill 1 ms 1 ms 1 ms
43
44Only the signal works everywhere. A receive timeout also works, but wakes
45every thread on every tick for the life of the process to catch something
46that happens once: at eight workers it doubled idle context switches, 255 to
47495 over thirty seconds.
48
49Each binary is built in one compiler invocation. Without object files a build
50cannot link objects compiled with different FEATURES, which was a real
51failure mode. It costs incremental rebuilds, 0.075 seconds becoming 1.56.
52
53A socket is read or written, never both. http_interest_set replaces an
54add-and-drop pair so that the invariant cannot be broken by accident, and the
55event layer hears about a change once instead of twice.
56
57Accept waits until the client's request has arrived: TCP_DEFER_ACCEPT on
58Linux, the accf_data filter on FreeBSD. Measured against a flood of silent
59connections, the filter serves more requests than no filter, not fewer, so it
60is not the denial of service vector that FreeBSD-SA-02:26 once described.
61
62
63Paths not taken
64---------------
65
66accf_http, the HTTP-aware accept filter. It duplicates a parser the tracker
67has anyway, puts more kernel code in the path, and holds a connection that
68sends "GET /" and stops, which accf_data hands straight over.
69
70recvmmsg and sendmmsg. No measurable gain on either FreeBSD machine, between
71-4 and +10 per cent, and they do not exist on Darwin.
72
73Link time optimisation. It cuts .text by 13 per cent, 93,497 bytes to 81,630,
74and changes throughput not at all. A tracker waits on memory and system
75calls, not on instruction fetch: the announce path is 6.6 KB of code against
76a 32 KB instruction cache at worst.
77
78A directory for object files. bmake has no pattern rules and the Makefile
79builds under FreeBSD's make today. Building each binary in one invocation
80clears the objects out of the tree without giving that up.
81
82A src/ directory. 52 files sharing an ot_ prefix are a module list rather
83than clutter, and build output was most of what made the tree look crowded.
84
85
86Platform differences that cost us code
87--------------------------------------
88
89SO_REUSEPORT means three different things. Linux spreads datagrams over the
90group; FreeBSD permits the bind and needs SO_REUSEPORT_LB to spread. Darwin
91hands every datagram to whichever socket bound last, so a group there starves
92all but one reader.
93
94Waking a blocked reader differs per kernel. see the table above.
95
96Deferring accept is TCP_DEFER_ACCEPT, or accf_data, or nothing.
97
98make is bmake on FreeBSD and GNU make elsewhere, and the Makefile has to
99build under both.
diff --git a/Makefile b/Makefile
index 3ce127e..2e52367 100644
--- a/Makefile
+++ b/Makefile
@@ -110,6 +110,12 @@ $(TESTBIN): $(TEST_SOURCES) $(TEST_UNITS) $(TEST_HEADERS)
110check: $(TESTBIN) $(BINARY) 110check: $(TESTBIN) $(BINARY)
111 ./$(TESTBIN) 111 ./$(TESTBIN)
112 112
113$(TESTBIN).poll: $(TEST_SOURCES) $(TEST_UNITS) $(TEST_HEADERS)
114 $(CC) -o $@ $(CFLAGS_test) -DOT_EVENT_FORCE_POLL $(TEST_SOURCES) $(TEST_UNITS)
115
116check-poll: $(TESTBIN).poll
117 ./$(TESTBIN).poll ot_event
118
113# The same unit tests under the sanitizers. Use clang over $(CC) because the 119# The same unit tests under the sanitizers. Use clang over $(CC) because the
114# two toolchains do not check the same things 120# two toolchains do not check the same things
115# Override CC_sanitize to use another compiler. 121# Override CC_sanitize to use another compiler.
@@ -153,13 +159,14 @@ check-tsan: opentracker.tsan
153# Everything, cheapest first, so a quick failure comes back quickly. 159# Everything, cheapest first, so a quick failure comes back quickly.
154check-all: 160check-all:
155 $(MAKE) check 161 $(MAKE) check
162 $(MAKE) check-poll
156 $(MAKE) check-sanitize 163 $(MAKE) check-sanitize
157 $(MAKE) check-integration 164 $(MAKE) check-integration
158 $(MAKE) check-tsan 165 $(MAKE) check-tsan
159 166
160.PHONY: all check check-sanitize check-integration check-tsan check-all clean install 167.PHONY: all check check-poll check-sanitize check-integration check-tsan check-all clean install
161 168
162BINARIES=opentracker opentracker.debug opentracker.integration opentracker.tsan proxy proxy.debug $(TESTBIN) $(TESTBIN).sanitize 169BINARIES=opentracker opentracker.debug opentracker.integration opentracker.tsan proxy proxy.debug $(TESTBIN) $(TESTBIN).sanitize $(TESTBIN).poll
163 170
164clean: 171clean:
165 rm -rf $(BINARIES) $(BINARIES:=.dSYM) *.o *~ 172 rm -rf $(BINARIES) $(BINARIES:=.dSYM) *.o *~
diff --git a/README b/README
index bb8a06e..c84c4e9 100644
--- a/README
+++ b/README
@@ -1,14 +1,8 @@
1This is opentracker. An open bittorrent tracker. 1This is opentracker. An open bittorrent tracker.
2 2
3You need libowfat (http://www.fefe.de/libowfat/).
4
5Steps to go: 3Steps to go:
6 4
7cvs -d :pserver:cvs@cvs.fefe.de:/cvs -z9 co libowfat 5git clone git://erdgeist.org/opentracker
8cd libowfat
9make
10cd ..
11cvs -d:pserver:anoncvs@cvs.erdgeist.org:/home/cvsroot co opentracker
12cd opentracker 6cd opentracker
13make 7make
14./opentracker 8./opentracker
@@ -24,7 +18,3 @@ sysctl kern.ipc.somaxconn=1024
24sysctl kern.ipc.nmbclusters=32768 18sysctl kern.ipc.nmbclusters=32768
25sysctl net.inet.tcp.msl=10000 19sysctl net.inet.tcp.msl=10000
26sysctl kern.maxfiles=10240 20sysctl kern.maxfiles=10240
27
28License information:
29
30Although the libowfat library is under GPL, Felix von Leitner agreed that the compiled binary may be distributed under the same beer ware license as the source code for opentracker. However, we like to hear from happy customers.
diff --git a/man4/opentracker.conf.4 b/man4/opentracker.conf.4
index b28879f..aa82e1d 100644
--- a/man4/opentracker.conf.4
+++ b/man4/opentracker.conf.4
@@ -100,6 +100,17 @@ group only where the answer is yes.
100.Pa tests/udp_scaling.py 100.Pa tests/udp_scaling.py
101in the source distribution sweeps the setting against a running tracker. 101in the source distribution sweeps the setting against a running tracker.
102Measure on the machine that will carry the traffic. 102Measure on the machine that will carry the traffic.
103.Pp
104On FreeBSD the
105.Xr accf_data 9
106filter has the kernel hold a new connection until data arrives on it, so the
107tracker is woken once with the request already there. opentracker uses it
108wherever it is available; enable it with
109.Cm options ACCEPT_FILTER_DATA
110in the kernel or
111.Va kld_list Ns = Ns Qq accf_data
112in
113.Xr rc.conf 5 .
103.Sh EXAMPLES 114.Sh EXAMPLES
104To specify the address opentracker will listen on for both TCP and UDP connections: 115To specify the address opentracker will listen on for both TCP and UDP connections:
105.Dl listen.tcp_udp 0.0.0.0:6969 116.Dl listen.tcp_udp 0.0.0.0:6969
diff --git a/opentracker.c b/opentracker.c
index 6160696..ec92fe0 100644
--- a/opentracker.c
+++ b/opentracker.c
@@ -180,12 +180,11 @@ static void handle_write(const int64_t sock) {
180 180
181 /* In a chunked transfer, wait for the next piece once everything queued has gone out */ 181 /* In a chunked transfer, wait for the next piece once everything queued has gone out */
182 if (chunked) 182 if (chunked)
183 http_interest_drop(sock, cookie, EVENT_OUT); 183 http_interest_set(sock, cookie, 0);
184 else if (keep_alive) { 184 else if (keep_alive) {
185 /* The reply is out and the client asked to keep the connection, so stop 185 /* The reply is out and the client asked to keep the connection, so go
186 waiting to write and go back to waiting for the next request */ 186 back to waiting for the next request */
187 http_interest_drop(sock, cookie, EVENT_OUT); 187 http_interest_set(sock, cookie, EVENT_IN);
188 http_interest_add(sock, cookie, EVENT_IN);
189 fdstate_set_expiry(sock, time_future_seconds(OT_CLIENT_TIMEOUT)); 188 fdstate_set_expiry(sock, time_future_seconds(OT_CLIENT_TIMEOUT));
190 } else 189 } else
191 http_close(sock); 190 http_close(sock);
@@ -199,8 +198,6 @@ static void handle_accept(const int64_t serversocket, struct ot_workstruct *ws)
199 198
200 while ((sock = socket_accept(serversocket, ip, &port)) != -1) { 199 while ((sock = socket_accept(serversocket, ip, &port)) != -1) {
201 200
202 socket_set_blocking((int)sock, 0);
203
204 if (!(cookie = (struct http_data *)calloc(1, sizeof(struct http_data)))) { 201 if (!(cookie = (struct http_data *)calloc(1, sizeof(struct http_data)))) {
205 close((int)sock); 202 close((int)sock);
206 continue; 203 continue;
@@ -342,6 +339,7 @@ static void listen_bind_one(ot_listen *entry) {
342 if (entry->proto == FLAG_TCP) { 339 if (entry->proto == FLAG_TCP) {
343 if ((entry->sock = socket_tcp_listen(entry->ip, entry->port, sharing)) == -1) 340 if ((entry->sock = socket_tcp_listen(entry->ip, entry->port, sharing)) == -1)
344 panic("socket_tcp_listen"); 341 panic("socket_tcp_listen");
342 socket_defer_accept((int)entry->sock);
345 } else { 343 } else {
346 if ((entry->sock = socket_udp_bind(entry->ip, entry->port, sharing)) == -1) 344 if ((entry->sock = socket_udp_bind(entry->ip, entry->port, sharing)) == -1)
347 panic("socket_udp_bind"); 345 panic("socket_udp_bind");
diff --git a/opentracker.xcodeproj/project.pbxproj b/opentracker.xcodeproj/project.pbxproj
deleted file mode 100644
index 3706a3f..0000000
--- a/opentracker.xcodeproj/project.pbxproj
+++ /dev/null
@@ -1,320 +0,0 @@
1// !$*UTF8*$!
2{
3 archiveVersion = 1;
4 classes = {
5 };
6 objectVersion = 44;
7 objects = {
8
9/* Begin PBXBuildFile section */
10 6520B7530D036AAF00A43B1F /* libowfat.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 6520B7520D036AAF00A43B1F /* libowfat.a */; };
11 653A320C0CE7F475007F0D03 /* ot_accesslist.c in Sources */ = {isa = PBXBuildFile; fileRef = 653A320B0CE7F475007F0D03 /* ot_accesslist.c */; };
12 653A56B50CE28EC5000CF140 /* ot_iovec.c in Sources */ = {isa = PBXBuildFile; fileRef = 653A56B40CE28EC5000CF140 /* ot_iovec.c */; };
13 654A80890CD832FD009035DE /* opentracker.c in Sources */ = {isa = PBXBuildFile; fileRef = 654A80840CD832FC009035DE /* opentracker.c */; };
14 654A808A0CD832FD009035DE /* scan_urlencoded_query.c in Sources */ = {isa = PBXBuildFile; fileRef = 654A80850CD832FC009035DE /* scan_urlencoded_query.c */; };
15 654A808B0CD832FD009035DE /* trackerlogic.c in Sources */ = {isa = PBXBuildFile; fileRef = 654A80870CD832FC009035DE /* trackerlogic.c */; };
16 65542D8B0CE078E800469330 /* ot_vector.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542D8A0CE078E800469330 /* ot_vector.c */; };
17 65542D930CE07CED00469330 /* ot_mutex.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542D8F0CE07CED00469330 /* ot_mutex.c */; };
18 65542D940CE07CED00469330 /* ot_stats.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542D910CE07CED00469330 /* ot_stats.c */; };
19 65542E750CE08B9100469330 /* ot_clean.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542E740CE08B9100469330 /* ot_clean.c */; };
20 65542EE80CE0CA6B00469330 /* ot_udp.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542EE70CE0CA6B00469330 /* ot_udp.c */; };
21 65542F920CE17CA900469330 /* ot_fullscrape.c in Sources */ = {isa = PBXBuildFile; fileRef = 65542F910CE17CA900469330 /* ot_fullscrape.c */; };
22 65B8DF3C0D0310D20017149E /* ot_http.c in Sources */ = {isa = PBXBuildFile; fileRef = 65B8DF3B0D0310D20017149E /* ot_http.c */; };
23 65FA33990E7EF09200F7D5A5 /* ot_livesync.c in Sources */ = {isa = PBXBuildFile; fileRef = 65FA33980E7EF09200F7D5A5 /* ot_livesync.c */; };
24 8DD76FB00486AB0100D96B5E /* opentracker.1 in CopyFiles */ = {isa = PBXBuildFile; fileRef = C6A0FF2C0290799A04C91782 /* opentracker.1 */; };
25/* End PBXBuildFile section */
26
27/* Begin PBXCopyFilesBuildPhase section */
28 8DD76FAF0486AB0100D96B5E /* CopyFiles */ = {
29 isa = PBXCopyFilesBuildPhase;
30 buildActionMask = 8;
31 dstPath = /usr/share/man/man1/;
32 dstSubfolderSpec = 0;
33 files = (
34 8DD76FB00486AB0100D96B5E /* opentracker.1 in CopyFiles */,
35 );
36 runOnlyForDeploymentPostprocessing = 1;
37 };
38/* End PBXCopyFilesBuildPhase section */
39
40/* Begin PBXFileReference section */
41 6520B7520D036AAF00A43B1F /* libowfat.a */ = {isa = PBXFileReference; lastKnownFileType = archive.ar; name = libowfat.a; path = ../libowfat/libowfat.a; sourceTree = SOURCE_ROOT; };
42 653A320A0CE7F475007F0D03 /* ot_accesslist.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_accesslist.h; sourceTree = "<group>"; };
43 653A320B0CE7F475007F0D03 /* ot_accesslist.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_accesslist.c; sourceTree = "<group>"; };
44 653A56AC0CE201FF000CF140 /* opentracker */ = {isa = PBXFileReference; explicitFileType = "compiled.mach-o.executable"; includeInIndex = 0; path = opentracker; sourceTree = BUILT_PRODUCTS_DIR; };
45 653A56B30CE28EC5000CF140 /* ot_iovec.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_iovec.h; sourceTree = "<group>"; };
46 653A56B40CE28EC5000CF140 /* ot_iovec.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_iovec.c; sourceTree = "<group>"; };
47 654A80840CD832FC009035DE /* opentracker.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = opentracker.c; sourceTree = "<group>"; };
48 654A80850CD832FC009035DE /* scan_urlencoded_query.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = scan_urlencoded_query.c; sourceTree = "<group>"; };
49 654A80860CD832FC009035DE /* scan_urlencoded_query.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = scan_urlencoded_query.h; sourceTree = "<group>"; };
50 654A80870CD832FC009035DE /* trackerlogic.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = trackerlogic.c; sourceTree = "<group>"; };
51 654A80880CD832FC009035DE /* trackerlogic.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = trackerlogic.h; sourceTree = "<group>"; };
52 65542D890CE078E800469330 /* ot_vector.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_vector.h; sourceTree = "<group>"; };
53 65542D8A0CE078E800469330 /* ot_vector.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_vector.c; sourceTree = "<group>"; };
54 65542D8F0CE07CED00469330 /* ot_mutex.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_mutex.c; sourceTree = "<group>"; };
55 65542D900CE07CED00469330 /* ot_mutex.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_mutex.h; sourceTree = "<group>"; };
56 65542D910CE07CED00469330 /* ot_stats.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_stats.c; sourceTree = "<group>"; };
57 65542D920CE07CED00469330 /* ot_stats.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_stats.h; sourceTree = "<group>"; };
58 65542E730CE08B9100469330 /* ot_clean.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_clean.h; sourceTree = "<group>"; };
59 65542E740CE08B9100469330 /* ot_clean.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_clean.c; sourceTree = "<group>"; };
60 65542EE60CE0CA6B00469330 /* ot_udp.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_udp.h; sourceTree = "<group>"; };
61 65542EE70CE0CA6B00469330 /* ot_udp.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_udp.c; sourceTree = "<group>"; };
62 65542F900CE17CA900469330 /* ot_fullscrape.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_fullscrape.h; sourceTree = "<group>"; };
63 65542F910CE17CA900469330 /* ot_fullscrape.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_fullscrape.c; sourceTree = "<group>"; };
64 65B8DF3A0D0310D20017149E /* ot_http.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_http.h; sourceTree = "<group>"; };
65 65B8DF3B0D0310D20017149E /* ot_http.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_http.c; sourceTree = "<group>"; };
66 65FA33970E7EF09200F7D5A5 /* ot_livesync.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ot_livesync.h; sourceTree = "<group>"; };
67 65FA33980E7EF09200F7D5A5 /* ot_livesync.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; path = ot_livesync.c; sourceTree = "<group>"; };
68 C6A0FF2C0290799A04C91782 /* opentracker.1 */ = {isa = PBXFileReference; lastKnownFileType = text.man; path = opentracker.1; sourceTree = "<group>"; };
69/* End PBXFileReference section */
70
71/* Begin PBXFrameworksBuildPhase section */
72 8DD76FAD0486AB0100D96B5E /* Frameworks */ = {
73 isa = PBXFrameworksBuildPhase;
74 buildActionMask = 2147483647;
75 files = (
76 6520B7530D036AAF00A43B1F /* libowfat.a in Frameworks */,
77 );
78 runOnlyForDeploymentPostprocessing = 0;
79 };
80/* End PBXFrameworksBuildPhase section */
81
82/* Begin PBXGroup section */
83 08FB7794FE84155DC02AAC07 /* opentracker */ = {
84 isa = PBXGroup;
85 children = (
86 65542D810CE0786F00469330 /* Headers */,
87 08FB7795FE84155DC02AAC07 /* Source */,
88 92762AC9104EDED700FDCB60 /* Libraries */,
89 C6A0FF2B0290797F04C91782 /* Documentation */,
90 653A56AD0CE201FF000CF140 /* Products */,
91 );
92 name = opentracker;
93 sourceTree = "<group>";
94 };
95 08FB7795FE84155DC02AAC07 /* Source */ = {
96 isa = PBXGroup;
97 children = (
98 654A80840CD832FC009035DE /* opentracker.c */,
99 653A320B0CE7F475007F0D03 /* ot_accesslist.c */,
100 65542E740CE08B9100469330 /* ot_clean.c */,
101 65542F910CE17CA900469330 /* ot_fullscrape.c */,
102 65B8DF3B0D0310D20017149E /* ot_http.c */,
103 65FA33980E7EF09200F7D5A5 /* ot_livesync.c */,
104 653A56B40CE28EC5000CF140 /* ot_iovec.c */,
105 65542D8F0CE07CED00469330 /* ot_mutex.c */,
106 65542D910CE07CED00469330 /* ot_stats.c */,
107 65542EE70CE0CA6B00469330 /* ot_udp.c */,
108 65542D8A0CE078E800469330 /* ot_vector.c */,
109 654A80850CD832FC009035DE /* scan_urlencoded_query.c */,
110 654A80870CD832FC009035DE /* trackerlogic.c */,
111 );
112 name = Source;
113 sourceTree = "<group>";
114 };
115 653A56AD0CE201FF000CF140 /* Products */ = {
116 isa = PBXGroup;
117 children = (
118 653A56AC0CE201FF000CF140 /* opentracker */,
119 );
120 name = Products;
121 sourceTree = "<group>";
122 };
123 65542D810CE0786F00469330 /* Headers */ = {
124 isa = PBXGroup;
125 children = (
126 653A320A0CE7F475007F0D03 /* ot_accesslist.h */,
127 65542E730CE08B9100469330 /* ot_clean.h */,
128 65542F900CE17CA900469330 /* ot_fullscrape.h */,
129 65B8DF3A0D0310D20017149E /* ot_http.h */,
130 653A56B30CE28EC5000CF140 /* ot_iovec.h */,
131 65FA33970E7EF09200F7D5A5 /* ot_livesync.h */,
132 65542D900CE07CED00469330 /* ot_mutex.h */,
133 65542D920CE07CED00469330 /* ot_stats.h */,
134 65542EE60CE0CA6B00469330 /* ot_udp.h */,
135 65542D890CE078E800469330 /* ot_vector.h */,
136 654A80860CD832FC009035DE /* scan_urlencoded_query.h */,
137 654A80880CD832FC009035DE /* trackerlogic.h */,
138 );
139 name = Headers;
140 sourceTree = "<group>";
141 };
142 92762AC9104EDED700FDCB60 /* Libraries */ = {
143 isa = PBXGroup;
144 children = (
145 6520B7520D036AAF00A43B1F /* libowfat.a */,
146 );
147 name = Libraries;
148 sourceTree = "<group>";
149 };
150 C6A0FF2B0290797F04C91782 /* Documentation */ = {
151 isa = PBXGroup;
152 children = (
153 C6A0FF2C0290799A04C91782 /* opentracker.1 */,
154 );
155 name = Documentation;
156 sourceTree = "<group>";
157 };
158/* End PBXGroup section */
159
160/* Begin PBXNativeTarget section */
161 8DD76FA90486AB0100D96B5E /* opentracker */ = {
162 isa = PBXNativeTarget;
163 buildConfigurationList = 1DEB928508733DD80010E9CD /* Build configuration list for PBXNativeTarget "opentracker" */;
164 buildPhases = (
165 8DD76FAB0486AB0100D96B5E /* Sources */,
166 8DD76FAD0486AB0100D96B5E /* Frameworks */,
167 8DD76FAF0486AB0100D96B5E /* CopyFiles */,
168 );
169 buildRules = (
170 );
171 dependencies = (
172 );
173 name = opentracker;
174 productInstallPath = "$(HOME)/bin";
175 productName = opentracker;
176 productReference = 653A56AC0CE201FF000CF140 /* opentracker */;
177 productType = "com.apple.product-type.tool";
178 };
179/* End PBXNativeTarget section */
180
181/* Begin PBXProject section */
182 08FB7793FE84155DC02AAC07 /* Project object */ = {
183 isa = PBXProject;
184 buildConfigurationList = 1DEB928908733DD80010E9CD /* Build configuration list for PBXProject "opentracker" */;
185 compatibilityVersion = "Xcode 3.0";
186 hasScannedForEncodings = 1;
187 mainGroup = 08FB7794FE84155DC02AAC07 /* opentracker */;
188 productRefGroup = 653A56AD0CE201FF000CF140 /* Products */;
189 projectDirPath = "";
190 projectRoot = "";
191 targets = (
192 8DD76FA90486AB0100D96B5E /* opentracker */,
193 );
194 };
195/* End PBXProject section */
196
197/* Begin PBXSourcesBuildPhase section */
198 8DD76FAB0486AB0100D96B5E /* Sources */ = {
199 isa = PBXSourcesBuildPhase;
200 buildActionMask = 2147483647;
201 files = (
202 654A80890CD832FD009035DE /* opentracker.c in Sources */,
203 654A808A0CD832FD009035DE /* scan_urlencoded_query.c in Sources */,
204 654A808B0CD832FD009035DE /* trackerlogic.c in Sources */,
205 65542D8B0CE078E800469330 /* ot_vector.c in Sources */,
206 65542D930CE07CED00469330 /* ot_mutex.c in Sources */,
207 65542D940CE07CED00469330 /* ot_stats.c in Sources */,
208 65542E750CE08B9100469330 /* ot_clean.c in Sources */,
209 65542EE80CE0CA6B00469330 /* ot_udp.c in Sources */,
210 65542F920CE17CA900469330 /* ot_fullscrape.c in Sources */,
211 653A56B50CE28EC5000CF140 /* ot_iovec.c in Sources */,
212 653A320C0CE7F475007F0D03 /* ot_accesslist.c in Sources */,
213 65B8DF3C0D0310D20017149E /* ot_http.c in Sources */,
214 65FA33990E7EF09200F7D5A5 /* ot_livesync.c in Sources */,
215 );
216 runOnlyForDeploymentPostprocessing = 0;
217 };
218/* End PBXSourcesBuildPhase section */
219
220/* Begin XCBuildConfiguration section */
221 1DEB928608733DD80010E9CD /* Debug */ = {
222 isa = XCBuildConfiguration;
223 buildSettings = {
224 COPY_PHASE_STRIP = NO;
225 GCC_DYNAMIC_NO_PIC = NO;
226 GCC_ENABLE_FIX_AND_CONTINUE = YES;
227 GCC_MODEL_TUNING = G5;
228 GCC_OPTIMIZATION_LEVEL = 0;
229 INSTALL_PATH = /usr/local/bin;
230 LIBRARY_SEARCH_PATHS = (
231 "$(inherited)",
232 "\"$(SRCROOT)/../libowfat\"",
233 );
234 PRODUCT_NAME = opentracker;
235 ZERO_LINK = YES;
236 };
237 name = Debug;
238 };
239 1DEB928708733DD80010E9CD /* Release */ = {
240 isa = XCBuildConfiguration;
241 buildSettings = {
242 DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
243 GCC_ENABLE_FIX_AND_CONTINUE = YES;
244 GCC_MODEL_TUNING = G5;
245 INSTALL_PATH = /usr/local/bin;
246 LIBRARY_SEARCH_PATHS = (
247 "$(inherited)",
248 "\"$(SRCROOT)/../libowfat\"",
249 );
250 PRODUCT_NAME = opentracker;
251 };
252 name = Release;
253 };
254 1DEB928A08733DD80010E9CD /* Debug */ = {
255 isa = XCBuildConfiguration;
256 buildSettings = {
257 ARCHS = "$(ONLY_ACTIVE_ARCH_PRE_XCODE_3_1)";
258 GCC_PREPROCESSOR_DEFINITIONS = (
259 WANT_ACCESSLIST_WHITE,
260 WANT_IP_FROM_QUERY_STRING,
261 WANT_FULLSCRAPE,
262 );
263 GCC_VERSION = com.apple.compilers.llvm.clang.1_0;
264 GCC_WARN_ABOUT_RETURN_TYPE = YES;
265 GCC_WARN_UNUSED_VARIABLE = YES;
266 HEADER_SEARCH_PATHS = ../libowfat/;
267 LIBRARY_SEARCH_PATHS = ../libowfat/;
268 ONLY_ACTIVE_ARCH_PRE_XCODE_3_1 = "$(NATIVE_ARCH_ACTUAL)";
269 PREBINDING = NO;
270 SDKROOT = "$(DEVELOPER_SDK_DIR)/MacOSX10.6.sdk";
271 };
272 name = Debug;
273 };
274 1DEB928B08733DD80010E9CD /* Release */ = {
275 isa = XCBuildConfiguration;
276 buildSettings = {
277 ARCHS = "$(ONLY_ACTIVE_ARCH_PRE_XCODE_3_1)";
278 DEAD_CODE_STRIPPING = NO;
279 GCC_PREPROCESSOR_DEFINITIONS = (
280 WANT_ACCESSLIST_WHITE,
281 WANT_IP_FROM_QUERY_STRING,
282 WANT_FULLSCRAPE,
283 );
284 GCC_VERSION = com.apple.compilers.llvm.clang.1_0;
285 GCC_WARN_ABOUT_RETURN_TYPE = YES;
286 GCC_WARN_UNUSED_VARIABLE = YES;
287 HEADER_SEARCH_PATHS = ../libowfat/;
288 LIBRARY_SEARCH_PATHS = ../libowfat/;
289 ONLY_ACTIVE_ARCH_PRE_XCODE_3_1 = "$(NATIVE_ARCH_ACTUAL)";
290 PREBINDING = NO;
291 SDKROOT = "$(DEVELOPER_SDK_DIR)/MacOSX10.6.sdk";
292 ZERO_LINK = NO;
293 };
294 name = Release;
295 };
296/* End XCBuildConfiguration section */
297
298/* Begin XCConfigurationList section */
299 1DEB928508733DD80010E9CD /* Build configuration list for PBXNativeTarget "opentracker" */ = {
300 isa = XCConfigurationList;
301 buildConfigurations = (
302 1DEB928608733DD80010E9CD /* Debug */,
303 1DEB928708733DD80010E9CD /* Release */,
304 );
305 defaultConfigurationIsVisible = 0;
306 defaultConfigurationName = Release;
307 };
308 1DEB928908733DD80010E9CD /* Build configuration list for PBXProject "opentracker" */ = {
309 isa = XCConfigurationList;
310 buildConfigurations = (
311 1DEB928A08733DD80010E9CD /* Debug */,
312 1DEB928B08733DD80010E9CD /* Release */,
313 );
314 defaultConfigurationIsVisible = 0;
315 defaultConfigurationName = Release;
316 };
317/* End XCConfigurationList section */
318 };
319 rootObject = 08FB7793FE84155DC02AAC07 /* Project object */;
320}
diff --git a/ot_http.c b/ot_http.c
index d4c7363..66f0102 100644
--- a/ot_http.c
+++ b/ot_http.c
@@ -6,6 +6,7 @@
6/* System */ 6/* System */
7#define _GNU_SOURCE 7#define _GNU_SOURCE
8#include <arpa/inet.h> 8#include <arpa/inet.h>
9#include <errno.h>
9#include <pthread.h> 10#include <pthread.h>
10#include <stdio.h> 11#include <stdio.h>
11#include <stdlib.h> 12#include <stdlib.h>
@@ -85,6 +86,12 @@ static void http_senddata(const int64_t sock, struct ot_workstruct *ws) {
85 buf_reset(&cookie->request); 86 buf_reset(&cookie->request);
86 87
87 written_size = write(sock, ws->reply, ws->reply_size); 88 written_size = write(sock, ws->reply, ws->reply_size);
89 /* A send buffer with no room in it is not a failure: nothing went out, so
90 the whole reply goes into the batch below and leaves when the socket
91 drains. Only a real error ends the connection here. */
92 if (written_size < 0 && (errno == EAGAIN || errno == EWOULDBLOCK || errno == EINTR))
93 written_size = 0;
94
88 if ((written_size < 0) || ((written_size == ws->reply_size) && !ws->keep_alive)) { 95 if ((written_size < 0) || ((written_size == ws->reply_size) && !ws->keep_alive)) {
89 http_close(sock); 96 http_close(sock);
90 return; 97 return;
@@ -111,11 +118,9 @@ static void http_senddata(const int64_t sock, struct ot_workstruct *ws) {
111 batch_add(cookie->batch, outbuf, ws->reply_size - written_size); 118 batch_add(cookie->batch, outbuf, ws->reply_size - written_size);
112 119
113 /* writeable short data sockets just have a tcp timeout */ 120 /* writeable short data sockets just have a tcp timeout */
114 if (!ws->keep_alive) { 121 if (!ws->keep_alive)
115 fdstate_set_expiry(sock, 0); 122 fdstate_set_expiry(sock, 0);
116 http_interest_drop(sock, cookie, EVENT_IN); 123 http_interest_set(sock, cookie, EVENT_OUT);
117 }
118 http_interest_add(sock, cookie, EVENT_OUT);
119 } 124 }
120} 125}
121 126
@@ -128,21 +133,16 @@ static void http_senddata(const int64_t sock, struct ot_workstruct *ws) {
128#define HTTPERROR_403_IP return http_issue_error(sock, ws, CODE_HTTPERROR_403_IP) 133#define HTTPERROR_403_IP return http_issue_error(sock, ws, CODE_HTTPERROR_403_IP)
129#define HTTPERROR_404 return http_issue_error(sock, ws, CODE_HTTPERROR_404) 134#define HTTPERROR_404 return http_issue_error(sock, ws, CODE_HTTPERROR_404)
130#define HTTPERROR_500 return http_issue_error(sock, ws, CODE_HTTPERROR_500) 135#define HTTPERROR_500 return http_issue_error(sock, ws, CODE_HTTPERROR_500)
131static void http_interest(const int64_t sock, struct http_data *cookie, int what) { 136/* A connection is either being read or being written, never both, so what it
137 waits for is set outright rather than added to and taken away again. The
138 event layer only hears about it when the answer changes. */
139void http_interest_set(const int64_t sock, struct http_data *cookie, int what) {
132 if (cookie->interest == what) 140 if (cookie->interest == what)
133 return; 141 return;
134 cookie->interest = what; 142 cookie->interest = what;
135 event_modify((int)sock, what, fdstate_generation((int)sock)); 143 event_modify((int)sock, what, fdstate_generation((int)sock));
136} 144}
137 145
138void http_interest_add(const int64_t sock, struct http_data *cookie, int what) {
139 http_interest(sock, cookie, cookie->interest | what);
140}
141
142void http_interest_drop(const int64_t sock, struct http_data *cookie, int what) {
143 http_interest(sock, cookie, cookie->interest & ~what);
144}
145
146ssize_t http_issue_error(const int64_t sock, struct ot_workstruct *ws, int code) { 146ssize_t http_issue_error(const int64_t sock, struct ot_workstruct *ws, int code) {
147 const char *error_code[] = {"302 Found", "400 Invalid Request", "400 Invalid Request", "400 Invalid Request", "402 Payment Required", 147 const char *error_code[] = {"302 Found", "400 Invalid Request", "400 Invalid Request", "400 Invalid Request", "402 Payment Required",
148 "403 Not Modest", "403 Access Denied", "404 Not Found", "500 Internal Server Error"}; 148 "403 Not Modest", "403 Access Denied", "404 Not Found", "500 Internal Server Error"};
@@ -187,7 +187,7 @@ ssize_t http_sendiovecdata(const int64_t sock, struct ot_workstruct *ws, size_t
187 187
188 /* If we came here, wait for the answer is over */ 188 /* If we came here, wait for the answer is over */
189 if (cookie->flag & STRUCT_HTTP_FLAG_WAITINGFORTASK) { 189 if (cookie->flag & STRUCT_HTTP_FLAG_WAITINGFORTASK) {
190 http_interest_drop(sock, cookie, EVENT_IN); 190 http_interest_set(sock, cookie, 0);
191 cookie->flag &= ~STRUCT_HTTP_FLAG_WAITINGFORTASK; 191 cookie->flag &= ~STRUCT_HTTP_FLAG_WAITINGFORTASK;
192 } 192 }
193 193
@@ -238,7 +238,7 @@ ssize_t http_sendiovecdata(const int64_t sock, struct ot_workstruct *ws, size_t
238 } 238 }
239 239
240 fdstate_set_expiry(sock, time_future_seconds(OT_CLIENT_TIMEOUT_SEND)); 240 fdstate_set_expiry(sock, time_future_seconds(OT_CLIENT_TIMEOUT_SEND));
241 http_interest_add(sock, cookie, EVENT_OUT); 241 http_interest_set(sock, cookie, EVENT_OUT);
242 return 0; 242 return 0;
243} 243}
244 244
@@ -333,7 +333,7 @@ static ssize_t http_handle_stats(const int64_t sock, struct ot_workstruct *ws, c
333 /* Clients waiting for us should not easily timeout */ 333 /* Clients waiting for us should not easily timeout */
334 fdstate_set_expiry(sock, 0); 334 fdstate_set_expiry(sock, 0);
335 fullscrape_deliver(sock, format); 335 fullscrape_deliver(sock, format);
336 http_interest_drop(sock, cookie, EVENT_IN); 336 http_interest_set(sock, cookie, 0);
337 return ws->reply_size = -2; 337 return ws->reply_size = -2;
338 } 338 }
339#endif 339#endif
@@ -428,7 +428,7 @@ static ssize_t http_handle_fullscrape(const int64_t sock, struct ot_workstruct *
428 /* Clients waiting for us should not easily timeout */ 428 /* Clients waiting for us should not easily timeout */
429 fdstate_set_expiry(sock, 0); 429 fdstate_set_expiry(sock, 0);
430 fullscrape_deliver(sock, format); 430 fullscrape_deliver(sock, format);
431 http_interest_drop(sock, cookie, EVENT_IN); 431 http_interest_set(sock, cookie, 0);
432 return ws->reply_size = -2; 432 return ws->reply_size = -2;
433} 433}
434#endif 434#endif
diff --git a/ot_http.h b/ot_http.h
index 0f351c1..ba16de4 100644
--- a/ot_http.h
+++ b/ot_http.h
@@ -33,9 +33,8 @@ struct http_data {
33 STRUCT_HTTP_FLAG flag; 33 STRUCT_HTTP_FLAG flag;
34}; 34};
35 35
36/* Add to or drop from what this connection is waiting to be told about */ 36/* What this connection is waiting for: EVENT_IN, EVENT_OUT, or nothing */
37void http_interest_add(const int64_t s, struct http_data *cookie, int what); 37void http_interest_set(const int64_t s, struct http_data *cookie, int what);
38void http_interest_drop(const int64_t s, struct http_data *cookie, int what);
39 38
40/* Close a connection and release everything that belongs to it */ 39/* Close a connection and release everything that belongs to it */
41void http_close(const int64_t s); 40void http_close(const int64_t s);
diff --git a/ot_socket.c b/ot_socket.c
index 47ed26d..48ffc27 100644
--- a/ot_socket.c
+++ b/ot_socket.c
@@ -8,6 +8,7 @@
8#include <errno.h> 8#include <errno.h>
9#include <fcntl.h> 9#include <fcntl.h>
10#include <netinet/in.h> 10#include <netinet/in.h>
11#include <netinet/tcp.h>
11#include <string.h> 12#include <string.h>
12#include <sys/socket.h> 13#include <sys/socket.h>
13#include <unistd.h> 14#include <unistd.h>
@@ -120,6 +121,26 @@ static int socket_bind_v6(int s, const ot_ip6 ip, uint16_t port, socket_sharing
120 return bind(s, (struct sockaddr *)&sa, sizeof(sa)); 121 return bind(s, (struct sockaddr *)&sa, sizeof(sa));
121} 122}
122 123
124/* Defer the listening socket reporting it to be ready to accept until enough
125 sensible data is available for an accept+read pair. Where available. */
126int socket_defer_accept(int s) {
127#if defined(TCP_DEFER_ACCEPT)
128 int patience = OT_CLIENT_TIMEOUT;
129
130 return setsockopt(s, IPPROTO_TCP, TCP_DEFER_ACCEPT, &patience, sizeof(patience));
131#elif defined(SO_ACCEPTFILTER)
132 struct accept_filter_arg filter;
133 const char name[sizeof(filter.af_name)] = "dataready";
134
135 memset(&filter, 0, sizeof(filter));
136 memcpy(filter.af_name, name, sizeof(name));
137 return setsockopt(s, SOL_SOCKET, SO_ACCEPTFILTER, &filter, sizeof(filter));
138#else
139 (void)s;
140 return -1;
141#endif
142}
143
123int socket_tcp_listen(const ot_ip6 ip, uint16_t port, socket_sharing sharing) { 144int socket_tcp_listen(const ot_ip6 ip, uint16_t port, socket_sharing sharing) {
124 int s = socket_v6(SOCK_STREAM); 145 int s = socket_v6(SOCK_STREAM);
125 146
diff --git a/ot_socket.h b/ot_socket.h
index f73a5c9..5bbffe1 100644
--- a/ot_socket.h
+++ b/ot_socket.h
@@ -38,6 +38,7 @@ int socket_tcp_connect(const ot_ip6 srcip, uint16_t srcport, const ot_ip6 ds
38int socket_is_connected(int s); 38int socket_is_connected(int s);
39int socket_accept(int s, ot_ip6 ip, uint16_t *port); 39int socket_accept(int s, ot_ip6 ip, uint16_t *port);
40int socket_set_blocking(int s, int blocking); 40int socket_set_blocking(int s, int blocking);
41int socket_defer_accept(int s);
41 42
42ssize_t socket_recv(int s, void *buf, size_t len, ot_ip6 ip, uint16_t *port); 43ssize_t socket_recv(int s, void *buf, size_t len, ot_ip6 ip, uint16_t *port);
43ssize_t socket_send(int s, const void *buf, size_t len, const ot_ip6 ip, uint16_t port); 44ssize_t socket_send(int s, const void *buf, size_t len, const ot_ip6 ip, uint16_t port);
diff --git a/tests/accept_filter_probe.py b/tests/accept_filter_probe.py
new file mode 100644
index 0000000..22f585a
--- /dev/null
+++ b/tests/accept_filter_probe.py
@@ -0,0 +1,110 @@
1"""What a deferred accept does to connections that never send anything.
2
3Those connections never reach opentracker, so its own client timeout cannot
4clean them up: whatever bound exists is the kernel's. This measures it, and
5then asks the question that follows from it -- whether enough of them keep
6the tracker from answering anybody else.
7
8 make opentracker
9 python3 tests/accept_filter_probe.py
10
11On FreeBSD the filter does nothing unless accf_data is loaded, so run it both
12ways to see whether the numbers differ:
13
14 kldload accf_data
15"""
16
17import os
18import socket
19import signal
20import subprocess
21import sys
22import tempfile
23import time
24import urllib.error
25import urllib.request
26
27TRACKER = os.environ.get("OPENTRACKER", "./opentracker")
28IDLE = int(os.environ.get("IDLE", "300"))
29PATIENCE = float(os.environ.get("PATIENCE", "90"))
30HASH = "".join("%%%02x" % b for b in range(1, 21))
31
32
33def start(port):
34 directory = tempfile.mkdtemp(prefix="ot-accf-")
35 with open(os.path.join(directory, "conf"), "w") as handle:
36 handle.write("listen.tcp 127.0.0.1:%d\nlisten.udp 127.0.0.1:%d\n" % (port, port))
37 log = open(os.path.join(directory, "log"), "w+b")
38 tracker = subprocess.Popen([TRACKER, "-f", os.path.join(directory, "conf")], stdout=log, stderr=log)
39 for _ in range(300):
40 try:
41 socket.create_connection(("127.0.0.1", port), timeout=0.2).close()
42 return tracker, log
43 except OSError:
44 time.sleep(0.02)
45 tracker.kill()
46 raise SystemExit("tracker never listened on %d" % port)
47
48
49def announce(port, timeout=5):
50 """One ordinary request, so we can see whether the tracker still serves."""
51 started = time.time()
52 try:
53 urllib.request.urlopen(
54 "http://127.0.0.1:%d/announce?info_hash=%s&peer_id=-OT0001-probe0000001&port=6881&left=0" % (port, HASH),
55 timeout=timeout,
56 ).read()
57 return time.time() - started, None
58 except Exception as problem: # noqa: BLE001 - reported, not swallowed
59 return time.time() - started, problem
60
61
62def main():
63 port = int(os.environ.get("PORT", "28000"))
64 tracker, log = start(port)
65 try:
66 print(" how long a connection that says nothing is held")
67 quiet = socket.create_connection(("127.0.0.1", port), timeout=5)
68 quiet.settimeout(PATIENCE)
69 started = time.time()
70 try:
71 held = "dropped after %.1f s" % (time.time() - started) if quiet.recv(1) == b"" else "sent us something"
72 except socket.timeout:
73 held = "still open after %.0f s" % PATIENCE
74 except OSError as problem:
75 held = "ended after %.1f s: %s" % (time.time() - started, problem)
76 quiet.close()
77 print(" %s" % held)
78
79 print(" and whether %d of them keep anybody else out" % IDLE)
80 before, problem = announce(port)
81 print(" a request before: %.0f ms%s" % (before * 1000, "" if problem is None else " (%s)" % problem))
82
83 idlers = []
84 for _ in range(IDLE):
85 try:
86 idlers.append(socket.create_connection(("127.0.0.1", port), timeout=2))
87 except OSError as problem:
88 print(" could only open %d of them: %s" % (len(idlers), problem))
89 break
90 print(" holding %d connections that have sent nothing" % len(idlers))
91
92 during, problem = announce(port, timeout=10)
93 print(" a request during: %.0f ms%s" % (during * 1000, "" if problem is None else " REFUSED: %s" % problem))
94
95 for idler in idlers:
96 idler.close()
97 after, problem = announce(port)
98 print(" a request after: %.0f ms%s" % (after * 1000, "" if problem is None else " (%s)" % problem))
99 finally:
100 tracker.send_signal(signal.SIGINT)
101 try:
102 tracker.wait(timeout=10)
103 except subprocess.TimeoutExpired:
104 tracker.kill()
105 tracker.wait()
106 log.close()
107
108
109if __name__ == "__main__":
110 main()
diff --git a/tests/integration/test_expiry.py b/tests/integration/test_expiry.py
index 5a21e57..5e6dc0a 100644
--- a/tests/integration/test_expiry.py
+++ b/tests/integration/test_expiry.py
@@ -27,6 +27,11 @@ class ExpirySweep(TrackerTestCase):
27 def test_the_sweep_takes_the_idle_and_leaves_the_answered(self): 27 def test_the_sweep_takes_the_idle_and_leaves_the_answered(self):
28 idle = socket.create_connection(("127.0.0.1", self.tracker.port), timeout=5) 28 idle = socket.create_connection(("127.0.0.1", self.tracker.port), timeout=5)
29 try: 29 try:
30 # Enough for a kernel that holds a connection back until data
31 # arrives to hand it over, and far too little to be answered, so
32 # the tracker has accepted it and is waiting for the rest
33 idle.sendall(b"GET /ann")
34
30 self.assertEqual(self.tracker.announce(self.hash_for("expiry-sweep"))[b"complete"], 1) 35 self.assertEqual(self.tracker.announce(self.hash_for("expiry-sweep"))[b"complete"], 1)
31 36
32 time.sleep(CLIENT_TIMEOUT + 3 * CHECK_INTERVAL) 37 time.sleep(CLIENT_TIMEOUT + 3 * CHECK_INTERVAL)
@@ -34,6 +39,6 @@ class ExpirySweep(TrackerTestCase):
34 self.assertEqual(self.tracker.torrent_count(), 1, "the tracker stopped serving while sweeping") 39 self.assertEqual(self.tracker.torrent_count(), 1, "the tracker stopped serving while sweeping")
35 40
36 idle.settimeout(5) 41 idle.settimeout(5)
37 self.assertEqual(idle.recv(64), b"", "a connection that never sent a request was not swept") 42 self.assertEqual(idle.recv(64), b"", "a connection that went quiet mid-request was not swept")
38 finally: 43 finally:
39 idle.close() 44 idle.close()
diff --git a/tests/syscount.c b/tests/syscount.c
new file mode 100644
index 0000000..ec45c15
--- /dev/null
+++ b/tests/syscount.c
@@ -0,0 +1,144 @@
1/* Counts the system calls a run makes, by standing in front of libc.
2
3 cc -O2 -shared -fPIC -o syscount.so tests/syscount.c -ldl
4 LD_PRELOAD=./syscount.so ./opentracker -f conf (DYLD_INSERT_LIBRARIES on macOS)
5
6 The tally goes to stderr when the process exits. Reading the source tells
7 you which calls a path should make; this tells you which it does make,
8 which is not always the same thing. */
9
10#define _GNU_SOURCE
11#include <dlfcn.h>
12#include <stdarg.h>
13#include <stdatomic.h>
14#include <stdio.h>
15#include <sys/socket.h>
16#include <sys/types.h>
17#include <unistd.h>
18
19#ifdef __linux__
20#include <sys/epoll.h>
21#endif
22
23#define COUNTER(name) static _Atomic long g_##name
24#define BUMP(name) atomic_fetch_add_explicit(&g_##name, 1, memory_order_relaxed)
25#define REPORT(name) fprintf(stderr, " %-14s %10ld\n", #name, atomic_load(&g_##name))
26
27COUNTER(accept);
28COUNTER(accept4);
29COUNTER(read);
30COUNTER(write);
31COUNTER(close);
32COUNTER(fcntl);
33COUNTER(recvfrom);
34COUNTER(sendto);
35COUNTER(epoll_ctl);
36COUNTER(epoll_wait);
37
38typedef int (*fn_accept)(int, struct sockaddr *, socklen_t *);
39typedef int (*fn_accept4)(int, struct sockaddr *, socklen_t *, int);
40typedef ssize_t (*fn_read)(int, void *, size_t);
41typedef ssize_t (*fn_write)(int, const void *, size_t);
42typedef int (*fn_close)(int);
43typedef int (*fn_fcntl)(int, int, ...);
44typedef ssize_t (*fn_recvfrom)(int, void *, size_t, int, struct sockaddr *, socklen_t *);
45typedef ssize_t (*fn_sendto)(int, const void *, size_t, int, const struct sockaddr *, socklen_t);
46
47static fn_accept real_accept;
48static fn_accept4 real_accept4;
49static fn_read real_read;
50static fn_write real_write;
51static fn_close real_close;
52static fn_fcntl real_fcntl;
53static fn_recvfrom real_recvfrom;
54static fn_sendto real_sendto;
55
56/* Resolving lazily keeps the library out of the way until something is
57 actually called, which matters because dlsym itself allocates */
58#define WRAP(name) \
59 if (!real_##name) \
60 real_##name = (fn_##name)dlsym(RTLD_NEXT, #name); \
61 BUMP(name)
62
63int accept(int s, struct sockaddr *addr, socklen_t *len) {
64 WRAP(accept);
65 return real_accept(s, addr, len);
66}
67
68#ifdef SOCK_NONBLOCK
69int accept4(int s, struct sockaddr *addr, socklen_t *len, int flags) {
70 WRAP(accept4);
71 return real_accept4(s, addr, len, flags);
72}
73#endif
74
75ssize_t read(int fd, void *buf, size_t count) {
76 WRAP(read);
77 return real_read(fd, buf, count);
78}
79
80ssize_t write(int fd, const void *buf, size_t count) {
81 WRAP(write);
82 return real_write(fd, buf, count);
83}
84
85int close(int fd) {
86 WRAP(close);
87 return real_close(fd);
88}
89
90int fcntl(int fd, int cmd, ...) {
91 va_list args;
92 long arg;
93
94 va_start(args, cmd);
95 arg = va_arg(args, long);
96 va_end(args);
97 WRAP(fcntl);
98 return real_fcntl(fd, cmd, arg);
99}
100
101ssize_t recvfrom(int s, void *buf, size_t len, int flags, struct sockaddr *from, socklen_t *fromlen) {
102 WRAP(recvfrom);
103 return real_recvfrom(s, buf, len, flags, from, fromlen);
104}
105
106ssize_t sendto(int s, const void *buf, size_t len, int flags, const struct sockaddr *to, socklen_t tolen) {
107 WRAP(sendto);
108 return real_sendto(s, buf, len, flags, to, tolen);
109}
110
111#ifdef __linux__
112typedef int (*fn_epoll_ctl)(int, int, int, struct epoll_event *);
113typedef int (*fn_epoll_wait)(int, struct epoll_event *, int, int);
114static fn_epoll_ctl real_epoll_ctl;
115static fn_epoll_wait real_epoll_wait;
116
117int epoll_ctl(int ep, int op, int fd, struct epoll_event *ev) {
118 WRAP(epoll_ctl);
119 return real_epoll_ctl(ep, op, fd, ev);
120}
121
122int epoll_wait(int ep, struct epoll_event *events, int max, int timeout) {
123 WRAP(epoll_wait);
124 return real_epoll_wait(ep, events, max, timeout);
125}
126#endif
127
128__attribute__((destructor)) static void report(void) {
129 fprintf(stderr, "\n system calls this run\n");
130 REPORT(accept);
131#ifdef SOCK_NONBLOCK
132 REPORT(accept4);
133#endif
134 REPORT(read);
135 REPORT(write);
136 REPORT(close);
137 REPORT(fcntl);
138 REPORT(recvfrom);
139 REPORT(sendto);
140#ifdef __linux__
141 REPORT(epoll_ctl);
142 REPORT(epoll_wait);
143#endif
144}
diff --git a/tests/t_event.c b/tests/t_event.c
index 5509f19..ad3948f 100644
--- a/tests/t_event.c
+++ b/tests/t_event.c
@@ -4,6 +4,7 @@
4 $id$ */ 4 $id$ */
5 5
6/* System */ 6/* System */
7#include <fcntl.h>
7#include <string.h> 8#include <string.h>
8#include <sys/socket.h> 9#include <sys/socket.h>
9#include <sys/time.h> 10#include <sys/time.h>
@@ -205,6 +206,41 @@ static void case_unknown_descriptor(void) {
205 CHECK_INT(event_del(4096), -1); 206 CHECK_INT(event_del(4096), -1);
206} 207}
207 208
209/* event_close exists so that the mechanisms which drop a closed descriptor
210 by themselves are not asked to do it twice. The one that keeps its own
211 list has to be told, and forgetting would leave the entry there to be
212 reported for ever, so what is checked here is the silence afterwards. */
213static void case_close_takes_it_out(void) {
214 int other[2], closed, count, i, strays = 0;
215
216 test_case("a closed descriptor is gone from the set and really closed");
217 pair_open();
218 CHECK(sv[0] != -1);
219 CHECK_INT(socketpair(AF_UNIX, SOCK_STREAM, 0, other), 0);
220 CHECK_INT(event_add(sv[0], EVENT_IN, 1), 0);
221 CHECK_INT(event_add(other[0], EVENT_IN, 2), 0);
222
223 closed = sv[0];
224 CHECK_INT(event_close(closed), 0);
225 sv[0] = -1;
226 CHECK_INT(fcntl(closed, F_GETFD), -1);
227
228 /* The survivor is made ready so the wait has something to come back with,
229 and anything left over from the closed one shows up beside it */
230 CHECK_INT((int)write(other[1], "x", 1), 1);
231 count = event_wait(ready, EVENTS, 200);
232 CHECK(count >= 1);
233 for (i = 0; i < count; ++i)
234 if (ready[i].fd != other[0])
235 ++strays;
236 CHECK_INT(strays, 0);
237
238 event_del(other[0]);
239 close(other[0]);
240 close(other[1]);
241 pair_close();
242}
243
208void test_event(void) { 244void test_event(void) {
209 CHECK_INT(event_init(), 0); 245 CHECK_INT(event_init(), 0);
210 CHECK(event_mechanism() != NULL); 246 CHECK(event_mechanism() != NULL);
@@ -217,6 +253,7 @@ void test_event(void) {
217 case_delete_from_the_middle(); 253 case_delete_from_the_middle();
218 case_timeout(); 254 case_timeout();
219 case_unknown_descriptor(); 255 case_unknown_descriptor();
256 case_close_takes_it_out();
220 257
221 event_deinit(); 258 event_deinit();
222} 259}
diff --git a/tests/t_socket.c b/tests/t_socket.c
index ab4b785..20a0c0a 100644
--- a/tests/t_socket.c
+++ b/tests/t_socket.c
@@ -78,15 +78,17 @@ static int sock_flag(int s, int level, int option) {
78} 78}
79 79
80/* Wait for one event, so a test never blocks forever on a lost packet */ 80/* Wait for one event, so a test never blocks forever on a lost packet */
81static int wait_for(int s, short events) { 81static int wait_for_ms(int s, short events, int timeout_ms) {
82 struct pollfd pfd; 82 struct pollfd pfd;
83 83
84 pfd.fd = s; 84 pfd.fd = s;
85 pfd.events = events; 85 pfd.events = events;
86 pfd.revents = 0; 86 pfd.revents = 0;
87 return poll(&pfd, 1, 2000) == 1; 87 return poll(&pfd, 1, timeout_ms) == 1;
88} 88}
89 89
90static int wait_for(int s, short events) { return wait_for_ms(s, events, 2000); }
91
90/* ------------------------------------------------------------ invariants */ 92/* ------------------------------------------------------------ invariants */
91 93
92/* Asked for the wildcard address, because a socket bound to one specific 94/* Asked for the wildcard address, because a socket bound to one specific
@@ -200,6 +202,42 @@ static void case_a_signal_ends_a_blocking_read(void) {
200 close(reader_socket); 202 close(reader_socket);
201} 203}
202 204
205/* What a deferred accept is for is a descriptor that already has the request
206 in it, so what is checked is the silence before: a client that has
207 connected but said nothing must not make the listener readable. Kernels
208 that cannot do this say so, and then there is nothing to check. */
209static void case_deferred_accept(void) {
210 struct sockaddr_in6 address;
211 char ip[16];
212 uint16_t port = 0;
213 int listener, client;
214
215 test_case("a connection that has sent nothing is held back from accept");
216 listener = socket_tcp_listen(ip_loopback, 0, SOCKET_EXCLUSIVE);
217 CHECK(listener != -1);
218 if (socket_defer_accept(listener) == -1) {
219 close(listener);
220 return;
221 }
222 bound_address(listener, ip, &port);
223
224 client = socket(AF_INET6, SOCK_STREAM, 0);
225 CHECK(client != -1);
226 memset(&address, 0, sizeof(address));
227 address.sin6_family = AF_INET6;
228 address.sin6_port = htons(port);
229 memcpy(&address.sin6_addr, ip_loopback, 16);
230 CHECK_INT(connect(client, (struct sockaddr *)&address, sizeof(address)), 0);
231
232 CHECK_INT(wait_for_ms(listener, POLLIN, 300), 0);
233
234 CHECK_INT((int)write(client, "GET / HTTP/1.0\r\n\r\n", 18), 18);
235 CHECK_INT(wait_for(listener, POLLIN), 1);
236
237 close(client);
238 close(listener);
239}
240
203/* An IPv4 peer must arrive in the v4 mapped form opentracker stores, however 241/* An IPv4 peer must arrive in the v4 mapped form opentracker stores, however
204 the kernel chose to report it. Linux fills a v4 mapped sockaddr_in6 here 242 the kernel chose to report it. Linux fills a v4 mapped sockaddr_in6 here
205 and the BSDs fill a plain sockaddr_in, so this is the case that tells the 243 and the BSDs fill a plain sockaddr_in, so this is the case that tells the
@@ -352,6 +390,7 @@ void test_socket(void) {
352 case_a_signal_ends_a_blocking_read(); 390 case_a_signal_ends_a_blocking_read();
353 case_udp_v4mapped(); 391 case_udp_v4mapped();
354 case_tcp_roundtrip(); 392 case_tcp_roundtrip();
393 case_deferred_accept();
355 case_connect_refused(); 394 case_connect_refused();
356 case_bind_failure(); 395 case_bind_failure();
357#ifdef WANT_SYNC_LIVE 396#ifdef WANT_SYNC_LIVE
generated by cgit v1.3 (git 2.53.0) at 2026-10-30 16:37:23 +0000