0

My tor relay uses ~50 Mbytes/s (25 MBytes/s read and 25 MBytes/s write). I would expect that each byte has to be decrypted and encrypted once. So I would expect my tor relay decrypts ~25 MBytes/s and encrypts ~25 MBytes/s.

The server receives < 10 SYN packets per second, which I assume is mostly setting up new TCP connections to other relays.

openssl seems to be able to do 150-350 MBytes/s (median IP-packet size to the node is 543), depending on whether tor uses AES-NI or not:

$ openssl speed -elapsed -evp aes-256-cbc
:
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-cbc     273508.06k   336348.35k   369346.47k   358322.18k   225869.82k   229485.23k

$ OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-256-cbc
:
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes                            
aes-256-cbc     141424.04k   177762.28k   160536.66k   148307.63k   154921.64k   188443.31k

Given this, I would have expected my tor-relay would run at 10-30% CPU. But it runs at 150% CPU (two cores at ~75%).

What explains this big difference?

0

Relays have more overhead than just TLS crypto operations. For example the scheduler uses a significant amount of CPU usage. There are also many memory copies and allocations involved in moving cells around in Tor between buffers.

Your benchmark uses AES-128, but Tor relay connections generally use AES-256 which should be slower than your AES-128 results. In addition to TLS, there is also the relay cell encryption which uses AES-128. Also when tor uses a bandwidth of for example 50 MiB/s, it uses 50 MiB/s for both incoming and outgoing bytes (not combined).

Two cores both at ~75% sounds unusual for a Tor relay, I would have expected it to be more like ~75% on one core and ~10% on the other core. Does top show the process at 150% (since the process CPU usage in top is the sum of all threads)?

For reference, with a server that gets about 730,000 KB/s in the openssl AES-128 benchmark, it has a maximum relay bandwidth of ~150 MiB/s at 100% CPU usage.

| improve this answer | |
  • I updated with AES-256 numbers. It is still very far from the expected. The 50 Mbytes/s is 25 M read + 25 M write, NOT 50 M read + 50 M write. I am surprised if the internal house keeping takes 3 times longer than AES does: The machine can copy >10 GBytes/s in RAM. Can you qualify that more? – Ole Tange Mar 30 at 18:21
  • Okay so the throughput is only 25 MB/s. Why would you be surprised if the internals take 3 times longer than AES does? AES is quite fast nowadays. If you want actual data you need to profile tor on your system. On the system I mentioned above, the CPU spends about 30% of its time doing socket reading/writing and TLS. Another ~30% is spent in the scheduler, and ~30% is processing the cells. Tor is also mostly single-threaded (threads are only used for small things such as circuit construction crypto), so the 70% CPU usage in a thread still doesn't seem right. – Steve Mar 30 at 18:45
  • Where do you see the 30% numbers? Can I ask my Tor process to tell me those, too? – Ole Tange Mar 30 at 18:52
  • Those are from profiling with ptrace and gperftools. Gperftools will probably give you better data here. I compiled it directly into tor, but you should be able to LD_PRELOAD it into tor without needing to recompile. When you view the data, it'll show you frequencies of what functions tor is in when it takes measurements. – Steve Mar 30 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.