0

My tor relay uses ~50 Mbytes/s (25 MBytes/s read and 25 MBytes/s write). I would expect that each byte has to be decrypted and encrypted once. So I would expect my tor relay decrypts ~25 MBytes/s and encrypts ~25 MBytes/s.

The server receives < 10 SYN packets per second, which I assume is mostly setting up new TCP connections to other relays.

openssl seems to be able to do 150-350 MBytes/s (median IP-packet size to the node is 543), depending on whether tor uses AES-NI or not:

$ openssl speed -elapsed -evp aes-256-cbc
:
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-cbc     273508.06k   336348.35k   369346.47k   358322.18k   225869.82k   229485.23k

$ OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-256-cbc
:
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes                            
aes-256-cbc     141424.04k   177762.28k   160536.66k   148307.63k   154921.64k   188443.31k

Given this, I would have expected my tor-relay would run at 10-30% CPU. But it runs at 150% CPU (two cores at ~75%).

What explains this big difference?

Improve this question

1 Answer 1

Reset to default
0

Relays have more overhead than just TLS crypto operations. For example the scheduler uses a significant amount of CPU usage. There are also many memory copies and allocations involved in moving cells around in Tor between buffers.

Your benchmark uses AES-128, but Tor relay connections generally use AES-256 which should be slower than your AES-128 results. In addition to TLS, there is also the relay cell encryption which uses AES-128. Also when tor uses a bandwidth of for example 50 MiB/s, it uses 50 MiB/s for both incoming and outgoing bytes (not combined).

Two cores both at ~75% sounds unusual for a Tor relay, I would have expected it to be more like ~75% on one core and ~10% on the other core. Does top show the process at 150% (since the process CPU usage in top is the sum of all threads)?

For reference, with a server that gets about 730,000 KB/s in the openssl AES-128 benchmark, it has a maximum relay bandwidth of ~150 MiB/s at 100% CPU usage.

Improve this answer
4
  • I updated with AES-256 numbers. It is still very far from the expected. The 50 Mbytes/s is 25 M read + 25 M write, NOT 50 M read + 50 M write. I am surprised if the internal house keeping takes 3 times longer than AES does: The machine can copy >10 GBytes/s in RAM. Can you qualify that more?
    – Ole Tange
    Mar 30, 2020 at 18:21
  • Okay so the throughput is only 25 MB/s. Why would you be surprised if the internals take 3 times longer than AES does? AES is quite fast nowadays. If you want actual data you need to profile tor on your system. On the system I mentioned above, the CPU spends about 30% of its time doing socket reading/writing and TLS. Another ~30% is spent in the scheduler, and ~30% is processing the cells. Tor is also mostly single-threaded (threads are only used for small things such as circuit construction crypto), so the 70% CPU usage in a thread still doesn't seem right.
    – Steve
    Mar 30, 2020 at 18:45
  • Where do you see the 30% numbers? Can I ask my Tor process to tell me those, too?
    – Ole Tange
    Mar 30, 2020 at 18:52
  • Those are from profiling with ptrace and gperftools. Gperftools will probably give you better data here. I compiled it directly into tor, but you should be able to LD_PRELOAD it into tor without needing to recompile. When you view the data, it'll show you frequencies of what functions tor is in when it takes measurements.
    – Steve
    Mar 30, 2020 at 18:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .