Securely hosting a Tor hidden service/site

Question

There are some detailed instructions for setting up a Tor site at torproject.org1. It seems that it would be most secure to use a dedicated machine for this task if possible. I've read elsewhere that thttpd might be preferable to Apache and its likely more secure to go with a GNI/Linux over Windows.

Any thoughts on this? Is there a "standard" distro and/or web-server that would be best-suited for this task? Or is that merely a matter of personal preference?

Also, what are appropriate and effective security practices, both technical and operational? What can we learn from the recent compromises of Freedom Hosting (perhaps the largest .onion hosting provider) and the Silk Road? According to Wired2, "the FBI yesterday acknowledged that it secretly took control of Freedom Hosting last July". The article notes that "[i]t’s not clear how the FBI took over the servers". However, the comment that "the bureau was temporarily thwarted when Marques somehow regained access and changed the passwords" is suggestive. As details come out in the trial, it would be prudent to identify relevant technical and/or operational failures.

In the case of the Silk Road, it's clear from the Maryland complaint3 that vulnerabilities in Tor were not instrumental in the site's compromise. Even so, did technical and/or operational failures contribute to the takedown, and how might they be corrected?

As new hidden service sites fill these market niches, at least some of the underlying technical and/or operational improvements may become public. There is much to learn about doing this right.

Answer 1

It's prudent to use open-source software, given the greater risk of backdoors in closed-source products. You want to thoroughly lock down remote access to the server. In my experience, servers are constantly hammered by login attempts. It's crucial to disable password-based ssh logins, allowing only key-based logins.

I've also seen thttpd recommended for hidden services, because it has a smaller attack surface. Unless you have good reason, and really know mysql and php, it's best to serve only static html. Vulnerabilities there are your major risk after securing login. Static html is also faster over Tor, because there's less back-and-forth in loading the site.

As David notes, it's best to run Tor and the service in separate machines, or at least in separate VMs. If you're using one machine, you want Tor on the host and the service in the VM, given the greater risk that the service will get hacked and try to take down the Tor client, rather than vice versa.

But the bigger question is where to site these machines. I have pondered this question for years. First, consider that, if someone wants to create a hidden service, it's reasonable to assume that they want it well "hidden". It's also reasonable to assume that the operator wants to remain anonymous, not linked to the hidden service. If OP doesn't care about those issues, I have no clue why they want to run a hidden service.

Regarding location, there are some seemingly contradictory requirements. For obvious reasons, it's imprudent to operate from one's home or place of business. However, it's also crucial to control physical access. Otherwise, it's impossible to rely on security practices such as full-disk encryption. Hardware can be hardened, of course, and one can use Mandos to protect against tinkering. But ultimately, physical access trumps everything else.

Controlling physical access to hosted servers is logistically difficult, especially when one wishes to remain anonymous. It's also very expensive. And then there's the need to trust ones partners and staff.

One workaround is using diskless servers as reverse proxies for actual content servers. However, it's still necessary to securely host the content servers. So that's not a real (or at least full) solution.

Perhaps that's why Freedom Hosting was so popular. Users figured that they must have some special knowledge. However, in retrospect, it's pretty clear that neither Freedom Hosting nor the Silk Road were very well secured.

Answer 2

Interesting discussion points thus far.

1. Use VMs as discussed. I prefer VMware because of the networking, cloning, console access, replication, etc. No need for an OSS hypervisor discussion. These VMs need to be online, replicating, and failing over with zero issues and for use by any person who is involved in the enterprise.
2. Your VMs are on RFC1918 IPs, on an isolated network, behind a firewall. pfsense.org appliance is fine. The firewall is your router and the only device your VMs route traffic through. Each VM has a function! Do not mix functionality. You have a firewall VM, a storage VM, a Tor VM, a hidden service VM, etc.
3. The datastore for the VMs is a FreeNAS VM, with encryption and compression, on the same host as the other VMs. This provides encrypted storage for all VMs and is the only VM that will need to be replicated offsite. The ESXi host connects to the FreeNAS: ESXi host <-> VMkernel Interface <-> NFS <-> FreeNAS VM (on local datastore).
4. The OS of your VMs are stripped down installs. What remains on the VMs is only what is needed to for that VM to perform its function. Remove all other binaries. Traceroute, ping, wget, fetch, netstat, gcc, tcpdump, ssh, vi, cat, ls, etc etc. There should be nothing on the production VMs that can be used to identify its location or real IPs, compile software, edit files, view files, fetch tools, etc. Your firewall should pay as much attention to the LAN as it does the WAN. This also includes stripping all un-necessary modules from PHP as many of the functions of the removed system binaries can be duplicated with PHP modules.
5. No SSH, FTP, telnet, etc. access to the VMs. The VMs are managed via the console of the VM. You can mount CDROM's, Floppies, etc. from the VM console for code pushes.
6. FreeNAS is replicated on a 4 hour RPO (the whole VM), after the initial sync the updated data would be minimal and would sync in minutes. You will have 4+ datacenters at any given time. All datacenters are replaced every 3 months. You will bring your VMs live at one of your 4 datacenters every week. You will delete the FreeNAS VM from the prior host each week. Rinse repeat. Your data is flowing from one data center to another each week via replication.
7. Downtime should be considered a huge warning sign to not return to a particular datacenter. Your cloned servers will provide no help to any investigation because your FreeNAS VM (containing all of your VMs) is encrypted. The extra paranoid would also have the OS of each VM encrypted also.

Your FreeNAS VM should be 5GB in size (or less) for easy portability. Thin provision your VMs and turn on compression on the FreeNAS datastore.

Answer 3

When running a hidden service the same precautions should be taken if you were running it over standard TCP/IP. Assuming the service is configured correctly then the only added risk is the Tor client implementation.

Depending on your resources you may want to setup a box only for the purpose of running the Tor client and pointing the service ports to another box running your service. Then block all traffic otherwise to and from that host by some means not controlled by the box running the service. (periodically reloading with an updated OS would be advised) If you don't have the ability to run two machines you could always virtualize that setup, something like ESXi would be great for such a setup.

Answer 4

It depends what attacks you want to defend against.

• Having your service private key stolen - this is a significant problem and something you should be trying hard to avoid. If someone steals the private key they can own your site, forever, and you can't ever get it back.
• Having your data etc, stolen, at least if you keep hold of your private key, you can just get all your users to change their passwords or something (might be tricky though, if an attacker has all their reset details, there is probably no way to easily avoid the attacker spoofing all your users forever)

I would strongly recommend

• Using a VM for your server
• using full-disc encryption on your server VM (although this may be defeated e.g. in the UK by authorities forcing you to reveal the key under threat of imprisonment)
• Having backups somewhere else out of the way, ideally admin'd by a trusted friend.

For the actual software itself, use whatever you are most comfortable with, but remember that unlike a "non-hidden" site, a security compromise is likely to be absolute and final.

Answer 5

Since mirimir already mentioned running tor and webserver on seperate computers, I will mention that Whonix is a great Debian-based GNU/Linux build for anonymity of users and servers. Also, having a LUKS (or like whole disk encryption Free And Open Sourced software) encrypted server will be a good idea (unless you use predecessors of DDR3, which is susceptible to Cold Boot Attack.)