1

I'm trying to compute the v3 onion address (hostname) from the hs_ed25519_secret_key file generated by tor itself in python. I know i can just copy that file to the hidden_service dir and start tor so that tor will generate the public key and hostname itself but i want to do it offline and in python preferably.

I already managed to get the hostname from the hs_ed25519_public_key file, but from the secret key I cannot make it working... I know the hs_ed25519_secret_key file is 96 bytes (where the first 32 are == ed25519v1-secret: type0 ==\x00\x00\x00 and can be trashed). But the remaining 64 bytes cannot all be the secret key since this should be only 32 bytes as far as I understand, otherwise it is considered to contain the 32 bytes public key which it obviously does not. I know some key expansion is done before storing it to hs_ed25519_secret_key file but there must be a way to get the public key (and hence the hostname) from the expanded format.

So I tried to just cut off the last 32 bytes too, but i couldn't get this to work (not even with the help of ChatGPT lol). Here is what I got so far:

import sys
import hashlib
import base64
from nacl.signing import SigningKey

def extract_pub_key_from_file(secret_key_path):
    with open(secret_key_path, 'rb') as f:
        content = f.read()[32:] # ignore the == ed25519v1-secret: type0 ==\x00\x00\x00
    signing_key = SigningKey(content[:-32]) # alternatively tried (content[32:]) here
    verifying_key = signing_key.verify_key
    return verifying_key.encode()

def generate_onion_address(pubkey, version=b'\x03'):
    checksum = hashlib.sha3_256(b".onion checksum" + pubkey + version).digest()[:2]
    onion_address = base64.b32encode(pubkey + checksum + version).decode().lower()
    return onion_address

secret_key_path = sys.argv[1]
public_key = extract_pub_key_from_file(secret_key_path)
if public_key:
    print("Extracted Public Key:", public_key)

onion_address = generate_onion_address(public_key)
print("Onion Address: {}.onion".format(onion_address))

As I took the public key and the secret key generated by tor itself, I know it has to work in some way. Tor does it when provided with only the secret key, but I just cannot figure out what part of the file contains the real secret key.

Does anyone have an idea on the exact content of the hs_ed25519_secret_key file or is my code just buggy?


Edit: I found this answer on how to do the expansion of the ed25519 secret key to fit the hs_ed25519_secret_key format. As far as I understand, the hs_ed25519_secret_key file does only contain the hashed (and manipulated) version of the "real" secret key. But tor still manages to restore the public key and hostname from it, if provided with hs_ed25519_secret_key. How is this done?

1 Answer 1

0

How is this done?

That's the Neat Part, You Don't.

What you think is the private key is actually the seed. From the seed, all other stuff that is needed is derived, but the seed is not really needed.

Things get much more clear when looking at this ed25519 key creation. enter image description here

src https://pynacl.readthedocs.io/en/latest/signing/

The length of the hs_ed25519_secret_key is 96 bytes (= 3 x 32 bytes) and the format is

  • 32 bytes that mean "== ed25519v1-secret: type0 ==\x00\x00\x00"
  • 32 bytes that are the private scalar a (as shown in the upper scheme)
  • 32 bytes that are the righthalf RH of the sha512

So tor does not need to unhash something because there is no interest in the seed. All you need for performing this ed25519 stuff is the either

  • the seed k
  • the sha512 sum of that seed, so its first 32 bytes and last 32 bytes or
  • the private scalar a and minimum the RH of the sha512 (so tor does; also storing the LH would be redundant)

And from the private scalar a, the pubkey A is generated. Both are used in the sign part. And the RH is used with the message M in the sha512 in the sign part, too.

So the private scalar a and righthalf RH are the most computed values that one can get. Just storing the seed means computation every time you need the private scalar a or the RH for signing. And because A is in the file hs_ed25519_public_key, when the signing part must be performed, there is no need for tor to spend computing power on the keygen. It can just read a, A and RH from the files and all the computing power can be used for the sign part.

p.s. I checked the python libs cryptography and pynacl but I couldn't find a way to give a as an input for A. So with this, you could also convert the file hs_ed25519_secret_key into hs_ed25519_public_key.

edit: I just tested the link from

this answer

and ran it (modified without the path stuff and keygen)

import base64
import hashlib
from cryptography.hazmat.primitives.asymmetric import ed25519


def expand_private_key(secret_key) -> bytes:
    hash = hashlib.sha512(secret_key).digest()
    hash = bytearray(hash)
    hash[0] &= 248
    hash[31] &= 127
    hash[31] |= 64
    return bytes(hash)


def onion_address_from_public_key(public_key: bytes) -> str:
    version = b"\x03"
    checksum = hashlib.sha3_256(b".onion checksum" + public_key + version).digest()[:2]
    onion_address = "{}.onion".format(
        base64.b32encode(public_key + checksum + version).decode().lower()
    )
    return onion_address


def verify_v3_onion_address(onion_address: str) -> list[bytes, bytes, bytes]:
    # v3 spec https://gitweb.torproject.org/torspec.git/plain/rend-spec-v3.txt
    try:
        decoded = base64.b32decode(onion_address.replace(".onion", "").upper())
        public_key = decoded[:32]
        checksum = decoded[32:34]
        version = decoded[34:]
        if (
            checksum
            != hashlib.sha3_256(b".onion checksum" + public_key + version).digest()[:2]
        ):
            raise ValueError
        return public_key, checksum, version
    except:
        raise ValueError("Invalid v3 onion address")


def create_hs_ed25519_secret_key_content(signing_key: bytes) -> bytes:
    return b"== ed25519v1-secret: type0 ==\x00\x00\x00" + expand_private_key(
        signing_key
    )


def create_hs_ed25519_public_key_content(public_key: bytes) -> bytes:
    assert len(public_key) == 32
    return b"== ed25519v1-public: type0 ==\x00\x00\x00" + public_key


def store_bytes_to_file(
    bytes: bytes, filename: str, uid: int = None, gid: int = None
) -> str:
    with open(filename, "wb") as binary_file:
        binary_file.write(bytes)
    return filename


def store_string_to_file(
    string: str, filename: str, uid: int = None, gid: int = None
) -> str:
    with open(filename, "w") as file:
        file.write(string)
    return filename


def create_hidden_service_files(
    private_key: bytes,
    public_key: bytes,
    hidden_service_dir: str,
) -> None:


    file_content_secret = create_hs_ed25519_secret_key_content(private_key)

    store_bytes_to_file(
        file_content_secret, f"{hidden_service_dir}/hs_ed25519_secret_key", None, None
    )

    file_content_public = create_hs_ed25519_public_key_content(public_key)
    store_bytes_to_file(
        file_content_public, f"{hidden_service_dir}/hs_ed25519_public_key", None, None
    )

    onion_address = onion_address_from_public_key(public_key)
    store_string_to_file(onion_address, f"{hidden_service_dir}/hostname", None, None)


if __name__ == "__main__":
    priv = ed25519.Ed25519PrivateKey.from_private_bytes(32*b"\x00")
    pub = priv.public_key().public_bytes_raw()
    create_hidden_service_files(
        priv.private_bytes_raw(),
        pub,
        ".",
    )

When you copy the hs_ed25519_secret_key file that is generated by this code in the tor directory, tor will create the same hostname file as this code. So what is going on? The function expand_private_key transfers the seed into the private scalar a (first 32 bytes of the return) (and the RH (unmodified last 32 bytes of the sha512 checksum / return)), where tor can create the pubkey and therefore the hostname, whereas the ed25519 function needs the seed (maybe there is a way to pass a and RH instead). Further, RH shouldn't matter here, because RH is not related. You can even set the RH in the hs_ed25519_secret_key file (so the last 32 bytes) to zero and tor is still happy with this file and it will work (disrespect security).

tl;dr

  • tor uses the format a + RH, so 64 bytes of relevant data and the first 32 bytes in hs_ed25519_secret_key are just for information.
  • the python functions needs a seed instead, where a and RH is derived from
  • the function expand_private_key makes some voodoo and generates from the seed the real private key a
  • it gets obvious that the python function are using the seed and not the private key, because ed25519.Ed25519PrivateKey.from_private_bytes(<32 bytes>) wants 32 bytes. And with only 32 bytes defined, ed25519 cannot work except these bytes are the seed

editedit: finally, the answer to your actual question. (Now I am a bit curious, what you want to achieve with this? I mean, this is not useful for vanity addresses. What do you want to do with this?)

def onion_address_from_public_key(pubkey):
    import base64
    import hashlib
    """ https://spec.torproject.org/address-spec

    Generates the onion address from the public key

    Parameters
    ----------
    pubkey : bytes
        The public key with a length of 32 bytes.

    Returns
    -------
    onion_address : string
        The onion address.

    """

    VERSION = b"\x03" 
    checksum = hashlib.sha3_256(b".onion checksum" + pubkey + VERSION).digest()[:2]
    onion_address = "{}.onion".format(base64.b32encode(pubkey + checksum + VERSION).decode().lower())
    return onion_address

class Ed25519:
    # stolen and modified from https://github.com/andreacorbellini/ecc/blob/master/scripts/ecdsa.py
    def __init__(self):
        """ https://www.rfc-editor.org/rfc/rfc7748#section-4.1

        Twisted Edwards Curve equation E: a*x^2 + y^2 = 1 + d*x^2*y^2
        Values taken from RFC7748

        Attributes
        ----------
        a, d : int
            Curve parameters.
        p : int
            Field (modulo p).
        n : int
            Order of generator Point ord(G).
        G : (int, int)
            Generator point.


        """
        self.a = -1
        self.d = 0x52036cee2b6ffe738cc740797779e89800700a4d4141d8ab75eb4dca135978a3
        self.p = 2**255 - 19
        self.n = 2**252 + 0x14def9dea2f79cd65812631a5cf5d3ed
        #self.cofactor = 8 # not used, just for completeness
        X = 0x216936d3cd6e53fec0a4e231fdd6dc5c692cc7609525a7b2c9562d608f25d51a
        Y = 0x6666666666666666666666666666666666666666666666666666666666666658
        self.G = (X, Y)

    # Modular arithmetic ######################################################
    def inverse_mod(self, k, p):
        """
        Returns the inverse of k modulo p. Nothing changed by me. Thank you,
        original author for this function.

        Parameters
        ----------
        k : int
            Must not be zero.
        p : int
            Must be a prime number.

        Returns
        -------
        int
            k modulo p.

        """
        if k == 0:
            raise ZeroDivisionError('division by zero')

        if k < 0:
            # k ** -1 = p - (-k) ** -1  (mod p)
            return p - self.inverse_mod(-k, p)

        # Extended Euclidean algorithm.
        s, old_s = 0, 1
        t, old_t = 1, 0
        r, old_r = p, k

        while r != 0:
            quotient = old_r // r
            old_r, r = r, old_r - quotient * r
            old_s, s = s, old_s - quotient * s
            old_t, t = t, old_t - quotient * t

        gcd, x = old_r, old_s

        assert gcd == 1
        assert (k * x) % p == 1
        
        return x % p

    # Functions that work on curve points #####################################
    def is_on_curve(self, P):
        """
        Checks if the given point is on the elliptic curve.

        Parameters
        ----------
        P : (int, int)
            Point to check.

        Returns
        -------
        bool
            True, when point is on curve.
            False, when point is not on curve.

        """
        if P is None:
            # None represents the point at infinity.
            return True

        x, y = P
        return ((self.a*x*x + y*y) % self.p == (1 + self.d*x*x*y*y) % self.p)

    def point_neg(self, P):
        """ https://en.wikipedia.org/wiki/Twisted_Edwards_curve#Addition_on_twisted_Edwards_curves

        Negative of (x1, y1) is (-x1, y1).

        Parameters
        ----------
        P : (int, int)
            Point to negate.

        Returns
        -------
        (int, int)
            Negative point.

        """
        assert self.is_on_curve(P)

        if P is None:
            # -0 = 0
            return None

        x, y = P
        result = (-x, y % self.p)

        assert self.is_on_curve(result)

        return result

    def point_add(self, P1, P2):
        """ https://en.wikipedia.org/wiki/Twisted_Edwards_curve#Addition_on_twisted_Edwards_curves
        https://en.wikipedia.org/wiki/Twisted_Edwards_curve#Doubling_on_twisted_Edwards_curves

        Adds two points according to the group law.

        Parameters
        ----------
        P1 : (int, int)
            Point 1.
        P2 : (int, int)
            Point 2.

        Returns
        -------
        (int, int)
            Point 3 = P1 + P2, when points are different.
            Point 3 = P1 + P1, when points are equal.

        """

        assert self.is_on_curve(P1)
        assert self.is_on_curve(P2)

        if P1 is None:
            # 0 + point2 = point2
            return P2
        if P2 is None:
            # point1 + 0 = point1
            return P1

        x1, y1 = P1
        x2, y2 = P2

        if x1 == x2 and y1 != y2:
            # point1 + (-point1) = 0
            return None

        if x1 == x2:
            # This is the case point1 == point2.
            x3 = 2*x1*y1 * self.inverse_mod(self.a*x1*x1 + y1*y1, self.p)
            y3 = (y1*y1 - self.a*x1*x1) * self.inverse_mod(2 - self.a*x1*x1 - y1*y1, self.p)
        else:
            # This is the case point1 != point2.
            x3 = (x1*y2 + y1*x2) * self.inverse_mod(1 + self.d*x1*x2*y1*y2, self.p)
            y3 = (y1*y2 - self.a*x1*x2) * self.inverse_mod(1 - self.d*x1*x2*y1*y2, self.p)

        result = (x3 % self.p, y3 % self.p)

        assert self.is_on_curve(result)

        return result

    def scalar_mult(self, k, P):
        """
        Calculates k * point with the double and point add algorithm.
        Nothing changed by me. Thank you, original author for this function.

        Parameters
        ----------
        k : int
            Multiplier.
        P : (int, int)
            Multiplicand.

        Returns
        -------
        (int, int)
            Product of scalar multiplication.

        """
        assert self.is_on_curve(P)

        if k % self.n == 0 or P is None:
            return None

        if k < 0:
            # k * point = -k * (-point)
            return self.scalar_mult(-k, self.point_neg(P))

        result = None
        addend = P

        while k:
            if k & 1:
                # Add.
                result = self.point_add(result, addend)

            # Double.
            addend = self.point_add(addend, addend)

            k >>= 1

        assert self.is_on_curve(result)

        return result


    # Conversion of private scalar to pubkey like tor does
    def pubkey_from_priv_scal(self, private_scalar):
        """ https://www.rfc-editor.org/rfc/rfc8032#section-5.1.5

        Converts the private scalar according to RFC8032 into the public key.

        Parameters
        ----------
        private_scalar : bytes
            The private scalar `a` (not the seed k).

        Returns
        -------
        bytes
            The public key `A`.

        """
        i = int.from_bytes(private_scalar, "little") # 3. Buffer as little-endian int
        x, y = self.scalar_mult(i, self.G)           #    Fixed-base scalar multiplication           
        y = int.to_bytes(y, 32, "little")            # 4. y-coord. as 32 bytes little-endian
        x = int.to_bytes(x, 32, "little")            #    (also needed for least sign. bit)
        if x[0] & 1:                                 #    When least sign. bit is 1, then
            y = bytearray(y)                         #    (make bytes accessible via index)
            y[31] |= 128                             #    make the most sign. bit 1.
            y = bytes(y)                             #    (back to bytes from the array)
        return y


c = Ed25519()

f = open("hs_ed25519_secret_key", "rb")
private_scalar = f.read()[32:64]

f = open("hs_ed25519_public_key", "rb")
pubkey_should = f.read()[32:64]

pubkey = c.pubkey_from_priv_scal(private_scalar)
print(pubkey_should == pubkey)

print(onion_address_from_public_key(pubkey))

2
  • Thanks for the long answer! I am not that deep in ed25519 that I would have found out something like this. Do you know any other library/tool that works offline and could handle a instead of the seed to get the public key?
    – SkUzy42
    Mar 8 at 13:38
  • @SkUzy42 Unfortunately, I haven't found a library. Fortunately, elliptic curve math is quiet easy and in a finite field not thaaat hard. I have lent some python code for the bitcoin curve secp256k1 and modified it that it fits to ed25519. I've tested it with a few hs_ed25519_secret_key files generated by the first code block of my answer. I will edit the answer and post some pure python code, without the need of any libs (except for the .onion address generator function).
    – NULL
    Mar 9 at 19:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .