is it possible for an exit node operator to reroute traffic through the tor network, despite all the disadvantages? What effect would that have anonymity and finally what would happen if the software directs some of the traffic to exit through that very node?
It's a lot of wasted time, it's easily detectable and would likely be flagged as malicious and get any such relay ejected from the Tor network for it's efforts.
It might result in easier deanonymization, as is always the case in Tor-over-Tor. The resulting paths might overlap resulting in it using a single relay or relays within the same family as both entry and exit giving a single entity the ability to deanonymize the user, a situation that Tor takes steps to avoid when constructing it's own paths. It would have no control over the second path and so relays might be used more than once in the same path.