No, this isn't true if you're just installing Tor Browser onto a USB and running it from the USB on a standard operating system.
While Tor Browser does everything it can to reduce it's forensic fingerprint, on most operating systems simply running a program that is stored on a USB on the operating system itself will not stop it leaving behind traces or the operating system itself recording details of the applications which were run and when.
You may be confusing this with using an operating system like Tails, which is run from a USB and loaded into RAM. It takes steps to avoid leaving forensic traces on the host computer and it pre-configured to use Tor and stop internet connections that don't go over Tor. You can find out more about the design of Tails on their website.