Which games can run from a USB SSD
Microsoft requires external drives to be USB 3.0 or faster with at least 128GB of space. Backward-compatible Xbox One, Xbox 360 and original Xbox games install and play directly from them. Games optimized for Series X|S can be stored on a USB drive, but they must be moved to the console's internal storage or an official expansion card (made by Seagate and WD_Black) before you can play them.
That still makes a portable SSD useful for new games: copying a game back from a fast SSD is much quicker than downloading it again, so you can keep a large library on the SSD and rotate the games you are playing onto the internal drive.
Don't pay for extra speed the Xbox can't use
The Xbox Series X|S USB ports run at 5Gbps (USB 3.1 Gen 1), which caps real transfers at roughly 400 MB/s. A drive rated at about 1,000 MB/s, such as the SanDisk Extreme or Crucial X9, already exceeds what the port can deliver, and 2,000 MB/s drives like the Samsung T9 or WD_Black P40 run no faster on the console.
For an Xbox-only drive, put the money into capacity instead. The Crucial X10 6TB holds a very large backward-compatible library in a pocket-sized drive.
How much extra space you need
The Xbox Series X has a 1TB internal SSD with about 802GB usable after the system files, and the Series S has 512GB with about 364GB usable. Large modern games take 100GB or more, so the internal drive fills after a handful of them. A 1TB or 2TB portable SSD roughly doubles or triples your library space, and a 4TB+ drive covers most collections.