While Scarlet/Violet seems to run more at 20fps in the open world
according to Digital Foundry You are not that far.
Sword & Shield also had issues, but especially in the Wild Area and less than the current games (even Arceus does better than the current games)
On top of that, the load times and graphics (the latter especially outside of the Wild Area) were okay, plus the frame rate wasn’t that bad either.
But the truth is that only Let’s Go games and D&P remakes have no major issues on the Switch, followed by S&S (because it’s Wild Area) and Arceus (because it’s open world).
But even though Arceus is at the bottom, the difference in terms of graphics and performance is absurdly large between that game and the new S&V games.
For me, the misery really started from the Switch with Pokémon, and Sun & Moon (+ Ultra variants) is still just fine, better than the Switch was but worse than the X&Y and ORAS before it.
I don’t count the problems with double and triple battles in 3ds games, which has been consistent across all Pokémon games on this system
Black(2) and White(2) were nicer that generation, X&Y showed a lot of improvement because the 3DS had more processing power and ORAS tweaked that a bit.
Sun & Moon went too far.
I’d also prefer that Gamefreak at least go back to how S&S was on Switch, not every game has to be unlockable by itself, and we don’t need a gimmick with every new game (except for Mega Pokémon, gimmicks after that were useless and got in the way more than that)
But that won’t happen, that’s the path they (unfortunately) want to follow.