That's correct.
A dedicated rimfire rifle with a detachable mag can have all the evil features you want, like pistol grips, flash hiders, folding stocks, etc.
What it can't have is a magazine over 10 rounds, unless you already possessed the magazine before 2000, in California.