Limit the number of street addresses in autofill?

So I have about two street addresses that I want to actually pre-fill.
The other 150 customer site addresses that I just deleted I really don’t want saved.
My experience is the feature is just toggled on or off.
Can I have the feature turned on, but only allow a finite number of addresses to actually be saved and reused?