I kept meaning to do this to my rifle/rifles that are smooth barrel, but every time I called a gunsmith, their turn around time was quoted as pretty lengthy... I finally gave up (after purchasing sights). I don't know what kind of sights these fellas are buying, but all the ones I found were very reasonable in price.
With the upcoming hunting season, most gunsmiths are going to be booked solid. While it is a bit of a challenge, I did eventually find an older savage 111 with irons on it. I, like some other folks on here, much prefer iron sights to scopes if and when the terrain fits. Now, I have 2 .30-06 rifles and use the one that fits my terrain the best. If (in the rare instances) I am on open terrain, I carry my a-bolt scope only gun. The rest of the time, I'll be carrying the savage.
The biggest issue I had, was actually FINDING a modern hunting rifle with open sights, they're a bit of a unicorn in my area. I could have "gotten by" with the .30-30, but I just prefer the .30-06 cartridge so now I have no need to keep wondering if I should put some sights on my browning...
The one issue you may encounter is if you wanted open sights AND a scope, then you'll need high rise mounts (as was mentioned). There is a bit of an accuracy loss with those, but my main reason for wanting a rifle with open sights is a couple years ago I fell on ice on my porch while leaving for hunting and knocked my scope off of zero, it caused me to gut shoot a deer and lose it. So there is a good argument in of itself for having a rifle with both scope and open sights... Good to always have a backup plan.
It's really your call, but personally for me, if I was going to an area where it is likely I'll be making a longer range shot (long range for me is 2-300 yards), I'd want the scope only rifle. If I were hunting in an area where it is not probable to make a long range shot, I'd opt for a high rise scope with see through mounts and iron sights. If I were hunting in an area where it would be highly unlikely to make a longer range shot, I would personally prefer the no-scope option...
Those scopes do add some weight too, at least the cheap scopes I can afford do.
It's all personal preference, but if you're confident in your open sight shooting and don't like scopes, then go for it, although you may be better off just finding a rifle with irons already on it. The other consideration is finding sights that will match the contour of your barrel, if the front sight is too high or too low, no amount of adjustment of that rear sight will compensate for it.