I like my Tikka T3, its the most accurate off-the-shelf bolt gun I've owned.

But if you are truly looking for a budget rifle, the chain sporting goods stores usually discount a basic bolt gun in August for hunting season. Any of the Remington, Savage, Ruger low-end models will work well.

I would suggest .30-06 for a first gun. Ammo is easy to find, has a lot of bullet choices and its easy to learn to reload for. I urge you to avoid the more esoteric calibers as their ammo is more expensive, and learning to reload on a belted magnum (.300 Win Mag as an example) isn't a good idea.