I suspect the RDAs for a lot of nutrients (such as potassium) are way out of whack, because they're assuming you'll be eating lots of bread and having nutrients get leached out of your body.
You're probably right. Heavy meat eaters (Eskimos( get by very well with limited Vitamin C and Calcium depletion seems to be prevented by protein for example. It wouldn't surprise me if actual requirements are vastly different in someone that avoids grains. Another factor is that the RDA is what's supposed to be required to prevent symptoms of diseases. Meeting the RDA doesn't guarantee optimum health.