I am no engineer or metallurgist but I have spent a quite a few years working metal so here are my thoughts on forged vs billet:
When a blank is forged the granular structure of the alloy is condensed. You get a denser grain structure in a forging regardless of what type of metal is used i.e. steel or aluminum.It is also aligned somewhat with the shape it is forged in. This will help with the way stresses are induced on the shape.That I would think is where one of the difference in tensile strength would come from.
Your chances of voids in the forging would be less also compared to billet. I have machined from billet/bar and have found inclusions in the billet.
I think the main reason billet lowers are popular now is for the simple fact they can be manufactured more easily than in the past. Billet bike and car parts are all the rage, why not firearms?
Think of what the cost of a lower would be if it was manually machined on a mill. I think the time for a competent machinest to complete a 80% lower is somewhere around 40 hours. And that is not including any time on exterior finish.
Nobody could afford a manually machined lower @ $80.00 per hr.
Now with a CNC machining center that time is probably cut to less than an hour.
Billet parts are only as good as the billet they are machined from. Usually billets are cut from a slab that came out of a rolling mill. The rolling mill can induce stresses to the billet that could possibly be detrimental. Look at the end cut on a piece of material and sometimes you can actually see the stresses.

Alloys used are going to play a huge part in the strength also. Most forged aluminum is in the 7000 series of alloys and if I am not mistaken it has better strength characteristics than 5000 or 6000 alloys that is typical for billet.
The machined lowers I have seen always seemed heavier and "chunkier" than a forged. I think with a forged you can minimize the structure and mass and still retain the strength.
I think you get finer contours as well.
Either way I dont think the strength is really all that critical in a lower. As the lock-up and firing stress is all pretty much contained in the bolt/barrel combination the lower just carries the buttstock and fire control parts.
Look at a Cavalry Arms plastic lower someday.

Mil-Spec is kind of a misnomer. People have the impression that mil-spec makes it better. Mil-Spec means that parts from a variety of manufactures will interchange. It also means that the firearm will work reliably in battlefield conditions.
Basicially it is a set of tolerances that the manufactures must adhere to. And on true Mil-Spec items it is usually a pretty broad set of tolerances, I have seen some in the .020 range. You can throw a tricycle through it with that kind of range.
To be truely Mil-Spec regardless of manufacturing method it would have to meet the tolerance range called out on the specs.
When armorers and 'smiths build a precision firearm they will mix and match parts, machine and massage parts closer to the minimum of the tolerance range to enhance mechanical fit and repeatability thus improving accuracy and grouping.
Tightening up the tolerances does come with a price. Usually when you tighten up tolerances you do so at the cost of reliability. Look at a well built match 1911 that is tight. The slide to frame fit is tight with no discernable play whatsover and it feels like buttered glass when you rack it. All it takes is a bit of debris to give you fits and that is not really what you want in a combat weapon. In a combat weapon you will always want to err on the reliable side.
Just my thoughts on your question.