It's the real key to the housing recovery.

If we know exactly how much shadow inventory of foreclosed properties will come to market, and we know the general demand, then we can get an idea of how much pain there is ahead in the still-fragile housing recovery.

The trouble is that the government doesn't report the number of homes held on bank's books or in private label securities, despite the fact that it counts just about everything else this country buys and sells.

The one, perhaps, positive outgrowth of government conservatorship of Fannie and Freddie is that now we can look far more deeply into their books with relative ease.

So we know that in Q2 Fannie Mae's REO inventory (real estate owned, i.e. repossessed) stood at 129,310, Freddie Mac's at 62,178 and FHA's at 44,850. Collectively 236,338 homes, which also happens to be an increase of 13 percent from Q1 and 74 percent from Q2 of 2009.

Now to the big banks.

They're harder to track because they report REOs in their quarterly earnings reports, but only as values of the properties, not actual numbers of properties. Thomas Lawler, an economist and housing consultant, estimates that by Q1 of this year banks owned $14.5 billion worth of REOs. If you assume a median price of $150,000 for each home, which I think is a bit high but he's assuming, then you get 97,000 properties.

Now to private label holdings.

Moody's estimates that private label repossessions stand at 203,665. So add it all up and you get close to 540,000 and then you add in other smaller banks and lenders that don't report anywhere we can find.

"My best guess right now is that REO held by Fannie, Freddie, and FHA, and other government entities, and banks and thrifts is just under 600-thousand, but unfortunately it is on the rise," says Lawler.