I'm assuming your real Twitter follower count is the follower count Twitter gives you, minus the number of spammer bots who follow you for no good reason, minus the number of random people who act like spammer bots for no discernible reason. I think you can form a very good estimate of that number by analyzing the social graph.
First, examine your Twitter feeds. Find the Twitter followers you converse with most frequently. They are considered known valid followers. Find known valid followers of these known valid followers. If these people are also following you, add them to your list of known valid followers, giving them a slightly smaller degree of validity. Continue until you exceed a threshold of iterations or fall past a threshold of validity.
The algorithm has three constants - the iterations threshold, a threshold degree of validity, and a threshold which defines frequent conversation. A value of 1 for that would consider anyone 100% valid if you @acknowledged them even once; a value of 100 or more sets a much more snobbish standard. You tweak the constants for a bit and you're done.