Is the SO tag ordering a good representation of programming language popularity?
My first guess would be that it's not and that it's largely populated by fans of Jeff (who would probably be .Net biased) and fans of Joel (who may be more C/C++ biased among others).
Is there a way to get unbiased programming language popularity stats?
I'm the creator of LangPop.com, which measures a number of metrics to try and gauge popularity. My philosophy is to simply try and measure a lot of different things and let people see the results, and judge for themselves. All of the metrics have flaws, biases and advantages, too.
In terms of SO tags, besides C#, they look fairly accurate, but there's a huge C# bias, so for the time being I don't think I will include SO in the LangPop results. I am keeping an eye on it, though. Another example of something I'm keeping my eye on, but that is still way too biased is GitHub, which has a huge number of Ruby projects.
Stackoverflow is an extremely biased sample, and the tags counts may further distort the figures. Because Joel Spolsky is a primary marketing tool for SO, and he favors Microsoft technology, it's greatly over-represented here. The bias is then amplified as people interested in other platforms sense that this site isn't for them and stop participating.
The Tiobe survey is widely cited as a means of gauging language popularity. Like any survey, it has flaws, but at least it was intended for the purpose of measuring popularity.
There are two good sites that track these sorts of things...
LangPop
TIOBE
Although I wish they showed C# right up there with Java, I'm not that surprised that most places are still using Java, or people are searching for Java, or there are lots of Java jobs. I don't have any particular reason to think they are being unfairly biased.
It may not be a good representation, apart form the reason you have posted, it would also be that certain techonolgies are new and there is very little help available elsewhere and would end up more on SO. Certain techonologies are quite stable and hence hence the questions would be less, you would always have expert nearby, or some dedicated communities/forum sites like the ones for Oracle or Microsoft products.
I would expect that SO tag ordering is a very, very rough guide to language popularity. Figure +/- a factor or 2 or 3, the tags with large numbers on them may reflect general reality. The tags with smaller numbers, however, may be more fully represented in other arenas.
Once a forum starts show a preference, people will be attracted to that forum more or less depending on how that preference matches their. Thus, there's a selection effect in addition to the ones you mentioned. There are many places to find help, but some languages have better resources preexisting on the network.
It is truly difficult to get unbiased programming language popularity stats. I suppose you could look at resumes to see what people advertise they know, but this is influenced by what they think will get someone to hire them. You could look at jobs advertised, but this also has biases. If people who know one language change jobs twice as often as people who know another language, then you'll see more jobs offered for the former, all other things being equal.
I think it will vary pretty heavily depending on how you define "popularity", but there probably will be no such thing as an "unbiased" representation.
Language popularity will vary greatly depending on which field/population you observe : Php is likely to be vastly used (if not neccesarily popular) by web developpers, Game developpers will probably tell you that C++ and Flash/AS are the be-all and end-all of programming, Windows developpers are probably widely adopting C# at the moment, ...
It might be possible to observe tendencies by looking ie at Job offers, Blogs, Forums, Stack Overflow, but you won't get much more than a tendency, and nothing really general enough to be of any use I think.
ohloh.net has some pretty good metrics. It reads from version control repositories for a bunch of projects and provides graphs.
Compare languages at Ohloh
