Procedural generation has been brought into the spotlight recently (by Spore, MMOs, etc), and it seems like an interesting/powerful programming technique.
My questions are these:
You should probably start with a little theory and simple examples such as the midpoint displacement algorithm. You should also learn a little about Perlin Noise if you are interested in generating graphics. I used this to get me started with my final year project on procedural generation.
Fractals are closely related to procedural generation.
Terragen and SpeedTree will show you some amazing possibilities of procedural generation.
Procedural generation is a technique that can be used in any language (it is definitely not restricted to procedural languages such as C, as it can be used in OO languages such as Java, and Logic languages such as Prolog). A good understanding of recursion in any language will strengthen your grasp of Procedural Generation.
As for 'serious' or non-game code, procedural generation techniques have been used to:
Procedural Content Generation wiki:
http://pcg.wikidot.com/
if what you want isn't on there, then add it ;)
Procedural content generation is now all written for the GPU, so you'll need to know a shader language. That means GLSL or HLSL. These are languages tied to OpenGL and DirectX respectively.
While my personal preference is for Dx11 / HLSL due to speed, an easier learning curve and Frank D Luna, OpenGL is supported on more platforms.
You should also check out WebGL if you want to jump right into writing shaders without having to spend the (considerable) time it takes to setup an OpenGL / DirectX game engine.
Procedural content starts with noise.
So you'll need to learn about Perlin noise (and its successor Simplex noise).
Shadertoy is a superb reference for learning about shader programming. I would recommend you come to it once you've given shader coding a go yourself, as the code there is not for the mathematically squeamish, but that is how procedural content is done.
Shadertoy was created by a procedural genius, Inigo Quilez, a product of the demo scene who works at Pixar. He has some youtube videos (great example) of live coding sessions and I can also recommend these.
If you want an example of a world generator simulation plates tectonics, erosion, rain-shadow, etc. take a look at: https://github.com/ftomassetti/lands
On top of that there is also a civilizations evolution simulator:
https://github.com/ftomassetti/civs
A blog full on interesting resource is:
dungeonleague.com/
It is abandoned now but you should read all its posts
I'm not an expert on this, but I can try and contribute a few answers:
NetHack and it's brethern are open source and rely heavily on procedural generation of levels (maps). Link to the downloads of it. If you are more interested in landscape/texture/cloud generation, I'd recommend you search Gamasutra and GameDev which have quite a few articles on those subjects.
AFAIK I don't think there is much difference between languages. Most of the code you see will be in C/CPP because it's still very much the official language of Game Developers, but you can use anything you want...
Well it depends if you have a project that can benefit from such technology. I saw procedural generation used in simulators for the army (which can be considered a game, although they are not very playable :)).
And a small note - my definition if procedural generation is anything generating a lot of data from a small amount of rules or patterns and lots of randomness, your results may vary :)
There is an excellent book about the topic:
http://www.amazon.com/Texturing-Modeling-Third-Procedural-Approach/dp/1558608486
It is biased toward non-real-time visual effects and animation generation, but the theory and ideas are usable outside of these fields, I suppose.
It may also worth to mention that there is a professional software package that implements a complete procedural workflow called SideFX's Houdini. You can use it to invent and prototype procedural solutions to problems, that you can later translate to code.
While it's a rather expensive package, it has a free evaluation licence, which can be used as a very nice educational and/or engineering tool.
Procedural generation is used heavily in the demoscene to create complex graphics in a small executable. Will Wright even said that he was inspired by the demoscene while making Spore. That may be your best place to start.
http://en.wikipedia.org/wiki/Demoscene
the most important thing is to analyze how roads, cities, blocks and buildings are structured. find out what all eg buildings have in common. look at photos, maps, plans and reality. if you do that you will be one step ahead of people who consider city building as a merely computer-technological matter.
next you should develop solutions on how to create that geometry in tiny, distinct steps. you have to define rules that make up a believable city. if you are into 3d modelling you have to rethink a lot of what you have learned so the computer can follow your instructions in any situation.
in order to not loose track you should set up a lot of operators that are only responsible for little parts of the whole process. that makes debugging, expanding and improving your system much easier. in the next step you should link those operators and check the results by changing parameters.
i have seen too many "city generators" that mainly consist of random-shaped boxes with some window textures on them : (
(More than 10 years later ...)
Procedural generation only means that code is used to generate the data instead of it being hand made. For example if you want to generate a forest with various trees you are not going to design each tree by hand, thus coding is more efficient to generate the variations. It could be the generation of the tree graphics, size, structure, placement ...
In general there is some kind of interation with a few rules, in addition to that you can add some randomness and logic of your own, combine all of these techniques ... Anything somewhat chaotic but not too chaotic can yield interesting results.
Here are a few notable techniques:
A few games famous for their procedural generation:
Have fun.
