Regarding MAC support, "The Mac and Windows operating systems have different methods of enabling this type of thumbnail, and in the case of the Mac OS, this support has been inconsistent from version to version so it hasn't been pursued [for Adobe InDesign]."
Windows
An Icon Handler is what you are looking for. It is also known as a Thumbnail Handler. Check out this sample example written as an active x control. Also, look up Property Handlers, which points to the latest and the best way of having dynamic metadata handled properly in windows.
Now, you won't be needing such dynamic solutions if you just want an icon to be associated with all your files. So, use them only when you'd want windows explorer to display an icon based on the file's content and not just the extension. That means when the file changes, the icon is also updated to reflect those changes. But, it doesn't necessarily have to be an image of that file itself as the thumbnail handler can generate any image based on what's in the file.
Since the property handler updates other metadata such as audio or video length, you can use all the metadata that Windows Explorer supports.
OS X
The Launch Services Database determines the icons for Mac OSX. But, it refers to a static icon file for all files handled by a registered application. So, it's not based on the extension. Rather, each file has attached metadata to determine its respective application. Although, extensions do help when no metadata exists, like when getting the file from a different OS or file system. The dynamic icon functionality in OSX seems to be provided by Finder.
Gnome
You can use a thumbnailer for Gnome. It's a very simple program, which has 3 command line arguments:
- input file name, the file you are describing with the thumbnail (or URI if you're accepting those instead)
- output file name, where you need to write the PNG
- size in pixels, a number that describes the maximum square image size you should produce (128 --> 128x128 or smaller)
In contrast to all the difficult to implement plugins on other systems, this system is simpler but doesn't support animation and a few other features.
KDE
Konqueror is the file manager here and displays the icons. So, it supports dynamic icons for some inbuilt types, but I can't say if they are hardcoded or are just plugins that you can write. Read: Embedded Components Tutorial for more info.
Also, there's this new or upcoming feature called Plasma that deals with icons and icon functionality. Here's the announcment.
You might have to dig into the source of Konqueror to underdstand how they did it for text files and other files that are already implemented.