Which files are more insecure than others?

Question

Regarding the fact (in case it's not always the proven fact, then an opinion) that there are types of files more safe for handling them after getting from the Internet than the other, I'd like to ask:

1. Which files are more insecure than others in contexts of handling them on a server after uploading or on a client computer after downloading?

2. What specifically makes those particular types of files insecure or more insecure than the others?

I don't mention a type of OS in order to not narrow down my question without strong reason. If it is the important factor for an answer then, please, specify it.

Answer 1

Typically the "insecurity" of files comes into play when they are accessed, not just by having them downloaded. Downloading a file unopened and having it pwn your computer is the stuff of Hollywood and science fiction (as far as I know, though there may be research into this too).

Thus, to understand the problem, the real question is not about the file types, but rather about the typical programs commonly used to access those files.

Thus, a .txt file should be relatively harmless, since it's accessed in raw form by simple text readers and not recognized as code in any way. (Though of course a 0day vuln found in e.g. Windows Notepad could change that....)

PDFs and Office docs are typically opened by Adobe Reader and MS Office programs, respectively - and these all recognize numerous types of dynamic content, e.g. Office Macros. Furthermore, there is a whole lot of data handling, parsing, translating, activating, and rendering going on, and there can be (and are, and were) many different vulns at each step of the way, in each of those programs (and of course others).

Same thing goes for media files - JPEGs and GIFs can have hidden code (see GIFAR), movie files can have embedded codecs, etc.

Of course, executable files and code (compiled or not) are the most dangerous, since they are trivially exploitable - the code is already there, and the user explicitly asks to excute it. This includes, e.g. on Windows exe, dll, com, bat, htm/html, js, vbs, jar, etc.
Of course source code (.java, .cs, .vb, .c, .cpp, etc) can also be an issue, since these are typically passed directly to a compiler - arguably one of the most complex pieces of common software - and gives execution instructions, not to mention the later output... It is not too complicated to obfuscate malicious code.

Choice of OS is not necessarily the most important factor here, though of course different files will execute differently on each OS - and each OS has its own share of unsafe filetypes.
More importantly, is if/when a malicious file is executed, what kind of damage can it do? An OS that defaults to minimal user privileges (compare Win7 to WinXP), will of course be that much safer.