Stripping Thierry Laronde's blazing fast TeX distribution, KerTeX, you can get a plain TeX distribution in under 2MB.
0) The tex executable is 1,229,008 bytes (after running strip on it). (I've modified the C sources a bit so your size may vary slightly.)
1) The plain.fmt file is 541,382 bytes.
Total: 1,770,390 bytes.
2) The rest is just fonts.
I'm using a custom DVI driver (forked from dvipng), and this is all I need to compile .tex source code and render .dvi code.
Type 1 fonts can add a few MBs (since there's so many of them...), but you can keep the total under 2MB. Each Type 1 font is 2 files: a .pfb file and a .tfm file. Each Type 1 font is about 35,000 bytes in total. For example, I'm using these Type 1 fonts: cmssbx10, cmssbx10, cmss10, cmssbx10, cmssi10, cmsl10, cmtt10, cmcsc10, cmmi10, cmsy10, cmbx10, eusm10, eusb10, eufm10, eufb10, cmr10, cmmi7, cmr7.
The KerTeX tex executable (which compiles .tex source code into .dvi), and my DVI driver (which renders .dvi code to the screen and to images) needs the plain.fmt file to work, and the .pfb and .tfm files for fonts. (Though I think tex only needs the TFM files?)
That's it.
(Actually KerTeX's tex executable also needs a KXPATH file in the fonts folder, for some reason, but it's only 6 bytes...)
(By the way, the KerTeX tex executable compiles small TeX files in about 2 milliseconds, so it's more than enough to compile and render .tex source in real time, since the DVI driver can be made pretty fast, too.)
I would say 5 MB is impossible today. 20 MB is very optimistic.
– Ulrike Fischer Mar 16 '11 at 15:59