The question might he better posed as "Why does fluorine have a greater affinity for just about everything (including boron) than oxygen?
The answer is first off, technically fluorine does not have a greater affinity for everything than oxygen. This answer explores counterexamples involving (early) transition metal complexes. However, in most settings the exceptionally low valence orbitals of fluorine and the availability of a vacancy in the fluorine atom to accept another electron facilitates the formation of bonding molecular orbitals with low energy. Meaning stronger, more stable bonds.