Let's look for a way for the OP to have the requested "Aha!" moment.
To me it looks like the diffusivity is independent of the particle's density. A 1 micron sphere of a dense metal like gold and a 1 micron thin spherical shell would have the same diffusivity as long as other forces were not present if for example the experiment were done in a microgravity environment.
Yes, under conditions where this is valid (smoke in air, bacteria in water) this is so.
This bothers me because I want to believe that since the same number of fluid atoms are hitting both particles transferring the same random amounts of momentum to them, why wouldn't the heavier particle diffuse more slowly?
So there's no conflict here. If the particle is at rest and a molecular collision knocks it to the side, the recoil velocity will be inversely proportional to the 1 micron particle's density.
The OP's "Aha!" will come when they realize they've tried to reconcile two different worlds. A microscopic model of individual collisions with atoms, and the collective effects of a fluid.
The diffusivity D can only be applied for time scales much longer than the time between individual collisions. The momentum imparted by each collision needs time to damp due to several more collisions.
If the lighter particle moves faster after one collision than the heavier one would, the subsequent collisions on average will slow it down a bit faster since their relative speeds are faster. On a macroscopic scale, that's where one can start thinking about the origin of the drag force.
What is that collision time? For a 1 micron particle in water or air its of order $\text{10}^{-19}$ and $\text{10}^{-16}$ seconds. So even if you are watching your particles using single pulses from nanosecond lasers, you're averaging over millions or billions of collisions.
It's time for the OP to throw away their microscopic thinking and start to embrace statistical mechanics and fluid dynamics.
A very good way to ease one's way into this picture slowly would be to read Life at Low Reynolds Number. E. M. Purcell
Am J. Phys. J45, 3 (1977); doi: 10.1119/1.10903