I had to look this up, but.. from the manual, it seems you have to check the 'As Instance' field of the Object Info node:
As Instance:
Output the entire object as single instance instead of
realized geometry. This allows instancing non-geometry object types,
because the output will contain an instance of the object.
It also works, if you put the light in a collection, and use the Collection Info node.

You can't Realize Instances, it seems, because the lamps are not geometry, and they disappear if you apply the GN modifier.