5

Let's say that I have a 1 dimensional signal that has data which comes from one of 2 different spectral bands. The measurements come evenly mixed between the two bands. I'm interested in a method of extending the frequency calculation by combining information from both signals.

To show an example, let's just say that I have a Red band and a Green Band. The detector arrays would look like this: RGRGRGRGRG. Let's say that I had 512 R, and 512 G, just to give a concrete example. As the detectors are neighboring each other, and most signals have some component of red and green, it seems likely that a common frequency calculation could be made by combining the bands, and possibly double the frequency range which can be detected. How could I best go about doing this?

PearsonArtPhoto
  • 1,610
  • 1
  • 16
  • 20
  • Do you mean spatial frequency (e.g. from a 1D slice of a still photo), temporal frequency (e.g. from a single pixel of a video), or some combination of the two (e.g. from a 1D slice of a video)? – datageist Aug 24 '11 at 01:43
  • @Data: All 3 would be interesting, but for the purpose of this question, let's say spatial. – PearsonArtPhoto Aug 24 '11 at 04:16
  • 1
    Is the idea to, say, increase the resolution of the image by combining information from the red and green channels [by exploiting knowledge about the physical sensor layout/characteristics]? – datageist Aug 24 '11 at 04:21
  • 2
    Yes, exactly. I'm hoping to see higher frequencies than could otherwise be detected. It seems like it must be possible to get info, essentially combining the bands to get a sensor with twice the resolution, thus being able to detect twice the frequency, but I'm at a bit of a loss at the exact method of doing so... – PearsonArtPhoto Aug 24 '11 at 04:25

1 Answers1

8

From a practical perspective, there's good news and there's bad news. The good news is that, in a sense, consumer grade digital cameras [at least] already do something like what you're describing, so there's a large amount of information available about the process, which is called demosaicing. The bad news is that, in "standard" designs, the various RAW color channels actually have lower resolution than the physical sensor array, and the color channels in non-RAW images have already been reconstructed.

The reason for this is that the physical sensors only detect light intensity (not color), and therefore each available physical "pixel" is paired with a color filter, according to some pattern. For example, the Bayer Filter is a common pattern which allocates 50% of the available spatial resolution to the green channel, but only 25% each to the red and blue channels. The various spatial patterns are called Color Filter Arrays (CFAs). In effect they model the idea of separate RGB sensors, just in a more cost-effective manner. If there is some way of exploiting "spectral crosstalk" from the materials used in a given CFA, it would be handled in the context of demosaicing.

That means, if you're dealing with a camera which uses that type of sensing scheme, the most reasonable thing you can hope for is access to the camera RAW data (i.e. un-demosaiced), from which you can apply a higher-quality demosaicing algorithm to reconstruct the image. From the wiki:

When one has access to the raw image data from a digital camera, one can use computer software with a variety of different demosaicing algorithms instead of being limited to the one built into the camera...The differences in rendering the finest detail (and grain texture) that come from the choice of demosaicing algorithm are among the main differences between various raw developers; often photographers will prefer a particular program for aesthetic reasons related to this effect.

The following link seems to be a good overview of various reconstruction algorithms:

A Study of Spatial Color Interpolation Algorithms for Single-Detector Digital Cameras

In higher-end cameras, however, the RAW color channels might not even be spatially interleaved. Two of the different higher-end schemes (beam splitting, temporal interleaving) are reviewed here.

One type of sensor that I think works more like what you had in mind is the Foveon X3 Sensor. The following paragraph from the link gives a good overview of the resolution issues:

For example, the dimensions of the photosite array in the sensor in the Sigma SD10 camera are 2268 × 1512, and the camera produces a native file size of those dimensions (times three color layers). This amounts to approximately 3.4 million three-color pixels. However, it has been advertised as a 10.2 MP camera by taking account of the fact that each photosite contains stacked red, green, and blue color sensing photodiodes, or pixel sensors (2268 × 1512 × 3). By comparison, the dimensions of the photosite array in the 10.2 MP Bayer sensor in the Nikon D200 camera are 3872 × 2592, but there is only one photodiode, or one pixel sensor, at each site. The cameras have equal numbers of photodiodes, and produce similar RAW data file sizes, but the Bayer filter camera produces a larger native file size via demosaicing.

I think this is what you're looking for, assuming I understood the question.

datageist
  • 4,897
  • 4
  • 32
  • 53