2

This is a very top level question with no code examples attached. I have an interest in using Mathematica to read raw files coming from a Canon camera, before they are converted into jpg's, where data is lost. Are there any existing non proprietary mathematica scripts, which can read a CR2 file from a Canon camera and display its image as it would if a jpeg were imported. At the very least a script that can display data about the image stored in the CR2 file to include intensity, size, pixel value, LRGB histograms, etc...

Alexey Popkov
  • 61,809
  • 7
  • 149
  • 368
Jon
  • 145
  • 7

2 Answers2

4

Yes! I have done this before with RAW files.

Use the dcraw command line utility and convert the RAW file to a TIFF. You'll need to use appropriate command line options to prevent any processing of the data.

dcraw -o 0 -D -T -6 infile.cr2

It'll output a TIFF file with un-demosaiced raw sensor data. You can read that with Mathematica.

EXIF data can be extracted using exiftool.

What I don't have a solution for yet is how to put any (Mathematica-processed) data back into RAW files (say, DNG) so it can be read into a RAW converter.

Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263
  • Unfortunately I already know how to do that part, but I do appreaceate the answer. I probably need to dig deeper into the RAW file format and go from there in trying to get mathematca ro read in the information. It will probably not be trivial, but there has got to be a way to do this in Mathematica. – Jon Feb 11 '14 at 22:31
  • 2
    @Jon I don't really understand what you are trying to do then. There's a simple way to get all the data into Mathematica, by using an external utility. Why are you looking for a complicated way to do the same without this utility? You'd just end up rewriting parts of dcraw. If you need decent performance, you'd probably end up rewriting it in C and using it through LibraryLink. There's no built-in way to read any of the myriads of RAW file types in Mathematica. – Szabolcs Feb 11 '14 at 22:53
  • 2
    Also, while I haven't worked with CR2, the situation is probably similar to Nikon NEFs: Nikon doesn't document their RAW file format. Though it's TIFF-based, there's enough difference that you can't get usable data with a TIFF reader in a simple way. If you're a company like Adobe who wants to include NEF support in their software, you need to use the NEF SDK from Nikon, which you will gain access to only after approval. Otherwise you'll have to reverse engineer the format on your own like the dcraw people did. – Szabolcs Feb 11 '14 at 22:57
  • 2
    Presumably to interpret the raw data properly you also need things like a bad pixel map, nonuniformity correction data (per-pixel gain and offset), color mapping curves, and perhaps various other things as well. How you extract that information from the camera firmware I have no idea, so while I suppose it's correct to say that information is lost in the conversion to JPEG, it sounds almost impossible to recover it from the raw sensor data without the correct manufacturer-supported software. – Oleksandr R. Feb 11 '14 at 23:06
  • 2
    @OleksandrR. The bad pixel map and non-uniformity is corrected for already, but to produce a properly converted image afterwards you're right that you need the curves. Actually there are good reasons for reading the raw data. I used it for two things: looking at cosmic ray tracks (I needed non-demosaiced data) and for banding noise reduction. My camera has a pretty annoying banding at high ISO settings ... and I did manage to improve on it by tailoring the method for this particular camera. But I couldn't put back the de-banded data into a RAW file so I can use a proper RAW converter ... – Szabolcs Feb 11 '14 at 23:33
  • @OleksandrR. ... and obtain a usable image! Because as you said that's both non-trivial and needs camera-specific data that's guarded by the manufacturer. – Szabolcs Feb 11 '14 at 23:34
  • Okay, I didn't know that the BPR and NUC are done already for the "raw data"--that makes things a lot easier. Worth a try for my camera, perhaps, which exhibits a very strange and unpleasant saturation effect (yellow, blocky region in the image) under bright lighting conditions. Did you see any cosmic rays travelling in the plane of the sensor, by the way? – Oleksandr R. Feb 12 '14 at 00:15
  • @OleksandrR. Yes, only those are interesting, and it takes a bit of luck to catch them. Just orient the sensor vertically and take a few long exposure pictures in bulb mode. Sometimes there'll be a tiny line on the image, still less than a millimeter long probably. – Szabolcs Feb 12 '14 at 00:38
  • Even though I don't know HOW to do it myself, I thought it was possible to use already developed C libraries in Mathematica using the C/C++ Language Interface. Am I wrong? – rhermans Feb 12 '14 at 15:35
  • @rhermans libraw is pretty much the same as dcraw. Yes, it's possible to call it through MathLink or LibraryLink too, it's just going to be a lot more work than calling it through the command line (i.e. dcraw) and in the end it will achieve pretty much the same thing. It's really not clear to me if the OP is asking to achieve something, or to achieve something in a particular way. – Szabolcs Feb 12 '14 at 15:40
  • Yes, you are correct, I am trying to reinvent the wheel on somethig that will require an enormous amount of code development in C or at the very least finding bits of already written code and compiling them. However, it is a path I still will go down. The end goal is to lengthy to describe here, but, it dawned on me that I am going to require a TIFF as the intermediate step anyways so I will end up using a shell script that executes dcraw and mathematica as you suggested – Jon Feb 12 '14 at 17:01
  • @Jon If you want to avoid TIFF files and want better integration, then the way to go is what rhermans said. Use libraw through LibraryLink. That should be straightforward, but quite a bit of work. If you do decide to go down that path, you can consider integrating it into the import/export framework for convenience. This will give you a solution of the same quality and convenience as built-in importers. – Szabolcs Feb 12 '14 at 17:58
  • While I have many years of experience in numerical method programming for engineering application, I am new in this type of programming for graphical applications. I would be in your debt if you could explain the difference between libraw and dcraw, as I suspect that one is based on the other. – Jon Feb 12 '14 at 23:59
  • @Jon dcraw is a command line tool. You can invoke it from a terminal or using Mathematica's Run command. libraw is a C library, which I think is based on the dcraw sources. You have to write a C program to use it. It doesn't involve any GUI/graphical programming (which I also don't do much), but it probably does involve reading a lot of libraw documentation. In this case you'd want to extend Mathematica to be able to use libraw directly from it, so you would also need to learn Mathematica's C interface. Your best choice would probably be using the LibraryLink ... – Szabolcs Feb 13 '14 at 00:49
  • ... @Jon LibraryLink API, but Mathematica also has another (older but more flexible) C interface called MathLink. So you'd need to learn how to use one of these two as well. None of this is difficult, but it is going to be a lot of work and it would probably take you (not more than) a few days if you're not used to this kind of programming. What you'd gain compared to calling dcraw through Run and using an intermediate TIFF format file: mainly you'd be able avoid the temporary TIFF file when invoking any function. This is why I said that it's probably not worth the effort. – Szabolcs Feb 13 '14 at 00:56
0

Well this is a duplicate post here

After some dig in version 13, I found a package that can do this indeed. And I have tried Canon and Sony, so far so good

<< ImageFileTools`

Here is a function ImageFileTools`Raw`RawGet, and its argument is this: enter image description here

Usage

enter image description here

yode
  • 26,686
  • 4
  • 62
  • 167