16

I have microscope videos that have a timestamp overlay to them, and it would be great if I could extract this for each frame to add to my metadata.

TextRecognize fails to accurately grab the numbers from the image that looks this (scaled by 4):

enter image description here

I have tried cropping in different ways, scaling, and several image processing methods (binarize, morphological operations, etc). I also tried prepending text to avoid the issue where TextRecognize does not detect the initial digits. I believe this to be a image quality problem.

I was wondering if anyone has a recommendation on how to clean this up so TextRecognize can detect at least the time in images like this.

rm -rf
  • 88,781
  • 21
  • 293
  • 472
NMSchneider
  • 548
  • 2
  • 9
  • Since you have a video, have you tried averaging nearby frames (assuming a high enough fps) to see if the clarity of the timestamp improves? – rm -rf Jul 02 '13 at 14:20
  • 4
    btw, doing this solely with TextRecognize might not be feasible, as it is a known limitation that it cannot identify disjoint letters/digits properly. A simple and robust solution might be to have templates of numbers 0-9, use ImageCorrelate with all 10 of them and identify which template gives you a max and its position. Since the time stamp has a structure, you can then map the digit sequence back to a time stamp. – rm -rf Jul 02 '13 at 14:28
  • Thank you @rm-rf. As for the averaging, because of the way the timestamp is overlaid in our microscope system, the average is of similar quality as a single image. I will try ImageCorrelate now. – NMSchneider Jul 02 '13 at 14:33
  • Looks like ImageCorrelate will work, thanks again! – NMSchneider Jul 02 '13 at 15:22
  • That's great! If you have a prototype of a working solution, please post it as an answer so that future visitors who have a similar problem will have a few leads to go on. This seems like a useful thing to document, as TextRecognize is pretty limited in what it can do. – rm -rf Jul 02 '13 at 15:56
  • Will do once I can. When I went to post, it told me I had to wait 8 hours since I have less than 10 reputation points. – NMSchneider Jul 02 '13 at 17:21
  • Oh yeah, the new user restrictions are a bummer (but are useful in spam control). No hurry :) Thanks! – rm -rf Jul 02 '13 at 17:30

1 Answers1

20

Since the timestamp will always look about the same. This allows us to crop a sample of each individual number to use as the kernel in ImageCorrelate (as suggested by @rm-rf).

I use the following method for the image above: 

 (* Define Where to Crop for kernelImages *)
imageCropRows = {11, 81};
imageCropColumns = {{726, 802}, {15, 46}, {53, 120}, {433, 499}, {280,
     345}, {1033, 1105}};

(* Grab Our kernelImages *)
kernelImages = 
 ImageTake[image, imageCropRows, #] & /@ imageCropColumns

Ultimately, this will give a kernel set that looks like:enter image description here

(* Apply ImageCorrelate, and make maxima white points *)
imageCorrelations = (ImageCorrelate[image, #, CosineDistance] // 
      ColorNegate // Binarize[#, 0.989] &) & /@ kernelImages

Returns a list of images that can be further processed to extract the location in order to reassemble the digit sequence, and ultimately the timestamp. Playing with threshold value in Binarize will help with sensitivity.

Combining MorphologicalComponents and Position should work to find where each copy of a kernel is found. For example, say we wanted to extract the location of the 3's (kernel index 4):

(* Use MorpologicalComponents to identify repeated number locations *)
results = MorphologicalComponents[#] & /@ imageCorrelations;

(* Find where each copy is located *)
Mean[#[[All, 2]]] & /@ (Position[results[[4]], #] & /@ 
   Range[1, Max[results[[4]]]])

Once you have the pixel location of all the digits, some logic can sort them, and reconstruct the timestamp.

This can applied to an arbitrary frame with reasonable results.

NMSchneider
  • 548
  • 2
  • 9