Image OCR scan using Power Automate and Vertex Ai vision (Google cloud Api)

[jkingno1]

Hi, 

I'm looking for suggestions on how to extract specific information from images using OCR or any other suggestions on how to with my following problem: 

Basically my goal is to have a folder of images scanned with OCR and then each individually renamed their Job Numbers (E.g. 26483) and if there is multiple of the same job to be named with a (1), (2), etc. on the end, e.g. 26483 (1), 26483 (2). The hard part I'm finding with OCR is it detects lots of text within the image and I'm not sure how to select that specific 'Job number' which is limited to 5 numbers only. Any help or suggestions would be much appreciated. 

[OkanAutomation]

Are the images in the same format?

[momlo · Accepted solution]

Hi @jkingno1 

Parse the text you get from OCR with regex, if you say the job no is always 5 digits, you can do something like this

(?<=text that appears before numbers>\d{5}(?=text that appears after the numbers>

example:

btw - i guess you tried, if not - give a shot for built in OCR in PAD, might save you some API calls to Google, and is free, if it works fine for your images (it does for few of my cases).

[jkingno1]

No, there generally always different and contain the job number within the image normally, either written or seen on a job card in the photo next to the Job itself. 

[jkingno1]

Hi @momlo ,

I have tried the built in OCR and it only works in extremely perfect situations... 

Ive had a look at regex but dont completely understand what it does and where to put the information i get.

Currently what i have done is setup the following flow and ive also got the image example ive tried to extract the text from. The output  is a long list as seen in the last screenshot in the txt file and obviously out of all that data all i want is the '33963' to rename the image that and then repeat for every other image. The hardest thing about this is that each 'image' is different but they all have a Job card or Job number somewhere within the image and is normally 99% of the time the only 5 Character number within the image. 

 

  

[VJR]

Hi @jkingno1 

I used your image and it extracted the given text using the out-of-the-box OCR action.

Can you check it with other images.

[jkingno1]

That was a good example, sometimes we get handwritten images and also plates with lots of different numbers on it. Something I just realized is the built in OCR likes Screenshots if the image file size is too large to recognized the text from, so to an extent you have proven to try use the built in one.

Lets assume I screenshot and replace every image file with the new screenshot file and power automate recognizes the text in the image. 

Do you or does anyone know how to differient changing text formats as I cant say "First 5 characters" as the image changes between different images which picks up multiple bits of text. What I want to do with the cloud api is have the JOB number in a specific colour and then this would consistently pick up only the Job Number.  Does Power automate how something that defines colour of text as well? Attached a screenshot of another example image. 

 

[VJR]

I don't think there's anything in-built in PAD to handle colors as you have specified...I would have suggested the AI Builder but since you are already using the Google Cloud API you could have that done by training the model or giving some color specifications in the model itself and simply call that from PAD.

In any case, did you post an incorrect image...someone can look into it if you have the right one.

[jkingno1]

The image I was testing is this one which gave the above (Image) variable value. 

[jkingno1]

Ive done some research and understand what you mean now and i have half got it working. Just wanted to say thanks!