Create text with GPT from email and attachments

[ns2023]

Hi everyone,

I built a flow which extracts information from emails, passes it through GPT and ultimately adds a row to an excel table with the defined variables. This part of the flow works exactly as intended.

I am now trying to add another action which will bring in the attachments of the email to the same scenario. I am having trouble getting the output of the "recognize text in an image or PDF document" to flow through the GPT action and extract what I need. 

Essentially I have a list of criteria that I want the GPT action to pull from the email body/subject and or the attachments. It is currently only working on the body/subject. 

What am I missing? Any insight would be greatly appreciated.

Thank you,

Nick

[Tjan]

@ns2023  Hi! I see that the action is skipped. This can have several reasons. Can you check the following

• Are there any attachments in the mail you are testing with? If there are none, this action would most probably be skipped because the for each is empty. 

Also make sure that the trigger includes the attachments when a mail arrives. 

If you don't mind, can you show the parameters of your trigger? I'll take a look into that to help you!

[ns2023]

Thank you, @Tjan !

Yes, the email I am testing on has an attachment. The recognize text action is even recognizing the text and converting it to JSON. But the GPT action doesn't seem to be picking it up.

Here are the email parameters:

This is a snippet of the output from the recognize text action. I am thinking I need another action which will only return the "text" field before passing it through the GPT action. There is way too many rows of information that is unnecessary to me, like the coordinates of the text. 

It also could very well have to do with the GPT action. I am not sure how to include the info from the recognize text action into the input text - it was throwing errors when I tried adding the apply to each. 

[Tjan]

@ns2023 Ah, I understand your situation more. I strongly recommend this blog for your solution: Re: Extract Data From PDFs and Images With GPT - Page 4 - Power Platform Community (microsoft.com).

I've tried it myself and it's a gem! @takolota  did a great job creating a solution that is able to output text that you can work with - by filtering out the unnecessary information. You can take his flow as a starting point and build upon it based on your wishes. 

[ns2023]

Thank you - definitely helpful and works great in it's current state. 

I am struggling to adjust the actions to fit for my use case. My goal is to use this flow to pull the attachments from incoming emails rather than a manual trigger / one drive link. 

[Tjan · Accepted solution]

@ns2023  What worked for me is to delete the actions that are not relevant. Afterwards you can add your trigger (when a new e-mail arrives). The attachment content that is outputted can then be used for the the OCR action (read text from document). The rest stays obviously the same — don't delete any of the actions below since I noticed they are linked with each other.

[ns2023]

Thank you, @Tjan ! I have it working now. The "get attachment" is what I was doing wrong. Now all text from body, subject and attachments flow through the GPT action. 

For anyone that may stumble across this and looking to do the same as me, here is what my flow looks like:

[Anand3]

Hi @ns2023 , please can you share the link to your flow or share as a ZIP package, that would be really helpfull.

[Anand3]

Hi @ns2023 Can you please specify what parameters or dynamic content you provided for the steps Get attachment and the recognise the text in the image or PDF. 

[ns2023]

Hi @Anand3 - see below. The Attachment Id on the Get Attachment action is expression: 

first(triggerOutputs()?['body/attachments'])?['id']