AI builder Prebuilt Vs Form processing

[DML1]

Hi Experts,

Pls guide me on the below,

My flow is to extract invoice date, invoice amount and tax% from Prebuilt invoice processing model. If any of the values not extracted by Prebuilt model then flow should use the form processing model which are trained and save it in SP doc library. Now the challenges are below,

1. Not able to find dynamic parameter for Tax% in Prebuilt model, found only tax amount 

2. Not sure on the condition for null value, if the extracted value is null from Prebuilt

3. Not only tax% if any of the value is null from Prebuilt model then use forms processing model

Regards,

DML

[JoeF-MSFT]

Hello @DML1 - thanks for the question.

You can use a Condition that checks whether a value returned by the invoice prebuilt model is null. Here is an example:

You will need to write null as an expression like this:

Here you will find more guidance on how you can combine the prebuilt model with a custom form processing model: https://docs.microsoft.com/en-us/ai-builder/prebuilt-invoice-processing#create-a-custom-invoice-proc...

I hope this helps!

[DML1 · Accepted solution]

thank you so much JoeF-MSFT, will definitely try this out and share my input if it works for me. 

I believe it will work for all the invoice fields except Tax%. Please help me what's the parameter to select for Tax% in Prebuilt model which is my first question in my post.

Once again thanks for the quick response JoeF-MSFT

[JoeF-MSFT]

You're welcome @DML1! The Tax% is currently not supported by the prebuilt. Here is the list of information the invoice prebuilt can extract today: https://docs.microsoft.com/en-us/ai-builder/prebuilt-invoice-processing#model-output

Could you perhaps calculate the percentage by extracting the subtotal amount and the tax amount and calculating what is the %?

[DML1]

Thanks for the confirmation on tax% Joe. Will definitely try with tax amount and subtotal amount.

I have a issue with your first solution, the condition is not working. When I try to extract details seperately through forms and Prebuilt it's working accurately but null condition not working.

Example - invoice date, invoice amount are accurate when I compare Prebuilt output and forms output. So to test whether null condition works perfectly I removed the dynamic value from form processing for both of the fields considering it will save the Prebuilt output in SharePoint but it shows blank in SP.

Pls guide

[DML1]

 

 

[DML1]

Null condition for multiple columns not working. Flow is overwriting the Prebuilt model output with forms output. This is what I finally found with different type of test done Joe. Pls guide

[JoeF-MSFT]

Thanks for sharing @DML1! To avoid overwriting values, you could do something similar like this:

1. Initialize a variable.
2. Check if the value from the invoice prebuilt AI model is empty or not.
3. If it's not empty, set the variable value to the one returned by the prebuilt.
4. If it's empty, then set the variable value to the one returned by the custom form processing model.
5. Use the value from the variable to save it in the destination

[DML1]

I have 7 fields, the above solution may be easy for one value, but if it's more than one then for every field need initialise and set variable. And I doubt this control condition works with multiple null condition, as it's not working.

[JoeF-MSFT]

Thanks for the context @DML1 - another way to approach this is by using the if function like this:

if(empty(outputs('Extract_information_from_invoices')?['body/responsev2/predictionOutput/result/fields/invoiceId/valueText']), 
outputs('Extract_information_from_forms')?['body/responsev2/predictionOutput/labels/Invoice_1c57ad2f06ae1b5ef457aa33e060c6ed/value'],
outputs('Extract_information_from_invoices')?['body/responsev2/predictionOutput/result/fields/invoiceId/valueText'])

Here I check if the 'invoice id' returned by the prebuilt model is empty. If it is, I write the 'invoice id' returned by the custom form processing model I trained, if not I use the one from the prebuilt. 

Please note that the field name for your custom form processing models will be different, so make sure you select  the right value output for the above expression. 

[DML1]

Same way I did Joe and it worked. Now I have new challenge, not able find where I am going wrong on nested if conditions. If seperately I use this expression it's working but when I combine it's not working. Pls check these below formula and tell me where I am going wrong in the last compose action.

Unable to process template language expression in action 'Tax_Amount_-_Compose' inputs at line '0' and column '0': 'The template language function 'replace' expects it's first parameter 'string' to be a string. The provided value is of type 'Null' 

Small correction in the last compose action, compose action name is Tax Amount - Compose

Regards,

DML

[DML1 · Accepted solution]

Jeo....hurray while drafting this issue for you to look into I found where I did wrong in the last action.

In the last compose action(Tax Amount - Compose) instead of considering the output Prebuilt model need to consider the output of Tax Amount compose action and it worked perfectly fine....yehhhhhh 🙂 very simple thing and I struggled for whole night to investigate this.