Handling Company Invoices & Bills With Microsoft Flow

[TomaszPoszytek]

I am running my own company for years. Every month I need to gather all cost bills and invoices (really 95% of them are sent via mail), put them in a single e-mail and send to my accountant. Previously every such e-mail I received, I had to flag, then copy its attachment to a draft mail that was sent by the end of month to my accountant. Manually it was a significant amount of work

Therefore I decided to built a Flow and to engage LUIS for help. The process is quite simple:

1. When a new e-mail with attachment arrives
2. Calculate intent 
3. If above 0.9 automaitcally process it
4. If above 0.7 assign me a task, that if approved pushes e-mail for processing
5. Flag mail
6. Forward it to accountancy
7. Copy attachments to OneDrive
8. Enter details to Excel file

LUIS (Language Understanding Intelligent Service)

I decided to utilize the power of Cognitive Services to automatically parse and verify if the received e-mail contains invoice or not. For this purpose I have created new app under: https://www.luis.ai/applications and then two Intents: Invoices and None:

Next, I created an Entity (type Simple) containing keywords (Role names) identifying invoice-related contents, such as: "invoice", "bill", "itinnerary", etc..., for languages in which I usually receive documents (English, Polish, German):

Then I used already received e-mails to train my model. And also, while adding, to bind keywords with Role names:

To do that simply hover keyword (1), then open the created Entity (2), and then select related Role name (3).

Finally I hit "Train", so that my model became ready to test:

Next I verified for some random mail bodies, if the model works fine and finally I've published it.

The Flow

Below is the initiation part of the Microsoft Flow I am using

1. When a new e-mail with attachments arrives
2. Convert its contents from HTML to pure text
3. Next, it's checking if body length is longer than 250 characters.
4. If yes, it's triming content for calculating prediction to only 250 characters - I noticed, that when trying to post all characters from e-mail, action for calculating prediction was failing. If not - no changes are made.
5. Next LUIS is taking care of calculating prediction,
6. And finally is converting the result to float value.

Next it's checking if the prediction value fits within tresholds:

1. If value is above 0.9 and the Top Scoring Intent Name is "Invoices" then...
2. it's pushing mail for automatic processing.
3. Otherwise it's checking if the value is above 0.7
4. If it is, it's assining me a task, to verify if the mail contains invoicing documents. Here I am using P1 action with custom outcomes: Approve, Reject, Not applicable. The last option here is for a scenario, where I receive e.g. an e-mail with a comment to already received invoice - therefore it is just bypassing the whole mechanisms and terminates the process.
5. Then evaluating the outcome.
6. Otherwise terminating the flow.

If the score is bewteen 0.7 and 0.9, Flow is moving into "Switch" action, to check outcome of the approval:

1. The "Switch" action evaluates outcome of the approval. For both "Approve" and "Reject" it's doing the following:
2. Adds new utterance to intent - either "Invoices" (if approved) or "None" (if rejected)
3. Then trains the model
4. If the outcome was "Not applicable" it is terminating Flow to not execute other logic.

Adding utterances to intents and training the model is done using P1 HTTP actions, that are using LUIS endpoints as described here: https://westus.dev.cognitive.microsoft.com/docs/services/5890b47c39e2bb17b84a55ff/operations/5890b47...

Next Flow is processing the mail itself:

1. It falgs the mail, so that I have it in my tasks and I can review if I need to pay it or not
2. It forwards it to accountancy
3. Finally, for each attachemnt it copies it to my OneDrive to a folder with current month-year name (if it doesn't exist - it's being created)
4. And enters information about the e-mail and attachment to an Excel file storing all processed e-mails.

Final words

This is how the Flow works - it saves me really lots of work. What I wish it is also able to do, is to read through the attachments to automatically discover the amount, payment due date etc... so that it would also be able to:

1. Create me a To-Do task for making the payment
2. Register cost information in Excel file so later I can see the values

However since the documents are mostly often PDF files, and having different formats, there is no easy way today to get these information using Microsoft Flow only. Possibly one day OCR present in Cognitive Services actions will allow also to process PDF files.

Here is the video that I recently posted covering this scenario!

I hope you find this process inspirational for covering your own needs. If you have any quesitons reach me out here or via my blog (https://poszytek.eu) or through Twitter (@TomaszPoszytek).

Thanks for reading! 😁

[becloudsmart]

Great post, for the pdf invoices, could you use AI Builder form processing feature?

https://powerapps.microsoft.com/en-us/blog/introducing-ai-builder-for-powerplatform/

[TomaszPoszytek]

Unfortunately not. I can't use AI Builder. Since that feature is dedicated to PowerApps. I can't utilize it inside Flow so far I know. Or may I be wrong?

[becloudsmart]

I have been able to utilise Business Card AI into flow to capture fields and push to Dynamics, so I am not sure but in general you can add PowerApps trigger and it can generate fields with values out of AI model. See this blog post at the bottom.

http://www.365knowledge.com/2019/06/10/ai-builder-in-powerapps-business-card-reader-example/