Form Processing Doesn't Recognize Fields

[jasonkhonlaw]

The AI Builder form processing doesn't work well enough to be usable.  I know it's in preview, however it really needs the ability to allow the user to create their own fields and then select an area of the document.  Even though my PDF invoices are originals (no scanned) and are very clear, it doesn't recognize many of the fields.  I can even select and copy and paste the text right out of the PDF, but the form processor doesn't recognize them.  It also doesn't recognize the table of line items as a table, even though some of the documents used in the analysis have many rows of data there.  It didn't even recognize the bill to address, which seems like a pretty common one.  Any suggestions? Please see attachment.

[JoeF-MSFT]

Hi @jasonkhonlaw,

Thanks for the feedback and providing a screenshot. We are actively working on Form Processing so improvements and new features are on its way.

To detect the fields, Form Processing will look for values that vary between the documents that you provide for training. For the case of the 'Billed to' field, does its value change across the documents you have uploaded?

[jasonkhonlaw]

No, the bill to wouldn't change in this instance since it is a supplier sending us an invoice, so the company name and address is the same.  However, I'd still want to pull this from the document and process it rather than hard coding it in the flow or whatever is using the model.

Thanks!

[JoeF-MSFT]

Makes sense @jasonkhonlaw. Today that value would need to change at training time. You could, for example, generate test documents to use for training that have that value changed.

Stay tuned for upcoming updates to Form Processing that will address these cases. 

[jasonkhonlaw]

So I actually went through the training PDFs that I fed the builder.  It looks like the bill to isn't the same in all of them.  Some have an attention line before our company name, others don't.  The headings do not have a ":" after them, if that makes any difference.  

Billed to

ATTENTION: CORPORATE HEADQUARTERS

Company Name

[Ashish2012]

Great scenarios, I guess Form processing works based on the major differences between two images.

[Tacsi]

I'm having a similar issue the fields just aren't being recognized or if they are it's very inconsistent. It's not workable at the moment. 

I am happy to share in PM

[BARD]

I'm having the same issue, lack of success. It recognizes a lot but I have several things it's skipping. I've created multiple models and it is, for instance, not capturing the Purchase Order Number but getting the date below it. It would be great if we could give AI a hint about what's where and that improve it or draw a box and say X will always be PO #.

[moutinhoabreu]

Here the same,

Training with a google invoice it only gets a few fields.

I thought a official invoice from this supplier could give me a clean reading, but it seems not.

Even the supplier name or invoice date isn't recognized.

Already tried this feature, form processing, since it was published and never got it working in a desirable way. 

So i gave up.

[markslosberg]

I am having the same general set of problems. I have uploaded 6 standard but relatively complex packing slips from a single supplier. It gets some of it but it isn’t reading the tables consistently.

It seems to be getting smarter (based on reading through this thread since Oct 2019) but not fast enough to be usable now.

In the case of my form, I think we are asking too much of the AI at this point. I suggest that it be modified to be more of a semi-automated process where I can help it analyze information better During these early releases where it hasn’t yet developed the necessary intelligence to do the whole job itself. Perhaps a 1st pass that provides me with what it thinks and the a human-centric opportunity to tell it what it missed based on “real human intelligence” and then it can combine that into a mixed mode model that combines both the AI and what I teach it.

I feel like I could help it be more accurate but it (she/he) doesn’t yet want to hear my opinion 😉

Happy to share the sample packing slips and the analyzed results if someone wants to look at it to improve the product or offer workarounds (but probably not out there on this open thread as there is some proper info on the packing slips).