Problem with ComputerVisionAPI (Microsoft Cognitive Services) - "InvalidImageUrl"

[Anonymous]

I'm trying to send an image to ComputerVisionAPI (Microsoft Cognitive Services) with a PowerApp but I always get an error.

I have tried with Camera image and a static image URL but both fail in the same manner.

I have a button with OnSelect:

ClearCollect(tagCollect, ComputerVisionAPI.TagImage("Image URL", "https://images.pexels.com/photos/36764/marguerite-daisy-beautiful-beauty.jpg").tags.name)

or

ClearCollect(tagCollect, ComputerVisionAPI.TagImage("Image Content", Camera1.Photo).tags.name)

This gives error "Invalid number of arguments".

So I remove the first argument and leave only the URL:

ClearCollect(tagCollect, ComputerVisionAPI.TagImage("https://images.pexels.com/photos/36764/marguerite-daisy-beautiful-beauty.jpg").tags.name)

or

ClearCollect(tagCollect, ComputerVisionAPI.TagImage(Camera1.Photo).tags.name)

Now the syntax seems right (no warnings), but when I press the button I get error "The service returned an error: ComputerVisionAPI!TagImage:{"code":"InvalidImageUrl","requestId":02d3d8ea-9521-4f0f-abe1-7d3dcf1bb88a","message":"Can't fetch the image."}

If I try the same image url on test console (https://westus.dev.cognitive.microsoft.com/docs/services/56f91f2d778daf23d8ec6739/operations/56f91f2...), it works.

The documentation (not sure if this is official or not http://www.carlosag.net/PowerApps/Connectors/Computer-Vision-API#_Toc1FFCAB37_DescribeImageParameter...) says that ComputerVisionAPI.TagImage should have two parameters, but it seems to accept only one. So instead of "image url", "[url]" I put just [url]. But it doesn't seem to work. What could be the problem? Has anyone else gotten this to work? 

[Anonymous]

Understanding the problem and a lot of reading, the problem is powerapps base64 image-format and all api, endpoints from computervision API, sharepoint, onedrive etc. are expecting binary file format.

[Supriya191]

@steeleyj Thanks for your suggestion. Can you please tell what are the parameters you are passing from powerapps to Azure function with an example. I am not able to pass the image url from my custom image gallery.

[PaulD1]

I got the ComputerVisionAPI (v2) working directly in PowerApps (without using FLOW) but only for describing (tagging) images.

The ComputerVisionAPI command DescibeImageContent will take as an argument the image property from the Add Picture control as follows:

ClearCollect(colImageDesc, ComputerVisionAPI.DescribeImageContent(UploadedImage2.Image).description)

This returns a table with a single row (I guess as a single image was submitted) with two columns - 'captions' and 'tags'. Each of the two columns contain an embedded table. In the code above I am capturing the returned table in a collection 'colImageDesc'.

Captions is a table that contains a single row with two columns: 'confidence' and 'text' where 'text' is a description/caption for the image and 'confidence' is a % representing how likely it is that the descption/caption is correct.

To display the caption, you can add a label with the following formula for the Text property (take the first row from colImageDesc, take the captions value (a table) then take the first row from the captions table and the entry from the text column).

First(First(colImageDesc).captions).text

To show the confidence, add another label with the following formula for the Text property:

Round(First(First(colImageDesc).captions).confidence,2)

I've used a round function to limit the display to just 2 decimal places.

Tags is a single column 'value' with multiple rows and lists tags (person, car, beach, etc.) representative of the image. There is no confidence rating returned, but it seems to list the tags from 'better' (most confident) to 'worse' (least confident).

An easy way to display this is to add a datatable and set the Items property as follows:

First(colImageDesc).tags

Given that the tags appear to be listed from 'most likely to apply to least likely to apply' you could use a FirstN function to limit to the first 5 results or so.

All the other ComputerVision commands appear to require 'format' and 'string' as arguments and as others here have found, there appears to be no way to pass these arguments directly from PowerApps in a way that the ComputerVision API will accept. 

Fingers crossed for an update to PowerApps and/or ComputerVision to get more of the commands (especially OCR, I can see a lot of potential applications for that) working within PowerApps without resorting to FLOW, Azure Functions, etc.

Hope this info helps someone.

Paul