Credit Consumption - AI Model Vs AI Prompt

[arun_kj]

Hi All,

I have two solutions to my requirement (Email classification to predefined categories based on mail content).

1) AI Model - I used one of the custom models named "Classify texts into custom categories" available in AI models in power automate website. I can store training data in a dataverse table and use it to train the model to classify mails into different categories.

2) AI Prompt - I have created a good custom prompt using "Create text with GPT using a prompt" available under AI prompts section in power automate website.

Please let me know below details if you know any of them:

1) Does both approach consume same amount of credits? If not, how many credits are used for each characters. I want to analyse which is better in financial aspect.

2) Is there any safety concern in using custom prompt or custom model. Is the data passed to it protected?

3) In ChatGpt, I have seen that even if I give a prompt in a conversation to generate the required result in a specific way it works as required. But if I use that chat one week later to give an input, the output generated may not have followed the instructions I gave initially. Since Microsoft uses Open AI resources, does the "Create text with GPT using a prompt" prompt has a similar memory issue? Is there a chance the classification accuracy change over time?

[Tjan · Accepted solution]

Hi @arun_kj , since you asked to look into the other articles, here I am! I will provide answers here underneath:

1) Does both approach consume same amount of credits? If not, how many credits are used for each characters. I want to analyse which is better in financial aspect.

No they do not consume the same amount of credits. The best thing that you can do to get a grasp of credit consumption is to navigate to the "AI Builder activity" page in Power Automate. Here you can view all consumptions including Prompt consumption (you can use the filter).

2) Is there any safety concern in using custom prompt or custom model. Is the data passed to it protected?

From what I understand the AI used is based on Azure OpenAI. This means that data is kept within your environment and not used for any AI testing. So yes, data is protected! 

3) In ChatGpt, I have seen that even if I give a prompt in a conversation to generate the required result in a specific way it works as required. But if I use that chat one week later to give an input, the output generated may not have followed the instructions I gave initially. Since Microsoft uses Open AI resources, does the "Create text with GPT using a prompt" prompt has a similar memory issue? Is there a chance the classification accuracy change over time?

If you want to do something about this you can play with the 'temperature' setting in prompts. If you set it on high (eg 1) it will provide answers that are not too consistent. If you set this value on low (eg 0) it will provide consistent results. This is not something you can do in ChatGPT, so play a bit with it. In your use case I would recommend putting it on 0. There's no need to set a higher temperature since your output is not required to be any creative (unlike blog posts for example).

[arun_kj]

@Tjan Thank you so much for your inputs!

[Tjan]

@arun_kj  No worries at all! If you have any additional questions regarding AI Builder, feel free to @ me in it.