How to group customer feedback with an LLM

What a language model can and cannot do

A language model can read hundreds of short messages and suggest themes faster than one person. It can notice that "the report is slow" and "the dashboard takes forever" describe one issue. It can also invent a theme no customer mentioned, merge unrelated messages or miscount.

Escuta Produto does not cluster feedback with AI, so this step runs outside the product, on an export you control. Treat the model as a first draft. Your reading of the messages decides the themes.

Export and clean the file first

Start with the CSV export described in how to analyze customer feedback in a spreadsheet. Before you export, confirm which fields your app sends with each item, using the widget docs. Then, before you send anything to a model, remove what it does not need:

• Names and email addresses.
• Phone numbers, account numbers or other identifiers that customers typed into a message.
• Any metadata field that identifies a person.

Keep the message text, the type, the date and the page URL if you need to locate a problem later. Give each row a row number, and use that number to check the model's answers against the source.

Write a clustering prompt

Give the model the rules, the output format and the input in one prompt. Be specific about the number of themes, the length of each name, and what to do with unclear messages. A workable starting point:

You are reviewing customer feedback for one software product.
Each row has a row number, a type and a message.
Propose at most 8 themes. Name each theme in 2 to 4 words, for example "slow report loading".
Assign every row to exactly one theme, or to "unclear" if the message does not fit any theme.
Return CSV with two columns: row, theme.
Then list each theme with two exact quotes from the rows that support it.
Do not invent a theme that no message supports.

The quotes are the most useful part. They let you check each theme in a minute.

Check the output against the source

Never trust a theme you have not checked. Run three checks:

1. Read a sample. Pick 30 rows at random, and compare each assigned theme with the message.
2. Verify the quotes. Search the export for each quoted sentence. A quote that does not appear in the file is a sign the model made it up.
3. Look at the unclear rows. If many land there, the theme list is too narrow.

Count the themes in your spreadsheet with a pivot table, not with the model's arithmetic. Counting belongs in the tool built for it. If the themes change a lot between runs on the same file, tighten the prompt or the theme limit before you act on any of them. Ask for a second run on a different sample of rows. Themes that appear in both runs with similar wording are more likely to be real, while a theme that appears once and then vanishes is usually noise. Drop it rather than act on it.

Decide what the themes are worth

A theme is a description, not a decision. Before you act, connect each theme to the rest of your work. How many distinct customers mention it? Are the messages bugs or ideas? Does the theme match a problem you already understand? The method in the problem behind a feature request is a good test: what is the customer trying to finish?

A small theme with a severe consequence still matters. One message about lost data can outweigh fifty mild complaints about wording, so read the severity of each theme as well as its size.

Keep the theme names in one shared list, and use them as labels in your internal notes so the team speaks the same language.

Privacy and customer data

Customer messages can be personal data, especially when they include names, emails or details about a person's business. Before you send them to any outside service:

• Read the provider's data terms, including whether submitted data is used to train models.
• Use the tool your company has approved for customer data, if you have one.
• Remove identifiers, and leave out anything sensitive such as health details, payment information or account credentials.
• Mention in your privacy policy if you process customer feedback this way.

Rules differ by country. Read feedback widgets, GDPR and LGPD for the basics, and ask a lawyer about your own situation. This is general guidance, not legal advice.

Bring the themes back into Escuta Produto

Escuta Produto has no tags and no AI analysis, so the themes live in internal notes:

1. Filter the product inbox to type idea or bug, and open the items that belong to a theme you care about.
2. Write the theme name in the internal note of those items, for example "theme: slow reports".
3. Use text search for the theme's key words to catch items the model missed.
4. Review the theme list in your weekly meeting, and revisit it when the product changes. The weekly feedback review template has a slot for it.

For the labeling rules that keep themes stable over time, see how to build a tagging system for customer feedback.