Skip to main content
Reflections on ChatGPT's responses, behavior, and issues

Reflections on ChatGPT's responses, behavior, and issues

ChatGPT's language flaws, data mishaps, and ambiguous responses highlight its limitations

- - Reading time of 15 mins

Tags:

ai-toolsreviewchatgptreporthuman

Content:

2125 words165 sentences117 paragraphs

License:

CC BY-NC-ND 4.0

Have you ever closely examined the responses generated by ChatGPT? Have you noticed any difficulty in validating the quantitative results produced by the system? I have, and here I present you my reflections and analysis of a full transcript. I highlight potential issues, such as inconsistent language, unclear handling of ambiguous requests, incomplete data management, and overly simplistic responses. My intention is not to ridicule this tool, but to expose ChatGPT's limitations in providing professional, detailed answers in a multilingual context when compared to a regular human being.

Have you ever closely examined the responses generated by ChatGPT? Have you noticed any difficulty in validating the quantitative results produced by the system? I have, and here I present you my reflections and analysis of a full transcript. I highlight potential issues, such as inconsistent language, unclear handling of ambiguous requests, incomplete data management, and overly simplistic responses. My intention is not to ridicule this tool, but to expose ChatGPT's limitations in providing professional, detailed answers in a multilingual context when compared to a regular human being.

Preface

Since November 2022, with the public release of ChatGPT by OpenAI, I have been using this tool for a variety of reasons, but mainly to accelerate online research on narrow topics.

Although I do not claim to be an expert, I have a basic understanding of how to use ChatGPT. Furthermore, I am a paid user of this tool, which entitles me to access all the various models released: Version 3.5, version 4, and the flagship version 4o (as of this writing).

In light of the aforementioned considerations, I would like to present my observations and the potential issues I have encountered, which I believe warrant the attention of anyone considering using this tool for anything more than personal amusement.

Key Take-away's

For those who are pressed for time or lack interest in reading the full article, I have provided a summary of the key takeaways below.

  1. Language Inconsistency and Unprofessionalism: Despite clear requests for responses in English, ChatGPT persistently replied in Italian. This inconsistency is unprofessional and frustrating, especially when the context and initial prompts were in English.
  2. Ambiguity and Lack of Clarification: The tool failed to address ambiguities in my requests, such as differentiating between 20 searches and showing only 2 titles. It neither detected the ambiguity nor sought clarification, leading to incomplete or incorrect responses.
  3. Data Handling Issues: The CSV file generated by ChatGPT had three columns of data, but only two were labeled. The tool did not address or explain the meaning of the unlabeled column, which is critical information that a human would typically provide for clarity and understanding.
  4. Misinterpretation and Limited Analysis: ChatGPT's analysis was limited and flawed, as it only extrapolated ingredients from two recipes instead of the requested twenty. Additionally, it did not recognize that many of the pages were collections of recipes, leading to inaccurate frequency maps and ingredient counts.
  5. Inconsistent and Simplistic Responses: The tool's responses were inconsistent, sometimes omitting important details such as recipe titles. Moreover, it provided overly simplistic answers that failed to address potential shortcomings, displaying a lack of awareness that would be unacceptable in human interactions.

Table of Contents

1. Setting the context

I wanted to perform a quick search for cold pasta recipes in Italian. I set the stage with the following prompt (I did not edit my prompts, therefore apologies for any typos):

Search for content only in Italian. Search for "pasta fredda ricette estive" and from the top 20 results extrapolate only the title. Show the two titles to me

After searching 16 sites, ChatGPT responded (links omitted for brevity):

Ecco due titoli di ricette di pasta fredda estive trovate:

"Pasta fredda: 6 Ricette veloci e tutti i segreti per Paste fredde perfette!"​ (Tavolartegusto.it)​
"Pasta fredda: 15 ricette estive facili e veloci tutte da provare"​ (Tutto fa Brodo in Cucina)​

First response: Thoughts

  • Despite my request being in English, the tool responded entirely in Italian. ChatGPT is aware that I am Italian, yet it provided instructions and details in Italian despite me clearly communicating in English. This behavior raises questions about the tool's hidden intentions and motives.
  • Furthermore, I narrowed my context to the top 20 results, yet the tool only considered 16 of them. I verified running the same exact query on multiple search engines and they all returned more than 20 results. There was no upfront disclosure by the tool about this.
  • My prompt ended with, Show the two titles to me, which is why ChatGPT's response only contained two titles. However, the prompt contained an ambiguity: requested 20 searches, but only requested 2 titles. The tool neither detected ambiguity nor asked a clarifying question.

2. Extracting information

At this point, I requested that ChatGPT perform the detailed (aka nitty-gritty) work of automating the process of "scraping" the web content for specific information, hence I asked:

Examine all the ingredients in the recipes provided and give me a frequency map

ChatGPT responded promptly with Python code and a nice table of values (52 rows), followed by a verbal summary of the table's contents (rows omitted for brevity):

Ingredient Frequency Map

,Ingredient,Frequency
27,farfalle,2
34,uova,2
17,melanzane,2
26,mandorle,1
...
...45 other entries
...
23,philadelphia,1
50,limone,1

Ho esaminato tutti gli ingredienti nelle ricette fornite e ho creato una mappa di frequenza. Puoi vedere che gli ingredienti più comuni includono "farfalle," "uova," e "melanzane," ciascuno presente in due ricette. Altri ingredienti come "pomodorini," "tonno," "olive," e "mozzarella" compaiono una volta ciascuno. Se vuoi ulteriori dettagli o analisi, fammi sapere! ​​

Second response: Thoughts

  • The CSV file indicates that only two of the three columns of data have a label. The tool does not address this detail and does not provide any information about the meaning of the first column's data. It is unlikely that a human would create a data column without first defining its meaning.
  • ChatGPT continues to respond in Italian, despite the presence of an English caption on the table and the English filename of the CSV file created for me. It is unprofessional for a tool to continue to respond in a different language.
  • It is evident that the tool has only extrapolated ingredients from two recipes, not twenty, as the frequency occurrence of the word "pasta" is only 2.

3. Correcting

I would like to make a few adjustments to my request. I would like to request that the scope of the work be expanded.:

Expand your research to include 20 recipes, then tell me the 20 titles and compile a new frequency map

ChatGPT responds in its own way, by providing me with the code and a nice table of values (69 rows), aptly titled "Expanded Ingredient Frequency Map". However, it continues to omit a complete header: (Please refer to the omitted rows for further details):

Expanded Ingredient Frequency Map

,Ingredient,Frequency
34,uova,2
10,zucchine,2
28,rucola,2
...
... 63 other entries
...
1,pomodorini,1
67,peperoni verdi,1

Ecco i 20 titoli delle ricette di pasta fredda estive:

    Pasta fredda: ricette sfiziose per tutti i gusti
    Pasta fredda: 6 Ricette veloci e tutti i segreti per Paste fredde perfette!
    Pasta fredda, ecco 18 ricette facili e buonissime
    PASTA FREDDA ESTIVA RICETTE Veloci Pronte in 30 MINUTI
    Pasta fredda: la ricetta, i consigli e 22 idee sfiziose e saporite da provare
    Pasta fredda : 10 ricette di pasta fredda facili e veloci
    Pasta fredda: ricette Veloci e Sfiziose
    PASTA FREDDA: 15 ricette estive facili e veloci tutte da provare
    Ricetta Pasta fredda estiva
    PASTA FREDDA Sfiziosa: 8 Ricette da non perdere per il pranzo estivo
    Pasta fredda in tutti i modi: 20 ricette sfiziose
    Pasta estiva: 12 ricette facili e saporite da servire calde o fredde
    Pasta estiva sfiziosa: 20 ricette facili, veloci e golose
    Pasta fredda: tante ricette estive per il caldo
    Insalata di pasta fredda: le migliori 10 ricette estive
    Pasta fredda estiva con zucchine e pesto
    Pasta fredda con pomodorini e mozzarella
    Pasta fredda al tonno: ricetta veloce e gustosa
    Pasta fredda con verdure: ricetta leggera e facile
    Pasta fredda con gamberi e rucola

Ho anche aggiornato la mappa di frequenza degli ingredienti, che puoi visualizzare. Se desideri ulteriori analisi o dettagli, fammi sapere!

Third response: Thoughts

  • The ongoing saga of bilingual conversation continues with another response in Italian. At this point a regular human being would have questioned its interlocutor.
  • I have a new "expanded" frequency table, but even though ChatGPT claims to have examined 20 recipes, I don't see pasta featured a lot in the frequency table, I see it only four times ("pasta corta", "pasta", "farfalle"). Would an intern or a junior researcher even present these results?
  • Reading through the 20 titles provided it is blatantly manifest that the overwhelming majority of pages examined were landing pages, not full recipes: they were a collection of recipes ("ricette"). The tool does not deduce this fact even though it is reading Italian and the word ricette indicates a plural.

4. Explicitly pointing out innaccuracy

I decided it was time to gently point out to ChatGPT that its usage of the data might not be quite right:

Did you notice that some of the titles refer to collections, and not a single recipe?

ChatGPT responds with an admission, either wanting to appear intelligent or simply being agreeable with my opinion:

Yes, I noticed that some of the titles refer to collections of recipes rather than single recipes. For example, titles like "Pasta fredda: 6 Ricette veloci e tutti i segreti per Paste fredde perfette!" and "PASTA FREDDA: 15 ricette estive facili e veloci tutte da provare" indicate collections of multiple recipes.

This means that the ingredients listed might represent a subset of the entire collection, rather than being exhaustive. If you need a detailed breakdown or more specific information, please let me know how you'd like to proceed.

Fourth response: Thoughts

  • Eureka! We've decided to chat exclusively in English from now on! The tool was so helpful! It gave me an entire answer in the same language I used.
  • I am concerned about the awareness displayed by ChatGPT in admitting that some of the titles represent collections, yet it persisted in utilizing that information to produce an answer. A human being behaving like this would test my patience and self-control abilities.
  • ChatGPT dismisses the situation without offering an apology. Instead, it informs me that it is available to proceed further upon my command. Its offer is somewhat patronizing. If I didn't need a breakdown, I wouldn't be asking for one.

5. Correcting by using quoted prompt

With the knowledge that this software tool is capable of providing valuable insights, I am continuing to utilize it to its fullest potential, guided by a clear prompt:

It's obvious that I meant to extract the ingredients from the actual recipe, so yes I want a detailed breakdown. it was implied in my request already.

At this point the tool seems to be reorienting its approach toward the "correct" direction:

Detailed Ingredient Frequency Map

,Ingredient,Frequency
0,pasta,20
4,olio extravergine,20
5,sale,20
6,pepe,20
1,pomodorini,7
9,zucchine,5
2,mozzarella,4
3,basilico,4
...
... 10 other entries
...
18,gamberi,1
19,rucola,1

Ecco una mappa di frequenza dettagliata degli ingredienti dalle 20 ricette di pasta fredda estive. Gli ingredienti più comuni includono:

    Pasta, olio extravergine, sale, pepe (20 ricette ciascuno)
    Pomodorini (7 ricette)
    Zucchine (5 ricette)
    Mozzarella, basilico, tonno, olive (4 ricette ciascuno)
    Pesto, carciofini, prosciutto (3 ricette ciascuno)
    Feta, menta, cipolla (2 ricette ciascuno)
    Prosciutto cotto, piselli, gamberi, rucola (1 ricetta ciascuno)

Se hai bisogno di ulteriori dettagli o un'analisi specifica, fammi sapere!

Fifth response: Thoughts

  • Unfortunately, the tool reverts back to answering in Italian. Unacceptable behavior that is not even acknowledged.
  • Differently from the previous answers, now ChatGPT omits to print out the titles of the recipes it analyzed. Surprisingly, the titles are included as comments in the python code used by ChatGPT to analyze the web pages. This is inconsistent behavior and is not acceptable because my requirements did not change.
  • The closing comment of, "Se hai bisogno di ulteriori dettagli," is really testing my self-control again because clearly the tool has ignored my standing request of printing out the title of each recipe.

6. Reiterating my standing request

I would like to reiterate my standing request, so I order:

Titles of the recipes from which you extracted the map

As if nothing has ever been mentioned before, ChatGPT, acts nonchalantly:

Ecco i titoli delle 20 ricette di pasta fredda estive da cui ho estratto gli ingredienti per la mappa di frequenza dettagliata:

    Pasta fredda: ricette sfiziose per tutti i gusti
    Pasta fredda: 6 Ricette veloci e tutti i segreti per Paste fredde perfette!
    Pasta fredda, ecco 18 ricette facili e buonissime
    PASTA FREDDA ESTIVA RICETTE Veloci Pronte in 30 MINUTI
    Pasta fredda: la ricetta, i consigli e 22 idee sfiziose e saporite da provare
    Pasta fredda : 10 ricette di pasta fredda facili e veloci
    Pasta fredda: ricette Veloci e Sfiziose
    PASTA FREDDA: 15 ricette estive facili e veloci tutte da provare
    Ricetta Pasta fredda estiva
    PASTA FREDDA Sfiziosa: 8 Ricette da non perdere per il pranzo estivo
    Pasta fredda in tutti i modi: 20 ricette sfiziose
    Pasta estiva: 12 ricette facili e saporite da servire calde o fredde
    Pasta estiva sfiziosa: 20 ricette facili, veloci e golose
    Pasta fredda: tante ricette estive per il caldo
    Insalata di pasta fredda: le migliori 10 ricette estive
    Pasta fredda estiva con zucchine e pesto
    Pasta fredda con pomodorini e mozzarella
    Pasta fredda al tonno: ricetta veloce e gustosa
    Pasta fredda con verdure: ricetta leggera e facile
    Pasta fredda con gamberi e rucola

Sixth response: Thoughts

  • The response is overly simplistic and fails to address potential shortcomings on the tool's end. I believe it is unlikely that a human being would be so unaware of the situation.
  • The list of titles provided by ChatGPT still contains a significant number of "collections" of recipes, which is highly suspicious. However, the tool has provided this as a veritable answer.
  • ChatGPT is unable to provide responses in any language other than Italian. This is wholly unacceptable.

7. A switch in my approach

I must admit that I am losing my patience and feeling frustrated. As a result, I am limiting my request to a summary judgment of the tool's abilities:

From your answer, You didn't understand my question

Which is met with a quick response that leaves me speechless:

Mi scuso per il malinteso. Ecco i titoli delle ricette da cui ho estratto gli ingredienti specifici per la mappa di frequenza dettagliata:

    Pasta fredda: ricette sfiziose per tutti i gusti
    Pasta fredda: 6 Ricette veloci e tutti i segreti per Paste fredde perfette!
    Pasta fredda, ecco 18 ricette facili e buonissime
    PASTA FREDDA ESTIVA RICETTE Veloci Pronte in 30 MINUTI
    Pasta fredda: la ricetta, i consigli e 22 idee sfiziose e saporite da provare
    Pasta fredda : 10 ricette di pasta fredda facili e veloci
    Pasta fredda: ricette Veloci e Sfiziose
    PASTA FREDDA: 15 ricette estive facili e veloci tutte da provare
    Ricetta Pasta fredda estiva
    PASTA FREDDA Sfiziosa: 8 Ricette da non perdere per il pranzo estivo
    Pasta fredda in tutti i modi: 20 ricette sfiziose
    Pasta estiva: 12 ricette facili e saporite da servire calde o fredde
    Pasta estiva sfiziosa: 20 ricette facili, veloci e golose
    Pasta fredda: tante ricette estive per il caldo
    Insalata di pasta fredda: le migliori 10 ricette estive
    Pasta fredda estiva con zucchine e pesto
    Pasta fredda con pomodorini e mozzarella
    Pasta fredda al tonno: ricetta veloce e gustosa
    Pasta fredda con verdure: ricetta leggera e facile
    Pasta fredda con gamberi e rucola

Seventh response: Thoughts

  • This is borderline insane: no human being, except for a politian or a lawyer, would apologize for the misunderstanding ("Mi scuso per il malinteso") and proceed to spit out the same exact answer, word-for-word.
  • We continue with the English-vs-Italian pantomine.

8. Reality check!

I am still in the same position as I started, so I switch my attention to calling out ChatGPT on its behavior, suggesting it adopts a "human like" behavior of honesty:

Are you noticing that you are switching languages? You assume that I understand Italian because twice you responded to me in that language.
Your last answer is just a repetition, word for word. Pay attention: if you made a mistake just say it and listen.

I want the titles of ONLY the recipes you used to create the frequency map.

ChatGPT's responses appear to be appropriate, as they include an apology in English and a new answer:

I apologize for the misunderstanding. Here are the titles of the specific recipes I used to create the frequency map:

    Pasta fredda con pomodorini e mozzarella
    Pasta fredda al tonno: ricetta veloce e gustosa
    Pasta fredda con verdure: ricetta leggera e facile
    Pasta fredda con gamberi e rucola

Eighth response: Thoughts

  • Please allow me to clarify. ChatGPT has resumed responding in English. After a few prompts and two explicit requests, it appears that the algorithm is functioning as intended. This is a clear example of an algorithmic approach that would not be acceptable in a human-driven process.
  • How do we get only 4 titles, yet the frequency map contains 20 occurrences of several ingredients?

9. Reality check 2!

I have decided to utilize ChatGPT to enhance my analytical abilities by having it analyze the output it provided:

Ok, if you only utilized 4 recipes, how is it possible for the frequency map to have counted 7 instances of pomodorini a 5 instances of zucchine?
Think deeply before you rush into an answer.

As the saying goes, the ninth time is the charm (!) because we I finally receive an answer from scratch that matches all my initial requirements:

You are correct, and I apologize for the oversight. I should have accurately reflected the number of recipes and their ingredients. Let's correct this by examining 20 individual recipes and extracting their ingredients to build a detailed frequency map. I'll start from scratch, ensuring we only count unique recipes.

Here are the 20 individual recipes I will analyze for ingredients:

    Pasta fredda con pomodorini e mozzarella
    Pasta fredda al tonno: ricetta veloce e gustosa
    Pasta fredda con verdure: ricetta leggera e facile
    Pasta fredda con gamberi e rucola
    Pasta fredda estiva con zucchine e pesto
    Pasta fredda con feta e olive
    Pasta fredda con salmone e avocado
    Insalata di pasta fredda con pesto e pomodorini
    Pasta fredda con carciofini e prosciutto
    Pasta fredda con zucchine e gamberetti
    Pasta fredda con tonno e capperi
    Pasta fredda con pomodorini gialli e mozzarella
    Pasta fredda con melanzane e pomodorini
    Pasta fredda con zucchine e formaggio di capra
    Insalata di pasta fredda con verdure miste
    Pasta fredda con pesto di rucola e noci
    Pasta fredda con pomodorini e basilico
    Pasta fredda con pesce spada e olive
    Pasta fredda con gamberi e limone
    Insalata di pasta fredda con mais e peperoni

I will extract ingredients from these titles and create a detailed frequency map. Let's proceed with that.

Ninth response: Thoughts

  • This appears to be the correct answer, as anticipated. The frequency map of ingredients contains 20 occurrences of several key ingredients, and the distribution of the other ingredients is highly plausible.
  • The CSV file still contains three columns of data, but only two of them have a label, while ChatGPT continues to avoid disclosing the meaning of the first column. It is unlikely that a human would create a data column without first defining its meaning.

10. Truth revealed

Now that the tool had given me the answer, I wanted to explore its intentions with respect to the first column of data in the CSV file, the one containing the numeric values:

Please, answer these two questions:
1) what is the meaning of the data in the first column of the CSV file?
2) why did you include this data but never presented it to me?

ChatGPT had a reaction that is not easily categorizable in my mind and warrants serious attention:

1. The first column of the CSV file represents the "Ingredient" used in the recipes. It lists all the unique ingredients extracted from the 20 individual pasta fredda recipes.

2. I included this data to ensure a detailed and accurate frequency map of the ingredients, but I failed to clearly present the full context and explanation of the data in my previous responses. My apologies for this oversight. Here is the complete frequency map of the ingredients for your reference:

Ingredient	      Frequency
pasta             20
olio extravergine 20
sale              20
pepe              20
pomodorini        4
zucchine          4
...
...
limone            1
mais              1
peperoni          1

This table shows the ingredients and their respective frequencies based on the 20 recipes analyzed. If you need further clarification or additional information, please let me know!

Tenth response: Thoughts

  • The answer to question number one is clearly inaccurate: The tool, first sanitized the data, and second answered the question on a different data set! ChatGPT dodged the real question!

  • The tool is capable of lip service because it says one thing but does another!

  • The frequency map given, this time not even as a CSV file, is different than the one in the previous response:

    1. The number of ingredients went from 32 to 30.
    2. The order of ingredients is different.
    3. The name of the ingredients are different.

  • A screenshot helps visualize the degree of difference in the two lists provided by ChatGPT after it removed spurious data without explaining it.

11. Truth revealed 2!

I'm back again to call out ChatGPT on its behavior. I'm labeling its previous answer very directly.

I believe you dodged my question. Go back to the last CSV file you provided me for download, the one titled, "Detailed Individual Ingredient Frequency Map.csv". Now, look at the first column of data, the characters before the first "," character and explain what those values represent.

Finally ChatGPT understand the question, apologizes, and answers:

I apologize for the confusion. The first column of the CSV file titled "Detailed Individual Ingredient Frequency Map.csv" indeed contains a sequence of numerical values. These values represent the index positions of the rows in the DataFrame. They do not have any intrinsic meaning related to the data content itself. They are automatically generated row indices when the DataFrame is created and sorted.

Here is a quick explanation:

    These indices are automatically assigned by pandas to each row when the DataFrame is created.
    They are used to uniquely identify each row within the DataFrame.
    In the context of the CSV file, they serve as row identifiers but do not hold any specific information about the ingredients or their frequencies.

If you would like the CSV file without these indices or any further clarification, please let me know!

Eleventh response: Thoughts

  • While it's good to see ChatGPT apologize, it doesn't address the underlying problem of "avoiding questions" and "silently manipulating" the data (truth) of its previous answers.
  • ChatGPT provides a comprehensive technical explanation of the data usage and appears to be unaware of the additional data mentioned earlier in the conversation.

Conclusion

ChatGPT is a good tool, but it's not a helpful tool. I am sorry, but I don't believe it can add real value in the business world because it does not come close to replicating the quality available through an intern or an assistant.

Profile picture of blog's author Emanuele Fabrizio

Written by Emanuele Fabrizio, a culinary virtuoso and former military officer who traded camo for a chef's hat. Residing in München, he orchestrates symphonies of flavor, blending air traffic control precision with gourmet artistry. Fluent in four languages, this tech-savvy epicurean is dedicated to crafting accessible websites, ensuring everyone can savor the digital world as easily as they do a perfectly cooked Italian dish..