mastodon.gamedev.place is one of the many independent Mastodon servers you can use to participate in the fediverse.
Mastodon server focused on game development and related topics.

Server stats:

5.7K
active users

Mario Zechner

So, I've been playing with ChatGPT and consorts in anger for the past 2 weeks.

Let me show you today's experience with ChatGPT (3.5 turbo).

I started out by creating a simple, self-contained HTML file with some JS, to parse out a subset of data from JSON. That worked fine, after giving ChatGPT a better idea of the data structure.

sharegpt.com/c/wVi0HAu

sharegpt.comCheck out this ShareGPT conversationThis is a conversation between a human and a GPT-3 chatbot. The human first asks: your are a programming assistant. you help creating simple, self contained HTML files with JavaScript to solve problems. Please write a program that takes a Spine .json file, extracts the skin names, and lists them in alphabetical order. The GPT-3 chatbot then responds: <div><p>Sure, here's a program written in JavaScript that extracts the skin names from a Spine JSON file and lists them in alphabetical order:</p><pre><div class="bg-black rounded-md mb-4"><div class="flex items-center relative text-gray-200 bg-gray-800 px-4 py-2 text-xs font-sans"><span>html</span><svg stroke="currentColor" fill="none" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="h-4 w-4" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"></svg>Copy code</div><div><code class="!whitespace-pre hljs language-html"><span class="hljs-meta"><!DOCTYPE <span class="hljs-keyword">html</span>></span> <span class="hljs-tag"><<span class="hljs-name">html</span>></span> <span class="hljs-tag"><<span class="hljs-name">head</span>></span> <span class="hljs-tag"><<span class="hljs-name">title</span>></span>Spine Skin Names<span class="hljs-tag"></<span class="hljs-name">title</span>></span> <span class="hljs-tag"><<span class="hljs-name">meta</span> <span class="hljs-attr">charset</span>=<span class="hljs-string">"UTF-8"</span>></span> <span class="hljs-tag"><<span class="hljs-name">meta</span> <span class="hljs-attr">name</span>=<span class="hljs-string">"viewport"</span> <span class="hljs-attr">content</span>=<span class="hljs-string">"width=device-width, initial-scale=1.0"</span>></span> <span class="hljs-tag"></<span class="hljs-name">head</span>></span> <span class="hljs-tag"><<span class="hljs-name">body</span>></span> <span class="hljs-tag"><<span class="hljs-name">h1</span>></span>Spine Skin Names<span class="hljs-tag"></<span class="hljs-name">h1</span>></span> <span class="hljs-tag"><<span class="hljs-name">script</span>></span><span class="javascript"> <span class="hljs-comment">// Load the Spine JSON file</span> <span class="hljs-title function_">fetch</span>(<span class="hljs-string">'spine.json'</span>) .<span class="hljs-title function_">then</span>(<span class="hljs-function"><span class="hljs-params">response</span> =></span> response.<span class="hljs-title function_">json</span>()) .<span class="hljs-title function_">then</span>(<span class="hljs-function"><span class="hljs-params">data</span> =></span> { <span class="hljs-comment">// Extract the skin names</span> <span class="hljs-keyword">const</span> skins = <span class="hljs-title class_">Object</span>.<span class="hljs-title function_">keys</span>(data.<span class="hljs-property">skins</span>); <span class="hljs-comment">// Sort the skin names in alphabetical order</span> skins.<span class="hljs-title function_">sort</span>(); <span class="hljs-comment">// Display the skin names in a list</span> <span class="hljs-keyword">const</span> list = <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">createElement</span>(<span class="hljs-string">'ul'</span>); skins.<span class="hljs-title function_">forEach</span>(<span class="hljs-function"><span class="hljs-params">skin</span> =></span> { <span class="hljs-keyword">const</span> listItem = <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">createElement</span>(<span class="hljs-string">'li'</span>); listItem.<span class="hljs-property">textContent</span> = skin; list.<span class="hljs-title function_">appendChild</span>(listItem); }); <span class="hljs-variable language_">document</span>.<span class="hljs-property">body</span>.<span class="hljs-title function_">appendChild</span>(list); }) .<span class="hljs-title function_">catch</span>(<span class="hljs-function"><span class="hljs-params">error</span> =></span> { <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">'Error:'</span>, error); }); </span><span class="hljs-tag"></<span class="hljs-name">script</span>></span> <span class="hljs-tag"></<span class="hljs-name">body</span>></span> <span class="hljs-tag"></<span class="hljs-name">html</span>></span> </code></div></div></pre><p>In this program, we first use the <code>fetch</code> function to load the Spine JSON file. Once the file is loaded, we extract the skin names using <code>Object.keys(data.skins)</code> and then sort them using <code>sort()</code>. Finally, we create a <code>ul</code> element to display the skin names in a list, and append it to the document body.</p></div>

Next, I got a little bolder, trying to make ChatGPT load the file(s) and pass it to an existing API, which i tried to describe to it, which coiuld display the file content in a graphical way.

It was a constant back and forth, without me being able to find a prompt that would make it do what it was asked. It's not a terribly hard use case either. But I figured it's bad at compositing simple tasks like that. So I started from scratch.

My goal was to get to a stage, where I can load 3 files and put the results into a JS map filename -> base64 encoded file content.

We'll here's the result.
sharegpt.com/c/gmye2Qc

sharegpt.comCheck out this ShareGPT conversationThis is a conversation between a human and a GPT-3 chatbot. The human first asks: you're a programming assistant that helps me write little self-contained tools in form of single .html files with some JavaScript. today's program should do the following: 1. let the user drag and drop a .json, .atlas, and a .png files onto the page, and validate that the user specified all 3 files. 2. load the contents of these 3 files and encode them as base64. the encoded data should have the octetstream mimetype in all 3 cases. we will extend this program later. we will . The GPT-3 chatbot then responds: <div><p>Sure, I can help you write that program. Here's a sample code that you can use as a starting point:</p><pre><div class="bg-black rounded-md mb-4"><div class="flex items-center relative text-gray-200 bg-gray-800 px-4 py-2 text-xs font-sans"><span>html</span><svg stroke="currentColor" fill="none" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="h-4 w-4" height="1em" width="1em" xmlns="http://www.w3.org/2000/svg"></svg>Copy code</div><div><code class="!whitespace-pre hljs language-html"><span class="hljs-meta"><!DOCTYPE <span class="hljs-keyword">html</span>></span> <span class="hljs-tag"><<span class="hljs-name">html</span>></span> <span class="hljs-tag"><<span class="hljs-name">head</span>></span> <span class="hljs-tag"><<span class="hljs-name">meta</span> <span class="hljs-attr">charset</span>=<span class="hljs-string">"UTF-8"</span>></span> <span class="hljs-tag"><<span class="hljs-name">title</span>></span>Drag and Drop Base64 Encoder<span class="hljs-tag"></<span class="hljs-name">title</span>></span> <span class="hljs-tag"></<span class="hljs-name">head</span>></span> <span class="hljs-tag"><<span class="hljs-name">body</span>></span> <span class="hljs-tag"><<span class="hljs-name">h1</span>></span>Drag and Drop Base64 Encoder<span class="hljs-tag"></<span class="hljs-name">h1</span>></span> <span class="hljs-tag"><<span class="hljs-name">div</span> <span class="hljs-attr">id</span>=<span class="hljs-string">"dropzone"</span>></span> <span class="hljs-tag"><<span class="hljs-name">p</span>></span>Drop your .json, .atlas, and .png files here.<span class="hljs-tag"></<span class="hljs-name">p</span>></span> <span class="hljs-tag"></<span class="hljs-name">div</span>></span> <span class="hljs-tag"><<span class="hljs-name">div</span> <span class="hljs-attr">id</span>=<span class="hljs-string">"output"</span>></span><span class="hljs-tag"></<span class="hljs-name">div</span>></span> <span class="hljs-tag"><<span class="hljs-name">script</span>></span><span class="javascript"> <span class="hljs-keyword">var</span> dropzone = <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">getElementById</span>(<span class="hljs-string">'dropzone'</span>); <span class="hljs-keyword">var</span> output = <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">getElementById</span>(<span class="hljs-string">'output'</span>); <span class="hljs-keyword">function</span> <span class="hljs-title function_">handleFiles</span>(<span class="hljs-params">files</span>) { <span class="hljs-comment">// Make sure the user dropped 3 files</span> <span class="hljs-keyword">if</span> (files.<span class="hljs-property">length</span> != <span class="hljs-number">3</span>) { output.<span class="hljs-property">innerHTML</span> = <span class="hljs-string">'Please drop all 3 files.'</span>; <span class="hljs-keyword">return</span>; } <span class="hljs-comment">// Validate that the files have the correct extensions</span> <span class="hljs-keyword">var</span> extensions = [<span class="hljs-string">'.json'</span>, <span class="hljs-string">'.atlas'</span>, <span class="hljs-string">'.png'</span>]; <span class="hljs-keyword">for</span> (<span class="hljs-keyword">var</span> i = <span class="hljs-number">0</span>; i < files.<span class="hljs-property">length</span>; i++) { <span class="hljs-keyword">var</span> file = files[i]; <span class="hljs-keyword">var</span> extension = <span class="hljs-string">'.'</span> + file.<span class="hljs-property">name</span>.<span class="hljs-title function_">split</span>(<span class="hljs-string">'.'</span>).<span class="hljs-title function_">pop</span>(); <span class="hljs-keyword">if</span> (extensions.<span class="hljs-title function_">indexOf</span>(extension) == -<span class="hljs-number">1</span>) { output.<span class="hljs-property">innerHTML</span> = <span class="hljs-string">'Please drop a .json, .atlas, and .png file.'</span>; <span class="hljs-keyword">return</span>; } } <span class="hljs-comment">// Read each file and encode as base64</span> <span class="hljs-keyword">var</span> results = []; <span class="hljs-keyword">var</span> reader = <span class="hljs-keyword">new</span> <span class="hljs-title class_">FileReader</span>(); reader.<span class="hljs-property">onload</span> = <span class="hljs-keyword">function</span>(<span class="hljs-params">e</span>) { <span class="hljs-keyword">var</span> base64 = <span class="hljs-title function_">btoa</span>(e.<span class="hljs-property">target</span>.<span class="hljs-property">result</span>); results.<span class="hljs-title function_">push</span>(<span class="hljs-string">'data:application/octet-stream;base64,'</span> + base64); <span class="hljs-keyword">if</span> (results.<span class="hljs-property">length</span> == <span class="hljs-number">3</span>) { output.<span class="hljs-property">innerHTML</span> = results.<span class="hljs-title function_">join</span>(<span class="hljs-string">'<br>'</span>); } }; reader.<span class="hljs-title function_">readAsBinaryString</span>(files[<span class="hljs-number">0</span>]); reader.<span class="hljs-title function_">readAsBinaryString</span>(files[<span class="hljs-number">1</span>]); reader.<span class="hljs-title function_">readAsDataURL</span>(files[<span class="hljs-number">2</span>]); } <span class="hljs-comment">// Handle the drop event</span> dropzone.<span class="hljs-title function_">addEventListener</span>(<span class="hljs-string">'drop'</span>, <span class="hljs-keyword">function</span>(<span class="hljs-params">e</span>) { e.<span class="hljs-title function_">preventDefault</span>(); <span class="hljs-title function_">handleFiles</span>(e.<span class="hljs-property">dataTransfer</span>.<span class="hljs-property">files</span>); }); <span class="hljs-comment">// Prevent the default dragover event</span> dropzone.<span class="hljs-title function_">addEventListener</span>(<span class="hljs-string">'dragover'</span>, <span class="hljs-keyword">function</span>(<span class="hljs-params">e</span>) { e.<span class="hljs-title function_">preventDefault</span>(); }); </span><span class="hljs-tag"></<span class="hljs-name">script</span>></span> <span class="hljs-tag"></<span class="hljs-name">body</span>></span> <span class="hljs-tag"></<span class="hljs-name">html</span>></span> </code></div></div></pre><p>This code sets up a drag-and-drop zone and checks that the user dropped all 3 files with the correct extensions. Then it reads each file and encodes it as base64 using the <code>FileReader</code> API. Finally, it displays the encoded data in the output div.</p><p>Note that this code uses the <code>btoa</code> function to encode binary data as base64, and the <code>data:application/octet-stream;base64</code> URI scheme to specify the octetstream mimetype. You can modify this code to handle other file types or mimetypes as needed.</p></div>

It took me about 25 minutes to get to that stage, mostly because ChatGPT response times are very bad, and because ChatGPT would just keep doing stupid things.

In that time, I could have written this myself, trice.

So, my take away is this:

If you think this will replace programmers anytime soon, I would like to get the phone number of your dealer.

If you give this to a junior dev without human supervision, you will forever have a junior dev churning out bad code that ignores any and all error states and corner cases.

Response times make this an inefficient tool to work on anything that's longer than a simple, self-contained function.

And even for the "single, self-contained function" use case, it usually shits the bed so hard the further you get away from "standard" problems, you'll spend more time massaging it via natural language input than writing things from scratch yourself.

Yes, GPT-4 is slightly better, given the bigger token window. But that's more than counter acted by the further increased response times. Also, it still shits the bed with anything that's "do a qsort in Python".

This is going to be a tool that will be used by many people, only to realize that diminishes their code comprehension and design skills. We'll have to unlearn all this again, once we see the real world negative impact it has on businesses and software quality.

ChatGPT/Copilot finally give programmers a chance to experience how non-technical people feel when trying to coerce programmers to create a thing.

You too can now throw imprecise natural language prompts at a magic coding box and get disillusioned by the results.