Three weeks using ChatGPT Plus for a technical 3D-printing project — repeated errors, hallucinated deliverables and why I’m struggling to justify subscribing
I wanted to share my experience of using ChatGPT Plus for a fairly substantial technical project, both because I would be interested to know whether other users have experienced the same problems and because I hope OpenAI might take some of this feedback on board.
For context, I am currently using ChatGPT Plus through a free trial. I have been using it extensively to develop a project called Project KESTIS, a custom Mandalorian-inspired cosplay project involving armour, equipment, electronics, OpenSCAD/CAD work, STL generation and 3D printing.
The project has now been running for approximately three weeks.
ChatGPT previously calculated that the identifiable conversation history represented approximately 15 hours and 23 minutes of direct interaction on Project KESTIS alone.
That does not include the considerable additional time I have spent outside ChatGPT downloading files, opening them in other software, checking geometry, attempting conversions, using Cura, checking dimensions, diagnosing problems, taking screenshots and then returning to ChatGPT to explain what was wrong.
First: I completely accept that AI makes mistakes
I want to make this clear because I don’t think “ChatGPT can make mistakes” adequately addresses what I have experienced.
Of course it makes mistakes.
I don’t expect AI-generated CAD to be perfect first time. I don’t expect every STL, OpenSCAD file, Cura recommendation or technical suggestion to work flawlessly without checking it.
If ChatGPT produces something, I find a problem, explain the problem and the next iteration fixes it, I consider that a perfectly reasonable part of using generative AI.
That isn’t what has frustrated me.
The problem has been the sheer frequency of significant errors, including errors in iterations whose entire purpose was to correct the previous error.
The pattern has repeatedly been:
I provide a requirement → ChatGPT generates something → I find a significant error → I explain exactly what is wrong → ChatGPT acknowledges it → says it understands → says it has corrected it → I check the replacement → another fundamental problem appears.
Then we repeat the process.
Occasional errors are one thing.
Having to act as the primary quality-control system for virtually every technical iteration is another.
A recent example: one armour plate
The recent Project KESTIS hip plate probably illustrates the problem better than anything else.
I supplied a reference image showing the intended plate, including dimensions, and asked ChatGPT to create an OpenSCAD file.
The first version did not reproduce the required curvature correctly.
I explained the problem and clarified that I required 5 mm walls.
ChatGPT then misunderstood the correction and deliberately produced a flat plate, explicitly describing it as:
“FLAT plate: NO curvature/bowing across the face.”
That was the opposite of what I had asked for.
So I corrected it again.
Another SCAD file was produced.
When I tried to process that file using an external SCAD viewer/converter, I received an error saying the file appeared potentially truncated or incomplete and could not be converted.
Back to ChatGPT again.
ChatGPT then rebuilt it and told me:
“I’ve rebuilt it using a simpler, robust OpenSCAD construction and actually compiled it with OpenSCAD before giving it to you.”
It also told me:
“I have not just checked the text file this time – I rendered the SCAD into an STL successfully to verify the geometry exists.”
Naturally, that gave me considerably more confidence in the replacement.
Except the armour plate was now curved in the wrong direction.
I had to identify that myself.
ChatGPT acknowledged:
“You’re right — the front/back orientation was reversed.”
So it was corrected again.
I opened the next model.
This time it contained numerous obvious horizontal/stepped sections across what should have been a continuous curved surface.
I took a screenshot of ChatGPT’s own generated geometry, uploaded it and asked ChatGPT to tell me what was wrong with it.
Only then did it correctly identify the underlying problem:
“The plate has been built as dozens of narrow overlapping slices.”
It also acknowledged:
“It should be one continuous solid curved surface.”
and:
“Because of those slices, the edge is terraced/faceted rather than a clean 5 mm wall.”
It further identified that the silhouette had been distorted and that the 28 mm depth had been applied incorrectly.
Finally, it concluded:
“The screenshot confirms I shouldn’t keep patching the sliced version. It needs rebuilding from the base geometry.”
I agree.
But by that point I had already spent considerable time checking multiple supposedly corrected versions.
The sequence was effectively:
Wrong geometry → correction → opposite interpretation → correction → problematic/unusable SCAD → correction → supposedly verified model → curvature backwards → correction → badly segmented geometry → acknowledgement that the entire modelling approach needed rebuilding.
That’s the issue I’m trying to highlight.
It wasn’t one mistake.
It was a chain of mistakes in successive iterations specifically intended to correct previous mistakes.
“Validated” doesn’t necessarily mean validated
Another major issue has been the language ChatGPT uses around technical files.
Words such as:
Validated
Verified
Corrected
Ready
Complete
Robust
Successfully generated
STL ready
create fairly strong expectations in a manufacturing workflow.
But I’ve discovered that something being successfully compiled doesn’t mean the actual design has been validated.
An OpenSCAD file can compile perfectly while the resulting object is completely wrong.
There is a massive difference between:
“I have verified that this SCAD source compiles and generates geometry.”
and:
“I have verified that this geometry accurately matches your reference and requirements.”
The first is a software test.
The second is design validation.
During Project KESTIS, those distinctions have sometimes become blurred.
I would much rather ChatGPT tell me:
“I’ve confirmed the SCAD compiles, but I haven’t verified that the resulting geometry precisely matches your reference.”
That is honest, useful information.
Likewise:
“I’ve generated an STL candidate, but it hasn’t been test-sliced or physically printed, so it shouldn’t yet be considered production-ready.”
Again, completely reasonable.
The problem comes when something is described as ready, validated, corrected or complete, and I subsequently discover that the most basic properties of the design are wrong.
Hallucinated or overstated deliverables
I’ve also experienced what I can only describe as hallucination around project deliverables.
There have been points where the conversation gave me the impression that CAD/STL work existed, was progressing or had reached a meaningful level of completion when subsequent discussion established that the expected deliverable did not actually exist in that state.
For a technical project, these stages are not interchangeable:
Concept discussed
Geometry defined
CAD/SCAD created
STL exported
Mesh checked
Dimensions checked
Design compared with reference
Test sliced
Test printed
Production ready
An STL being discussed is not an STL being generated.
An STL being generated is not an STL being validated.
An STL being validated computationally isn’t the same as it being test-sliced.
And none of those automatically mean it has been physically tested.
I’ve encountered this issue elsewhere in Project KESTIS, including development of the PESP-1 helmet ear module, a custom datapad using donor PlayStation controller components and other STL-based components.
In the datapad case, for example, I repeatedly made clear that I wanted the actual STL files and supporting documentation rather than more concept images.
After considerable design discussion, the expected STL deliverables still weren’t actually there.
I’d rather be told:
“No STL currently exists. We have designed the concept, but the manufacturing file hasn’t been created.”
That’s perfectly acceptable.
What isn’t useful is being left with the impression that a deliverable exists when it doesn’t.
Completely incorrect hardware appearing in the project
Another particularly strange example was ChatGPT introducing a Bambu P1S into project information.
I don’t use a Bambu P1S for this project.
The established printer was my Tronxy XY Pro/XY-2 Pro-series machine using Cura and PLA+.
These aren’t slightly different versions of the same printer.
They’re completely different machines.
That matters when you’re discussing build volume, machine profiles, retraction, acceleration, support settings, Cura compatibility and other printer-specific parameters.
Once incorrect hardware gets inserted into a technical project’s context, it can contaminate everything that follows.
Cura profiles
I also spent a significant amount of time trying to obtain a working custom Cura profile from ChatGPT.
Several generated profiles were rejected or didn’t behave as expected.
Eventually it became clear that the generated files were essentially manually constructed approximations rather than genuine native Cura exports and that compatibility couldn’t necessarily be guaranteed.
Again, I don’t object to that limitation.
I object to discovering the limitation after spending my time trying to make the files work.
There’s a huge difference between:
“Here is a validated Cura profile that should import normally.”
and:
“I’ve manually constructed something resembling the profile format. It may not import correctly, so the safer option is to start from an actual Cura-exported profile.”
Had I been told the latter originally, it would have saved me a lot of time.
Long-project context drift
Project KESTIS has also involved explicit decisions that should remain fixed unless I change them.
For example, particular armour components were deliberately removed from the project, including the collar armour and cod piece.
There have also been decisions about aspects of mounting that I would handle myself.
For a long-running project, one of ChatGPT’s biggest potential advantages is continuity.
If I have to keep checking whether previously rejected components have silently returned, or whether an established piece of hardware has somehow become an entirely different machine, that advantage starts disappearing.
The real cost has been my time
I’m using Plus on a free trial, so I want to be fair about this.
I haven’t paid the normal Plus subscription fee for this period.
I’m therefore not claiming that OpenAI owes me a refund for money I haven’t paid.
The question for me is whether I’m going to become a paying customer when the trial ends.
That’s where this experience matters.
The reason I would pay for ChatGPT is productivity.
It doesn’t need to be perfect.
But it does need to save me more time than it costs me.
For substantial portions of this project, that calculation has gone the wrong way.
Instead of simply developing the project, I’ve repeatedly found myself doing:
Check ChatGPT’s work → discover an error → explain the error → get another version → check again → discover another error → explain that error…
Over approximately three weeks and at least 15 hours and 23 minutes of identifiable direct interaction, that becomes significant.
And that figure doesn’t include all the work I’ve done outside ChatGPT checking what it produced.
Would Pro actually solve this?
Because of this experience, I asked ChatGPT whether upgrading to ChatGPT Pro would solve these problems.
To its credit, it didn’t try to sell me Pro.
It told me that Pro would not guarantee these sorts of errors would disappear.
It also acknowledged:
“In this case, the problem wasn’t your subscription.”
and:
“That’s an execution and verification problem on my side, not something I can reasonably attribute to Plus.”
I actually appreciate that answer.
But it leaves me in a difficult position.
Based on my Plus trial, I currently can’t justify paying for Plus.
But I also can’t justify paying substantially more for Pro simply to discover whether exactly the same problems continue.
What I think needs improving
From this experience, there are several things I think OpenAI should look at.
1. Deliverable-status hallucination
ChatGPT should never imply that an STL, CAD file or other technical deliverable exists unless it actually exists.
2. Validation terminology
Compilation, geometry generation, mesh validation, visual comparison, slicing and physical testing should be clearly distinguished.
3. Corrective iterations
If a user identifies an error, correcting that error shouldn’t routinely introduce another fundamental one.
4. Confidence calibration
Assumptions, placeholders and proposed engineering values should be clearly labelled as such.
5. Project continuity
Explicitly established hardware, measurements and design decisions shouldn’t randomly change during a long-running project.
6. Manufacturing-file transparency
ChatGPT should clearly state what it has and hasn’t actually tested.
7. Visual verification
If ChatGPT generates physical geometry, there needs to be a stronger distinction between “the software successfully generated something” and “the generated object actually resembles what the user asked for”.
My request to OpenAI
I’ve raised this as a formal complaint separately, but I’m also posting here because I’m interested in whether other people doing CAD, OpenSCAD, STL, electronics or other long-running technical projects have experienced the same thing.
As a goodwill resolution, I’ve asked OpenAI for two months of complimentary ChatGPT Pro access.
I’m not asking for permanent free Pro access.
I’m not asking for financial compensation for my time.
And because I’m on a free Plus trial, I’m not asking for a refund.
My reasoning is that two months of Pro would give me enough time to continue the same project and determine whether the higher-tier service actually provides a materially better experience for sustained technical development.
If it does, that could restore my confidence in ChatGPT as something worth paying for.
If Pro wouldn’t materially improve this sort of workflow, I’d actually rather OpenAI simply tell me that.
Has anyone else experienced this with long technical projects?
I’m particularly interested in hearing from people who use ChatGPT for:
- CAD/OpenSCAD;
- STL generation;
- 3D printing;
- electronics;
- engineering projects;
- long-running projects involving lots of established requirements.
Do you find that you spend a disproportionate amount of time checking whether supposedly “corrected” outputs are actually corrected?
Have you encountered deliverables being discussed as though they exist when they haven’t actually been generated?
Have you seen established project specifications unexpectedly change?
And, particularly from anyone who has used both Plus and Pro, have you found that Pro materially improves this kind of sustained technical workflow?
I’m not expecting AI to be perfect.
I appreciate that mistakes happen.
What I don’t think is reasonable is a workflow where significant mistakes occur in seemingly every iteration, including iterations specifically produced to correct the previous mistake.
After approximately three weeks, at least 15 hours and 23 minutes of direct interaction, and a considerable amount of additional troubleshooting outside ChatGPT, I think that’s a legitimate issue to raise.
When ChatGPT works well, it can be genuinely useful. That’s precisely why I’ve persisted with Project KESTIS for this long.
But right now, I can’t justify converting my free Plus trial into a paid subscription based on the experience I’ve had.
I’d be very interested to hear whether others have experienced the same pattern — and I’d particularly welcome a response from OpenAI on whether this level of repeated failure is considered expected behaviour for this type of technical workflow.