Quality Control

Shaka444

How will we deal with quality control in a new world where anyone can make an app?
Serious question.
Haiku is inherently a clean, easy to understand OS. Coding standards, code formatting guidelines etc.. should still apply.
I guess in an ideal world, the "kids" could run amuk and make all kinds of crazy apps and the "adults" (core devs) would check the code. Unfortunately that ship has sailed and the Pirate ship is out in the deep blue sea on it's own.
Arrr!!

clasqm

We are very far away from a situation where anyone can make an app. Perhaps the pool has enlarged a little, but "anyone"? Not a chance. If you can write sufficiently precise prompts to end up with an app, with all possible edge cases covered, you are smart enough to have done it the old-fashioned way. This is just faster. And not "anyone" is that smart.

I was a professor for thirty years. Every year I'd see these bright-eyed new students come in and I'd think "about one-sixth of you will walk away with a degree". Believe me, five-sixths of the human race is not going to string along requirements in a way that will end up in an app.

As for quality control, what I see developing is having different AIs verify each other's work. And they are brutal!

Atomozero

clasqm As a huge tech enthusiast but a terrible developer, the quality of the software I write can only get better 🤣 Tools like the ones the pirate crew uses can just as easily wreck a program or take it to levels never seen before. What I see is truly essential is testing; human testing of the application is what can really guarantee software quality 💪🦀

Kevin

That's a great question, and has so many aspects to it. Quality control for OS dev, quality control for small apps built by users, quality control for SaaS software, quality control for desktop applications, and quality control for enterprise software all may have different needs and solutions.

I do think we are going to enter uncharted territory. I've seen even non-technical, or barely technical people, start to build apps or use AI agents to automate work. Quality control is going to become very important.

For my personal projects, I always start with a coding style guide to try to build clean code. For Haiku apps, this is mostly Haiku's coding style (though I loose the line length restrictions). Other types of projects will have their own requirements. For example on web apps I will define if I want to use an ORM or write manual SQL, I will instruct it not to write code that slams the database (i.e. select statements in a foreach over 100k objects), etc.

I also have general guidelines of "always choose proper/structurally sound decesion instead of prioritizing quick fixes or work arounds", etc.

Then I do a research phase. I look at other similar software to identify what I do and don't like, whats "the good, the bad, the ugly" about how each software implements a feature I am interested in, etc.

Then with all that I can come up with a plan, and use the LLM to implement against the plan.

I still manually test, but also can use the LLM to do larger stress tests. Hypothetically, if I was writing a new bit torrent app I could give the LLM a list of several hundred torrents and have it downloaded them all 80 times, while adjusting parameters such as bandwidth and concurrency limits, and doing checksums on all the files.

I've started to spin up a local AI which I will use for code reviews and "code to plan" reviews. "Code to plan" is my term for "did the LLM really execute against the plan? Or take shortcuts to just make it work? Or did it do something nonsense"

I've found that workflow produces pretty good results.

In the professional world, it's a little more invovled.

dcbeckman

The word vibe coding is almost cringe to me at this point. What I, and most of the people I know that are using AI, are doing is Agentic Engineering. And I do not mean to disparage vibe coders. The power of any user to immediately make changes that benefit their own life and workflow is great. But without a technical background and some underlying knowledge of coding and the Software Development Life Cycle, most "vibe coded" projects should never be made public in any meaningful way. And most aren't. Quality comes with human knowledge and the willingness to test, test, test. AI Agents are just another tool in the arsenal. But agents should never be relied on 100%.

Shaka444

One thing we can probably do is make a "Code Review Team" with a mailing list. Anyone who wants to can join- even just to lurk/learn.