Our response to it (Iraq war, forever wars, etc.) combined with the realization that the USA are be "the baddies" and we've been lied to since forever, probably might have been the thing that set all the dominos up.

COVID was the straw that broke the camel's back. Had we _not_ had the disastrous response to 9/11, I suspect we could've weathered COVID better (like the rest of the world has.)

If you want to do this for fun or for learning? Absolutely! I did my CS Masters thesis on SQL JOINS and tried building my own new JOIN indexing system (tl;dr: mine wasn't better). Learning is fun! Just don't recommend people build production systems like this.

Is this article trolling? It feels like trolling. I struggle to take an article seriously that conflates databases with database management systems.

A JSON file is a database. A CSV is a database. XML (shudder) is a database. PostgreSQL data files, I guess, are a database (and indexes and transaction logs).

They never actually posit a scenario in which rolling your own DBMS makes sense (the only pro is "hand rolled binary search is faster than SQLite"), and their "When you might need" a DBMS misses all the scenarios, the addition of which would cause the conclusion to round to "just start with SQLite".

It should basically be "if you have an entirely read-only system on a single server/container/whatever" then use JSON files. I won't even argue with that.

Nobody - and I mean nobody - is running a production system processing hundreds of thousands of requests per second off of a single JSON file. I mean, if req/sec is the only consideration, at that point just cache everything to flat HTML files! Node and Typescript and code at all is unnecessary complexity.

PostgreSQL (MySQL, et al) is a DBMS (DataBase Management System). It might sound pedantic but the "MS" part is the thing you're building in code:

concurrency, access controls, backups, transactions: recovery, rollback, committing, etc., ability to do aggregations, joins, indexing, arbitrary queries, etc. etc.

These are not just "nice to have" in the vast, vast majority of projects.

"The cases where you'll outgrow flat files:"

Please add "you just want to get shit done and never have to build your own database management system". Which should be just about everybody.

If your app is meaningfully successful - and I mean more than just like a vibe-coded prototype - it will break. It will break in both spectacular ways that wake you up at 2AM and it will break in subtle ways that you won't know about until you realize something terrible has happened and you lost your data.

Didn't we just have this discussion like yesterday (https://ultrathink.art/blog/sqlite-in-production-lessons)?

It feels like we're throwing away 50 years of collective knowledge, skills, and experience because it "is faster" (and in the same breath note that nobody is gonna hit these req/sec.)

I know, it's really, really hard to type `yarn add sqlite3` and then `SELECT * FROM foo WHERE bar='baz'`. You're right, it's so much easier writing your own binary search and indexing logic and reordering files and query language.

Not to mention now you need a AGENTS.md that says "We use our own home-grown database nonsense if you want to query the JSON file in a different way just generate more code." - NOT using standard components that LLMs know backwards-and-forwards? Gonna have a bad time. Enjoy burning your token budget on useless, counter-productive code.

This is madness.

The whole point of LLM-based code execution is, well, I can just type in any old language it understands and it ought to figure out what I mean!

A "skill" for searching a pdf could be :

* "You can search PDFs. The code is in /lib/pdf.py"

or it could be:

* "Here's a pile of libraries, figure out which you want to use for stuff"

or it could be:

* "Feel free to generate code (in any executable programming language) on the fly when you want to search a PDF."

or it could be:

* "Solve this problem <x>" and the LLM sees a pile of PDFs in the problem and decides to invent a parser.

or any other nearly infinite way of trying to get a non-deterministic LLM to do a thing you want it to do.

At some level, this is all the same. At least, it rounds to the same in a sort of kinda "Big O" order-of-magnitude comparison.

On the other hand, I also agree, but I can definitely see present value in trying to standardize it because humans want to see what is going on (see: JSON - it's highly desirable for programmers to be able to look at a string representation of data than send opaque binary over the wire, even though to a computer binary is gonna be a lot faster).

There is probably an argument, too, for optimization of context windows and tokens burned and all that kinda jazz. `O(n)` is the same as `O(10*n)` (where n is tokens burned or $$$ per second or context window size) and that doesn't matter in theory but certainly does in practice when you're the one paying the bill or you fill up the context window and get nonsense.

So if this is a _thoughtful_ standard that takes that kinda stuff into account then, well, great! It gives a benchmark we can improve and iterate upon.

With some hypothetical super LLM that has a nearly infinite context window and a cost/tok of nearly zero and throughput nearing infinity, you can just say "solve my problem" and it will (eventually) do it. But for now, I can squint and see how this might be helpful.

Software is not the same as building in the physical world where we get economies of scale.

Building 1,000 bridges will make the cost of the next incremental bridge cheaper due to a zillion factors, even if Bridge #1 is built from sticks (we'll learn standards, stable, fundamental engineering principles, predicable failure modes, etc.) we'll eventually reach a stable, repeatable, scalable approach to build bridges. They will very rarely (in modernity) catastrophically fail (yes, Tacoma Narrows happened but in properly functioning societies it's rare.)

Nobody will say "I want to build a bridge upside-down, out of paper clips and can withstand a 747 driving over it". Because that's physically impossible. But nothing's impossible in software.

Software isn't scalable in this way. It's not scalable because it doesn't have hard constraints (like the laws of physics) - so anything goes and can be in scope; and since writing and integrating large amounts of code is a communication exercise, suffers from diseconomies of scale.

Customers want the software to do exactly what they want and - within reason - no laws of physics are violated if you move a button or implement some business process.

Because everyone wants to keep working the way they want to work, no software project (even if it sounds the same) is the same. Your company's bespoke accounting software will be different than mine, even if we are direct competitors in the same market. Our business processes are different, org structures are different, sales processes are different, etc.. So they all build different accounting software, even if the fundamentals (GaaP, double-entry bookkeeping, etc.) are shared.

It's also the same reason why enterprise software sucks - do you think that a startup building expense management starts off being a giant mess of garbage? No! IT starts off simple and clean and beautiful because their initial customer base (startups) are beggars and cannot be choosers, so they adapt their process to the tool. But then larger companies come along with dissimilar requirements and, Expense Management SaaS Co. wins that deal by changing the product to work with whatever oddball requirements they have, and so on, until the product essentially is a bunch of config options and workflows that you have to build yourself.

(Interestingly, I think these products become asymptotically stuck - any feature you add or remove will make some of your customers happy and some of your customers mad, so the product can never get "better" globally).

We can have all the retrospectives and learnings we want but the goal - "Build big software" - is intractable, and as long as we keep trying to do that, we will inevitably fail. This is not a systems problem that we can fix.

The lesson is: "never build big software".

(Small software is stuff like Bezos' two pizza team w/APIs etc. - many small things make a big thing)

I am surprised on the lack of creativity when doing these projects. Why don't they start 5 small projects building the same thing and let them work for a year. At the end of the year you cancel one of the projects, increasing the funding in the other four. You can do that every year based on the results. It may look like a waste but it will significantly increase your chances of succeeding.

Build 1000 JSON parsers and tell me if the next one isn't cheaper to develop with "(we'll learn standards, stable, fundamental engineering principles, predicable failure modes, etc.)"

>Software isn't scalable in this way. It's not scalable because it doesn't have hard constraints (like the laws of physics)

Uh, maybe fewer but none is way to far. Get 2 billion integer operations per second out of a 286, the 500 mile email, big data storage, etc. Physical limits are everywhere.

>It's also the same reason why enterprise software sucks.

The reason enterprise software sucks is because the lack of introspection and learning from the garbage that went before.

Walk through most any suburban American neighborhood and you'll primarily see neutral shades of white, gray, beige, or the occasional muted blues and greens. Sometimes someone will be daring and paint their house in a deep, dark blue or purple (or even black) but that feels relatively rare.

If near the ocean, typical "seaside pastels" come into view.

What's the backstory to the Faroes' colors? Are they set by some local entity/government? Left up to the homeowners? Was there a push to make them colorful? Do the locals have a particular eye for color composition? Did someone help them?

Why are American homes so bland and the Faroes' so delightfully colorful?

So many questions!

Splitting into services siloed each team and allowed them to move independently and by side effect, faster. Not due to inherent properties of (micro) services but because one goes faster by removing the things that slow you down.

As a startup, you do not have this problem. You likely will _never_ have this problem as your default future state is 'dead'.

Do the simplest thing that can possibly work - build a monolith using tools you know that give you the ability to predictably, rapidly iterate on the product.

After you hit some semblance of product/market fit and need to scale, you can do that. Scaling is a solved problem. Premature scaling is not.

[1. This is a joke. Kubernetes wasn't even a thought in Google's eye at this point in history]*

Not quite. Amazon's problem was that they were experiencing too much impedance between teams, and some teams were even siloing themselves to the extent they were creating problems for everyone around them. Bezos' diktat was intended to break through a bunch of petty office politics bullshit that was dragging down the company and preventing teams from delivering their work. The diktat boiled down to basically ordering everyone to grant access to the service they owned and provided to external teams that need it, no exception, and would be held liable if they failed to provide it, intentionally or not

1) Deep expertise and network with potential customers

2) Ability to empathize and connect with your customers and sell the thing

3) Ability to build some sort of MVP

The deep expertise and empathy will allow you to hone in on the problem you know your customers will face, and so you can narrow down the "paradox of choice" of different features/products you can build. It'll give you a much better starting point for experimentation and iteration. Start small. Talk to these folks. Get people to validate your idea. If you don't have an existing network where you can talk to 30-50 customers (100 is better!), this will be a problem.

The empathy and salespersonship will allow you to get folks to give you feedback and/or pre-sell the thing. Maybe do consulting for them to earn revenue and testimonials. If you can't sell this, you're going to have a hard time.

Finally, you need to be able to build something that delivers value to your customers. You don't have to be the world's greatest programmer (actually, if you were that might hurt you more than it helps!) but you do need some programming chops. Maybe it's automation around an existing process (a chatbot that does something small). Or even a spreadsheet. Maybe it's a full-blown MVP. Either way, you don't have the revenue to pay a developer, and sharing context with someone that you can afford is likely very very difficult.

If you don't have these things, this will be really, really tough. If you do, it will still be very tough, but you have a much better chance of being successful.

Find a network of solopreneurs - either something local to you (where you can meet up periodically) or an online group. This will be invaluable as you run into challenges they solved, and then you can pay it forward for the people that will walk your path later. And, having a place to vent with empathetic folk will help keep you sane.

If you need any help, I'm happy to share my experiences solo-bootstrapping my company to 20+ people. Just email me matt [dot] rogish [at] gmail

Good luck!

Would love to connect to learn more about your experiences- I'm at matt@puresomni.com. Is there a best email to reach you at?

Cheers,

Matt

We needed to replace our washing machine when we bought the house, so I did the research and got the best front-loader we could afford. Turns out, due to $reasons, it vibrates throughout the whole house (it's on the second floor) and although it's working fine, every time we do the laundry we shudder at the annoyance. Now, we eventually will have to settle for a lower-performing top loader but I'm still not 100% certain that it won't cause the same problem, so we feel stuck. Keep the good, but frustrating, washer, or take a risk that another almost thousand dollar washer will have the same problem. Outsourcing these decisions and annoyances to a landlord is appealing.

Fighting entropy is a great way to put it - we had things come up that resulted in our inability to tend to the garden, and in a month, it's overwhelmed with weeds. That's my weekend project. Could we pay someone to clear it? Sorta. Service providers don't like (for obvious reasons) small projects, so nobody will return our calls when we say "It's like 2-3 hours of weeding". Could we find someone (say, a college student or something?) to do it? Probably, but it'd be almost more work to find and vet someone to do the work than it is to just do it ourselves. And, there goes a weekend. A weekend that, if we were still in the City, would be spent doing anything else.

I wouldn't trade it for almost anything, but it's not clearly the best solution for everyone. I would totally pay the equivalent of a "condo fee" for someone to manage all this nonsense.

This is why washing machines are usually on the ground floor in UK homes - usually in the kitchen, where they're close to the plumbing, despite there being very little overlap between food preparation and washing clothes.