I’ve decided to create a new SQL documentation website for H2: sqldock.com/h2
Why?

1. I was tired of scrolling the large page of functions to find the funciton I wanted. I also wanted examples of that funciton being used.
2. For qStudio we provide tooltips that say what a function does AND links to actual vendor documentation. Well every so often vendors move those docs. By hosting it myself I can guarantee the doc link won’t be broken and that I won’t have to change links every other year. I can standardise the path and always send users there for docs.

Hopefully others find it useful. Let me know what you think.

Everyone seems to jump straight to hibernate and not consider alternatives and when to use each. Let’s look at this afresh. We want to create a SAAS or self-hosted app and for any modified entites to be persisted. Entites being users, teams and widgets. Imagine something like a simple github json API server where users belong to a team, users or teams may have repos etc.

Typical Operations

• Create a user/team.
• Get a user and the list of teams they are linked to.
• For a given widget, check if the user has access permission.
• Keep database updated as new columns or tables are added.

Options:

• Hibernate – Java Class Driven. Fully mapped objects, generate schema from objects, query everything through repositories i.e. hibernate layer.
• Hibernate Hybrid – For most save items, update items use hibernate. For some complex or optimized selects use raw SQL with automapping to JSON where possible.
  • Automap requires either creating java classes and letting Object->JSON happen
  • Manual conversion / automatic resultset conversion.
• Raw SQL – Thin Wrapper. Create database manually. Insert / Select writing raw SQL tables. Probably use a very thin wrapper that autoconverts resultsets to JSON.
• JOOQ – Database Driven – Based on database generate type safe java access layers.
• Blob Storage (Async) – Literally do not run any SQL commands. Only periodically save down all java state required. I mean you could even serialize everthing to one binary file. Or serialize large chunks and store them as strings to database.

Decision Pivot Points:

• If SQL querying not needed and no relations within or no insight into inner data = Blob
• If many standard CRUD operations over different entities = Hibernate
• If many customized selects = JOOQ – to provide type checking.
• Never RawSQL. You may as well use a very thin wrapper.
• Other options? Write your own wrapper to persistance?

Outcome for me

Unfortunately we already adopted hibernate before giving it much thought. In hindsight we should have evaluated JOOQ as we have 2-3 entities which have modified SQL queries for half the operations.

Update:2023-11-23 I tried to convert parts of code base to JOOQ. JOOQ had lots of features and great docs on versioning, locking, storing, CRUD. The main parts lacking were nicer generted POJOs with primitives for “NOT NULL” and soft deletes. Without soft deletes I can’t use it as primary persistance only for SQL queries to be type safe.

These quotes captures my worry with Hibernate:
“John, I thought we were entering a new era of software development. I thought we were doing a light-year’s jump. But, after four years, it seems like we are still dealing with all the same problems, only from a different angle. I had to learn Hibernate architecture, configuration, logging, naming strat…………….ritance mappings, replicating object between two different datastores, detached objects and automatic versioning, connection release modes, stateless session interface, taxonomy of collection persistence, cache levels, lazy or eager fetching and many, many more. Even with everything I know, it seems like we’ve failed badly. It’s a software fiasco! Ultimate failure! Disaster! Armageddon!”
“so we decided to rewrite it as plain SQL statements. What’s funny is that writing plain SQL was actually the fastest way of doing it, so we decided to do it as our last option.”

I would suggest one idea, to use Blob storage for individual entires that you don’t need to “look inside”. There’s no point converting JSON to java objects to nested entities, if you never need to access those entities within Java. Just store the blob.

Podcasts I listen to:

• The Rest Is Politics + Leading
• Lennys Podcast
• Practical Founder Podcast

Below I will make notes on specific podcasts and what I took away from them:

• Idea to Startup – Find your wedge – You MUST be solving a bleeding neck problem else most people won’t care and won’t tell others.
• AirPlane.dev – SaaS Club 370 – Very similar product to PulseUI. His elevator pitch at start is really slick.
  • About Pulse? What’s it do? Who’s it for? What’s the main problem it’s trying to solve?
  • Half the software in the world is internal tooling, for every line of code behind your favourite app, there’s another line powering internal platforms and operations.
    • Massively important to success of business
    • Chronically not invested in as nobody wants to work on these tools
  • Our goal is to make it faster to build software tooling for these businesses.
  • Founder fit – Will you be happy working on this for 10 years?
  • We are doing all the boring bits…for us it’s fun as we are doing it at scale.
  • “You could build thi syourself but do you really want to”
• Lenny – Wise.com – How to drive word of mouth. Think this was good need to recheck.
• Saastr – Loom – Building flywheel. Users didn’t know they could share URLS! they needed to add a share button.

I’m on quite a few slack channels and I got a message today on the radpanda/kafka slack:

Which if you follow to their website (Aiven):

1. They describe themselves as: “The trusted open source data platform for everyone”
2. They offer hosting on AWS/Azure/Google for all the new hip open source data technologies. Kafka, Flink, Influx, Redis.
3. Under technology partners page, all their partners are hosting firms.
4. They are not partnered with any of the firms that build the software. None of the management team has worked on or in open source.

For end consumers this is good, get reliable hosting from a “big firm” that supplies many other systems that you may use. For bigtech it’s a win as it will be hosted on their cloud. For startup/tech companies this is deadly. Why would anyone wait to host on their platform?

This feels like the next evolution of the spat between technology companies and bigtech aka google/amazon. Previous disagreements have forced some technology companies/startups to change away from open source licensing e.g. Elastic but the big guys also received bad press for doing it. Now with an smaller intermediary party, the bad press has only a positive effect in advertising them more.

Half the work in building a new SAAS offering is building and supporting the general platform, user logins, system backup, recovery etc. If this middle company spreads that cost over multiple offerings and doesn’t pay to contribute any open source back, they can supply it faster and cheaper short term.

As I look at startups or new tech companies like questdb, redpanda, timeplus, materialize, redis, superset, clickhouse….if a company like this starts offering their tech early I think they could get a lot, if not most the business while the other company pays the cost of development. In some cases as we seen they are on the open source slack channel engaging customers away….

What do you think?

I have an old page: legendary quotes.

As I gather new ones, I will add new ideas here and prune them over time:

“Just recently I came to a realization, people and organisations exist in a competency / incompetency matrix.

The competent orgs with incompetent people are your blue chips and “traditional” companies. They ship stuff, have customers, grow along with the overall market and economy and are able to survive most non-existential threats and down-turns and even existential ones from time to time. All, or most, the competency is institutional, recorded in implicit knowledge held by people and various tools and processes. Unless those companies lose that knowledge for some reason, things are good.” – hef19898 – On HN

“When making life decisions, going in the direction of more money can be wise. However, we must keep in mind that when we choose money, we don’t choose much. We just decide to decide later.”

“Upper limit of complexity is infinity? An engineer can hold a whole complicated system in his head and work on it. When he leaves and needs to pass it on, he won’t be able to communicate it all. Complexity we can sustain over time is less than today.” https://www.youtube.com/watch?time_continue=2841&v=pW-SOdj4Kkk

Norris Numbers – average amount of code an untrained programmer can write before he or she hits a wall. Clift estimates this as 1,500 lines. Beyond that the code becomes so tangled that the author cannot debug or modify it without herculean effort.

… the very language that one could use to make a comparison is taken away, and the possibility of benchmarking on a different infrastructure is an endeavor akin to rewriting everything from scratch.
https://cerebralab.com/Is_a_billion-dollar_worth_of_server_lying_on_the_ground

““The choice isn’t between automation and non-automation,” said Erik Brynjolfsson, the director of M.I.T.’s Initiative on the Digital Economy. “It’s between whether you use the technology in a way that creates shared prosperity, or more concentration of wealth.””

“People are taking the piss out of you everyday. They butt into your life, take a cheap shot at you and then disappear. They leer at you from tall buildings and make you feel small. They make flippant comments from buses that imply you’re not sexy enough and that all the fun is happening somewhere else. They are on TV making your girlfriend feel inadequate. They have access to the most sophisticated technology the world has ever seen and they bully you with it. They are The Advertisers and they are laughing at you.

You, however, are forbidden to touch them. Trademarks, intellectual property rights and copyright law mean advertisers can say what they like wherever they like with total impunity.

Fuck that. Any advert in a public space that gives you no choice whether you see it or not is yours. It’s yours to take, re-arrange and re-use. You can do whatever you like with it. Asking for permission is like asking to keep a rock someone just threw at your head.

You owe the companies nothing. Less than nothing, you especially don’t owe them any courtesy. They owe you. They have re-arranged the world to put themselves in front of you. They never asked for your permission, don’t even start asking for theirs.” -Banksy

“Every internal decision about look and function answers the questions “What does the customer need?” and “How can we help them be more productive?” (Not “How can we give them what they’re asking for?” because that isn’t the right question to answer.)” – source.

Articles:

The most plausible explanation I’ve seen for people moving off programming onto different jobs:
https://whoisnnamdi.com/never-enough-developers/

A case where tight feedback loops don’t help.
https://brianlui.dog/2020/05/10/beware-of-tight-feedback-loops/

What Color is Your Function?

THE TYRANNY of STRUCTURELESSNESS
https://www.jofreeman.com/joreen/tyranny.htm

psychohistory is real:
https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0237458&type=printable

code:

digraph G {

subgraph cluster_0 {
label = “Me”;
node [style=filled];
pension salary dividend mbalance ISA stocks expenses;
color=blue
}

subgraph cluster_1 {
node [style=filled];
NIC VAT CapitalGains IncomeTax Corporation;
label = “Gov”;
color=red
}

subgraph cluster_2 {
node [style=filled];
profits revenue cbalance employee;
label = “Company”;
}

mbalance -> pension;
IncomeTax -> pension [label=”tax relief”];
mbalance -> stocks;
stocks -> mbalance;
stocks -> CapitalGains [label=”10% tax”];
ISA -> mbalance;
mbalance -> costs;
revenue -> pension [label=”40K limit”];
revenue -> VAT [label=”14.5% VAT on revenue”];
cbalance -> dividend [label=”pay shareholders”];
“UK customer” -> revenue [label=”pays for services\n+20% VAT”];
customer -> revenue;
revenue -> profits;
profits -> cbalance;
revenue -> costs;
cbalance -> employee;
profits -> Corporation [label=”19% of profits”];
employee -> NIC;
employee -> salary;
salary -> IncomeTax [label=”20-45% tax”];
costs -> expenses [label=””];
pension -> mbalance;
dividend -> mbalance [label=”2K allowance”];
dividend -> IncomeTax;
salary -> mbalance;
mbalance -> ISA [label=”20K annually”];
}