Solving the 1+N Query Problem
Comments
vilterp
kstrauser
Side note: I strongly prefer referring to this as the "1+N problem" as the author did here. I didn't understand what people were grousing about when they talked about "N+1".
N+1: You're already doing N queries. Is adding 1 more that big of a deal?
1+N: This should have been 1 query, but somehow you blew it up into that one plus N more.
I'd seen that query antipattern plenty of times and knew what it was bad, but didn't realize that's what people meant by "N+1", which I thought must mean something different.
libria
You're not the only one. I never stopped to delve into what this N+1 problem was b/c I assumed it was never an issue for me. All these years and this is the 1st time I've finally understood what they were saying.
However, after going back and forth with LLM on it just now, I feel like "1+N" is just a coding mistake, not a perplexing multi-faceted, engineering problem to be solved. Experience or a slow application would teach you to find a better way to get that info and then you move on.
red_admiral
This is what you get when you let your ORM loose on the database without understanding JOINs. Especially, the bit where something like 'book.author.name' that looks like a simple field dereference actually is a method call on an ORM proxy object (book), via python's __getattr__ or similar, that fires off a new query if the data you want is not loaded yet.
Some ORMs let you specify the extent of the data that you want, like Hibernate has its own Hibernate Query Language.
At some point you are better off just writing SQL yourself, though. Even without join problems, if you ask an ORM to get the person with user id 123 and all you want is their name, the ORM cannot know that unless to tell it, and so you end up with a 'SELECT *' type query.
tehlike
This is true but not super true in case of linq and related providers like efcore. Even nhibernate linq would do this.
cnity
ORMs are great. They make the easy queries remain easy and the harder queries impossible.
quibono
Nice, and I understand why using `getAuthorNames` solves the N+1 here.
But... isn't this solving the problem by removing most of what makes it an issue in the first place? I imagine most people use ORMs for the SQL <-> native class data sync capability. And this assumes one would run the Acadia query instead.
FWIW I'm not trying to be negative, it's just my general impression is that these N+1 usually occur because people _want_ direct object access and _want_ to write loops, and _want_ to access fields and have the underlying SQL be sorted by the ORM.
lobofta
As far as I understand Acadia gives you Acadia <-> Native class data sync, only just Haskell and Elm at the moment unfortunately.
I'd be willing to rewrite queries in some other language that transpiles to SQL if it allows me to do all the queries I want and gives me full compile time type support for db access in return.
The policies look interesting too by the way, but they don't solve a major IMO.
cnity
This is my experience too. The solution is to train people to stop wanting to solve data query problems in the application layer.
mrkeen
Compare with the prior art of N+1 queries of 2014:
https://github.com/facebook/Haxl/blob/main/example/sql/readm...
jbverschoor
Wasn't this 'solved' by Hibernate ages ago?
> [Datalog] is a subset of Prolog that lacks recursion
Datalog does allow for recursion — a common example is graph reachability:
reachable(a, b) :- edge(a, b). reachable(a, c) :- edge(a, b), reachable(b, c).
'Base datalog' guarantees termination by requiring all input relations to be finite. Notably this means that it doesn't have numerical operations like addition or multiplication, since `plus(a, b)` or `times(a, b)` would be infinite relations.
More practical Datalog engines like Souffle (https://souffle-lang.github.io/) have numerical operations but don't guarantee termination.
Recursive queries are not needed by most applications, but maybe Acadia could allow them (compiling to recursive CTEs) by proving that recursion only goes through finite relations.