I appreciated this, but it's probably worth mentioning: when you say AlphaFold 3, you're talking about AlphaFold 2.

TFA announces AlphaFold 3.

Post: "Unlike AlphaFold 3, which predicts only a small, preselected subset of ligands, RosettaFold All Atom predicts a much wider range of small molecules"

TFA: "AlphaFold 3...*models large biomolecules such as proteins, DNA and RNA*, as well as small molecules, also known as ligands"

Post: "they also use the structure predicting model as the denoising model in a diffusion process...Presumably, DeepMind is working on this problem as well."

TFA: "AlphaFold 3 assembles its predictions using a diffusion network, akin to those found in AI image generators."

And that tech just got $1b in funding.

Coming up with ideas is cheaper than executing the ideas. Predicting a wide range of molecules okay-ish is cheaper than predicting a small range of molecules very well.

They also had a headline for Alphazero that convinced everyone that they crushed Stockfish and that classical chess engines were stuff of the past, when in fact it was about 50 elo better than the Stockfish version they were testing against, or roughly the same as how much Stockfish improves each year.

That's what I thought. They go from "predicting all of life's molecules" to "it's a 50% improvement...and we HOPE to...transform drug discovery..."

Seems unfortunately typical of Google these days: "Gemini will destroy GPT-4..."

IIRC the next best models all have all been using AlphaFold 2's methodology, so that's still a massive improvement.

Edit: I see now that you're probably objecting to the headline that got edited on HN.

That's pretty good. Based on the previous performance improvements of Alpha-- models, it'll be nearing 100% in the next couple of years.

I suspect https://www.isomorphiclabs.com/ is the reason.

There are 3 basic ways to fund research.

- Taxes - most academic research

- begging - research charities

- profits - companies like Google.

Sometimes the lines get blurred - but I don't think you can expect Google to release as much of their work for free as people who are paid via central taxes.

I'd be willing to bet that OpenFold has an implementation inside of a few months.

https://openfold.io/

Openness and collaboration could have far-reaching implications for public health and well-being but there are lots of aspects of being open

> the community at large

Which community? Not any I'm part of.

The closer it gets to enabling full drug discovery, the closer it also gets to enabling bioterrorism. Taking it to the extreme, if they had the theory of everything, I don't think I'd want it to be made available to the whole world as it is today.

On a related note, I highly recommend The Talos Principle 2, which really made me think about these questions.

I am not aware of anybody currently criticiszing AF2's abilities outside of its training set. In fact the most recent papers (written by crystallographers) they are mostly arguing about atomic-level details of side chains at this point.

problem is biomolecules, are "chaperoned" to fold properly, only specific regions such as, alpha helix, or beta pleatedsheet will fold de novo.

Chaperone (protein)

https://en.wikipedia.org/wiki/Chaperone_(protein)

>So the criticism towards AlphaFold 2 will likely still apply? For example, it’s more accurate for predicting structures similar to existing ones, and fails at novel patterns?

Yes, and there is simply no way to bridge that gap with this technique. We can make it better and better at pattern matching, but it is not going to predict novel folds.

> Makes me feel a bit uncomfortable.

Why? Do compilers which can't bootstrap themselves also make you uncomfortable due to dependencies on pre-built artifacts? I'm not saying you're unjustified to feel that way, but sometimes more abstracted systems are quicker to build and may have better performance than those built from the ground up. Selecting which one is better depends on your constraints and taste

Consider that humans also learn from other humans, and sometimes surpass their teachers.

A bit more comfortable?

Classic essay in this vein:

>Can a biologist fix a radio? — Or, what I learned while studying apoptosis

https://www.cell.com/cancer-cell/pdf/S1535-6108(02)00133-2.p...

>However, if the radio has tunable components, such as those found in my old radio (indicated by yellow arrows in Figure 2, inset) and in all live cells and organisms, the outcome will not be so promising. Indeed, the radio may not work because several components are not tuned properly, which is not reflected in their appearance or their connections. What is the probability that this radio will be fixed by our biologists? I might be overly pessimistic, but a textbook example of the monkey that can, in principle, type a Burns poem comes to mind. In other words, the radio will not play music unless that lucky chance meets a prepared mind.

I’d say it’s always been the case for medicine, when people first used medicines, the intention was never to fully understand what happens, just save a life, eliminate or reduce symptoms.

Now we’ve built explainable systems like computers and software, we try to overlay that onto everything and it might not work.

To quote Alan Watts, humans like to try square out wiggly systems because we’re not great and understanding wiggles.

First time reading a Deep Mind PR? This is literally their modus operandi.

How to make the share price go up…surprised?

Marketing vs. Reality :)

Accuracy can be assessed two main ways: computationally and experimentally. Computationally, they would compare the predicted structures and interactions with known data from databases like PDB (Protein Database). Experimentally, they can use tools like x-ray crystallography and NMR (nuclear magnetic resonance) to obtain the actual molecule structure and compare it to the predicted result. The outcomes of each approach would be fed back into the model for refining future predictions.

https://www.rcsb.org/

We're this much closer to being hacked?

When I competed in CASP 20 years ago (and lost terribly) I predicted that the next step to improve predictions would be to develop empirically fitted force fields to make MD produce accurate structure predictions (MD already uses empirically fitted force fields, but they are not great). This area was explored, there are now better force fields, but that didn't really push protein structure prediction forward.

Another approach is fully differentiable force fields- the idea that the force field function itself is a trainable structure (rather than just the parameters/weights/constants) that can be optimized directly towards a goal. Also explored, produced some interesting results, but nothing that woudl be considered transformative.

The field still generally believes that if you had a perfect force field and infinite computing time, you could directly recapitulate the trajectories of proteins folding (from fully unfolded to final state along with all the intermediates), but that doesn't address any practical problems, and is massively wasteful of resources compared to using ML models that exploit evolutionary information encoded in sequence and structures.

In retrospect I'm pretty relieved I was wrong, as the new methods are more effective with far fewer resources.

This version has Isomorphic Labs far more in the focus of the press release, which seems to be now the commercial arm more or less licensing access out.

The new AlphaFold server does not do everything the paper says AlphaFold 3 says it does. You cannot predict docking with the server! That is the main interest of pharma companies, 'does our medication bind to the target protein?'. From the FAQ: 'AlphaFold Server is a web-service that offers customized biomolecular structure prediction. It makes several newer AlphaFold3 capabilities available, including support for a wider range of molecule type' - that's not ALL AlphaFold3 capabilities. Isomorphic prints the money with those additional capabilities.

It's hilarious that Google says they don't allow this for safety reasons, pure OpenAI fluff. It's just money.

I wonder what the license for RoseTTAFold is. On github you have:

https://github.com/RosettaCommons/RoseTTAFold/blob/main/LICE...

But there's also:

https://files.ipd.uw.edu/pub/RoseTTAFold/Rosetta-DL_LICENSE....

Which is it?

[flagged]

as soon as google tries to think commercially this will shut down so the longer it stays pure research the better. google is bad with productization.

There is a biannual structural prediction contest called CASP [1], in which a set of newly determined structures is used to benchmark the prediction methods. Some of these structures will be "novel", and so can be used to estimate the performance of current methods on predicting "structure that we have no baseline to start with".

CASP-style assessments are something that should done for more research fields, but it's really hard to persuade funders and researchers to put up the money and embargo the data as required.

[1] https://en.wikipedia.org/wiki/CASP

Speaking of physics, we should borrow the quote "Shut up and calculate" to describe the situation: it works so use it now and worry about the explanations later.

I've never trusted those predicted binding energies. If you have predicted a ligand/protein complex and have high confidence in it and want to study the binding energy I really think you should do a full MD simulation, you can pull the ligand-protein complex apart and measure the change in free energy explicitly.

Also, and this is an unfounded guess only, the problem of protein / ligand docking is quite a bit more complex than protein folding - there seems to be a finite set of overall folds used in nature, while docking a small ligand to a big protein with flexible sidechains and even flexible large-scale structures can have induced fits that are really important to know and estimate, and I'm just very sceptical that it's going to be possible to in a general fashion ever predict these accurately by the AI model with the limited training data.

Though you just need some hints, then you can run MD sims on them to see what happens for real.

Thanks for this informative video summary. As a layperson, with a BS in Chemistry, it was quite helpful in understanding main bulletpoints of this accomplishment.

That sucks a bit. I was just wondering why they are touting that 3rd party company in their own blog post, who commercialise research tools, as well. Maybe there are some corporate agreements with them that prevents them from opening the system...

Imagine the goodwill for humanity for releasing these pure research systems for free. I just have a hard time understanding how you can motivate to keep it closed. Let's hope it will be replicated by someone who doesn't have to hide behind the "responsible AI" curtain as it seems they are now.

Are they really thinking that someone who needs to predict 11 structures per day are more likely to be a nefarious evil protein guy than someone who predicts 10 structures a day? Was AlphaFold-2 (that was open-sourced) used by evil researchers?

Also no commercial use, from the paper:

> AlphaFold 3 will be available as a non-commercial usage only server at https://www.alphafoldserver.com, with restrictions on allowed ligands and covalent modifications. Pseudocode describing the algorithms is available in the Supplementary Information. Code is not provided.

Or OpenFold, which is the more literal reproduction of AlphaFold 2: https://github.com/aqlaboratory/openfold

The AI call is rolling fast, I see similarities with cryptography in the 90s.

I have a history to tell for the record, back in the 90s we developed a home banking for Palm (with a modem), it was impossible to perform RSA because of the speed so I contacted the CEO of Certicom which was the unique elliptic curve cryptography implementation at that time. Fast forward and ECC is everywhere.

Not just unfortunate, but doesn't this make it completely untrustable? How can you be sure the data was not modified in any way? How can you verify any results?

in other words, this has been converted to a novelty, and has no use for scientific purposes.

The second amendment prevents the government's overreaching perversion to restrict me from having the ability to print biological weapons from the comfort of my couch.

Google has no such restriction.

The logical consequence is to put all scientific publications under a license that restricts the right to train commercial ai models on them.

Science advances because of an open exchange of ideas, the original idea of patents was to grant the inventor exclusive use in exchange for disclosure of knowledge.

Those who did not patent, had to accept that their inventions would be studied and reverse engineered.

The „as a service“ model, breaks that approach.

This turns it into a tool that deserves to be dethroned by another group, frankly. What a strange choice.

Well, it's because you can design deadly viruses using this technology. Viruses gain entry to living cells via cell-surface receptor proteins whose normal job is to bind signalling molecules, alter their conformation and translate that external signal into the cellular interior where it triggers various responses from genomic transcription to release of other signal molecules. Viruses hijack such mechanisms to gain entry to cells.

Thus if you can design a viral coat protein to bind to a human cell-surface receptor, such that it gets translocated into the cell, then it doesn't matter so much where that virus came from. The cell's firewall against viruses is the cell membrane, and once inside, the biomolecular replication machinery is very similar from species to species, particularly within restricted domains, such as all mammals.

Thus viruses from rats, mice, bats... aren't going to have major problems replicating in their new host - a host they only gained access to because some nation-state actors working in collaboration on such gain-of-function research in at least two labs on opposite sides of the world with funds and material provided by the two largest economic powers for reasons that are still rather opaque, though suspiciously banal...

Now while you don't need something like AlphaFold3 to do recklessly stupid things (you could use directed evolution, making millions of mutatad proteins, throwing them at a wall of human cell receptors and collecting what stuck), it makes it far easier. Thus Google doesn't want to be seen as enabling, though given their prediliction for classified military-industrial contracting to a variety of nation-states, particularly with AI, with revenue now far more important than silly "don't be evil" statements, they might bear watching.

On the positive side, AlphaFold3 will be great for fields like small molecular biocatalysis, i.e. industrial applications in which protein enzymes (or more robust heterogenous catalysts designed based on protein structures) convert N2 to ammonia, methane to methanol, or selectively bind CO2 for carbon capture, modification of simple sugars and amino acids, etc.

If you're a scientist who works in protein folding (or one of those other areas) and strongly believe that science's goal is to produce falsifiable hypotheses, these new approaches will be extremely depressing, especially if you aren't proficient enough with ML to reproduce this work in your own hands.

If you're a scientist who accepts that probabilist models beat interpretable ones (articulated well here: https://norvig.com/chomsky.html), then you'll be quite happy because this is yet another validation of the value of statistical approaches in moving our ability to predict the universe forward.

If you're the sort of person who believes that human brains are capable of understanding the "why" of how things work in all its true detail, you'll find this an interesting challenge- can we actually interpret these models, or are human brains too feeble to understand complex systems without sophisticated models?

If you're the sort of person who likes simple models with as few parameters as possible, you're probably excited because developing more comprehensible or interpretable models that have equivalent predictive ability is a very attractive research subject.

(FWIW, I'm in the camp of "we should simultaneously seek simpler, more interpretable models, while also seeking to improve native human intelligence using computational augmentation")

The frontier in model space is kind of fluid. It's all about solving differential equations.

In theoretical physics, you know the equations, you solve equations analytically, but you can only do that when the model is simple.

In numerical physics, you know the equations, you discretize the problem on a grid, and you solve the constraint defined by the equations with various numerical integration schemes like RK4, but you can only do that when the model is small and you know the equations, and you find a single solution.

Then you want the result faster, so you use mesh-free methods and adaptive grids. It works on bigger models but you have to know the equations, finding a single solution to the differential equations.

Then you compress this adaptive grid with a neural network, while still knowing the governing equations, and you have things like Physics Informed Neural Networks ( https://arxiv.org/pdf/1711.10561 and following papers) where you can bound the approximation error. This method allows solve all solutions to the differential equations simultaneously, sharing the computations.

Then when knowing explicitly your governing equations is too complex, so you assume that there are some governing stochastic equations implicitly, which you learn the end-result of the dynamic with a diffusion model, that's what this alpha-fold is doing.

ML is kind of a memoization technique, analog to hashlife in the game of life, that allows you reuse your past computational efforts. You are free to choose on this ladder which memory-compute trade-off you want to use to model the world.

As a steelman, wouldn't the abundance of infinitely generate-able situations make it _easier_ for us to develop strong theories and models? The bottleneck has always been data. You have to do expensive work in the real world and accurately measure it before you can start fitting lines to it. If we were to birth an e.g. atomically accurate ML model of quantum physics, I bet it wouldn't take long until we have mathematical theories that explain why it works. Our current problem is that this stuff is super hard to manipulate and measure.

It depends whether the value of science is human understanding or pure prediction. In some realms (for drug discovery, and other situations where we just need an answer and know what works and what doesn’t), pure prediction is all we really need. But if we could build an uninterpretable machine learning model that beats any hand-built traditional ‘physics’ model, would it really be physics?

Maybe there’ll be an intermediate era for a while where ML models outperform traditional analytical science, but then eventually we’ll still be able to find the (hopefully limited in number) principles from which it can all be derived. I don’t think we’ll ever find that Occam’s razor is no use to us.

In case it's not clear, this does not "beat" experimental structure determination. The matches to experiment are pretty close, but they will be closer in some cases than others and may or may not be close enough to answer a given question about the biochemistry. It certainly doesn't give much information about the dynamics or chemical perturbations that might be relevant in biological context. That's not to pooh-pooh alphafold's utility, just that it's a long way from making experimental structure determination unnecessary, and much much further away from replacing a carefully chosen scientific question and careful experimental design.

It means we now have an accurate surrogate model or "digital twin" that can be experimented on almost instantaneously. So we can massively accelerate the traditional process of developing mechanistic understanding through experiment, while also immediately be able to benefit from the ability to make accurate predictions, even without needing understanding.

In reality, science has already pretty much gone this way long ago, even if people don't like to admit it. Simple, reductionist explanations for complex phenomena in living systems don't really exist. Virtually all of medicine nowadays is empirical: try something, and if you can prove its safe and effective, you keep doing it. We almost never have a meaningful explanation for how it really works, and when we think we do, it gets proven wrong repeatedly, while the treatment keeps working as always.

It makes me think about how Einstein was famous for making falsifiable real-world predictions to accompany his theoretical work. And, sometimes it took years for proper experiments to be run (such as measuring a solar eclipse during the breakout of a world war).

Perhaps the opportunity here is to provide a quicker feedback loop for theory about predictions in the real world. Almost like unit tests.

Many of our existing physical models can be decomposed into "high-confidence, well tested bit" plus "hand-wavy empirically fitted bit". I'd like to see progress via ML replacing the empirical part - the real scientific advancement then becomes steadily reducing that contribution to the whole by improving the robust physical model incrementally. Computational performance is another big influence though. Replacing the whole of a simulation with an ML model might still make sense if the model training is transferrable and we can take advantage of the GPU speed-ups, which might not be so easy to apply to the foundational physical model solution. Whether your model needs to be verified against real physical models depends on the seriousness of your use-case; for nuclear weapons and aerospace weather forecasts I imagine it will remain essential, while for a lot of consumer-facing things the ML will be good enough.

"Best methods" is doing a lot of heavy lifting here. "Best" is a very multidimensional thing, with different priorities leading to different "bests." Someone will inevitably prioritize reliability/accuracy/fidelity/interpretability, and that's probably going to be a significant segment of the sciences. Maybe it's like how engineers just need an approximation that's predictive enough to build with, but scientists still want to understand the underlying phenomena. There will be an analogy to how some people just want an opaque model that works on a restricted domain for their purposes, but others will be interested in clearer models or unrestricted/less restricted domain models.

It could lead to a very interesting ecosystem of roles.

Even if you just limit the discussion to using the best model of X to design a better Y, limited to the model's domain of validity, that might translate the usage problem to finding argmax_X of valueFunction of modelPrediction of design of X. In some sense a good predictive model is enough to solve this with brute force, but this still leaves room for tons of fascinating foundational work. Maybe you start to find that the (wow so small) errors in modelPrediction are correlated with valueFunction, so the most accurate predictions don't make it the best for argmax (aka optimization might exploit model errors rather than optimizing the real thing). Or maybe brute force just isn't computationally feasible, so you need to understand something deeper about the problem to simplify the optimization to make it cheap.

Physicists like to retroactively believe that our understanding of physical phenomena preceded the implementation of uses of those phenomena, when the reality is that physics has always come in to clean up after the engineers. There are some rare exceptions, but usually the reason that scientific progress can be made in an area is that the equipment to perform experiments has been commoditized sufficiently by engineering demand for it.

We had semiconductors and superconductors before we understood how they worked -- on both cases arguably we still don't completely understand the phenomena. Things like the dynamo and the electric motor were invented by practice and later explained by scientists, not derived from first principles. Steam engines and pumps were invented before we had the physics to describe how they worked.

The most moneyed and well-coordinated organizations have honed a large hammer, and they are going to use it for everything, and so almost certainly future big findings in the areas you mention, probabilistically inclined models coming from ML will be the new gold standard.

But yet the only thing that can save us from ML will be ML itself because it is ML that has the best chance to be able to extrapolate patterns from these blackbox models to develop human interpretable models. I hope we do dedicate explicit effort to this endeavor, and so continue the human advances and expanse of human knowledge in tandem with human ingenuity with computers at our assistance.

It's interesting to compare this situation to earlier eras in science. Newton, for example, gave us equations that were very accurate but left us with no understanding at all of why they were accurate.

It seems like we're repeating that here, albeit with wildly different methods. We're getting better models but by giving up on the possibility of actually understanding things from first principles.

A new-ish field of "mechanistic interpretability" is trying to poke at weights and activations and find human-interpretable ideas w/in them. Making lots of progress lately, and there are some folks trying to apply ideas from the field to Alphafold 2. There are hopes of learning the ideas about biology/molecular interactions that the model has "discovered".

Perhaps we're in an early stage of Ted Chiang's story "The Evolution of Human Science", where AIs have largely taken over scientific research and a field of "meta-science" developed where humans translate AI research into more human-interpretable artifacts.

A few things:

1. Research can then focus on where things go wrong

2. ML models, despite being "black boxes," can still have brute-force assessment performed of the parameter space over covered and uncovered areas by input information

3. We tend to assume parsimony (i.e Occam's razor) to give preference to simpler models when all else is equal. More complex black-box models exceeding in prediction let us know the actual causal pathway may be more complex than simple models allow. This is okay too. We'll get it figured out. Not everything is closed-form, especially considering quantum effects may cause statistical/expected outcomes instead of deterministic outcomes.

Interesting times indeed. I think the early history of medicines takes away from your observation though. In the 19th and early 20th century people didn't know why medicines worked, they just did. The whole "try a bunch of things on mice, pick the best ones and try them on pigs, and then the best of those and try a few on people" kind of thing. In many ways the mice were a stand in for these models, at the time scientists didn't understand nearly as much about how mice worked (early mice models were pretty crude by today's standards) but they knew they were a close enough analog to the "real thing" that the information provided by mouse studies was usefully translated into things that might help/harm humans.

So when you're tools can produce outputs that you find useful, you can then use those tools to develop your understanding and insights. As a tool, this is quite good.

I asked a friend of mine who is chemistry professor at a large research university something along these lines a while ago. He said that so far these models don't work well in regions where either theory or data is scarce, which is where most progress happens. So he felt that until they can start making progress in those areas it won't change things much.

> What happens when the best methods for computational fluid dynamics, molecular dynamics, nuclear physics are all uninterpretable ML models?

A better analogy is "weather forecasting".

This is the topic of epistemology of the sciences in books such as "New Direction in the Philosophy of Mathematics" [1] and happened before with problems such as the four color theorem [2] where AI was not involved.

Going back to the uninterpretable ML models in the context of AlphaFold 3, I think one method for trying to explain the findings is similar to the experimental methods of physics with reality: you perform experiments with the reality (in this case AlphaFold 3) to came up with sound conclusions. AI/ML is an interesting black-box system.

There are other open discussions on this topic. For example, can our human brain absorbe that knowledge or it is limited somehow with the scientific language that we have now?

[1] https://www.google.com.ar/books/edition/New_Directions_in_th...

[2] https://en.wikipedia.org/wiki/Four_color_theorem

In physics, we already deal with the fact that many of the core equations cannot be analytically solved for more than the most basic scenarios. We've had to adapt to using approximation methods and numerical methods. This will have to be another place where we adapt to a practical way of getting results.

Reminds me of the novel Blindsight - in it there's special individuals who work as synthesists, whos job it is to observe and understand and then somehow translate back to "lay person" the seemingly undecipherable actions/decisions of advanced computers and augmented humans.

I'd say it's not new. Take fluid dynamics as an example, the navier stokes equations predict the motion of fluids very well but you need to approximately solve them on a computer in order to get useful predictions for most setups. I guess the difference is the equation is compact and the derivation from continuum mechanics is easy enough to follow. People still rely on heuristics to answer "how does a wing produce lift?". These heuristic models are completely useless at "how much lift will this particular wing produce under these conditions?". Seems like the same kind of situation. Maybe progress forward will look like producing compact models or tooling to reason about why a particular thing happened.

I think it likely that instead of replacing existing methods, we will see a fusion. Or rather, many different kinds of fusions - depending on the exact needs of the problems at hand (or in science, the current boundary of knowledge). If nothing else then to provide appropriate/desirable level of explainability, correctness etc. Hypothetically the combination will also have better predictive performance and be more data efficient - but it remains to be seen how well this plays out in practice. The field of "physics informed machine learning" is all about this.

Is alphafold doing model generation or is it just reducing a massive state space?

The current computational and systems biochemistry approaches struggle to model large biomolecules and their interactions due to the large degrees of freedom of the models.

I think it is reasonable to rely on statistical methods to lead researchers down paths that have a high likelihood of being correct versus brute forcing the chemical kinetics.

After all chemistry is inherently stochastic…

Our metaphors and intuitions were crumbling already and stagnating. See quantum physics: sometimes a particle, sometimes a wave, and what constitute a measurement anyway?

I’ll take prediction over understanding if that’s the best our brains can do. We’ve evolved to deal with a few orders of magnitude around a meter and a second. Maybe dealing with light-years and femtometer/seconds is too much to ask.

> Does this decouple progress from our current understanding of the scientific process - moving to better and better models of the world without human-interpretable theories and mathematical models / explanations?

Replace "human-interpretable theories" with "every man interpretable theories", and you'll have a pretty good idea of how > 90% of the world feels about modern science. It is indistinguishable from magic, by the common measure.

Obtuse example: My parents were alive when the first nuclear weapon was detonated. They didn't know that they didn't know this weapon was being built, let alone that it might have ignited the atmosphere.

With sophisticated enough ML, that 90% will become 99.9% - save the few who have access to (and can trust) ML tools that can decipher the "logic" from the original ML tools.

Yes, interesting times ahead... indeed.

"better and better models of the world" does not always mean "more accurate" and never has.

We already know how to model the vast majority of things, just not at a speed and cost which makes it worthwhile. There are dimensions of value - one is accuracy, another speed, another cost, and in different domains additional dimensions. There are all kinds of models used in different disciplines which are empirical and not completely understood. Reducing things to the lowest level of physics and building up models from there has never been the only approach. Biology, geology, weather, materials all have models which have hacks in them, known simplifications, statistical approximations, so the result can be calculated. It's just about choosing the best hacks to get the best trade off of time/money/accuracy.

This is a key but secondary concern to many of us working in molecular geneticist who will use AlphaFold 3 to evaluate pair-wise interactions. We often have genetic support for an interaction between proteins A and B. For example, in a study of genetic variation in responses of mice to morphine I currently have two candidate proteins that interact epistatically, suggesting a possible “lock and key” model—-the mu opiate receptor (MOR) and FGF12. I can now evaluate the likelihood of a direct molecular interaction between these proteins and possible amino acids substitutions that account for individuals difference.

In other words I bring a hypothesis to AF3 and ask for it to refute or affirm.

For me the big question is how do we confidently validate the output of this/these model(s).

You are conflating the whole scientific endeavor to a very specific problem to which this specific approach is effective at producing results that fit with the observable world. This has nothing to do with science as a whole.

My argument is: weather.

I think it is fine & better for society to have applications and models for things we don't fully understand... We can model lots of small aspects of weather, and we have a lot of factors nailed down, but not necessarily all the interactions.. and not all of the factors. (Additional example for the same reason: Gravity)

Used responsibly. Of course. I wouldn't think an AI model designing an airplane that no engineers understand how it works is a good idea :-)

And presumably all of this is followed by people trying to understand the results (expanding potential research areas)

We need to advance mechanistic interpretability (field reverse engineering neural networks) https://www.youtube.com/watch?v=P7sjVMtb5Sg https://www.youtube.com/watch?v=7t9umZ1tFso https://www.youtube.com/watch?v=2Rdp9GvcYOE

To paraphrase Kahan, it's not interesting to me whether a method is accurate enough or not, but whether you can predict how accurate you can be. So, if ML methods can predict that they're right 98% of times then we can build this in our systems, even if we don't understand how they work.

Deterministic methods can predict result with a single run, ML methods will need ensemble of results to show the same confidence. It is possible at the end of day that the difference in cost might not he that high over time.

Science has always given us better, but error prone tooling to see further and make better guesses. There is still a scientific test. In a clinical trial, is this new drug safe and effective.

Perhaps an ai can be made to produce the work as well as a final answer, even if it has to reconstruct or invent the work backwards rather than explain it's own internal inscrutable process.

"produce a process that arrives at this result" should be just another answer it can spit out. We don't necessarily care if the answer it produces is actually the same as what originally happened inside itself. All we need is that the answer checks out when we try it.

No, science doesn't work that way. You can just calculate your way to scientific discoveries, you got to test them in the real world. Learning, both in humans and AI, is based on the signals provided by the environment. There are plenty of things not written anywhere, so the models can't simply train on human text to discover new things. They learn directly from the environment to do that, like AlphaZero did when it beat humans at Go.

In order for that not to happen (uninterpretable ML models) some research on symbolic distillation, aka symbolic regression

https://arxiv.org/abs/2006.11287

https://www.science.org/doi/10.1126/sciadv.aay2631

I think at some point, we will be able to produce models that are able to pass data into a target model and observe its activations and outputs and put together some interpretable pattern or loose set of rules that govern the input-output relationship in the target model. Using this on a model like AlphaFold might enable us to translate inferred chemical laws into natural language.

Even if we don’t understand the models themselves, you can still use them as a basis for understanding

For example, I have no idea how a computer works in every minute detail (ie, exactly the physics and chemistry of every process that happens in real time), but I have enough of an understanding of what to do with it, that I can use it as an incredibly useful tool for many things

Definitely interesting times!

> Does this decouple progress from our current understanding of the scientific process?

Thank God! As a person who uses my brain, I think I can say, pretty definitively, that people are bad at understanding things.

If this actually pans out, it means we will have harnessed knowledge/truth as a fundamental force, like fire or electricity. The "black box" as a building block.

I believe it simply tells us that our understanding of mechanical systems, especially chaotic ones, is not as well defined as we thought.

https://journals.aps.org/prresearch/abstract/10.1103/PhysRev...

> What happens when...

I can only assume that existing methods would still be used for verification. At least we understand the logic used behind these methods. The ML models might become more accurate on average but they could still throw out results that are way off occasionally, so their error rate would have to become equal to the existing methods.

I wonder if ML can someday be employed in deciphering such black box problems; a second model that can look under the hood at all the number crunching performed by the predictive model, identify the pattern that resulted in a prediction, and present it in a way we can understand.

That said, I don’t even know if ML is good at finding patterns in data.

The models are learning an encoding based on evolutionary related and known structures. We should be able to derive fundamental properties from those encodings eventually. Or at least our biophysical programmed models should map into that encoding. That might be a reasonable approach to look at the folding energy landscape.

Perhaps related, the first computer-assisted mathematics proof: https://en.wikipedia.org/wiki/Four_color_theorem

I'm sure that similar arguments for and against the proof apply here as well.

In terms of docking, you can call the conventional approaches "physically-based", however, they are rather poor physical models. Namely, they lack proper electrostatics, and, most importantly, basically ignore entropic contributions. There is no reason for concern.

I can only hope the models will be sophisticated enough and willing to explain their reasoning to us.

Might be easier to come up with new models with analytic solutions if you have a probabilistic model at hand. A lot easier to evaluate against data and iterate. Also, I wouldn't be surprised if we develop better tools for introspecting these models over time.

Perhaps for understanding the structure itself, but having the structure available allows us to focus on a coarser level. We also don't want to use quantum mechanics to understand the everyday world, and that's why we have classic mechanics etc.

These processes are both beyond human comprehension because they contain vast layers of tiny interactions and also not practical to simulate. This tech will allow for exploration for accurate simulations to better understand new ideas if needed.

I'm not a scientist by any means, but I imagine even accurate opaque models can be useful in moving the knowledge forward. For example, they can allow you to accurately simulate reality, making experiments faster and cheaper to execute.

We could be entering a new age of epicycles - high accuracy but very flawed understanding.

There will be an iterative process built around curated training datasets - continually improved, top tier models, teams reverse engineering the model's understanding and reasoning, and applying that to improve datasets and training.

This is a neat observation. Slightly terrifying, but still interesting. Seems like there will also be cases where we discover new theories through the uninterpretable models—much easier and faster to experiment endlessly with a computer.

I think it creates new studies, such as diagnosing these models behaviors without the doctor having an intricate understanding of all of the model's processes/states just like with natural organisms

As a tool people will use it as any other tool, by experimenting, testing, tweaking and iterating.

As a scientific theory for fundamentally explaining the nature of the universe, maybe it won't be as useful.

I would assume that given enough hints from AI and if it is deemed important enough humans will come in to figure out the “first principles” required to arrive at the conclusion.

every time the two systems disagree, it's an opportunity to learn something. both kinds of models can be improved with new information, done through real-world experiments

Hook the protein model up to an LLM model, have the LLM interpret the results. Problem solved :-) Then we just have to trust the LLM is giving us correct interpretations.

We will get better with understanding black boxes, if a model can be compressed into simple math formula then it's both easier to understand and to compute.

Is it capable of predictions though? Ie can it accurately predict the folding of new molecules? Otherwise how do you distinguish accuracy from overfitting.

What happens if we get to the stage of being able to simulate every chemical and electrical reaction in a human brain, is doing this torture or wrong?

Whatever it is if we needed to we could follow each instruction through the black box. It’s never going to be as opaque as something organic.

Next decade we will focus on building out debugging and visualization tools for deep learning , to glance inside the current black box