## Tackling the curse of dimensionality with physics-informed neural networks

77 points
1/20/1970
6 days ago
by jhoho

### SeanAnderson

> For instance, we solve nontrivial nonlinear PDEs (one HJB equation and one Black-Scholes equation) in 100,000 dimensions in 6 hours on a single GPU using SDGD with PINNs

100,000 dimensions? I thought there were like... 11 tops? https://imagine.gsfc.nasa.gov/science/questions/superstring....

(edit: oops, I misunderstood the context of dimensions here. my bad. thanks)

6 days ago

### Legend2440

The way ML people use "dimension", each free parameter is an extra dimension. A high-resolution 2D image is considered to have millions of dimensions - one for each pixel.

6 days ago

### tnecniv

That’s not just ML, that’s linear algebra. Moreover, across fields in math, the dimension of an object is defined differently, but it’s always a very fundamental property of an object that captures loosely how complex an object is. Often, that complexity is related to how many numbers you need to describe said object.

6 days ago

### reedf1

These are all expressions of the same concept. The confusion is that physicists are describing the dimensionality of specific systems (space, spacetime, superstring theory, supergravity) - that doesn't mean that this limits the dimensionality of other systems (which is often where confusion lies between laymen).

6 days ago

### atoav

And each pixel is what, a RGB-vector?

6 days ago

### PartiallyTyped

We usually do HxWxC, for height, width, and channels, so each pixel is addressed via the two first dims of the input, and then it has 3 channels. Of course, you can transpose the tensor to CxHxW or CxWxH. Different ordering behaves differently with respect to memory locality.

6 days ago

### dartos

In the context of neural networks, kind of. It’s 3 numbers on the input layer, but that could influence N parameters in later layers.

6 days ago

### polynomial

Yes, each pixel is a vector of the rasterized vector space.

5 days ago

### hommelix

Thanks for clarifying the dimensions.

6 days ago

### relyks

Physics uses vectors, which are multi-dimensional and can represent a lot more than 11 dimensions... Machine learning typically uses feature vectors, which are basically just lists of numerical properties.

6 days ago

### lapinot

To add to the other answers: dimensions also appear in higher-order PDE: you typically go from an nth order 1-dimensional equation (an equation mentioning the n-th derivative of your unknown function) to an n-dimensional 1st order equation. There is a general pattern of adding parameters to have several simple problems instead of one complicated.

6 days ago

### Eliezer

Iiii do not think you should be able to solve the Schrodinger equation with thousands of dimensions in general on a non-quantum computer, what with that being a quantum-mechanical equation some of whose solutions would reflect quantum-hard problems?

6 days ago

### bowsamic

What? The Schrodinger equation is just a linear differential equation. I can solve it here right now by hand if the Hamiltonian is time-independent, even for millions of dimensions.

|psi(t)> = exp(-iHt/hbar) |psi(t=0)>

No offence but this sounds like you've never actually studied any quantum mechanics. The difficulty of solving such equations is purely just the difficulty of solving any complex set of PDEs, as the article talks about. There is nothing about the Schrodinger equation that means it's not computable on a classical computer (EDIT nor would it be easier on a quantum computer).

6 days ago

### scentoni

Calculating the mean field of a thousand electrons is easy for a classical computer. Calculating the exchange and correlation energies of a thousand interacting electrons is not. Quantum computers would have an enormous advantage there.

6 days ago

### selimthegrim

The time evolution or trotter evolution has some advantages on a quantum computer

6 days ago

### bowsamic

There of course may be advantages to specific calculations on quantum computers but that isn't because the Schroedinger equation itself is somehow "a quantum equation"

6 days ago

### abecedarius

The example they try it on is a quantum harmonic oscillator, i.e. the potential V(x) is just the squared norm of x. That's analytically solvable. Also,

> in the case of the Schro ̈dinger equation, due to the separable nature of its network structure, we employ a separate neural network for each dimension.

which doesn't sound very general. Admittedly in my quick skim of this paper I didn't follow much.

6 days ago