Optimization of Neural Networks with Linear Solvers

How to optimize nonlinear neural networks in more than one dimension using linear solvers.

Tim Forster
Towards Data Science

Photo by Sam Moghadam Khamseh on Unsplash

Note: If you don’t have a medium subscription, you can read the article for free here (if you have a subscription, please continue reading here, thanks! 🥰)

Recently, I stumbled over a problem that required me to create a model which takes more than one input feature and predicts a continuous output.

Then, I needed to get the best possible output from that model, which in my case was the lowest possible numerical value. So, in other words, I needed to solve an optimization problem.

The issue was (and I only realized it at that stage) that the environment I was working in did not allow me to use nonlinear things or sophisticated frameworks— so no neural networks, no nonlinear solvers, nothing...

But, the model I created worked well (considering the low number of data points I trained it on), and I did not want to delete all my codes and start from scratch with a linear model.

So, after a cup of coffee ☕, I decided to use this nonlinear model I already trained to generate a number of small linear ones. Then I could use a linear solver to solve the optimization problem.

Create an account to read the full story.

The author made this story available to Medium members only.
If you’re new to Medium, create a new account to read this story on us.

Or, continue in mobile web

Already have an account? Sign in

Responses (3)

What are your thoughts?