Member-only story

Limited-Memory Broyden-Fletcher-Goldfarb-Shanno Algorithm in ML.NET

A short theoretical introduction to the L-BFGS algorithm and its implementation using ML.NET.

Robert Krzaczyński
Towards Data Science
4 min readMar 8, 2020

By 500photos.com from Pexels

Some time ago I published an article about the implementation of Naive Bayes using ML.NET. Continuing this series today I would like to introduce you to the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno method. I will start with a theory and an explanation of what this method is about and what it is used for.

Some facts

The creator of the algorithm is Jorge Nocedal. It was created at the Optimization Center, a joint venture of Argonne National Laboratory and Northwestern University. The original source code was written in FORTRAN. It is referred to as software for large-scale unconstrained optimization. As you might expect from the name, this method is similar to BFGS, but it uses less memory. Therefore, it is perfect for large datasets.

L-BFGS

This is an algorithm from the Quasi-Newton family of methods. These are algorithms for finding local extrema of functions, which are based on Newton’s method of finding stationary points of functions.

Time for some math

Towards Data Science
Towards Data Science

Published in Towards Data Science

Your home for data science and AI. The world’s leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.

Robert Krzaczyński
Robert Krzaczyński

Written by Robert Krzaczyński

Software Engineer specializing in .NET/C#. Enthusiast of Machine Learning. bush-dev.com

No responses yet

What are your thoughts?