Adaptive Meta-heuristically Intelligent Particle (AMI-P)

Red Stars — Track of best solutions. Blue Star — AMI-Particle

The AMI Particle

AMI-P is my newly discovered optimization method. The AMI Particle iteratively searches for a global optimum using a single particle which intelligently learns and adapts to both convex and non-convex n-dimensional search spaces.

As shown below, AMI-P converges to the global optimum in both unimodal and multimodal objective functions. This research is due in part to a mathematical identity I’ve discovered. This identity grants the AMI particle a balance of exploration and exploitation by imploring adaptive n-dimensional rotations constrained within the feasible search space of the objective function.

A unique, but notable, trait of the AMI particle is that it’s derivative-free; hence, AMI-P is gradient-free. This allows the AMI particle to converge in higher dimensions in linear-time with respect to the dimensions. Below Animated Performance section are performance results for 100,000- & 1,000,000-dimension unimodal and multimodal functions.

All results shown below are the worst-case performances after 200 individual runs on a MacBook Pro using Matlab. These are very promising results for the field of Applied Mathematics. Mathematical optimization methods are key factors in Artificial Intelligence (neural networks), Mechanics, Molecular Engineering, Civil Engineering, Operations Research, Geophysics, Control Engineering, Electrical Engineering, and, last but not least, Economics and Finance.

Animated Performance of AMI-Particle

For each test function, a 2-dimensional animated performance plot was generated using Matlab. The animation shows the particle exploring the search space while remaining ‘attracted’ to the areas it learns to exploit.

The particle converges to the global optimum less than ~0.1/s each time; however, I slowed down the animated GIFs for demonstration purposes.

Below the animated plots are AMI-Particle performance results in 100,000- and 1,000,000-dimension tests for the Sphere, Rastrigin, and Griewank functions.

AMI-Particle — Sphere Function

https://www.sfu.ca/~ssurjano/griewank.html

AMI Particle Sphere Test — 2-D where xi ∈ [-100, 100], for all i = 1, …, D (100 epochs)

AMI-Particle — Rastrigin Function

https://www.sfu.ca/~ssurjano/rastr.html

AMI Particle Rastrigin Test — 2-D where xi ∈ [-5.12, 5.12], for all i = 1, …, D (100 epochs)

AMI-Particle — Griewank Function

https://www.sfu.ca/~ssurjano/griewank.html

AMI Particle Griewank Test — 2-D where xi ∈ [-600, 600], for all i = 1, …, D (250 epochs)

AMI-Particle — Goldstein-Price Function

https://www.sfu.ca/~ssurjano/goldpr.html

AMI Particle Goldstein-Price Test — 2-D where xi ∈ [-2, 2], for all i = 1, 2 (300 epochs)

AMI-Particle — Dimensionality Tests

100,000-dimension Convergence Tests

AMI-Particle 100,000-d Sphere Test — Worst-Case Performance Results of 200 Runs (~ 3/s)

AMI-Particle 100,00-d Rastrigin Test — Worst-Case Performance Results of 200 Runs (~ 3/s)

AMI-Particle 100,000-d Griewank Test — Worst-Case Performance Results of 200 Runs (~3/s)

1,000,000-dimension Convergence Tests

AMI-Particle 1,000,000-d Sphere Test — Worst-Case Performance Results of 200 Runs (~ 9/s)

AMI-Particle 1,000,000-d Rastrigin Test — Worst-Case Performance Results of 200 Runs (~ 9/s)

AMI-Particle 1,000,000-d Griewank Test — Worst-Case Performance Results of 200 Runs (~ 9/s)

What’s Next?

With the plethora of real-world problems solved using different mathematical optimization methods increasing, the promising results of the Adaptive Meta-heuristically Intelligent Particle method are motivating to continue with my research. I believe this method could drastically improve performance and decrease computational resources needed for neural networks in Artificial Intelligence and Deep Learning.