Neuroevolution - genetic algorithms for neural networks

I recently gave a workshop at the International Summer School on Deep Learning in Gdańsk, Poland. It was a hands-on introduction to neuroevolution, a technique which attempts to find the optimal architecture and hyperparameters for a neural network using genetic algorithms.

Workshop exercises

You can find materials for the workshop exercises here.

Workshop recording

References

Stanley, K. O., & Miikkulainen, R. (2002). Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation, 10(2), 99–127. doi.org/10.1162/106365602320169811

Stanley, K. O., & Miikkulainen, R. (2002). Efficient Evolution of Neural Network Topologies. In Proceedings of the 2002 Congress on Evolutionary Computation, CEC 2002 (Vol. 2, pp. 1757–1762). IEEE. doi.org/10.1109/CEC.2002.1004508

Stanley, K. O., & Miikkulainen, R. (2002). Efficient Reinforcement Learning Through Evolving Neural Network Topologies. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2002), pp. 569–577. doi.org/10.1109/CEC.2002.1004508

Gauci, J., & Stanley, K. O. (2010). Autonomous Evolution of Topographic Regularities in Artificial Neural Networks. Neural Computation, 22(7), 1860–1898. doi.org/10.1162/neco.2010.06-09-1042

Stanley, K. O., D'Ambrosio, D. B., & Gauci, J. (2009). A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks. Artificial Life, 15(2), 185–212. doi.org/10.1162/artl.2009.15.2.15202

Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., … Hodjat, B. (2017). Evolving Deep Neural Networks. arxiv.org/abs/1703.00548

Zoph, B., & Le, Q. V. (2016). Neural Architecture Search with Reinforcement Learning. arxiv.org/abs/1611.01578

Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., & Dean, J. (2018). Efficient Neural Architecture Search via Parameter Sharing. arxiv.org/abs/1802.03268

Wistuba, M. (2019). Deep Learning Architecture Search by Neuro-Cell-Based Evolution with Function-Preserving Mutations. In Lecture Notes in Computer Science (Vol. 11052 LNAI, pp. 243–258). doi.org/10.1007/978-3-030-10928-8_15

Liang, X., Lin, L., Shen, X., Feng, J., Yan, S., & Xing, E. P. (2017). Interpretable Structure-Evolving LSTM. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), pp. 1010–1019. doi.org/10.1109/CVPR.2017.234