Difference between revisions of "GP SSM"

From Robotics
Jump to: navigation, search
(Papers which are more oriented toward fluids)
m (Basic Gaussian Process Info)
 
(32 intermediate revisions by 2 users not shown)
Line 7: Line 7:
  
 
=== Basic Gaussian Process Info ===
 
=== Basic Gaussian Process Info ===
* Rasmussen and Williams
+
* [[Media:RasumussenWilliamsBook.pdf | Gaussian Processes For Machine Learning]], by Rasmussen and Williams
 +
* [[Media:ModellingControlDynamicSystemsGPs.pdf | Modeling and Control of Dynamic Systems Processes Using Gaussian Processes Models]] by J. Kocijan
  
 
=== Web Links ===
 
=== Web Links ===
Line 36: Line 37:
 
* R. Frigola, F. Lindsten, T.B. Schon, C.E. Rasmussen, [[Media:IdentificationGPSSMwithParticalEM.pdf | Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM]]
 
* R. Frigola, F. Lindsten, T.B. Schon, C.E. Rasmussen, [[Media:IdentificationGPSSMwithParticalEM.pdf | Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM]]
 
<hr />
 
<hr />
 +
* Z.Y. Wan and T.P. Sapsis, [[Media:ReducedSpaceGPR.pdf | Reduced-Space Gaussian Process Regression for Data-Driven Forecast of Chaotic Dynamical Systems]], ''arXiv:1611.01583''
 
* T. Beckers, J. Umlauft, and S. Hirsche, [[Media:ModelBasedGPRControl.pdf | Stable Model-Based Control with Gaussian Process Regression for Robot Manipulators]],
 
* T. Beckers, J. Umlauft, and S. Hirsche, [[Media:ModelBasedGPRControl.pdf | Stable Model-Based Control with Gaussian Process Regression for Robot Manipulators]],
 
* A. Marco, P. Hennig, S. Schaal, S. Trimpe, [[Media:DesignLQRKernels.pdf | On the Design of LQR Kernels for Efficient Controller Learning]], ''arXiv:1709.07089v1''
 
* A. Marco, P. Hennig, S. Schaal, S. Trimpe, [[Media:DesignLQRKernels.pdf | On the Design of LQR Kernels for Efficient Controller Learning]], ''arXiv:1709.07089v1''
Line 44: Line 46:
 
== Deep Learning ==
 
== Deep Learning ==
  
=== Papers on Deep Learning ===
+
=== Papers on Deep Learning ''Theory'' ===
* Soatto Paper
+
* This is the paper that we seek to understand in this reading group.
 +
** A. Achille and S. Soatto, [[Media:InvarianceDisentanglement.pdf | Emergence of Invariance and Disentanglement in Deep Representations]], ''arXiV:1706.01350v2'', Oct. 2017.
 +
* Here are some background papers (i.e., links to many of the references in the Achille and Soatto paper)
 +
** A. Achille and S. Soatto [https://arxiv.org/pdf/1611.01353.pdf  Information Dropout: Learning Optimal Representations Through Noisy Computation]
 +
** A. Alemi, I. Fischer, K. Dillon, and K. Murphey, [https://arxiv.org/abs/1612.00410 Deep Variational Information Bottleneck], ''arXiV:1612.00410''
 +
** F. Anselmi, L. Rosasco, T. Poggio, [https://arxiv.org/pdf/1503.05938.pdf On Invariance and Selectivity in Representation Learning]
 +
** Y. Bengio [https://www.iro.umontreal.ca/~lisa/pointeurs/TR1312.pdf Learning Deep Architectures for AI], 2009.
 +
** J. Bruna and S. Mallat, [http://www.cmap.polytechnique.fr/scattering/scattering_cvpr2011.pdf Classification with Scattering Operators], ''CVPR'', 2011.
 +
** P. Chaudhari, A. Choromanska, S. Soatto, Y. LeCun, C. Baldassi, C. Borgs, J. Chayes, L. Sagun, R. Zecchina [https://arxiv.org/abs/1611.01838 Entropy-SGD: Biasing Gradient Descent into Wide Valleys], ''Proceedings of the International Conference on Learning Representations (ICLR)'', 2017.
 +
** N. Tishby and N. Zaslavsky, [[Media:DeepLearningBottleneck.pdf | Deep Learning and the Information Bottleneck Principle]], ''arXiv:1503.02406'', March, 2015.
 +
** D.-A. Clevert, T. Unterthiner, and S. Hochreiter, [https://arxiv.org/abs/1511.07289 Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)], ''arXiv:1511.07289''.
 +
** L. Dinh, R. Pascanu, S. Bengio, Y. Bengio [https://arxiv.org/abs/1703.04933 Sharp Minima Can Generalize For Deep Nets], ''arXiv:1703.04933''
 +
 
 +
* P. Chauderi, A.Oberman, S. Osher, S. Soatto, G. Ccarlier, [[Media:DeepNetPDEs.pdf | Deep Relaxation: Partial Differential Equations for Optimizing Deep Neural Networks]], ''arXiV:1704.04932v2''
 +
 
 +
==== Review-like papers ====
 +
* J. Schmidhuber, [https://arxiv.org/pdf/1404.7828.pdf Deep Learning in Neural Networks: An Overview], ''arXiV:404.7828v4'', Oct. 2014.
 +
* S. Mallat, [[Media:UnderstandingDeepCNNs.pdf | Understanding Deep Convolutional Networks]], ''Phil. Trans. Royal Society, A, vol. 374, May 15, 2017.
 +
* M.D. Zeiler and R. Fergus, [https://arxiv.org/abs/1311.2901  Visualizing and Understanding Deep Convolutional Networks]
 +
 
 +
==== Papers on Scattering Networks ====
 +
''Scattering Networks'' are proposed by Stephan Mallat (of wavelet fame) to understand why deep nets work so well.
 +
* S. Mallat [[Media:GroupInvariantScattering.pdf | Group Invariant Scattering]], 2012.
 +
* J. Anden, S. Mallat, [[Media:DeepScatteringSpectrum.pdf | Deep Scattering Spectrum]], 2015.
 +
* J. Bruna, S. Mallat, [[Media:InvariantScatteringCNNs.pdf | Invariant Scattering Convolution Networks]]
 +
 
 +
==== Classics ====
 +
* K. Hornik, M. Stinchcombe, H. White, [[Media:HSH.pdf | Universal Approximation of an Unknown Mapping and Its Derivatives Using Multi-Layer Feedforward Networks]], ''Neural networks'', vol. 3, 1990.
  
 
=== Web Links ===
 
=== Web Links ===
 +
* [http://deeplearning.net Deeplearning.net]; [http://deeplearningbook.org/ On-line Deep Learning TextBook]
  
 
== Koopman Spectral Method ==
 
== Koopman Spectral Method ==
  
 
=== Papers on Koopman Spectral methods ===
 
=== Papers on Koopman Spectral methods ===
* S. Brunton, J. Proctor, N. Kutz, [[Media:DiscoveringEquationsFromData.pdf | Discovering Governing Equations from Data: Sparse Identification of Nonlinear Dyanmical Systems]], ''arXiv:1509.03580v1''
+
* S. Brunton, J. Proctor, N. Kutz, [[Media:DiscoveringEquationsFromData.pdf | Discovering Governing Equations from Data: Sparse Identification of Nonlinear Dyanmical Systems]], ''arXiv:1509.03580v1'' (also, [[Media:BPK_PNAS.pdf | PNAS Version of the paper]]).
 
* S. L. Brunton, B.W.Brunton, J.L. Proctor, J.N.  Kutz, [[Media:KoopmanInvariantSubspaces.pdf | Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control]], ''PLOS One'', vol. 11, no. 2, 2016.
 
* S. L. Brunton, B.W.Brunton, J.L. Proctor, J.N.  Kutz, [[Media:KoopmanInvariantSubspaces.pdf | Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control]], ''PLOS One'', vol. 11, no. 2, 2016.
 
* M. Budisic, R. Mohr, I. Mezic, [[Media:AppliedKoopmanism.pdf | Applied Koopmanism]], ''Chaos'', vol. 22, 2012.
 
* M. Budisic, R. Mohr, I. Mezic, [[Media:AppliedKoopmanism.pdf | Applied Koopmanism]], ''Chaos'', vol. 22, 2012.
Line 62: Line 92:
 
* I. Mezic and A. Surana, [[Media:PeriodicKoopmanModeDecomposition.pdf | Koopman Mode Decomposition for Periodic/Quasi-Periodic Time Dependence]], ''IFAC Papers Online, 48-18, pp. 690697, 2016.
 
* I. Mezic and A. Surana, [[Media:PeriodicKoopmanModeDecomposition.pdf | Koopman Mode Decomposition for Periodic/Quasi-Periodic Time Dependence]], ''IFAC Papers Online, 48-18, pp. 690697, 2016.
 
* H. Schaeffer, [[Media:LearningPDEs.pdf | Learning Partial Differential Equations via Data Discovery and Sparse Optimization]], ''J. Royal Society, Proceedings A'', 2017
 
* H. Schaeffer, [[Media:LearningPDEs.pdf | Learning Partial Differential Equations via Data Discovery and Sparse Optimization]], ''J. Royal Society, Proceedings A'', 2017
 +
* J.N. Kutz, X. Fu, S.L. Brunton, [[Media:MultiresolutionDMD.pdf | Multi-resolution Dynamic Mode Decomposition]], ''arXiv:1506.00564''
  
 
==== Papers which are more oriented toward control ====
 
==== Papers which are more oriented toward control ====
Line 67: Line 98:
 
* A. Surana, [[Media:KoopmanObserverBasedSynthesis.pdf | Koopman Based Operator Synthesis for Control-Affine Noninear Systems]], ''Proc. IEEE Conf. Decision and Control'', 2016
 
* A. Surana, [[Media:KoopmanObserverBasedSynthesis.pdf | Koopman Based Operator Synthesis for Control-Affine Noninear Systems]], ''Proc. IEEE Conf. Decision and Control'', 2016
 
* A. Surana and A. Banaszuk, [[Media:LinearObserverSynthesis.pdf | Linear Observer Synthesis for Nonlinear Systems Using Koopman Operator Framework]], ''IFAC Papers Online'', 49-18, pp. 716-723, 2016.
 
* A. Surana and A. Banaszuk, [[Media:LinearObserverSynthesis.pdf | Linear Observer Synthesis for Nonlinear Systems Using Koopman Operator Framework]], ''IFAC Papers Online'', 49-18, pp. 716-723, 2016.
 +
* E. Kaiser, N. Kutz, and S. Brunton, [[Media: sparseMPC.pdf | Sparse identification of nonlinear dynamics for model predictive control in the low-data limit]] ''arXiv preprint'' arXiv:1711.05501 (2017).
  
 
==== Papers which are more oriented toward fluids ====
 
==== Papers which are more oriented toward fluids ====
 
* I. Mezic, [[Media:AnalysisFluidFlows.pdf | Analysis of Fluid Flows via Spectral Properties of the Koopman Operator]], ''Annual Review of Fluids,'' vol. 45, 357-378, 2013.
 
* I. Mezic, [[Media:AnalysisFluidFlows.pdf | Analysis of Fluid Flows via Spectral Properties of the Koopman Operator]], ''Annual Review of Fluids,'' vol. 45, 357-378, 2013.
* [[Media:FlowModelReductionReview.pdf | Model Reduction for Flow Analysis and Control]]
+
* C.W. Rowley and S.T.M. Dawson, [[Media:FlowModelReductionReview.pdf | Model Reduction for Flow Analysis and Control]], ''Annual Review Fluids'', 49:387-417, 2017.
 
* M.S Hemati, C.W. Rowley, E.A. Deem, L.N. Cattafesta, [[Media:DebiasingDMD.pdf | De-biasing the Dynamic Mode Decomposition for applied Koopman spectral analysis of noisy data sets]], ''arXiv:1502.03854v2''
 
* M.S Hemati, C.W. Rowley, E.A. Deem, L.N. Cattafesta, [[Media:DebiasingDMD.pdf | De-biasing the Dynamic Mode Decomposition for applied Koopman spectral analysis of noisy data sets]], ''arXiv:1502.03854v2''
 
* J.H. Tu, [[Media:DMDApplicationsTheory.pdf | Dynamic Mode Decomposition, Theory and Applications]], Ph.D. Thesis, Princeton, 2013.
 
* J.H. Tu, [[Media:DMDApplicationsTheory.pdf | Dynamic Mode Decomposition, Theory and Applications]], Ph.D. Thesis, Princeton, 2013.
 +
* S Bagheri, [[Media:ShearFlowsThesis.pdf | Analysis and Control of Transitional Shear Flows Using Global Modes]], Ph.D. Thesis, Royal Inst. Technology, Sweden, 2010.
  
 
==== Some Early Papers ====
 
==== Some Early Papers ====
 +
* Koopman's original paper: [[Media:KoopmanPNAS.pdf | Dynamical Systems of Continuous Spectra]], ''PNAS'' , vol. 18, 1932.
 +
* J. Ding, [[Media:PointSpectrum.pdf | The Point Spectrum of Frobenius-Perron and Koopman Operators]], ''Proc. AMS,'' vol. 126, no. 5, 1998.
 
* I. Mezic and A. Banaszuk, [[Media:ComparisonSystems.pdf | Comparison of Systems with Complex Behavior]], ''Physica D'', vol. 197, pp. 101-133, 2004.
 
* I. Mezic and A. Banaszuk, [[Media:ComparisonSystems.pdf | Comparison of Systems with Complex Behavior]], ''Physica D'', vol. 197, pp. 101-133, 2004.
 
* Y. Lan and I. Mezic, [[Media:LinearizationInTheLarge.pdf | Linearization in the Large of Nonlinear Systems and Koopman Operator Spectrum]], ''Physica D'', 2013
 
* Y. Lan and I. Mezic, [[Media:LinearizationInTheLarge.pdf | Linearization in the Large of Nonlinear Systems and Koopman Operator Spectrum]], ''Physica D'', 2013
Line 80: Line 115:
 
==== Other Papers ====
 
==== Other Papers ====
 
* E.Berger, M. Sastuba, D. Vogt, B. Jung, H.B. Amor, [[Media:EstimationRobotPerturbations.pdf | Estimation of Perturbations in Robotic Behavior using Dynamic Mode Decomposition]], ''Advanced Robotics,'' vol. 25, no. 5, 2015.
 
* E.Berger, M. Sastuba, D. Vogt, B. Jung, H.B. Amor, [[Media:EstimationRobotPerturbations.pdf | Estimation of Perturbations in Robotic Behavior using Dynamic Mode Decomposition]], ''Advanced Robotics,'' vol. 25, no. 5, 2015.
 +
* S. Wang, Z. Qiao, [[Media:NuclearNormDMD.pdf | Nuclear Norm Regularized Dynamic Mode Decomposition]], ''IET Signal Processing'', 2016.

Latest revision as of 17:43, 15 June 2018

This page gathers references and materials related to the study of

  • Gaussian Process (GP) State Space Models (SSM)
  • Deep Learning
  • Koopman Spectral Methods.

Gaussian Process Approaches

Basic Gaussian Process Info

Web Links

Papers on GP-SSMs


Deep Learning

Papers on Deep Learning Theory

Review-like papers

Papers on Scattering Networks

Scattering Networks are proposed by Stephan Mallat (of wavelet fame) to understand why deep nets work so well.

Classics

Web Links

Koopman Spectral Method

Papers on Koopman Spectral methods

Papers which are more oriented toward control

Papers which are more oriented toward fluids

Some Early Papers

Other Papers