The concave-convex procedure
- PMID: 12689392
- DOI: 10.1162/08997660360581958
The concave-convex procedure
Abstract
The concave-convex procedure (CCCP) is a way to construct discrete-time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically. This procedure can be applied to almost any optimization problem, and many existing algorithms can be interpreted in terms of it. In particular, we prove that all expectation-maximization algorithms and classes of Legendre minimization and variational bounding algorithms can be reexpressed in terms of CCCP. We show that many existing neural network and mean-field theory algorithms are also examples of CCCP. The generalized iterative scaling algorithm and Sinkhorn's algorithm can also be expressed as CCCP by changing variables. CCCP can be used both as a new way to understand, and prove the convergence of, existing optimization algorithms and as a procedure for generating new algorithms.
Similar articles
-
A proof of convergence of the concave-convex procedure using Zangwill's theory.Neural Comput. 2012 Jun;24(6):1391-407. doi: 10.1162/NECO_a_00283. Epub 2012 Feb 24. Neural Comput. 2012. PMID: 22364501
-
CCCP algorithms to minimize the Bethe and Kikuchi free energies: convergent alternatives to belief propagation.Neural Comput. 2002 Jul;14(7):1691-722. doi: 10.1162/08997660260028674. Neural Comput. 2002. PMID: 12079552
-
A novel neural dynamical approach to convex quadratic program and its efficient applications.Neural Netw. 2009 Dec;22(10):1463-70. doi: 10.1016/j.neunet.2009.03.020. Epub 2009 Apr 8. Neural Netw. 2009. PMID: 19410427
-
Stochastic reasoning, free energy, and information geometry.Neural Comput. 2004 Sep;16(9):1779-810. doi: 10.1162/0899766041336477. Neural Comput. 2004. PMID: 15265322
-
An alternative recurrent neural network for solving variational inequalities and related optimization problems.IEEE Trans Syst Man Cybern B Cybern. 2009 Dec;39(6):1640-5. doi: 10.1109/TSMCB.2009.2025700. Epub 2009 Aug 4. IEEE Trans Syst Man Cybern B Cybern. 2009. PMID: 19661003
Cited by
-
CALIBRATING NON-CONVEX PENALIZED REGRESSION IN ULTRA-HIGH DIMENSION.Ann Stat. 2013 Oct 1;41(5):2505-2536. doi: 10.1214/13-AOS1159. Ann Stat. 2013. PMID: 24948843 Free PMC article.
-
Information Geometry for Radar _target Detection with Total Jensen-Bregman Divergence.Entropy (Basel). 2018 Apr 6;20(4):256. doi: 10.3390/e20040256. Entropy (Basel). 2018. PMID: 33265347 Free PMC article.
-
Multitask learning for host-pathogen protein interactions.Bioinformatics. 2013 Jul 1;29(13):i217-26. doi: 10.1093/bioinformatics/btt245. Bioinformatics. 2013. PMID: 23812987 Free PMC article.
-
Big-data and machine learning to revamp computational toxicology and its use in risk assessment.Toxicol Res (Camb). 2018 May 1;7(5):732-744. doi: 10.1039/c8tx00051d. eCollection 2018 Sep 1. Toxicol Res (Camb). 2018. PMID: 30310652 Free PMC article. Review.
-
Joint Power Allocation and Hybrid Beamforming for Cell-Free mmWave Multiple-Input Multiple-Output with Statistical Channel State Information.Sensors (Basel). 2024 Sep 27;24(19):6276. doi: 10.3390/s24196276. Sensors (Basel). 2024. PMID: 39409316 Free PMC article.
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources
