Jump to content

Logarithmically concave function

From Wikipedia, the free encyclopedia

In convex analysis, a non-negative function f: Rn → R+ is logarithmically concave[1] (or log-concave for short) if its domain is a convex set, and if it satisfies the inequality

f(θx+(1−θ)y)≥f(x)θf(y)1−θ

for all x, y ∈ dom f and 0 < θ < 1. If f is strictly positive, this is equivalent to saying that the logarithm of the function, log ∘ f, is concave; that is,

log⁡f(θx+(1−θ)y)≥θlog⁡f(x)+(1−θ)log⁡f(y)

for all x, y ∈ dom f and 0 < θ < 1.

Examples of log-concave functions are the 0-1 indicator functions of convex sets (which requires the more flexible definition), and the Gaussian function.

Similarly, a function is log-convex if it satisfies the reverse inequality

f(θx+(1−θ)y)≤f(x)θf(y)1−θ

for all x, y ∈ dom f and 0 < θ < 1.

For non-negative discrete functions f: Z → R+, it is log-concave [2] if

f(k)2≥f(k+1)f(k−1)

Properties

[edit]
  • A log-concave function is also quasi-concave. This follows from the fact that the logarithm is monotone implying that the superlevel sets of this function are convex.[3]
  • Every concave function that is nonnegative on its domain is log-concave. However, the reverse does not necessarily hold. An example is the Gaussian function f(x) = exp(−x2/2) which is log-concave since log f(x) = −x2/2 is a concave function of x. But f is not concave since the second derivative is positive for |x| > 1:
f″(x)=e−x22(x2−1)≰0
  • From above two points, concavity ⇒ log-concavity ⇒ quasiconcavity.
  • A twice differentiable, nonnegative function with a convex domain is log-concave if and only if for all x satisfying f(x) > 0,
f(x)∇2f(x)⪯∇f(x)∇f(x)T,[3]
i.e.
f(x)∇2f(x)−∇f(x)∇f(x)T is
negative semi-definite. For functions of one variable, this condition simplifies to
f(x)f″(x)≤(f′(x))2

Operations preserving log-concavity

[edit]
  • Products: The product of log-concave functions is also log-concave. Indeed, if f and g are log-concave functions, then log f and log g are concave by definition. Therefore
log⁡f(x)+log⁡g(x)=log⁡(f(x)g(x))
is concave, and hence also f g is log-concave.
  • Marginals: if f(x, y): Rn+m → R is log-concave, then
g(x)=∫f(x,y)dy
is log-concave (see Prékopa–Leindler inequality).
  • This implies that convolution preserves log-concavity, since h(x, y) = f(x − y) g(y) is log-concave if f and g are log-concave, and therefore
(f∗g)(x)=∫f(x−y)g(y)dy=∫h(x,y)dy
is log-concave.

Log-concave distributions

[edit]

Log-concave distributions are necessary for a number of algorithms, e.g. adaptive rejection sampling. Every distribution with log-concave density is a maximum entropy probability distribution with specified mean μ and Deviation risk measure D.[4] As it happens, many common probability distributions are log-concave. Some examples:[5]

Note that all of the parameter restrictions have the same basic source: The exponent of non-negative quantity must be non-negative in order for the function to be log-concave.

The following distributions are non-log-concave for all parameters:

Note that the cumulative distribution function (CDF) of all log-concave distributions is also log-concave. However, some non-log-concave distributions also have log-concave CDF's:

The following are among the properties of log-concave distributions:

  • If a density is log-concave, so is its cumulative distribution function (CDF).
  • If a multivariate density is log-concave, so is the marginal density over any subset of variables.
  • The sum of two independent log-concave random variables is log-concave. This follows from the fact that the convolution of two log-concave functions is log-concave.
  • The product of two log-concave functions is log-concave. This means that joint densities formed by multiplying two probability densities (e.g. the normal-gamma distribution, which always has a shape parameter ≥ 1) will be log-concave. This property is heavily used in general-purpose Gibbs sampling programs such as BUGS and JAGS, which are thereby able to use adaptive rejection sampling over a wide variety of conditional distributions derived from the product of other distributions.
  • If a density is log-concave, so is its survival function.[5]
  • If a density is log-concave, it has a monotone hazard rate (MHR), and is a regular distribution since the derivative of the logarithm of the survival function is the negative hazard rate, and by concavity is monotone i.e.
ddxlog⁡(1−F(x))=−f(x)1−F(x) which is decreasing as it is the derivative of a concave function.

See also

[edit]

Notes

[edit]
  1. ↑ Khademi, Abbas (2026-11-18). "The convexity zoo: a taxonomy of function classes in optimization". Optimization. 75 (15). Taylor & Francis: 4215–4266. doi:10.1080/02331934.2026.2672525. ISSN 0233-1934.
  2. ↑ Johnson, O., 2007. Log-concavity and the maximum entropy property of the Poisson distribution. Stochastic Processes and their Applications, 117(6), pp.791-802.
  3. 1 2 Boyd, Stephen; Vandenberghe, Lieven (2004). "Log-concave and log-convex functions". Convex Optimization. Cambridge University Press. pp. 104–108. ISBN 0-521-83378-7.
  4. ↑ Grechuk, Bogdan; Molyboha, Anton; Zabarankin, Michael (May 2009). "Maximum Entropy Principle with General Deviation Measures". Mathematics of Operations Research. 34 (2): 445–467. doi:10.1287/moor.1090.0377.
  5. 1 2 See Bagnoli, Mark; Bergstrom, Ted (2005). "Log-Concave Probability and Its Applications" (PDF). Economic Theory. 26 (2): 445–469. doi:10.1007/s00199-004-0514-4. S2CID 1046688.
  6. 1 2 Prékopa, András (1971). "Logarithmic concave measures with application to stochastic programming" (PDF). Acta Scientiarum Mathematicarum. 32 (3–4): 301–316.

References

[edit]
  • Barndorff-Nielsen, Ole (1978). Information and exponential families in statistical theory. Wiley Series in Probability and Mathematical Statistics. Chichester: John Wiley \& Sons, Ltd. pp. ix+238 pp. ISBN 0-471-99545-2. MR 0489333.
  • Dharmadhikari, Sudhakar; Joag-Dev, Kumar (1988). Unimodality, convexity, and applications. Probability and Mathematical Statistics. Boston, MA: Academic Press, Inc. pp. xiv+278. ISBN 0-12-214690-5. MR 0954608.
  • Pfanzagl, Johann; with the assistance of R. Hamböker (1994). Parametric Statistical Theory. Walter de Gruyter. ISBN 3-11-013863-8. MR 1291393.
  • Pečarić, Josip E.; Proschan, Frank; Tong, Y. L. (1992). Convex functions, partial orderings, and statistical applications. Mathematics in Science and Engineering. Vol. 187. Boston, MA: Academic Press, Inc. pp. xiv+467 pp. ISBN 0-12-549250-2. MR 1162312.