Mathematics

Questions tagged [stochastic-approximation]

This tag is for questions about stochastic approximation which are a family of methods of iterative stochastic optimization algorithms that attempt to find zeroes or extrema of functions which cannot be computed directly, but only estimated via noisy observations.

84 questions
13
votes
0 answers

Convergence of a Stochastic Process - Am I missing something obvious?

In the paper On the Convergence of Stochastic Iterative Dynamic Programming Algorithms (Jaakkola et al. 1994) , the authors claim that a statement is "easy to show". I am starting to suspect that it isn't easy and they might have just wanted to…
Felix Benning
  • 3,187
8
votes
1 answer

Variance converging to zero implies weak convergence to delta measure?

I have the following question: Suppose that I have a sequence of random variables $X_n$ such that all moments exists and are finite. I have that $E[X_n]\to a$, where $a$ is a finite number and also $\text{Var}(X)\to 0$. We may also assume that a…
Liealgebrabach
  • 622
7
votes
1 answer

Proof for convergence of stochastic gradient descent to a local optimum for non convex functions

Let's say I have a (multivariable) function $F(x) : \mathbb{R}^n \rightarrow \mathbb{R}$ , which I would like to minimize. There are no assumptions made on $F$ , besides it being differentiable and having a gradient of the form $$\nabla F(x) \propto…
Markus Peschl
  • 387
7
votes
1 answer

Simple application of Donsker's theorem

I am trying to do exercise 5.15 in Moerter's book "Brownian Motion". It seems quite easy, but I can't solve it somehow: Suppose $S(j)_j$ is a SRW on the integers, started at zero. Show that: $$…
Liealgebrabach
  • 622
5
votes
0 answers

Consequence of Dvoretzky Stochastic Approximation Theorem

I am having some problems trying to apply Dvoretzky Stochastic Approximation Theorem to one Lemma used in a paper I found about the proof of convergence of some reinforcement learning temporal difference methods. Jaakkola & Jordan & Singh claim that…
Kareit
  • 51
5
votes
1 answer

Algorithms for Stochastic Continuous Optimization

Question I have a continuous optimization problem of the form $$ \max_{x \in \mathbb R^n} f(x), $$ where $f:\mathbb R^n \rightarrow \mathbb R$ is mostly smooth and bounded above. The standard approximation algorithms for these kinds of problems,…
Frank Seidl
  • 1,016
5
votes
2 answers

How likely are two events to occur at the same time?

Let's think of two events $1$ and $2$. Both events happen randomly $n_1$/$n_2$-times during a given time $T$ and last for a time of $t_1$/$t_2$. What is probability $P$, that both events happen simultaneously at some moment? EXAMPLE 1: $T = 60$ min…
lukas.simon
  • 77
5
votes
0 answers

Finding a unique strong solution

I am brushing up on my stochastic approximation. I am having a hard time with the following problem. I have the equation $dX_t = \ln(1+ X_t^2)dt + X_tdB_t$ $X_0 = x$ , with $x \in\mathbb R$ I know that this equation has a unique, strong solution.…
Phil.D
  • 51
4
votes
0 answers

Time Discretization

I wonder why we work with constant discretization in Time Discretization of numerical approximation for numerical scheme if we take not necessarily constant Discretization Is the numerical scheme (Such : Euler, Runge-Kutta4, Euler-maruyama, milstein…
Educ
  • 4,956
4
votes
0 answers

Is there something like "stochastic induction"?

I'm trying to prove convergence of a stochastic approximation-like algorithm. I have two questions about prove-techniques when working with randomness. 1. For a non-random sequence $(a_t)_t$ one could prove $a_t \rightarrow a$ by induction as…
Nick Halden
  • 118
4
votes
1 answer

Convergence of a real sequence (stochastic approximation)

Let $(\gamma_n)_{n \geq 1}$ be a sequence of positive real numbers satisfying $$\sum_{n \in \mathbb{N}}\gamma_n = + \infty \text{ and }\sum_{n \in \mathbb{N}}\gamma_n^2 < + \infty$$ I would like to know if the sequence $(u_n)_{n \geq 1}$ defined by…
mfrt
  • 138
4
votes
0 answers

difference of independent Rayleigh random variables

How do I find the probability density distribution (pdf) of the difference of independent Rayleigh random variables (whose probability density functions are known)? Assume $X$, $Y \sim \text{Rayleigh}(\sigma)$, $X$ and $Y$ are i.i.d. Then what…
Ricky
  • 41
3
votes
0 answers

Maximize or find an upper bound of the function $kx^{k-1}\exp(-\mu(x^k-x))$

I was programming some random variable simulation using the acceptance-rejection method and I encounter with the Weibull$(k,\lambda)$ distribution. This random variable is posible to simulate with this method when $k \geq 1$. To do this I use a…
user111216
  • 269
3
votes
0 answers

Application of Gram-Charlier expansion for Swaption pricing with drift extension

I am doing a project for university and I'm stuck at some point. The aim is to implement the Gram-Charlier expansion into the 2-factor Hull-White model with Drift extension. I already found out how to implement it without drift extension but I'm not…
cleoser
  • 31
3
votes
0 answers

Polynomial chaos expansion and ODEs

I originally asked this question on stats.SE but I didn't even get a handful of views. So I figured that here is probably a more appropriate site to ask. I am trying to figure out how to use PCE to quantify uncertainty in solutions of ODEs. So if I…
rom
  • 851
  • 6
  • 15
1
2 3 4 5 6