I am having a hard time grasping the limit of sets as used in $\sigma$ algebra and probability space. What does the notation $\lim_{n\rightarrow\infty}S_n=S$ mean when all these $S_n$ are sets. I have some intuitive idea. But I understand limit in real analysis with the $\delta$-$\epsilon$ concept. So what is the rigorous definition of limit applied to sets, a definition parallel to the $\delta$-$\epsilon$ notation as applied to real analysis?
Citing one or two examples will be great.
When all you have is the raw set structure, the only limit concept that really makes sense is:
$S$ is the limit of the sequence $S_1, S_2, S_3,\ldots$ iff $$ \forall x\; \exists N\in\mathbb N\; \forall n>N : x\in S\Leftrightarrow x \in S_n $$
In other words every possible element is either in all but finitely many $S_n$ (in which case it is in the limit set too), or in only finitely many $S_n$ (in which case it is not in the limit set). If there is even one $x$ that is both present in infinitely many of the $S_n$ and absent from infinitely many of them, then the sequence does not have a limit.
This notion corresponds to a pointwise limit of indicator functions.
If the sequence of sets is increasing, then the limit is simply the union of all the sets. If it is decreasing then the limit is the intersection of all the sets.
We have (see e. g. Billingsley, Probiability and measure, 3rd edition, page 52), that the limes superior and the limes inferior are defined - as for any ordered space - on the power set:
Definition. Let $X$ be a set, $(S_n)$ a sequence in $\mathfrak P(X)$. We define
(1) The limes superior of $(S_n)$ is the set which contains all elements which are frequently in $S_n$, that is in infinitely many $S_n$: $$ \limsup S_n := \bigcap_{n\ge 1} \bigcup_{k\ge n} S_k $$ (2) The limes inferior of $(S_n)$ is the set which contains all elements which are finally in $S_n$, that is in all but finitely many $S_n$: $$ \liminf S_n := \bigcup_{n\ge 1} \bigcap_{k\ge n} S_k $$ (3) If $\liminf S_n$ and $\liminf S_n$ coincide, we say, $(S_n)$ converges and write $$ \lim S_n := \limsup S_n = \liminf S_n. $$
Example: If $S_n$ is montonically increasing, then $\bigcup_{k\ge n} S_k$ is equal for all $n$, hence $\limsup S_n = \bigcup_{k\ge 1} S_k$, on the other side, $\bigcap_{k\ge n} S_k = S_n$, hence $\liminf S_n = \bigcup_{n\ge 1} S_n$, that is $\lim S_n = \bigcup_n S_n$.
If you take a sequence $x_1,x_2,x_3,\ldots$ of real numbers then you can find $\displaystyle \liminf_{n\to\infty} x_n$ and $\displaystyle \limsup_{n\to\infty} x_n$. If these are equal to each other then you can call this value $\displaystyle \lim_{n\to\infty} x_n$.
The same idea can be applied apply to sets. $\displaystyle \liminf_{n \rightarrow \infty} S_n = \bigcup_{n \ge 1} \bigcap_{j \geq n} S_j$ and $\displaystyle\limsup_{n \rightarrow \infty} S_n = \bigcap_{n \ge 1} \bigcup_{j \geq n} S_j$ and if these are equal call the resulting set $\displaystyle \lim_{n \rightarrow \infty} S_n$.
As with sequences of real numbers, most sequences of sets (in a handwaving sense) will not have a limit.
[This is from "A Probability Path" by Resnick]
Supremum, infimum, limsup, liminf, limit of a sequence of sets can be defined via indicator functions.
Supremum and infimum of a seq of sets:
Consider sets ${ A _n \subseteq X . }$ We say ${ \sup \lbrace A _1, A _2, \ldots \rbrace = A }$ if and only if ${ \sup \lbrace 1 _{A _1} (x), 1 _{A _2} (x), \ldots \rbrace = 1 _A (x) }$ pointwise.
Note that
$${ \sup \lbrace 1 _{A _1} (x), 1 _{A _2} (x), \ldots \rbrace = {\begin{cases} 1 \quad \text{ if } x \text{ lies in some } A _n \\ 0 \quad \text{ otherwise} \end{cases}} = 1 _{\cup _{ n = 1} ^{\infty} A _n} (x) . }$$
Hence we have
$${ \sup \lbrace A _1, A _2, \ldots \rbrace = \cup _{ n = 1} ^{\infty} A _n . }$$
Similarly we have
$${ \inf \lbrace A _1, A _2, \ldots \rbrace = \cap _{ n = 1} ^{\infty} A _n . }$$
Note that this notion of supremum and infimum agrees with the usual notion of supremum and infimum with respect to the partial order ${ \subseteq . }$
The union is the smallest set containing all the sets in the sequence (least upper bound), and the intersection is the largest set contained in all the sets of the sequence (greatest lower bound).
Limsup and liminf of a seq of sets:
Consider sets ${ A _n \subseteq X . }$ We say ${ \limsup _{n \to \infty} A _n = A }$ if and only if ${ \limsup _{n \to \infty} 1 _{A _n} (x) = 1 _A (x) }$ pointwise.
Note that
$${ {\begin{align} &\, \limsup _{n \to \infty} 1 _{A _n} (x) \\ = &\, \lim _{j \to \infty} \left( \sup _{n \geq j} 1 _{A _n} (x) \right) \\ = &\, \lim _{j \to \infty} 1 _{ \cup _{n = j} ^{\infty} A _n } (x) \end{align}} }$$
and by monotone convergence
$${ {\begin{align} &\, \lim _{j \to \infty} 1 _{ \cup _{n = j} ^{\infty} A _n } (x) \\ = &\, \inf _{j \geq 1} 1 _{ \cup _{n = j} ^{\infty} A _n } (x) \\ = &\, 1 _{\cap _{j \geq 1} \cup _{n = j} ^{\infty} A _n } (x) . \end{align}} }$$
Hence we have
$${ \limsup _{n \to \infty} A _n = \cap _{j \geq 1} \cup _{n = j} ^{\infty} A _n = \lbrace x : x \in A _n \text{ for infinitely many } n \rbrace . }$$
Similarly we have
$${ \liminf _{n \to \infty} A _n = \cup _{j \geq 1} \cap _{n = j} ^{\infty} A _n = \lbrace x : x \in A _n \text{ for all but finitely many } n \rbrace . }$$
Limit of a seq of sets:
Consider sets ${ A _n \subseteq X . }$ We say ${ \lim _{n \to \infty} A _n = A }$ if and only if ${ \lim _{n \to \infty} 1 _{A _n} (x) = 1 _A (x) }$ pointwise.
Note that ${ \lim _{n \to \infty} A _n = A }$ if and only if
$${ \limsup _{n \to \infty} A _n = \liminf _{n \to \infty} A _n = A , }$$
as needed.