Julia set
Julia set, set of values in range of holomorphism of function, but out of holomorphism of some its integer iteration.
Julia set is often defined with symbol $J$ or $\mathbb J$. The name of the function can be indicated either as the subscript or in the parenthesis immediately after this symbol.
Let $f$ be holomorphic function defined at some $C\in \mathbb C$.
Then
$\mathbb J(f) = \{ z \in C : \exists ~ n\in \mathbb N_+ ~:~ f^n(z) \bar \in C\}$
where the upper subscript after the name of the function indicates the number of iteration. On this case, the number of iteration is supposed to be integer.
Fatou set
The complementary set to Julia set is called Fatou set; symbol $\mathbb F$ is used to denote it:
$\mathbb F(f) = \{ z \in C : \forall ~ n\in \mathbb N_+ ~,~ f^n(z) \in C\}$
in such a way that
$\mathbb J(f) \cup \mathbb F(f)=C$
At least for positive integer $n$, the following relations hold:
\[f^n(\mathbb J(f)) = \mathbb J(f)\]
\[ f^n(\mathbb F(f)) = \mathbb F(f)\]
Fractal behavior
Usually, the Julia set of any non–trivial function with at least one singularity shows complicated, fractal behaviour; the similar structures reproduce again and again, displaced and scaled.
Science about properties of these fractals, and, in particular, those of the Julia sets and the Fatou sets, is called Complex dynamics.
Non–integer iterates
The definition of the Julia set above implies that the function $f$ is iterated integer number of times. In principle, similar set can be considered, assuming, that the number of iterate $n$ can take also non–integer values.
References
http://en.wikipedia.org/wiki/Julia_set