Uncertainties and changes are pervasive characteristics of modern systems involving interactions between humans, economics, nature and technology. These systems are often too complex to allow for precise evaluations and, as a result, the lack of proper management (control) may create significant risks. In order to develop robust strategies we need approaches which explic- itly deal with uncertainties, risks and changing conditions. One rather general approach is to characterize (explicitly or implicitly) uncertainties by objec- tive or subjective probabilities (measures of confidence or belief). This leads us to stochastic optimization problems which can rarely be solved by using the standard deterministic optimization and optimal control methods. In the stochastic optimization the accent is on problems with a large number of deci- sion and random variables, and consequently the focus ofattention is directed to efficient solution procedures rather than to (analytical) closed-form solu- tions. Objective and constraint functions of dynamic stochastic optimization problems have the form of multidimensional integrals of rather involved in- that may have a nonsmooth and even discontinuous character – the tegrands typical situation for "hit-or-miss" type of decision making problems involving irreversibility ofdecisions or/and abrupt changes ofthe system. In general, the exact evaluation of such functions (as is assumed in the standard optimization and control theory) is practically impossible. Also, the problem does not often possess the separability properties that allow to derive the standard in control theory recursive (Bellman) equations.