Frank D. Grosshans 
Algebraic Homogeneous Spaces and Invariant Theory [PDF ebook] 

The invariant theory of non-reductive groups has its roots in the 19th century but has seen some very interesting developments in the past twenty years. This book is an exposition of several related topics including observable subgroups, induced modules, maximal unipotent subgroups of reductive groups and the method of U-invariants, and the complexity of an action. Much of this material has not appeared previously in book form. The exposition assumes a basic knowledge of algebraic groups and then develops each topic systematically with applications to invariant theory. Exercises are included as well as many examples, some of which are related to geometry and physics.