The books listed here either have some chapter(s) on hierarchical planning, or are specifically devoted to hierarchical planning. Since there so many versions of books like newer editions that come with different ISBNs etc. only titles and authors are provided, but no bibtex entries (as also publishers might change depending in which version one refers to).
Ordered from most to least recent:
Ghallab, Nau, Traverso book series
Artificial Intelligence – A Modern Approach
by S. Russell, P. Norvig, Prentice Hall, 2010.
(Only approx. 10/1300 pages about hierarchical planning, more is spent on non-hierarchical planning.)
Intelligent Planning – A Decomposition and Abstraction Based Approach
by Q. Yang, Springer, 1997.
(The entire book is about hierarchical planning.)
Practical Planning: Extending the Classical AI Planning Paradigm
by David E. Wilkins, Morgan Kaufmann, 1988.
(Among others, it focuses on SIPE, an early HTN planning system.)
Readings in Planning
by James Allen, James Hendler, and Austin Tate, Morgan Kaufmann, 1990.
(Summarizes work on AI planning and reprints key papers from the 1960s to the early 1990s up to and including work on hierarchical partial-order planners such as NOAH, NONLIN and SIPE. See table of contents and index.)
Should any book be missing, please reach out!