reftex-toc-keep-other-windows

reftex-toc-keep-other-windows is a variable defined in `reftex-vars.el'.
Its value is

t

Documentation:
Non-nil means, split the selected window to display the *toc* buffer.
This helps to keep the window configuration, but makes the *toc* small.
When nil, all other windows except the selected one will be deleted, so
that the *toc* window fills half the frame.

You can customize this variable.