reftex-index-phrases-case-fold-search

reftex-index-phrases-case-fold-search is a variable defined in `reftex-vars.el'.
Its value is

t

Documentation:
Non-nil means, searching for index phrases will ignore case.

You can customize this variable.