reftex-index-math-format

reftex-index-math-format is a variable defined in `reftex-vars.el'.
Its value is

"$%s$"

Documentation:
Format of index entries when copied from inside math mode.
When `reftex-index-selection-or-word' is executed inside TeX math mode,
the index key copied from the buffer is processed with this format string
through the `format' function. This can be used to add the math delimiters
(e.g. `$') to the string.
Requires the `texmathp.el' library which is part of AUCTeX.

You can customize this variable.