If you want to use text in math, you need to specify it as text. On the other hand, you can define the Sanskrit digits to work in text and math without further hassle.
\documentclass{article}
\usepackage{amsmath}
\usepackage{fontspec}
\usepackage{polyglossia}
\setmainlanguage{sanskrit}
\setmainfont{ITF Devanagari}[NFSSFamily=devn]
\newfontfamily{\latinfont}{Latin Modern Roman}
\DeclareSymbolFont{sanskritdigits}{\encodingdefault}{devn}{m}{n}
\Umathcode०="7 \symsanskritdigits "0966 \Umathcode१="7 \symsanskritdigits "0967
\Umathcode२="7 \symsanskritdigits "0968 \Umathcode३="7 \symsanskritdigits "0969
\Umathcode४="7 \symsanskritdigits "096A \Umathcode५="7 \symsanskritdigits "096B
\Umathcode६="7 \symsanskritdigits "096C \Umathcode७="7 \symsanskritdigits "096D
\Umathcode८="7 \symsanskritdigits "096E \Umathcode९="7 \symsanskritdigits "096F
\begin{document}
$\dfrac{\text{जी.श}}{५}$ $\dfrac{३४५}{८९}$
{\latinfont Text:} ०१२३४५६७८९
{\latinfont Math:} $०१२३४५६७८९$
\end{document}

unicode-mathpackage. I would suggest looking at the documentation here: http://texdoc.net/texmf-dist/doc/latex/unicode-math/unicode-math.pdf and investigating how to get what you want. If you do succeed, provide an answer to your question for the benefit of those who follow. – Don Hosek Dec 15 '20 at 15:13\frac{\text{/जी.श}}{\text{५}}$should work (\textfromamsmathpackage) – David Carlisle Dec 15 '20 at 15:28