Lulatex - Unicode character error

Question

I'm using a \LaTeX document using both Devnagari and Latin fonts.

The command used is: lualatex -shell-escape -interaction nonstopmode

However, despite selecting the appropriate fonts. The minimal code is:

I also tried using the polyglossia package, (as per https://tex.stackexchange.com/a/24731/50653), but still no difference.

\documentclass[10pt]{article}
\usepackage{polyglossia}
\setdefaultlanguage{english}
\setotherlanguage{hindi}
% From https://tex.stackexchange.com/a/37251
\usepackage{fontspec}
\setmainfont{Noto Serif} % sets the roman font
\setsansfont{Noto Sans} % sets the sans font
\setmonofont{Fira Code} % sets the monospace font
\defaultfontfeatures{Ligatures=TeX} % makes this a feature for all selected fonts
\newfontfamily\hindifont[Script=Devanagari]{Noto Serif Devanagari}
\newfontfamily\hindifontsf[Script=Devanagari]{Noto Sans Devanagari}
\begin{document}
All the places marked with red triangles are \emph{sites} from which archaeologists have found evidence of hunter-gatherers. (Hunter-gatherers lived in many more places. Only some are shown on the map). Many sites were located near sources of water, such as rivers and lakes.
येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।
\end{document}

The obtained result is:

While one expects the Devanagari characters to be rendered as well.

I had earlier also tried

\documentclass[10pt]{article}
\usepackage[hindi, english]{babel}
% From https://tex.stackexchange.com/a/37251
\usepackage{fontspec}
\setmainfont{Noto Serif} % sets the roman font
\setsansfont{Noto Sans} % sets the sans font
\setmonofont{Fira Code} % sets the monospace font
\defaultfontfeatures{Ligatures=TeX} % makes this a feature for all selected fonts
\begin{document}
All the places marked with red triangles are \emph{sites} from which archaeologists have found evidence of hunter-gatherers. (Hunter-gatherers lived in many more places. Only some are shown on the map). Many sites were located near sources of water, such as rivers and lakes.
येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।
\end{document}

Answer 1

There are two issues at play here: The primary problem is that the selected font does not contain the characters in question, the other issue is that using fontspec fontloading commands without explicitly specifying a script and language default to Latin script and therefore does not shape indic scripts correctly.

If you do not want to worry about the right script/language parameters for fontspec, you can use babel's fontspec wrapper \babelfont to specify the fonts for your languages. For this to work correctly (especially for language specific shaping to work) you have to indicate language changes in your document using \selectlanguage, \foreignlanguage or the otherlanguage / otherlanguage* environments. If you write a whole paragraph in another language you can use \selectlanguage{new-language}, for parts of a paragraph there is \foreignlanguage{new-language}{text-in-new-language}.

You can find details about all this in the babel manual, section 1.2 (Multilingual documents), section 1.7&1.8 (Basic/Auxiliary language selectors) and section 1.14 (Selecting fonts).

\documentclass[10pt]{article}
\usepackage[hindi, english]{babel}
\babelfont{rm}{Noto Serif} % sets the roman font
\babelfont{sf}{Noto Sans} % sets the sans font
\babelfont{tt}{Fira Code} % sets the monospace font
\babelfont[hindi]{rm}{Noto Serif Devanagari}
\babelfont[hindi]{sf}{Noto Sans Devanagari}
\begin{document}
All the places marked with red triangles are \emph{sites} from which archaeologists have found evidence of hunter-gatherers. (Hunter-gatherers lived in many more places. Only some are shown on the map). Many sites were located near sources of water, such as rivers and lakes.
\foreignlanguage{hindi}{येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।}
More English.
\selectlanguage{hindi}
येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए। येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।
येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए। येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।
येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए। येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।
\selectlanguage{english}
And English again.
\end{document}

As Davislor mentioned in a comment, the following requires a pretty modern TeX installation (updated in February 2020 or later) and therefore might not work on your system or break if you try to compile your document on another system:

If you really, really can't insert such language switching commands, you only use one language from every script system and you do not care about proper paragraph spacing in Devanagari text (e.g. because it is mostly latin) you can also use onchar=ids fonts: Replace

\usepackage[hindi, english]{babel}

with

\usepackage[english]{babel}
\babelprovide[import, onchar=ids fonts]{hindi}

Then babel automatically switches the font and hyphenation patterns to Hindi when it encounters Devanagari characters. (Of course, you can switch english and hindi if you mostly have Hindi text.)

Answer 2

Welcome to TeX.SX!

The LuaTeX shaper dev2 has been a bit troublesome with Indic scripts, but there's been another version called deva out, as mentioned in https://tex.stackexchange.com/a/362738/90407. However, recently, LuaTeX has additionally support to use the HarfBuzz shaper instead of the built-in one. Both variants should give comparable results, albeit with slight differences. In both cases we need to create font switches for the new font family however. Using polyglossia we can add another language and define new font families for that language which will handle all the font switching between sans-serif and serif automatically:

LuaTeX deva

\documentclass{article}
\usepackage{polyglossia}
\setdefaultlanguage{english}
\setotherlanguage{hindi}
\usepackage{fontspec}
% define Devanagari script using `deva' backend
\newfontscript{Devanagari}{deva,dev2}
\setmainfont{Noto Serif}
\setsansfont{Noto Sans}
% Create font switches for Devanagari font
% These are hard-coded into polyglossia, if we name them differently
% we get:
%     Package polyglossia Error: The current latin font [...] does
%     not contain the "Devanagari" script!
%     Please define \devanagarifont with \newfontfamily command.
\newfontfamily{\devanagarifont}{Noto Serif Devanagari}[Script=Devanagari]
\newfontfamily{\devanagarifontsf}{Noto Sans Devanagari}[Script=Devanagari]
\defaultfontfeatures{Ligatures=TeX}
\begin{document}
All the places marked with red triangles are \emph{sites} from which archaeologists have found evidence of hunter-gatherers. (Hunter-gatherers lived in many more places. Only some are shown on the map). Many sites were located near sources of water, such as rivers and lakes.
\texthindi{येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।}
All the places marked with red triangles are \emph{sites} from which archaeologists have found evidence of hunter-gatherers. (Hunter-gatherers lived in many more places. Only some are shown on the map). Many sites were located near sources of water, such as rivers and lakes.
\texthindi{\textsf{येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।}}
\end{document}

The result is:

LuaTeX with HarfBuzz (LuaHBTeX)

Alternatively, we don't define a new script but simply ask HarfBuzz to do the heavy lifting:

\documentclass{article}
\usepackage{polyglossia}
\setdefaultlanguage{english}
\usepackage{fontspec}
\setmainfont{Noto Serif}
\setsansfont{Noto Sans}
% Create font switches for Devanagari font
% These are hard-coded into polyglossia, if we name them differently
% we get:
%     Package polyglossia Error: The current latin font [...] does
%     not contain the "Devanagari" script!
%     Please define \devanagarifont with \newfontfamily command.
\newfontfamily{\devanagarifont}{Noto Serif Devanagari}[Script=Devanagari,Renderer=Harfbuzz]
\newfontfamily{\devanagarifontsf}{Noto Sans Devanagari}[Script=Devanagari,Renderer=Harfbuzz]
\defaultfontfeatures{Ligatures=TeX}
\begin{document}
All the places marked with red triangles are \emph{sites} from which archaeologists have found evidence of hunter-gatherers. (Hunter-gatherers lived in many more places. Only some are shown on the map). Many sites were located near sources of water, such as rivers and lakes.
\texthindi{येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।}
All the places marked with red triangles are \emph{sites} from which archaeologists have found evidence of hunter-gatherers. (Hunter-gatherers lived in many more places. Only some are shown on the map). Many sites were located near sources of water, such as rivers and lakes.
\texthindi{\textsf{येह एक प्रयोग है हिन्दी की लिखावट देखने के लिए।}}
\end{document}

Which yields:

---

Taking my browsers rendering as reference, this looks quite okay to me.

---

If you don't care about proper spacing, you can do this without polyglossia, with something like this:

\usepackage[Devanagari]{ucharclasses}
\setTransitionsFor{Devanagari}{\devanagarifont}

However, this will only switch the font and do away with any other language adjustments otherwise implemented by polyglossia.