Typing in ' and then " results in LyX automatically converting these into ' and ”. The code changes into '{}'' so that the output is right single quote and then right double quote ’ ”. Without the {}, just as ''', the output would be right double quote and then right single quote ” ’. The same is true for ``{}`.
This is supposed to become left single quote and then left double quote ‘ “ and not left double quote and then left single quote “ ‘.
This works fine as expected in pdflatex.
However, LuaTex ignores the {} and treats the sequence as ''' or ```.
The fix one side is simple, if the user types ' and then ", change the code from '{}'' to '{}\textdblquoteright. As a matter of fact, change ALL cases of typed " to \textdblquoteright.
This is a problem in typical sentences like the following sentence I would type:
Brian said, ``Alice turned to me and yelled,Stop!'"`
Lyx displays:
Brian said, “Alice turned to me and yelled, ‘Stop!’”
Lyx's code:
Brian said, ``Alice turned to me and yelled, `Stop!'{}''
pdflatex output:
Brian said, “Alice turned to me and yelled, ‘Stop!’”
LuaTex output (incorrect):
Brian said, “Alice turned to me and yelled, ‘Stop!”’
Solving the left quote problem is very hard. If you have three quotations as in the following, there is no easy way out other than manually coding \textdblquoteleft. The following is fine, because double quotes come first:
Lucy said, “‘Sting’ is Gordon's stage name.”
The following requires typing out \textdblquoteleft to display correctly in LuaTex:
Alice said, “Lucy said, ‘“Sting” is Gordon's stage name.’”
As it is, LuaTex would display this as:
Alice said, “Lucy said, “‘Sting” is Gordon's stage name.”’