A solution is to redefine \sectionmark and \subsectionmark. This can be done as follows:
\usepackage{fancyhdr}
\pagestyle{fancy}
\renewcommand{\subsectionmark}[1]{\markright{#1}}
\renewcommand{\sectionmark}[1]{\markboth{#1}{}}
This will produce the same look as in the pictures, but now without the section/subsection numbers and in ordinary letters instead of capital letters. If you still would want the section/subsection numbers, then a posibbility is:
\usepackage{fancyhdr}
\pagestyle{fancy}
\renewcommand{\subsectionmark}[1]{\markright{\thesubsection \quad #1}}
\renewcommand{\sectionmark}[1]{\markboth{\thesection \quad #1}{}}
There is one important thing to note. The two \renewcommand MUST appear AFTER the command \pagestyle{fancy}. Besides this there is infinitely many possibilities for choosing a formating of these 'reminders' should look.
What the commands \markright and \markboth do is that, \markright sets the value of \rightmark and \markboth sets the value of both \leftmark and \rightmark. These commands \leftmark and \rightmark can thus be used to customize the headder and footer, e.g. as follows:
\lfoot{}
\rfoot{}
\cfoot{ \thepage }
\rhead{ {\scshape \leftmark} }
\lhead{ {\scshape \rightmark} }
\chead{}
The reason why you would want to set the \rightmark to be blank when setting the \leftmark as done in the solutions with \renewcommand above, is that you would not want, e.g., to list the name of the last subsection of the previous section untill you create a subsection in the new section.
If you have choosen a documentclass that supports chapters then you should do the above construction with \chaptermark replacing \sectionmark and \sectionmark replacing \subsectionmark.