We can steal the code for dashed names from IEEEtran.bst.
The heart of the "dahsed" feature is the function
FUNCTION {name.or.dash}
{ 's :=
oldname empty$
{ s 'oldname := s }
{ s oldname =
{ "---" }
{ s 'oldname := s }
if$
}
if$
}
A call to that function needs to be added after each function call that would print the primary name of an entry.
The necessary changes to turn apalike.bst into apalike-dashed.bst are
--- apalike.bst 2010-12-10 10:19:51.000000000 +0100
+++ apalike-dashed.bst 2019-01-28 17:29:20.877333300 +0100
@@ -1,3 +1,9 @@
+%% apalike-dashed.bst
+%% https://tex.stackexchange.com/q/472240/35864
+%% 2019-01-28 MW
+%% modification of apalike.bst with dashed functionality from IEEEtran.bst
+%% original header follows
+%% --------------------------------------------------------------------------
% BibTeX `apalike' bibliography style (version 0.99a, 8-Dec-10), adapted from
% the `alpha' style, version 0.99a; for BibTeX version 0.99a.
%
@@ -76,7 +82,7 @@
#3 'after.block :=
}
-STRINGS { s t }
+STRINGS { s t oldname }
FUNCTION {output.nonnull}
{ 's :=
@@ -479,9 +485,22 @@
" \cite{" * crossref * "}" *
}
+FUNCTION {name.or.dash}
+{ 's :=
+ oldname empty$
+ { s 'oldname := s }
+ { s oldname =
+ { "---" }
+ { s 'oldname := s }
+ if$
+ }
+ if$
+}
+
FUNCTION {article}
{ output.bibitem
format.authors "author" output.check
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -504,9 +523,11 @@
{ output.bibitem
author empty$
{ format.editors "author and editor" output.check
+ name.or.dash
editor format.key output
}
{ format.authors output.nonnull
+ name.or.dash
crossref missing$
{ "author and editor" editor either.or.check }
'skip$
@@ -537,6 +558,7 @@
FUNCTION {booklet}
{ output.bibitem
format.authors output
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -553,9 +575,11 @@
{ output.bibitem
author empty$
{ format.editors "author and editor" output.check
+ name.or.dash
editor format.key output
}
{ format.authors output.nonnull
+ name.or.dash
crossref missing$
{ "author and editor" editor either.or.check }
'skip$
@@ -588,6 +612,7 @@
FUNCTION {incollection}
{ output.bibitem
format.authors "author" output.check
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -615,6 +640,7 @@
FUNCTION {inproceedings}
{ output.bibitem
format.authors "author" output.check
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -644,6 +670,7 @@
FUNCTION {manual}
{ output.bibitem
format.authors output
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -660,6 +687,7 @@
FUNCTION {mastersthesis}
{ output.bibitem
format.authors "author" output.check
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -676,6 +704,7 @@
FUNCTION {misc}
{ output.bibitem
format.authors output
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -690,6 +719,7 @@
FUNCTION {phdthesis}
{ output.bibitem
format.authors "author" output.check
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -706,6 +736,7 @@
FUNCTION {proceedings}
{ output.bibitem
format.editors output
+ name.or.dash
editor format.key output % special for
output.year.check % apalike
new.block
@@ -724,6 +755,7 @@
FUNCTION {techreport}
{ output.bibitem
format.authors "author" output.check
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
@@ -740,6 +772,7 @@
FUNCTION {unpublished}
{ output.bibitem
format.authors "author" output.check
+ name.or.dash
author format.key output % special for
output.year.check % apalike
new.block
You can download apalike-dashed.bst from https://gist.github.com/moewew/fe37f610e010abb45ac14f4f318393f3
\documentclass[british]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{babel}
\usepackage{natbib}
\usepackage{filecontents}
\begin{filecontents}{\jobname.bib}
@book{appleby:a,
author = {Humphrey Appleby},
title = {On the Importance of the Civil Service I},
year = {1980},
publisher = {Oxford University Press},
}
@book{appleby:b,
author = {Humphrey Appleby},
title = {On the Importance of the Civil Service II},
year = {1980},
publisher = {Clarendon Press},
}
\end{filecontents}
\begin{document}
\cite{appleby:a,appleby:b}
\bibliographystyle{apalike-dashed}
\bibliography{\jobname}
\end{document}

If you want the dash only for same author and same year, you could use
FUNCTION {name.or.dash}
{ 's :=
oldname empty$
{ s 'oldname :=
year 'oldyear :=
s }
{ s oldname =
year oldyear =
and
{ "---" }
{ s 'oldname :=
year 'oldyear :=
s }
if$
}
if$
}
instead of the definition above and you would have to declare the new variable oldyear
STRINGS { s t oldname oldyear }
I have never seen a style that only replaces authors from the same year, so I would not recommend doing that.