After some playing around, I discovered the root of the problem was the titles loading-option of tocloft. Remove that option and it worked as expected. Searching the site revealed this question, tocloft package interfering with ToC page style?, and Mico's answer to it.
Specifically, Mico notes this:
The manual of the tocloft package has the following to say about this option:
The titles option causes the titles of the ToC, LoF, and LoT lists to be
typeset using the default LATEX methods. This can be useful, for example,
when the tocloft and fncychap packages are used together and the 'fancy'
chapter styles should be used for the ToC, etc., titles.
This very explicitly points to the problem and the solution as well. The titles option is intended for use, when one wishes to use some of the tocloft features, but not actually set the toc using the tocloft features. The solution? remove the titles option.
\documentclass[11pt,letterpaper]{article}
\usepackage{tocloft}
\title{Hello world \vspace{-0.5cm}}
\begin{document}
\maketitle
\setlength\cftaftertoctitleskip{-100pt}
\tableofcontents
\section{A}
A stuff
\section{B}
B stuff
\end{document}
\cftaftertoctitleskipfeature gets broken by the[titles]option oftocloft. That is because (see Mico's answer at http://tex.stackexchange.com/questions/137299/tocloft-package-interfering-with-toc-page-style) thetitlesoption purposefully bypasses thetocloftmechanisms for setting thetoc,lof,lot, etc. – Steven B. Segletes Aug 08 '16 at 18:54tocloftwithout the[titles]option, if you wish to use the\cftaftertoctitleskipmechanism. – Steven B. Segletes Aug 08 '16 at 19:00