You can use selinput; for this particular case, a cyrillic letter can be used, together with a restricted set of encodings to look for.
\documentclass{article}
\usepackage{selinput}
\SelectInputEncodingList{utf8,cp1251}
\SelectInputMappings{
cyrd={д},
}
\begin{document}
\inputencodingname
\end{document}
If the file is encoded in UTF-8, compiling it will produce
utf8
If it's encoded in CP1251,
cp1251
will be produced.
However, your problem is different and not really solvable.
If your package has, say
\renewcommand{\chaptername}{Глава}
and is encoded in CP1251, there will be no way to use this in a UTF-8 encoded document, because TeX uses the input encoding only when expanding macros. It is completely irrelevant when TeX stores a definition.
In this case, \chaptername will expand to the sequence of characters
<C3><EB><E0><E2><E0>
(each <..> denotes a character code) with category code 13, that will be invalid in a UTF-8 encoded document.
You could use
\RequirePackage[utf8,cp1251]{inputenc}
\inputencoding{cp1251}
in the sergiokapone.sty file and
\protected@edef\chaptername{Глава}
in the code, that would make LaTeX expand \chaptername to
\IeC{\CYRG}\IeC{\cyrl}\IeC{\cyra}\IeC{\cyrv}\IeC{\cyra}
which is independent of the encoding, and the main document would use
\usepackage{sergiokapone}
\inputencoding{utf8} % or cp1251
or, possibly, selinput as shown before.
This is however not what I'd recommend. The best way to proceed is to make .sty files encoding independent; the above \renewcommand should be
\renewcommand{\chaptername}{\CYRG\cyrl\cyra\cyrv\cyra}
selinput– egreg Jan 03 '15 at 13:34inputencshould be left to the user IMHO. Personally I wouldn't bother and would just leave it... – cgnieder Jan 03 '15 at 14:28