The mentioned Algorithm-Style Package for LyX (version 0.3) does not provide LyX module support since it was written for an older LyX version without module support. My approach creates a LyX module that can be used with every document class. Hence, it circumvents the necessity of the original package to use a certain document class.
The instructions below will install the package and add module support. Actually they replace the instructions of Readme.txt of the package. The files algorithm.inc and article-algo.layout of the original package are not needed anymore. $HOME will stand for your home directory.
- unpack the archive
algo-0.3.tar.gz,
- copy the file
algorithm.inc to $HOME/.lyx/layouts and rename it to algolyx.module,
insert the following lines at the very beginning of the file algolyx.module
#\DeclareLyXModule{Algorithm support for LyX}
#DescriptionBegin
#This module provides a text style to write algorithms using the
#packages "algorithm" and "algorithmic".
#DescriptionEnd
(Now algolyx.module became the needed module.)
in LyX run "Tools > Reconfigure" and restart LyX,
copy the file algolyx.sty to the directory $HOME/texmf/tex/latex (needs to be created possibly) and run texhash (or alternatively, copy the file algolyx.sty into each directory that contains a LyX document using this module).
For your document, add the module "Algorithm support for LyX" under "Document settings > Modules". In an algorithm float ("Insert > Float > Algorithm") choose the text style "Algorithm" and write an algorithm. The indentation is controlled by using nesting of lists, that is, "Increase depth" Alt+Shift+Right and "Decrease depth" Alt+Shift+Left. Each statement starting without any keyword must be preceded by a star *. Here is an example:

For more examples, have a look at my answer here or the example provided by algo-0.3.tar.gz (to get it working with the modified setup, you need to add the module "Algorithm support for LyX" and change the document class to "article").
algopackage, which I linked to in my question, doesn't work for me – inspectorG4dget May 23 '13 at 14:16~/Library/Application Support/LyX-2.0/layoutinstead of~/.lyx/layoutsince you are using Mac? – e-birk May 25 '13 at 17:17texhashI had to Reconfigure and restart lyx again. Also on debian I had to putalgolyx.styunder/usr/local/share/texmf/as root.Now it works, thanks.
– Kasper Ziemianek Jan 02 '15 at 13:17