Drawing local neuro-fuzzy architecture

Question

I am trying to draw some sort of neural network with additional nodes and connections. I have the following code so far, which I mostly copied from around here.

\documentclass[border=0.125cm]{standalone} \usepackage{tikz}
\usetikzlibrary{positioning}
\begin{document}

\tikzset{every neuron/.style={ circle, draw, minimum size=1cm },
neuron missing/.style={ draw=none, scale=3, text height=0.2cm, execute
at begin node=\color{black}$\vdots$ }, }

\begin{tikzpicture}[x=1.5cm, y=1.5cm, >=stealth]

\foreach \m/\l [count=\y] in {1,2,missing,3} \node [every
neuron/.try, neuron \m/.try] (input-\m) at (0,2.5-\y) {};

\foreach \m [count=\y] in {1,2,missing,3} \node [every neuron/.try,
neuron \m/.try ] (hidden-\m) at (2,2.5-\y) {};

\foreach \m [count=\y] in {1,2,missing,3} \node [every neuron/.try,
neuron \m/.try ] (output-\m) at (4,2.5-\y) {};

\foreach \l [count=\i] in {1,2,K} \draw [<-] (input-\i) -- ++(-1,0)
node [above, midway] {$x_\l$};

\foreach \l [count=\i] in {1,2,K} \node [above] at (input-\i.north)
{$I_\l$};

\foreach \l [count=\i] in {1,2,L} \node [above] at (hidden-\i.north)
{$H_\l$};

\foreach \l [count=\i] in {1,2,M} \node [above] at (output-\i.north)
{$S_\l$};

\foreach \l [count=\i] in {1,2,M} \draw [->] (output-\i) -- ++(1,0) node
[above, midway] {$w_\l$};

\foreach \i in {1,...,3} \foreach \j in {1,...,3} \draw [->]
(input-\i) -- (hidden-\j);

\foreach \i in {1,...,3} \foreach \j in {1,...,3} \draw [->]
(hidden-\i) -- (output-\j);

\foreach \l [count=\x from 0] in {Input, Hidden, Softmax} \node
[align=center, above] at (\x*2,2.5) {\l \\ layer};

\end{tikzpicture}

\end{document}

This leads to the following image:

But in the end I would like to achieve the following (or something similar):

I am not married to the exact design, I just need a well-arranged and comprehensible visual model of the network architecture outlined above.

Answer 1

Like this:

For exercise I rewrote your MWE in more concise form and add missing parts frame around x inputs and L outputs as well nodes L and output neuron. for orientation in nodes see its comments:

\documentclass[tikz, border=0.125cm]{standalone}
\usetikzlibrary{calc, fit, positioning, quotes}% new libraries

\begin{document}
\tikzset{
  every neuron/.style={circle, draw, minimum size=8mm},
neuron missing/.style={draw=none, scale=3, text height=0.2cm, 
                       execute at begin node=\color{black}$\vdots$}, 
  layer labels/.style={above, align=center}
        }
    \begin{tikzpicture}[x=16mm, y=16mm, >=stealth]
% neuron nodes with part of labels
\foreach \m [count=\y] in {1,2,missing,3}
\foreach \j/\l in {0/I, 2/H, 4/S}
{ 
\ifnum\y<3
\node [every neuron/.try, neuron \m/.try,
       label=$\l_\y$] (n\j\m) at (\j,2.5-\y) {};
\else
\node [every neuron/.try, neuron \m/.try] (n\j\m) at (\j,2.5-\y) {};
\fi       
}
% neuron labels not included in neuron nodes
\foreach \l/\k in {I_K/0, H_L/2, S_M/4}
\node [above] at (n\k3.north) {$\l$};
% inputs
\foreach \l [count=\i] in {1,2,K} 
\draw [<-] (n0\i.west) -- ++(-1.1,0) node (in\i) [above, midway] {$x_\l$};
\node (input) [draw, inner ysep=2mm, yshift=-2mm, fit=(in1) (in3)] {};
% w and L outputs
\foreach \l [count=\i] in {1,2,M}
\draw [->] (n4\i.east) -- ++(1.6,0) 
    node (wout\i) [above, midway] {$x_\l$}
    node (Lout\i) [right, draw, minimum size=8mm, label=$L_\i$] {};% Local Model
\node [neuron missing] at ($(Lout2)!0.5!(Lout3)$) {};
\node (woutput) [draw, inner ysep=2mm, yshift=-2mm, fit=(wout1) (wout3)] {};
% output
\node (output) [every neuron,right=16mm] at ($(Lout1.east)!0.5!(Lout3.east)$) {};
\draw [->] (output.east) to["$\hat{y}$"] ++(1.1,0);
% neurons interconection
 \foreach \i in {1,2,3}
 \foreach \j in {1,2,3}
{
\draw [->] (n0\i) -- (n2\j);
\draw [->] (n2\i) -- (n4\j);
}
\foreach \j [count=\i] in {1,2,M}
\draw [->] (Lout\i.east) to ["$\hat{y}_\j$"] (output);
% neuron layers labels
\foreach \l [count=\x from 0] in {Input, Hidden, Softmax} 
\node [layer labels] at (\x*2,2.2) {\l \\ layer};
\node [layer labels] at (6,2.2)    {Local \\ Model};
\node [layer labels] at (8,2.2)    {Output \\ Agregation};
% x-fit L-fit conections
\draw[dashed,->] (input.south)   -- ++ (0,-1)   -| (Lout3)  
                        node [pos=0.25,above] {$\underline{x}$};
\draw[dashed,->] (woutput.south) -- ++ (0,-0.5) -| (output) 
                        node [pos=0.75,right] {$\underline{w}$};
   \end{tikzpicture}
\end{document}