As you note, the problem is a bad interaction with one of your packages. To manually overcome it, I define \mystrut which is a rule that sinks low enough and rises high enough so that adding it to a top row entry and to a bottom row entry, suitably grows the height of the brackets.
However, I also give this solution as an alternative approach, if you wish to control the vertical and horizontal spacing between matrix entries, which may be of use when displaying \textstyle fractions, as you do.
After giving your original solution, and the modified solution with the strut fix, I then give a solution that uses my under-construction tabstackengine.sty package, currently posted at Writing a table with equally spaced columns, based on the widest column
\documentclass{article}
\usepackage{amsmath, amssymb, mathtools, fouriernc}
\usepackage{tabstackengine}
\newcommand{\vecd}[1]{\mathbf{#1}}
\newcommand\mystrut{\rule[-1.5ex]{0ex}{4ex}}
\begin{document}
ORIGINAL
\[
\vecd{x}^{(k+1)} = \begin{bmatrix*}[r] 0 & \tfrac{1}{4} & -\tfrac{1}{4} \\[3pt] \tfrac{1}{3} & 0 & -\tfrac{1}{3} \\[3pt] -\tfrac{1}{5} & -\tfrac{1}{5} & 0 \end{bmatrix*} \vecd{x}^{(k)} + \begin{bmatrix*}[r] 4 \\ -1 \\ 1 \end{bmatrix*}
\]
MODIFIED ORIGINAL
\[
\vecd{x}^{(k+1)} = \begin{bmatrix*}[r] \mystrut0 & \tfrac{1}{4} & -\tfrac{1}{4} \\[3pt] \tfrac{1}{3} & 0 & -\tfrac{1}{3} \\[3pt] -\tfrac{1}{5} & -\tfrac{1}{5} & 0\mystrut \end{bmatrix*} \vecd{x}^{(k)} + \begin{bmatrix*}[r] 4 \\ -1 \\ 1 \end{bmatrix*}
\]
TABSTACKENGINE (1.8baselineskip vertical \& 1.8ex intercolumngap)
\setstackgap{L}{1.8\baselineskip}
\setstackTABgap{1.8ex}
\[
\mathbf{x}^{(k+1)} =
\bracketMatrixstack[r]{
0 & \frac{1}{4} & -\frac{1}{4}\\
\frac{1}{3} & 0 & -\frac{1}{3}\\
-\frac{1}{5} & -\frac{1}{5} & 0
}
\mathbf{x}^{(k)} + \bracketVectorstack[r]{4\\-1\\1}
\]
\end{document}

! Undefined control sequence. l.5 \vecdand! LaTeX Error: Environment bmatrix* undefined.Please correct your example – David Carlisle Oct 23 '13 at 13:54fourierncpackage. Perhaps Djjerome can edit his title or question to reflect that point. – Steven B. Segletes Oct 23 '13 at 17:22