As you can read in the reference to Wiki or into a lot of good mathematical logic textbooks, there are some basic concepts in play :
formal system
language.
Propositional logic is based on a specific language; first-order logic (or predicate logic) is based on a "more wide" (i.e.more expressive) language.
In both cases we need formation rules, to build "correct expression", like terms and formulas.
Then we need transformation rules, usually called inference rules (e.g modus ponens): at least one, ususally more than one, and axioms (zero or more).
With this in place, we have a calculus or proof system: the basic concept of a proof system is that of derivation (from axioms or assumptions) of theorems.
Upo to now we have introduced the syntax; then we add the semantics that allows us to "give meaning" to terns (they have denotation) and formula (that stay for sentences).
In first-order logic (or predicate calculus) a predicate is a symbol; when we interpret it, the "standard" semantics for predicates are subset of the domain of our interpretation.
If we "apply" f-o logic to arithemetic, we may use a predicate like $\le$, and a term like $0$, in order to build a formula like :
$x \le 0$.
This formula is constructed with the binary predicate $\le$ (a binary realtion), and it holds for all numbers that are less or equal than $0$. So, in the domain of the natural numbers, this formula will be true for a number $k$ iff
$k \in \{ n \in \mathbb{N} : n \le 0 \} = \{ 0 \}$.
Added
For a "formal" approach, you can see Heinz-Dieter Ebbinghaus & Jörg Flum & Wolfgang Thomas, Mathematical logic (1984), Ch.XII : Characterizing First-Order Logic :
some results, due to Lindstrom, [...] show that first-order logic occupies a unique place among logical systems [...] :
(a) There is no logical system with more expressive power than first-order logic, for which both the compactness theorem and the Löwenheim-Skolem theorem hold.
(b) There is no logical system with more expressive power than first-order logic, for which the Löwenheim-Skolem theorem holds and for which the set of valid sentences is enumerable.
Where :
Definition 1.1. A logical system $\mathscr L$ consists of a function $L$ and a binary relation $\vDash_{\mathscr L}$.
$L$ associates with every symbol set $S$ a set $L(S)$, the set of $S$-sentences of $\mathscr L$.
The following properties are required:
(a) If $S_0 \subset S_1$, then $L(S_0) \subset L(S_1)$.
(b) If $\mathfrak A \vDash_{\mathscr L} \varphi$ (i.e., if $\mathfrak A$ and $\varphi$ are related under $\vDash_{\mathscr L}$), then, for some $S, \mathfrak A$ is an $S$-structure and $\varphi \in L(S)$.
(c) (Isomorphism property) If $\mathfrak A \vDash_{\mathscr L} \varphi$ and $\mathfrak A \cong \mathfrak B$ then $\mathfrak B \vDash_{\mathscr L} \varphi$.
(d) (Reduct property) If $S_0 \subset S_1, \varphi \in L(S_0)$, and $\mathfrak A$ is an $S_1$-structure then $\mathfrak A \vDash_{\mathscr L} \varphi$ iff $\mathfrak A \upharpoonright S_0 \vDash_{\mathscr L} \varphi$.
[...] in the case of [first-order logic] $\mathscr L_I$ we choose $L_I$ to be the function which assigns to a symbol set $S$ the set $L_I(S) := L_0^S$ of first-order $S$-sentences, and we take $\vDash_{\mathscr L}$ to be the usual satisfaction relation between structures and first-order sentences.