Let's start off with the informal notions that have to be crystallized by any formal definition. The way I teach this in freshman physics is that a wave is something that has superposition, while a particle is something that you can't have half of. Conceptually, it's important to discard all the other psychological baggage that comes with these terms. For example, students tend to think that particles should have mass, and waves shouldn't, but this is wrong. People also tend to think that a particle has a trajectory, but this is not so, as shown by the Heisenberg uncertainty principle.
More formally, I think physicists usually define a wave to be something that obeys a hyperbolic partial differential equation, although this is a little too strict for QM. The interpretation is that we have an equation for which solutions to Cauchy problems exist and are unique. Typically a hyperbolic PDE has oscillatory solutions (as opposed to, e.g., the heat equation, which is parabolic). This sort of fits for quantum mechanics, but not quite. The nonrelativistic time-independent Schrodinger equation is an example of a hyperbolic PDE in quantum mechanics. However, the time-dependent Schrodinger equation (TDSE) actually does not fit the technical criteria, but it shows similar behavior, such as having oscillatory solutions and having existence and uniqueness for initial-value problems.
A hyperbolic PDE does not have to be linear. However, in quantum mechanics the Schrodinger equation is always exactly linear. This fits the conceptual motivation that waves obey superposition. Possibly a more appropriate way to capture the idea is that all states in quantum mechanics are connected. This is expressed by Hardy's axiom 5. So, e.g., you can have a state where the cat is dead, and a state where it's alive, and in Hardy's language "there exists a continuous reversible transformation" between the live and dead states.
Quantum mechanics doesn't actually have anything fundamentally to do with waves. If you look at an axiomatization such as the ones by Mackey or Hardy, they don't even require the existence of space. For example, you can have a quantum computer, which is just modeled as an interacting collection of qubits.
In nonrelativistic systems, quantum mechanics embodies the notion of a particle only very indirectly, through the fact that you assume a Hilbert space, and the Hamiltonian is unitary. This guarantees that wavefunctions can be normalized, and that they will stay normalized. Connecting this to the notion of a particle is an indirect interpretation, but the idea is that if we start with one electron, we will keep on having one electron -- not half an electron. Extending this to more than one particle is straightforward in the nonrelativistic case.
In relativistic systems, $E=mc^2$ says that in general we can create and destroy particles, so solutions are not going to be states of good particle number. However, the eigenvalues of the particle number operator are going to be integers. In systems like the Wightman axiomatization, this seems to come about in an indirect and tricky way, since the axioms don't directly mention anything about creation and annihilation operators. The axioms are, however, strong enough to prove things like spin-statistics, which implicitly refers to what we would think of as particles. In general, the conceptual advice I hear from people who are wise about QFT (which I am not) is that QFT is about fields, not particles. The particles are a secondary concept, not a primary one.
Summary
Wave-particle duality was thought in 1927, at the birth of quantum mechanics, to be a fundamental notion in QM, but actually modern axiomatizations of quantum mechanics don't refer to waves or particles. These concepts come up indirectly. The wave notion is embodied by the fact that, for many quantum-mechanical systems that play out in a background of space, the solutions are oscillatory, and also in the fact that we can always continuously transform any state to any other (as in wave superposition). The particle notion is embodied in QFT by the fact that particle number operators have integer eigenvalues.
References
Hardy, "Quantum theory from five reasonable axioms," http://arxiv.org/abs/quant-ph/0101012
Mackey, The Mathematical Foundations of Quantum Mechanics, 1963