Read Numerical Operations On Functions
When you work numerically, you should always ask yourself if the results depend or not on your working precision (WorkingPrecision).
You should only trust calculations that are stable with respect to the parameters used for the discretization of the problem.
Particularly with WorkingPrecision, it should be high enough that small changes in WorkingPrecision will give similar results, with variations within your acceptable tolerances.
In your case clearly, you are looking at an artefact of poor parameters for the discretization, like the small default (WorkingPrecision), as the results look different for different choices for WorkingPrecision values.
Check other options like: MaxStepFraction, MaxSteps, MaxStepSize, Method, PrecisionGoal, StartingStepSize, etc.
LogLogPlot[
Evaluate[
ReplaceAll[
P[x]/(P[x] + Q[x]),
First@NDSolve[
{
P'[x] + (P[x] + Q[x])^(1/2)* P[x] == -10^3 P[x] - P[x]^2,
Q'[x] + (P[x] + Q[x])^(1/2)* Q[x] == 10^3 P[x] + P[x]^2,
P[0.1] == 10^-3, Q[0.1] == 10
}
, { P[x], Q[x] }
, { x, 0.1, 10^5 }
, WorkingPrecision->#*$MachinePrecision
]
]& /@ Echo@PowerRange[1,8,2]
]
, {x, 0.1, 10^5}
, PlotRange -> {{10^-2,10^5},{10^-40, 10}}
, PlotPoints -> 1000
, PlotLegends->Evaluate[Row[{"WorkingPrecision: ", #, "\[Times] $MachinePrecision"}]& /@ PowerRange[1,8,2]]
]
