0

I am looking for an activation function that squashes $\mathbb{R}$ to $[0,100]$.

Currently I am using $$f(x) = \frac{100}{1+e^{-x}}$$ but this does not evenly distribute the values across the interval, they are skewed to both ends. By this I mean that all values above $x=5$ map to $\approx 100$ and all values below $x=-5$ map to $\approx 0$, which we can see if we plot the graph. I would like a function which widens the range of inputs which map to values in $(0,100)$.

Can anyone point me towards such a function please.

Ѕᴀᴀᴅ
  • 34,263
lioness99a
  • 4,943

3 Answers3

2

Any continuous increasing function $f: \mathbb R \to [0, 100]$ will "squash" most of $\mathbb R$. Supposing that $f$ surjects onto $(0, 100)$ -- making "maximal use of the available space" -- you have real numbers $r_1 = f^{-1}(0.005)$ and $r_2 = f^{-1}(99.995)$, and the entire infinite real line except for the part $[r_1, r_2]$ is squashed into one ten-thousandth of the interval $[0, 100]$.

Mees de Vries
  • 26,947
2

Modify your activation function by introducing a scale parameter $b$: $$ f(x)=\frac{100}{1+e^{-x/b}} $$ The larger $b$ is, the more spread out is the range of $x$ values that avoid getting slammed into the extremes of the interval $(0,100)$. Experiment with $b$ to get the behavior you want. Alternatively, if possible, scale the values being fed into the activation function to prevent them from getting too large.

grand_chat
  • 38,951
0

Such a function, not "squashing" $\mathbb R$, can be viewed as $100$x a cumulative distribution function of a uniform distribution on $\mathbb R$. However, such a distribution can't exist as is shown here:

Why isn't there a uniform probability distribution over the positive real numbers?

bsbb4
  • 3,636