You would get 75 % loss if you took $1-\frac{final~relative~velocity^2}{initial~relative~velocity^2}$. Instead, try $\displaystyle 1-\frac{KE_{1,f}+KE_{2,f}}{KE_{1,i}+KE_{2,i}}$.

Doing what you mentioned would give me this:

$1-\frac{KE_{1,f}+KE_{2,f}}{KE_{1,i}+KE_{2,i}}$

$KE_{1,i}=\frac{1}{2} 2\times 5^2= 25\,J$

$KE_{2,f}=\frac{1}{2} 1\times \left(-10\right)^2=50\,J$

$KE_{1,f}= \frac{1}{2} 2\times \left(-2.5\right)^2= 6.25\,J$

$KE_{2,f}=\frac{1}{2} 1\times \left( 5\right)^2= 12.5\,J$

Therefore:

$1-\frac{KE_{1,f}+KE_{2,f}}{KE_{1,i}+KE_{2,i}}=1-\frac{6.25+12.5}{25+50}=1-\frac{18.75}{75}=0.75$

Which is exactly what I did in my notes book!.