I would be worried!
Most of the time, a fish corpse will last no more than 48 hours on the bank; foxes, crows, rats will eat it. Only otters and foxes are strong enough drag the body up the bank.
Otters have different types of 'patrol' or ranges. A dog otter may move around his territory, river to lake, over into female territories, whereas a female with cubs is likely to stay put. They will take the available food source, so if fish are within reach they will take them. They will also take coots, moorhens, and I have seen them take greylag geese and a year old Swan.
Just because you still have silver fish does not mean the otters have moved on. In fact if you have loads of silvers it could be a sign that the bigger fish are not there; perch and pike will munch on silvers, the ill, slow and dying, and keep excess numbers down.