If memory serves me right ..........once the cable has been unhooked from the brake shoe lever , there is a u clip that retains the cable ferrule in the backplate.
If you have removed the u clip......it is most likeley that it is just corrosion holding it firm.
You may have to get angry with it
