Halifax says the drop is the biggest annual fall since 2011 as mortgage rates continue to rise for many.