On the left pane you have Browse and Radio, then (title) YOUR MUSIC and at the bottom you see Local Files. Click on this, you will see this list will all ...
I fixed this problem by opening the local files folder, (by default /Users/*username*/Music/Spotify) and deleting all the files and folders in that folder.